Cisco SMA - Apply Data Storage Time

[maraz]

Hello,  If I want to use the Apply data Storage Time function the dokumentation states the following:

"Important: From Secure Email and Web Manager 13.6.2 version, the Splunk database is no longer used for email tracking data.
All new email tracking data is stored in the Lucene database.
When you use the 'Apply Data Storage Time' option, the Splunk database that contains the email tracking data gets deleted automatically.

Action: Make sure you take a backup of the email tracking data (if required).
You can use the backupconfig command in the CLI to perform the backup action. For more information, see Scheduling Single or Recurring Backups."

What does that mean? If I have installed the SMA with a version after 13.6.2 then I am safe because it is already using the Lucene database?

If I have done an upgrade from version 12 (in several steps) up to version 14, then all my message tracking will be lost because I am using the Splunk database? 

[REJR77]

Hi,

Just got this problem today. I opened a TAC case and looks that the TAC can remove the Splunk DB via a remote access tunnel.

From what I understand since 13.6 the Tracking is stored in the new Lucene DB so if you remove the Splunk DB you will only loose tracking info before your upgrade.

HTH

[maraz]

But did you ask Cisco if that is the case? Because our customer need to have the full tracking info intact.

[REJR77]

Hi,

Actually to delete the Splunk DB, we just needed to activte the Apply Data Storage Time for Email tracking. This will delete the splunk DB. From what I understand all tracking collected before the new Lucene DB is lost.

Question to Cisco TCA:

For my information, is it correct that from 13.6 the new tracking DB is a Lucene DB, so if we remove the Splunk DB we will only lose data before the upgrade to recent release? Or Do the old DB is somehow copied to the Lucene DB?

Answer

Yes, correct,  you will not lose any date from after you have upgraded to the version 13.6.2.

[maraz]

Great, I interpret the way you do it i. e. all tracking before 13.6.2 will be lost.

[charella]

A little Clarification.

History:

- Splunk is the original DB used for message Tracking.

- 2018 ESA 12.x and 202 SMA 13.6 introduced Lucene.

- Once the applicable version's loaded and onward, no data gets written to the splunk DB. It is still present and working.

- The plan: Using the "Disk Management size" method of deleting old data as new data gets ingested, the Splunk DB eventually gets pushed out of the SMA

- Message Tracking data retention has always been based on the Size of the allocated space in "Disk Management."

- This new feature "Apply Data Storage Time" offers the option to manipulate the setting based on "number of days." Since it utilizes the new Lucene DB, the use of this option is/was not compatible with Splunk and will remove the Splunk DB if selected.

Aside from this, the splunk would be retained until the upgrade to 15.

------------ 15.0 will delete the SPLUNK DB Permanently -------------

This is the true date of termination. There will be warnings during upgrade to 15 as well as noted in the release notes that Splunk DB files will get deleted.

Who should be concerned? If you waited until 2023 to upgrade to 13.6 or newer, then you may have to wait for your data in spunk db to age out before upgrading to 15.x.

---- Changes in 15!!!

15.0 SMA Tracking lucene has introduced a new single log line method of writing the data producing faster search times once converted.

This will also undergo a similar transition as the existing Lucene data ages out and gets replaced with the new SLL storage method. When completed, the tracking performance will experience a drastic improvements in speed.