Customers are always on the look out to purchase additional storage for meeting their end to end  requirments such as regulatory compliance needs, advanced analytics ,cost etc. To tailor the increased storage and data retention needs , splunk cloud have introduced a new features called as DDSS: Dynamic Data Self storage and DDAA:Dynamic Data Active Archive

 Let us discuss each one of the feature in detail

DDSS

Self storage gives you the flexibility to move the data out of splunk cloud.This data once moved from the splunk cloud system ,it is under your control

Points to take into consideration regarding DDSS:

  1. Amazon S3 bucket should be in the same region as that of the splunk cloud instance
  2. Raw data is exported and in jounal.gz or tar.gz format.
  3. Since the self storage is not managed by splunk cloud,customer is responsible for monitoring purposes
  4. When restoring the data into splunk cloud no additional license cost is incurred since the data is already indexed

After the retention period,frozen data bucket  will be written (raw data) into the S3 bucket specified  .Since the data is not in the splunk cloud to make it searchable you need to spin up aws instance, rebuild all the buckets and then search it

DDAA

Dynamic Data Active Archive(DDAA) is a service of archiving index data in Splunk Cloud, when subscribed to this service in Splunk Cloud, we can set a searchable retention(HOT+WARM+COLD) in days and when searchable retention are met data can roll over to DDAA which can be later restored for period upto 30 days.

We can set DDAA storage retention when creating a new index or can edit the existing one just like below. For Dynamic Data Storage options we have to select Splunk Archive Radio Button and set the retention in days and save it.

When data from search retention is rolled out to DDAA and it needs to be retrieved, we can restore the data from UI itself we don’t need any separate Splunk instance to archive the data like in Data
 Storage Self Storage (DDSS). Data in DDAA can be restored in less amount of time depending on the data size which needs to be restored.

Restoring of the index data from DDAA –

When the index is set to send data to DDAA storage the options in the index will change we can see the Restore option, which is used to restore the specific amount of data from DDAA storage, click on Restore and we can add Time Range Earliest to Latest time data we want to restore. After selecting the time range and adding other expected attributes we can check the size of the data which is going to be restored and click Restore it will take up to 6 – 24 hrs to restore data depending on the size of the data as shown in below fig.

When data is restored we can search the data in index=gis_adobe(depending on each index name).

Note – The restored data will be separated from the actual index data bucket and the restored data will be available in the Splunk Cloud instance for 30 days after that the restored data will be deleted.

The  two options provided by the Splunk cloud for data retention can be summed up from the diag below:

If you still have any doubt regarding data retention option in splunk cloud, Feel free to Ask your Doubts in the Comment Section below ,give your ratings and Don’t Forget to Follow us on 👍 Social Networks.| Happy Splunking 😉