Latest Updates

random

Implement Split Indexing in Splunk

Split indexing is used to point some events from the same source to alternate index based on field value.

“This usecase is related to customer query, where customer wants to divide data from single source into two internal teams for dashboarding and analysis purpose. Due to some internal team data exchange security policy, data needs to be hidden so that every team could see data related to their work only.”

Team A is handling “error data” and Team B is handling “Info data”


Solution:
We have one field log_level and it has two values info and error. Now, what we want to do is we need to send info logs to index=info and error logs to index=error.

Data onboarded for this usecase , contains two fields log_level and msg.
Note: Msg field contains, data as per info and error.
Note: Split indexing should be done at index time.
Create props.conf - Create sourcetype for data to be indexed and for transforms configurations.

  • [split_csv]                                             
  • TRANSFORMS-index = routing_to_index          //Transforms.conf initialisation 

Create transforms.conf - To route data to alternate index based on log_level field value.

  • [routing_to_index]
  • DEST_KEY = _MetaData:Index    // send the portion of data to index mentioned in FORMAT{error}
  • REGEX = ^error                            //It will search for regex pattern (error keyword) and inde
  • FORMAT = error                          //mention index name where error log to be store

Create inputs.conf - To monitor the onboarded data file.

  • [monitor:///opt/split_testing.csv]
  • index = info
  • sourcetype = split_csv
  • disabled = false

Overall output will be shown as following.

Info messages will go to index=info. It will show only info logs from log_level field as error log already split to error index.


Error message will go to index=error



That’s all! As you can see the messages are getting indexed into their respective indexes.

If you are still facing issue regarding this topic Feel free to Ask Doubts in the Comment Box Below and Don't Forget to Follow us on πŸ‘ Social Networks, happy Splunking >πŸ˜‰
Implement Split Indexing in Splunk Reviewed by Avotrix.Author on Tuesday, December 10, 2019 Rating: 5
All Rights Reserved by Avotrix © 2018 - 2019
Developed and created by Avotrix

Contact Form

Name

Email *

Message *

Powered by Blogger.