In this blog we are explaining the different types of lookups in splunk.
Why do we need lookup?
In splunk search query, we sometimes get values which may not clearly convey the meaning of the field. For example, we may get a field which lists the value of product id as a numeric result. These numbers will not give us any idea of what kind of product it is. But if we list the product name along with the product id, that gives us a good report where we understand the meaning of the search result.
Such linking of values of one field to a field with same name in another dataset using equal values from both the data sets is called a lookup process.
Advantage of lookup?
we retrieve the related values from two different data sets.
Three types of lookup
1. CSV lookup
2. KV store lookup
3. Automatic lookup
CSV LOOKUP
CSV lookup pulls data from CSV files. It populates the event data with fields and represents it in the static table of data. Therefore, it is also called as a “static lookup”. There must be at least two columns representing field with a set of values. They can have multiple instances of the same value.
CSV lookups are best for small sets of data. The general workflow for creating a CSV lookup in Splunk Web is to upload a file, share the lookup table file, and then create the lookup definition from the lookup table file.
Go to setting>lookups
Click on lookup table files
Crate a new lookup table file
Add new
After save it the lookup has been created.
CSV lookup is also called as static lookup because csv file represent static tables of data.Each column in csv table is interpreted as a potential value of field.
For csv we have created a lookup table now we are creating lookup definition for price.csv.
Add new
After saving the lookup has created Prices_lookup
Upload a another csv file product.csv
Add new lookup definition
Save it
Three important commands for lookup are inputlookup, outputlookup and lookup
Inputlookup: it is use to search the content of the table. A lookup table can be csv lookup or kv store lookup
Write a query |inputlookup Product.csv
What we are trying to do here is matching the value of Code field from the search output with the lookup table and output new field product_name from lookup table and output new field product_name from lookup table mactching with the code.
sourcetype=vendor_sales|stats count by Code VendorID|lookup Prices_lookup Code OUTPUT product_name
sourcetype=vendor_sales|stats count by Code VendorID|lookup Prices_lookup Code OUTPUT product_name,price
sourcetype=vendor_sales|stats count by Code VendorID|lookup Prices_lookup Code OUTPUT product_name,price|table product_name VendorID count
sourcetype=vendor_sales|stats count by Code VendorID|lookup Prices_lookup Code OUTPUT product_name,price|table product_name VendorID count|lookup Product_lookup Code
KV STORE LOOKUP & OUTPUTLOOKUP
It populates your event data with fields pulled from your App Key Value Store (KV Store) collections. This lookup matches the fields in your event to fields in a KV store.
KV Store lookups can be invoked through REST endpoints or by using the following search commands: lookup, inputlookup, and outputlookup.
login to splunk
create a collections.conf in local
Stanza:
source="tutorialdata.zip:*" | outputlookup kvstorelookup
|inputlookup kvstorelookup
Now transform.conf
Stanza for transforms.conf
AUTOMATIC LOOKUP
Automatic lookup helps to configure a new lookup to run automatically or edit an existing one.
Add new automatic lookup
source=access* |stats values(price) As price BY productName
Here are the three types of Lookups.
If you are still facing an issue Feel free to Ask Doubts in the Comment Section Below and Don’t Forget to Follow us on 👍 Social Networks. Happy Splunking 😉
