Skip to main content

Splunk - Add more fields to events using Lookups


In this article we will see how to join multiple datasets in Splunk using Lookup. Consider below two datasets:
Vendor Sales Data: Provides the detail of which vendors have sold what products.



Prices Data: Master data of the products like product name, price, sale price and code for each product.




Code is the common field between 2 data sets

Requirement is to display the Products sold by the vendors along with the vendor details and product details To get this information we would need to join the two datasets on a common field which is "Code" Splunk provides lookup command in order to do that.

I already have uploaded the vendor sales data to the index "index2". You can refer to my blog 
"Upload a dataset to Splunk" for the steps to upload the data. Next step is to upload the prices data, which will be used as a lookup table to get the information on additional fields.

Goto Settings -> Lookups -> Lookup table files -> New Lookup Table File

Leave the destination app field to default.
Upload your lookup file "Prices.csv"
Provide a destination, for example "prices.csv"

Hit Save

Goto Settings -> Lookups -> Lookup definitions -> New Lookup Definition

Leave the destination app as default "search"
Provide a name, for example "prices"
Select a type -> File-based
Select the lookup file you uploaded "prices.csv"


Go back to Search & Reporting app

Now we can use a command similar to below to join the two datasets using lookup command on the Common field "Code"

index=index2 source="tutorialdata.zip:./vendor_sales/vendor_sales.log" | lookup prices Code | table AcctID, VendorID, Code, productId, product_name












Comments

Post a Comment

Popular posts from this blog

Configure Oracle ASM Disks on AIX

Configure Oracle ASM Disks on AIX You can use below steps to configure the new disks for ASM after the raw disks are added to your AIX server by your System/Infrastructure team experts: # /usr/sbin/lsdev -Cc disk The output from this command is similar to the following: hdisk9 Available 02-T1-01 PURE MPIO Drive (Fibre) hdisk10 Available 02-T1-01 PURE MPIO Drive (Fibre) If the new disks are not listed as available, then use the below command to configure the new disks. # /usr/sbin/cfgmgr Enter the following command to identify the device names for the physical disks that you want to use: # /usr/sbin/lspv | grep -i none This command displays information similar to the following for each disk that is not configured in a volume group: hdisk9     0000014652369872   None In the above example hdisk9 is the device name and  0000014652369872  is the physical volume ID (PVID). The disks that you want to use may have a PVID, but they must not belong to a volume group. PVID must be cleared for
2 comments
Read more

Adding New Disks to Existing ASM Disk Group

Add Disks to Existing ASM Disk Group In this blog I will show how to add new disks to an existing ASM Disk group. This also contains the steps to perform the migration from existing to the new storage system. In order to add the disk to the ASM disk group, you will first need to configure these disk using the operating system commands. I have provided the steps to configure the disks on AIX system in my blog " Configure Oracle ASM Disks on AIX" Adding New Disks to DATA Disk Group (Storage Migration for DATA Disk Group) Login to your ASM instance $ sqlplus / as sysasm If the name of the new disk is in different format from the existing disk, the modify the asm_diskstring parameter to identify the new disks. In my below example /dev/ora_data* is the format of the existing disks and /dev/new_disk* is the naming format of the newly configured disks. You should not modify this parameter unless the naming format changes. SQL> alter system set asm_diskstring = '/dev/ora_data*
Post a Comment
Read more

Installing Splunk on AWS EC2 - Red Hat Linux

Installing Splunk on AWS EC2 - Red Hat Linux In this article we will see how to install Splunk on Red Hat Linux using Amazon AWS EC2 instance. We would first need to create a EC2 instance on Amazon AWS. Steps to Create EC2 Instance Login to your AWS console.  Under Services -> Click on EC2 Click on Launch Instance Choose an Amazon Machine Image (AMI). In my case case, I am using "Red Hat Enterprise Linux 8 (HVM)" that is available as free tier. Choose an Instance Type - General purpose - t2.micro, as this is eligible as free tier. Click on Review and Launch In the next step, it will show your EC2 instance configuration. Click on "Launch". Another window will open saying to "Select an existing key pair or create a new key pair". You will need to create a new key pair or choose an existing if you already have one. Key-pair file is a .pem file which is used to connect to your AWS EC2 instance using password less authentication. Download key pair file and C
Post a Comment
Read more