Skip to main content

Searching, Analyzing and Visualizing in Splunk

Searching, Analysing and Visualizing in Splunk



In this article we will see how to Analyse and Visualize a dataset using Splunk "Search & Reporting App"

We have already uploaded a dataset (.csv file) that contains information on confirmed cases of Covid-19 across different countries.

Refer to my blog "Adding dataset to Splunk" to learn about uploading a CSV dataset file to Splunk.

The dataset is free to download from https://data.humdata.org

Once you have finished loading your dataset, Click on Search & Reporting. 

Check that your dataset is uploaded successfully. In my case, I have uploaded the dataset in an index with the name "index2" and the name of my csv file is "time_series_covid19_confirmed_global.csv". 

As my index "index2" contains the information of other dataset as well, so I will put another filter to include the source name.
Based on when the dataset was uploaded, you can adjust the drop down which I have left to default "All time"
All the events (records) from the dataset are displayed in the Events tab.
On the left side of the result all the fields which are extracted by Splunk engine from the csv file are displayed.

Analyzing the Dataset 

index="index2" source="time_series_covid19_confirmed_global.csv"






Understanding the Dataset Fields:

Country_Region: Name of the Country or Region
Case Count: There are several date fields in the dataset, which represents the total confirmed cases of Covid-19 for that Country till the date mentioned in the field name. For example: field name 6_10_20 holds the confirmed cases for each country until 10th June 2020.
Lat & Long Fields: Latitude and Longitude coordinates of each country.

In the next step, we will use the table command to show the data of the interested fields in a tabular format.

As the dataset I have used is until 10th June 2020, so I will be using only the last date field 6_10_20 which contains the total confirmed cases till 10th June 2020.

index="index2" source="time_series_covid19_confirmed_global.csv" | table Country_Region 6_10_20 Lat Long




We will do the following in next step of the query:

Rename the field 6_10_20 to "ConfirmedCases"
Sort the output table in descending order with ConfirmedCases field
Use the head command to pick only top 20 countries 
Again sort the output with country_Region field


index="index2" source="time_series_covid19_confirmed_global.csv" | table Country_Region 6_10_20 | rename 6_10_20 as ConfirmedCases | sort - ConfirmedCases | head 20 | sort Country_Region ConfirmedCases



Click on Save As to save the output table to a New Splunk Dashboard with the Title "Covid-19 Dashboard" and Panel Title as "Top 20 Countries by Case Count - Tabular"

Next Click on Visualization and change the chart type to Column Chart

Click on Save As to save the output table to the existing Splunk Dashboard that we just created in the previous step "Covid-19 Dashboard" and Panel Title as "Top 20 Countries by Case Count - Column Chart"


Let's view our Dashboard. Click on Dashboards tab and further click on "Covid-19 Dashboard". The dashboard should look like this:


Further I have modified my query using geostats command to plot the confirmed case count on the world map using coordinates provided in lat and long columns (latitude and longitude)


index="index2" source="time_series_covid19_confirmed_global.csv" | rename 6_10_20 as ConfirmedCases | table Country_Region ConfirmedCases Lat Long | sort - ConfirmedCases | geostats avg(ConfirmedCases) by Country_Region latfield=Lat longfield=Long


Save the result to the same dashboard "Covid-19". Dashboard should look like below:














Comments

Post a Comment

Popular posts from this blog

Configure Oracle ASM Disks on AIX

Configure Oracle ASM Disks on AIX You can use below steps to configure the new disks for ASM after the raw disks are added to your AIX server by your System/Infrastructure team experts: # /usr/sbin/lsdev -Cc disk The output from this command is similar to the following: hdisk9 Available 02-T1-01 PURE MPIO Drive (Fibre) hdisk10 Available 02-T1-01 PURE MPIO Drive (Fibre) If the new disks are not listed as available, then use the below command to configure the new disks. # /usr/sbin/cfgmgr Enter the following command to identify the device names for the physical disks that you want to use: # /usr/sbin/lspv | grep -i none This command displays information similar to the following for each disk that is not configured in a volume group: hdisk9     0000014652369872   None In the above example hdisk9 is the device name and  0000014652369872  is the physical volume ID (PVID). The disks that you want to use may have a PVID, but they must not belong to a volu...
2 comments
Read more

Adding New Disks to Existing ASM Disk Group

Add Disks to Existing ASM Disk Group In this blog I will show how to add new disks to an existing ASM Disk group. This also contains the steps to perform the migration from existing to the new storage system. In order to add the disk to the ASM disk group, you will first need to configure these disk using the operating system commands. I have provided the steps to configure the disks on AIX system in my blog " Configure Oracle ASM Disks on AIX" Adding New Disks to DATA Disk Group (Storage Migration for DATA Disk Group) Login to your ASM instance $ sqlplus / as sysasm If the name of the new disk is in different format from the existing disk, the modify the asm_diskstring parameter to identify the new disks. In my below example /dev/ora_data* is the format of the existing disks and /dev/new_disk* is the naming format of the newly configured disks. You should not modify this parameter unless the naming format changes. SQL> alter system set asm_diskstring = '/dev/ora_data*...
Post a Comment
Read more

Load records from csv file in S3 file to RDS MySQL database using AWS Data Pipeline

 In this post we will see how to create a data pipeline in AWS which picks data from S3 csv file and inserts records in RDS MySQL table.  I am using below csv file which contains a list of passengers. CSV Data stored in the file Passenger.csv Upload Passenger.csv file to S3 bucket using AWS ClI In below screenshot I am connecting the RDS MySQL instance I have created in AWS and the definition of the table that I have created in the database testdb. Once we have uploaded the csv file we will create the data pipeline. There are 2 ways to create the pipeline.  Using "Import Definition" option under AWS console.                    We can use import definition option while creating the new pipeline. This would need a json file which contains the definition of the pipeline in the json format. You can use my Github link below to download the JSON definition: JSON Definition to create the Data Pipeline Using "Edit Architect" ...
1 comment
Read more