Skip to main content

Configure a universal forwarder to monitor a log file on Linux

Configure a universal forwarder to monitor a log file on Linux


In this article we will see how to configure Splunk forwarder to monitor a log file and ingest the log file events to Splunk. In this example we will perform the steps to monitor a file on the same server where Splunk enterprise is installed.

  • Login to the Linux server using PuTTY on which Splunk is installed.
  • If not already set, then set SPLUNK_HOME variable to the directory where Splunk is installed.
    • export SPLUNK_HOME=/opt/splunk
  • Go to the directory $SPLUNK_HOME/etc/system/local
    • cd $SPLUNK_HOME/etc/system/local
  • Modify the file inputs.conf and append the below lines at the end of the file.
            [monitor:///var/log/messages_copy]
            index=main
            sourcetype=server_log


    • Value of monitor tells the file(s) that you want Splunk to monitor for the events. You can specify a single or multiple files using wild cards. In this example I am using a copy of system file /var/log/messages. I will later add some events in this finally and we will see how Splunk reads these events.
    • Value of index parameter tells the name of the index in which the information of the events will be stored.
    • Sourcetype is a classification of data input, i.e, type of data input being provided.
  • Save the file inputs.conf
  • Restart Splunk
    • #cd $SPLUNK_HOME/bin
    • #./splunk stop
    • #./splunk start

Verify that Splunk is Reading events from the file and the events are visible on Splunk console using below query:

  • First I did a test for searching the events, but no events were found as the file /var/log/messages_copy was not found. See the screenshot below:


  • In the next step, I will create the file /var/log/messages_copy

  • Now check the same query to again and you should be able to see the events for the log file /var/log/messages_copy. 


Comments

Post a Comment

Popular posts from this blog

Configure Oracle ASM Disks on AIX

Configure Oracle ASM Disks on AIX You can use below steps to configure the new disks for ASM after the raw disks are added to your AIX server by your System/Infrastructure team experts: # /usr/sbin/lsdev -Cc disk The output from this command is similar to the following: hdisk9 Available 02-T1-01 PURE MPIO Drive (Fibre) hdisk10 Available 02-T1-01 PURE MPIO Drive (Fibre) If the new disks are not listed as available, then use the below command to configure the new disks. # /usr/sbin/cfgmgr Enter the following command to identify the device names for the physical disks that you want to use: # /usr/sbin/lspv | grep -i none This command displays information similar to the following for each disk that is not configured in a volume group: hdisk9     0000014652369872   None In the above example hdisk9 is the device name and  0000014652369872  is the physical volume ID (PVID). The disks that you want to use may have a PVID, but they must not belong to a volu...
2 comments
Read more

Adding New Disks to Existing ASM Disk Group

Add Disks to Existing ASM Disk Group In this blog I will show how to add new disks to an existing ASM Disk group. This also contains the steps to perform the migration from existing to the new storage system. In order to add the disk to the ASM disk group, you will first need to configure these disk using the operating system commands. I have provided the steps to configure the disks on AIX system in my blog " Configure Oracle ASM Disks on AIX" Adding New Disks to DATA Disk Group (Storage Migration for DATA Disk Group) Login to your ASM instance $ sqlplus / as sysasm If the name of the new disk is in different format from the existing disk, the modify the asm_diskstring parameter to identify the new disks. In my below example /dev/ora_data* is the format of the existing disks and /dev/new_disk* is the naming format of the newly configured disks. You should not modify this parameter unless the naming format changes. SQL> alter system set asm_diskstring = '/dev/ora_data*...
Post a Comment
Read more

Load records from csv file in S3 file to RDS MySQL database using AWS Data Pipeline

 In this post we will see how to create a data pipeline in AWS which picks data from S3 csv file and inserts records in RDS MySQL table.  I am using below csv file which contains a list of passengers. CSV Data stored in the file Passenger.csv Upload Passenger.csv file to S3 bucket using AWS ClI In below screenshot I am connecting the RDS MySQL instance I have created in AWS and the definition of the table that I have created in the database testdb. Once we have uploaded the csv file we will create the data pipeline. There are 2 ways to create the pipeline.  Using "Import Definition" option under AWS console.                    We can use import definition option while creating the new pipeline. This would need a json file which contains the definition of the pipeline in the json format. You can use my Github link below to download the JSON definition: JSON Definition to create the Data Pipeline Using "Edit Architect" ...
1 comment
Read more