Latest Updates

random

Indexing the Data from a centralised Rsyslog server into Splunk


In computing, syslog is a standard for message logging. It allows separation of the software that generates messages, the system that stores them, and the software that reports and analyses them. Each message is labelled with a facility code, indicating the software type generating the message, and assigned a severity level.
Rsyslog is an open-source software utility used on UNIX and Unix-like computer systems for forwarding log messages in an IP network. It is also used on scenario where one has many data in multiple hosts and one wants to gather the data into a central server which can be later accessed from it.
In our case we will be gathering many sets of /var/log/secure data from multiple VM hosts running on Linux OS to a single VM Host and then forwarding that to Splunk using a UniversalForwarder.

Prerequisites before installing and configuring rsyslog:
1. Open the port for rsyslog traffic. By default UDP 514 port is used for rsyslog.
 firewall-cmd --zone=public --add-port=514/udp --permanent
2.      If we want to use any other port and if SELinux is enabled then following steps need to be followed:
a.      yum install policycoreutils-python
b.      semanage port -a -t syslogd_port_t -p tcp 10514
where 10514 is desired port number.

Configuration of Rsyslog:-

1. Rsyslog is usually installed and already running in CentOS/RHEL. In order to check if the status is up and running, run the following command:-
           systemctl status rsyslog



If the service is not running by default, run the following command to start rsyslog daemon.
           systemctl start rsyslog



If rsyslog has been installed at all use the following command to run it and the start it as well.

            yum install rsyslog

2.  Take a backup of /etc/rsyslog.conf file on client as well as server by using the following command:-
cp myfile.txt myfilecopy.txt



The view of rsyslog.conf -
3. Login to the /etc/rsyslog.conf  file and in the server uncomment the following lines:-

4.  On client add the server ip and port as required by your standards as shown below:-

5. Restart rsyslog server after making changes in rsyslog.conf files on server and client.
         systemctl restart rsyslog


Indexing into splunk:-

Now, as we can see that rsyslog has been configured and that the syslogs from the different VM’s are being sent to the centralised server, we can now index the rsyslog data into splunk.

As usual it’s now time to make the inputs.conf file to monitor the rsyslog files. It’s usually best to keep the centralised rsyslog server as well as the splunk universal forwarder on the same instance. This is mainly due to ease of work.

1. In the inputs.conf, add the paths of which logs you want to monitor, the index and the sourcetype.

2. Now it’s time to create the props.conf to extract the host name from the logs that will be indexed into splunk as it will help us identify from which host the individual log has been originated.


3. Now create 2 different apps to place the inputs.conf and props.conf respectively. We use the app name Avo_FW_rsys for inputs.conf and Avo_SH_Rsyslog_parse for props.conf. After creating the app add both the .conf files into their individual default folders respectively. 

4. After the above step place the app in the /etc/deployment-apps folder in the deployment server and then push the app towards the splunk universal forwarder by using the following command. The class name is where you want to push the bundle to the deployer, cluster master or forwarder. The app named Avo_FW_rsys will be pushed towards the forwarder and Avo_SH_Rsyslog_parse will be pushed towards the deployer.

/opt/splunk/bin/splunk reload deploy-server –class

5. From the deployer the Avo_SH_Rsyslog_parse app needs to be sent towards the search head cluster so use the following command for it to happen.
/opt/splunk/bin/splunk apply shcluster-bundle –target :8089

Be sure to check which ip in the search head cluster is the captain as well. To check this go to the UI of any search head in the cluster > settings > Search Head Clustering and you will find who is the captain. After the bundle is pushed towards the search head cluster click on begin rolling restart which is present in the top left corner on the same page where you found the ip address of the captain of the search head cluster.

6. Now, after the above steps go to your search head cluster and check if you can see your rsyslog logs or not.

That’s all! Rsyslog is now configured as a centralised log server, can collect logs from remote clients and it has also been indexed into splunk as well.

If you are still facing issue regarding this topic Feel free to Ask Doubts in the Comment Box Below and Don't Forget to Follow us on 👍 Social Networks, happy Splunking >😉
Indexing the Data from a centralised Rsyslog server into Splunk Reviewed by Avotrix.Author on Friday, December 06, 2019 Rating: 5
All Rights Reserved by Avotrix © 2018 - 2019
Developed and created by Avotrix

Contact Form

Name

Email *

Message *

Powered by Blogger.