Using Syslog-ng to monitor and forward log files to QRadar

Posted on November 27, 2018 Updated on December 20, 2018

Although it is possible to use SSH/SFTP/SCP to collect log files with QRadar, that option may not be the most appropriated for every company. Those collection methods, called pull-collection, requires an extra account in the system (so QRadar can login to collect the log). Moreover, the pull-collection methods collect the logs in an interval (usually, 5 minutes), meaning that logs may take up to 5 minutes to arrive in QRadar.

One alternative for collecting logs stored in files is using the syslog-ng service, which is already pre-installed in the majority of the newer Linux releases. This service can be used to monitor a specific file (in smaller intervals, such as 10 seconds) and in case of any changes, send the logs to QRadar. The configuration is fairly simple, here are the steps:

1. Check if the syslog-ng is installed:

   $ service syslog-ng status

   if not installed, install using:

   $ apt-get install syslog-ng

2. Edit the syslog-ng configuration file (usually at: /etc/syslog-ng/syslog-ng.conf), inserting the following lines:
   

   source s_logfile1 {
   file("[[[PATH_TO_FILE]]]" follow_freq(10) flags(no-parse));
   };
   
   destination d_qradar { 
   tcp("[[[QRADAR_COLLECTOR_IP]]]" port(514) template("<$PRI>$DATE $HOST $MESSAGE\n"));
   };
   
   log { 
   source(s_logfile1); 
   destination(d_qradar);
   };

   The first line adds a “file monitoring” rule to the specific file. You can update the file monitoring frequency (in seconds) by changing the follow_freq parameter.
   The second line adds a syslog destination, if you already had syslog-ng configured for another log monitoring, this line may not be necessary.
   The third line adds a logging action from the source file to the destination syslog recipient.

3. Restart the syslog-ng service

   $ service syslog-ng restart

4. Done! You can now check on QRadar if logs are coming.
5. If you need to troubleshoot, you can check the syslog-ng logs by using the following commands:

   $ syslog-ng
   $ service syslog-ng status