Centralized logging in a hybrid environment (Windows/Linux) – Step 4

Friday, March 11th, 2011

Step 4 – Setting up the Linux Clients

We still need to implement the Linux clients into our logging network. This could be Linux workstations or servers. Currently, rsyslog is already shipped with many Linux Distributions as the default syslog daemon already. Though, often a rather old version is installed (mostly v3), this will be sufficient for our current goals.

For our example we will use a Fedora system and alter the configuration accordingly. In the end, rsyslog should use the logs, that are logged locally and forward them via TCP to the central log server.

Please note: The example has been made with Fedora 13. Some steps may vary with other Linux distributions. Since there are so many possibilities, we cannot list them all. So we only concentrate on one version.

Step 4.1:

First of all, we need a terminal or command line and go into root mode with su. We do not need the root rights desperately, but it is a bit easier to achieve our goals.

Step 4.2

We need to alter the configuration accordingly. We will use the vi text-editor to do so. If you are not familiar with vi, you should consider getting some knowledge first with the vi man page.

You can find the default configuration file in

/etc/rsyslog.conf

This configuration file will be loaded once rsyslog is started.

Open the configuration file by the following command:

vi /etc/rsyslog.conf

Your terminal should look like this:

centralized_monitoring_4001

Step 4.3

Under the header "#### MODULES ####" you can see the loaded modules. The ones we need are already loaded. For sending syslog, we do not need a extra module to be loaded. Basically, the imuxsock.so is listening to the local log socket and imklog.so provides the support for kernel logging. More is currently not needed.

When going further down in the configuration, we see parts that are called UDP and TCP syslog reception. They are uncommented. We don’t need them, so we leave them that way. As you can see, there is the command to load a module (introduced my $ModLoad) and then a specific command that is given to the module which tells it to run as a syslog server on a specific port.

Step 4.4

When scrolling further down in the configuration file, we will find the header "#### RULES ####".

centralized_monitoring_4002

Here, some rules are defined about which logs are stored in which location. These rules have a basic format. Basically, this looks like this:

facility.severity storage_location

The facility roughly determines, what has generated the log message. The severity tells you, how urgent the message is. And the storage location determines, where to store the logs. In the default config, there are several rules for some important log files. They will all be stored in the folder /var/log.

We need to add another rule for forwarding all log messages via TCP syslog. For now, go to the last of the rules and insert the following:

*.* @@central_server:10514

This rule means that all messages, no matter the facility or severity, will be forwarded via TCP to the central server on port 10514. We need to choose port 10514, since our central server will listen to TCP syslog on this port.

In the configuration file, this should look like this:

centralized_monitoring_4003

You can now save and quit vi.

Step 4.5

Now that we have altered our configuration according to our needs (which hasn’t been too much), we still have one more thing to do. We need to restart the service.

Now that we are back on the command line, we can easily use the following command:

service rsyslog restart

This will tell the rsyslog service to stop and start again. Thus, the configuration will be loaded with the changes we made.

centralized_monitoring_4004

Step 4 – finished

We are now finished with setting up rsyslog correctly. It will take all the logs it receives and forwards them to our central log server via TCP on port 10514. In addition to that, the default log rules still exist and will log messages to the local harddrive.

Though, there could be many more scenarios and configurations. We decided to keep it simple for a basic setup.

<< Go back to the main page

Centralized logging in a hybrid environment (Windows/Linux)

Thursday, March 3rd, 2011

Centralized logging in a hybrid environment (Windows/Linux)

Created 2011-03-11 by Florian Riedl

This article will describe how to setup centralized logging in a hybrid environment. Basically, we will have various major steps, that show different configuration of several clients, which forward their log data to a central loghost. There, everything will be stored into a database and processed further for alerting.

To describe the situation basically, we want all machines on the network send their log data to a central syslog server (if possible). For the central log server we take a windows machine running MonitorWare Agent (www.mwagent.com). Here we can receive syslog, monitor local log files and the Windows EventLog. Data shall be stored into a database and several email alerts shall be configured. The other steps describe the configuration of simple Windows workstations and servers, as well as Linux servers.

For TCP transmission we will use port 514 (default) for UDP and port 10514 for TCP. We want to use TCP mainly, because it ensures the transmission of the syslog messages. This is due to UDP being connectionless and thus it can occur (and will) that messages get lost.

The Client machines in this example consist of several different types of machines. We have regular Windows Workstations. Here we will use EventReporter (www.eventreporter.com). In addition to our central server, we have some other Windows Servers which will get MonitorWare Agent as well and some Linux machines which have rsyslog (www.rsyslog.com) installed. These machines will send their log messages via TCP syslog to the central server.

Additionally to these clients, we will mention some other devices and appliances (just roughly), like firewalls, switches and routers.

Step 1:

This is the first and biggest step. We will configure the central server first. The reason is simple. If this is already running, we can setup the clients and it will directly start logging everything. We assume, this is a Windows Server where MonitorWare Agent is installed. The central log server shall provide the following functionality:

  • syslog receiver with TCP (for devices that can send TCP syslog)
  • syslog receiver with UDP (for devices that can only send UDP syslog)
  • monitor the local Windows EventLog
  • monitor local textfile-logs
  • store all log messages into a database
  • send email alerts to an admin on error or critical log messages

Continue reading on Step 1

Step 2:

In step 2 we will set up the regular Windows clients. These are usually the workstations the people work on. We will use EventReporter here. It can pull all log messages from the Windows EventLog and forward them via TCP syslog. Thus the following functionality is mandatory:

  • monitor the local Windows EventLog
  • forward all log data via TCP syslog

Continue reading on Step 2

Step 3:

Now we will configure the other Windows servers. Again, we will use MonitorWare Agent because it has the most functionality. We need the following functions to be setup here:

  • monitor the local Windows EventLog
  • monitor local textfile-logs
  • forward all log data via TCP syslog

Continue reading on Step 3

Step 4:

Now we get to the Linux servers. Here we need to use a completely different product – rsyslog. For a first-time user, this might look a bit strange. The configuration we want to have here needs the following:

  • monitor local log messages
  • monitor local textfile-logs
  • forward all log data via TCP syslog

Continue reading on Step 4

Step 5:

This is rather just a note on other devices and appliances that are not yet covered. Often devices (like routers, firewalls or switches) have the possibility to send log data to a syslog server. Usually, this only works via UDP and some machines are even capable of sending logs via TCP. Since there is such a huge mass of different systems and devices, we cannot give correct steps for everything. Please refer to the user manual that came with the device or contact the manufacturer for information about how to configure the devices for sending syslog.

If you already know how to configure it, let it send it’s log messages to the central server on port 514 for UDP or (if possible) port 10514 for TCP.

Conclusion

We now have a setup that stores all the log data that machines on the network will generate to a central database for storage. Most of the clients on the network send their log data securely via TCP to the central log storage. Some machines were rather quick to set up, others needed more effort. Usually the effort rises with the amount of features that will be used. Thus we thought of this setup to be quite simple.

If you have any remarks or ideas of improvement for this guide, please let us know and send an email to info@adiscon.com.