How can I get rid of control characters and linefeeds?

How can I get rid of control characters and linefeeds?

Created 2011-02-17 by Florian Riedl

Some syslog sources create strange message formats. In many cases, this is of no concern. In other cases, this can disturb reviewing logs. For example, if you are using the InterActive SyslogViewer, control characters in the message create strange placeholder characters (usually this looks like a box, because Windows cannot display those control characters. This is disturbing the view of course and makes it hard to read the log message correctly. This can happen with devices like firewalls or something similar, but with Windows Events as well. Here is an example of the InterActive SyslogViewer struggling with tab characters in the log message:

Control Characters in InterActive SyslogViewer
Click to enlarge.

In this case, we want to remove the control characters from the messages. Therefore we have to use the property replacer on the message property. With the property replacer, we can easily do that. Basically, we want to replace a control character with a space. In the best case, we now have a space where the control character was, but usually a control character has already a space before and/or behind it. But we do not want to have multiple spaces in the middle of a line, so we need to compress them as well.

Now that we have made the decission about what we have to do, we can create the property replacer command. We take the property for the message:

%msg%

This is just the property, but not the property replacer yet. The property replacer looks basically like this:

%msg:::%

Usually, this is used to substract a substring from a string with FromPos and ToPos. Not in this case, but here is a short description anyway. After the first colon comes the FromPos parameter and after the second colon comes ToPos. Now after the third colon, this is important for us now, come the options. Since we need to replace the control characters we need the option spacecc and for compressing space we use compressspace. The property replacer string should look like this now:

%msg:::spacecc,compressspace%

As you can see, multiple options are comma separated.

But where do we use this property replacer now? You can use it in nearly every action that is available. Here is an example of a “Forward via Syslog” Action.

Control Characters 01

In the field Message Format you can see the property %msg%. This needs to be replaced by our property replacer. In the end, it should look like this:

Control Characters 02

Now, all the control characters like tabs or linefeeds will be replaced by a space and multiple spaces will be compressed to a single space.

For more information about properties and the property replacer, please refer to the manual at the chapter for Event Properties in the Reference section.