File Monitor
Previous  Top  Next

The file monitor monitors the content of a text file just as the event monitor monitors the NT event log. Its purpose is to gather vital information that is stored in system text files. Many applications do not write events to the event log but to a text file. This is also the case with many Microsoft applications (for example the WINS log).

The file monitor can also gather Internet Information Server (Windows' web server) log files. This is very useful for monitoring web activity and detecting attacks.

filemonitor
Figure 1: File Monitor Properties


filemonitor_advancedoptions
Figure 2: File Monitor Advanced Options


File and path name

Here, you must type the name of the file to be monitored. To select a file from any specified location, press the browse button. Once a complete file name is specified, exactly that file is monitored.

The file name is never changed automatically. However, many systems generate changing log files. For example, Internet Information Server (IIS) can be configured to change the log file every day. Therefore, each day's log file has a different name.

To support changing log file names, there are replacement characters available within the file name. These are:

Character
Meaning
%y
Year with two digits (e.g. 2002 becomes "02")
%Y
Year with 4 digits
%m
Month with two digits (e.g. March becomes "03")
%M
Minute with two digits
%d
Day of month with two digits (e.g. March, 1st becomes "01")
%h
Hour as two digits
%S
Seconds as two digits. It is hardly believed that this ever be used in reality.   
%w
Weekday as one digit. 0 means Sunday, 1 Monday and so on.
%W
Weekday as three-character string. Possible values are
"Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat".
This replacement character is most useful for DHCP log files.
%generatedfilename%
It contains the fully generated filename (Can be useful for filtering).
%msgsep%
Only available if enable in the advanced option of the File Monitor. This value contains the current used messageseperator. This is usefull if you want to reconstruct messages where the seperator is part of the message.
%msgseplast%
Only available if enable in the advanced option of the File Monitor. This value contains the last used messageseperator. This is usefull if you want to reconstruct messages where the seperator is part of the message.

Character Replacement Table

Please note: the replacement characters are case sensitive!

For example, daily Internet Information Server log files are named "exyymmdd.log", with yy being the 2 digit year, mm the month and dd the day of month. To generate the same name with file monitor, use the following name "ex%y%m%d.log".

Please note that there is no replacement character for the monthly week number (1st week, 2nd week). As such, the weekly log file setting of IIS is not supported.


TimeMode Used for Filename

Select the time mode used for the log file to be monitored with this drop-down list. Available options are:

1.Local Time: log file is monitored based on local time.  
2.UTC: log file is monitored based on universal coordinated time. UTC was formerly referred to as "GMT" (Greenwich Mean Time) and is the basis of the international time zone system.     


----------------------------->


Advanced Options


Max Bytes per Message

Maximum value of bytes that the file monitor reads per line. If a message is larger then this value, the message splits into multiple parts.


Compress spaces

This option compresses sequences of spaces found inside the message to a single one.


Remove Control Characters

Removes control characters like CR and LF(carriage return and line feed).


Ignore empty lines

As the name allready says, this option discards empty lines within the logfile.


Remove leading space

If there are any leading spaces in the file, this option removes them.


Save Message Seperator in Property

If this option is enabled, the current and last used message seperator will be saved into the properties %msgsep% and %msgseplast%.


Reset Message Seperator after each run

If enabled, the message seperator values will be resetted after the File Monitor has finished a run (reached the end of a file).


Message separation sequence

The customizeable separation sequence when this option is enabled. The file monitor splits messages by this value. If it is disabled crlf (carriage return line feed) is used.
If using multiple separation sequences, the comparison operation will be held as an OR operation. That means, that either this value or another value has to be true, so a message can be split. This is especially important for logfiles with different log formats.

To date, the following characters are available:

Character
Meaning
\r
carriage return
\n
line feed



<-----------------------------


Allow Directories or read multiple files

This is the new Multiple Files feature which means you can now read an array of files. This will require a wildcard in the filename.


Use Wildcard in Filename

This option allows you use * as random character sequence in the filename.
Please note: this character can only be used in Filename and not in the filepath.


Keep reading the current opened file until a new is created
This has been added to define if the Service shall continuously read an open logfile until a new file (depending on the configured filename) is available. This Options is helpful for such cases where you don't know when a new logfile is generated and the old one is closed.


Report an Error if the File was not found

As the name says, if this setting is enabled, an error is reported in the Windows Eventlog if the file was not found.


Skip all lines on Startup

If this option is enabled, the File Monitor will skip all new lines of a logfile during startup. Please note that this will only work in SingleFile Mode.


Check Interval

This interval is in miliseconds. After the specified interval the file monitor checks the file for new records.

We recommend a value of 60000 milliseconds for the "Check Interval". With that setting, the file monitor checks for new records every 60 seconds. Larger periods can be specified for occasionally connected systems or if email delivery with few emails per day is intended.

Very security-aware environments might use a shorter interval. The MonitorWare Agent 3.0 is specifically designed to limit the burden on the monitored system. As such, resource usage is typically low, even with frequently run file monitor checks. However, we recommend not running the file monitor more often than once a second.


Overrun Prevention Delay

This property allows configuring a delay after generating an event. The time for the delay is in miliseconds.

If run at a value of zero, the MonitorWare Agent 3.0 generates events as fast as the machine permits. We have seen scenarios where routers and receivers are not able to keep up with this rate, resulting in packet loss. In addition, the CPU of the reporting machine is run at 100% - which is not a problem because MonitorWare Agent 3.0 runs at a low priority. However, with even a 1-millisecond delay, there is no noticeable CPU activity even when large bursts of events are forwarded. At one millisecond, MonitorWare Agent 3.0 can still generate 1000 events per second.

The default setting is an overrun protection of five millisecond, which allows roughly 200 events per second. This should be sufficient for even very busy servers.


Logfile Type

Select the type of the log file to be monitored with this drop-down list. Available options are:

1.Standard - a standard text log file  
2.W3C Web Server logfile - log files in the W3C web server compliant format.     


Last Line Value

This value contains the last read line of the FileMonitor Service. The file Monitor Service reads a configured file continuesly line by line and everytime there is a new line, this value is incremented.


Syslog Facility

The Syslog facility to be assigned to events created by the service. Most useful if the message shall be forwarded to a Syslog daemon.


Syslog Priority

The Syslog priority to be assigned to events created by the service. Most useful if the message shall be forwarded to a Syslog daemon.


Resource ID

The Resource ID to be assigned to events created by the service. Most useful if the message shall be forwarded to a Syslog daemon.


Syslog Tag Value

The Syslog tag value to be assigned to events created by the service. Most useful if the message shall be forwarded to a Syslog daemon.


Default Ruleset Name

Name of the rule set to be used for this service. The Rule Set name must be a valid Rule Set.


Further Reading

Please visit our white paper on monitoring IIS logs.