Blue Light
Space shortcuts
Child pages
  • Logcheck administration
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Versions

The information on this page was developed and tested on Debian 6.0 (Squeeze) with logcheck 1.3.13.

References

  1. Main documentation:
        Online: http://logcheck.org/docs/
        As installed: /usr/share/doc/logcheck and /usr/share/doc/logcheck-database.
            The .gz files may conveniently be read using zcat and less.  For example:
            zcat /usr/share/doc/logcheck-database/README.logcheck-database.gz | less
  2. logcheck man page (HTML format): http://linux.die.net/man/8/logcheck

 

When logcheck mails a message

It means logcheck is not configured to filter it out.

Should the message be filtered out?

If the message shows there's an issue to be resolved, filtering the message out would hide essential information.

If the message is generated rarely, it is not worth the work of filtering it out.  Server boot messages are a good example.

Often it is not clear from the message itself whether it shows there's an issue to be be resolved or not.  Research work is required.

If the message should be filtered out ... 

Which log(s) is logcheck getting messages from?

logcheck searches whichever logs are listed in /etc/logcheck/logcheck.logfiles.  By default these are /var/log/syslog and /var/log/auth.log.

Which directory is logcheck loading its filter files from?

Normally this is /etc/logcheck/ignore.d.server.  This can be changed in the configuration file – either /etc/logcheck/logcheck.conf or as specified using logcheck's -c option. 

The rest of this article assumes logcheck is loading its filter files from /etc/logcheck/ignore.d.server.

What is a filter?

A filter is a regular expression matching the message to be filtered out.  Logcheck filters are in /etc/logcheck/ignore.d.server/<package name> files, one filter per line.

Which files to put locally developed filters in?

Locally developed filters can most easily be kept in /etc/logcheck/ignore.d.server/local-<package name> files.

The alternatives are harder to administer.  Adding to the as-installed files means those files will not be upgraded when logcheck is upgraded.  A single file for all local filters does not allow installing a local filter file at the same time as a package is installed or upgraded (the format of the log messages may change with the upgrade).

How to develop a filter?

Find a similar example

For example, suppose logcheck was mailing something like

May 23 10:08:33 LS1 nmbd[1199]: *****

The part in green is usually the name of a daemon or command.

Find files with filters for similar messages by

cd /etc/logcheck/ignore.d.server && grep --files-with-matches nmbd *

Choose one of the filters in the file(s) listed as the basis for developing the new one.  If no files are listed, take any filter – most (all?) messages match the message format above up to the last ":" except for the nmbd string.

Structure of a typical filter

A filter is a regular expression, specifically an egrep regular expression, for example looking like this:

^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ (openvpn|ovpn-[._[:alnum:]-]+)\[[[:digit:]]+\]:( ([-_.@[:alnum:]]+/)?[.[:digit:]]{7,15}:[[:digit:]]{2,5})? Replay-window backtrack occurred \[[[:digit:]]+\]$

  1. To ensure a complete match, filters begin with ^ and end with $ .
  2. The grey part matches from start of the line through the time stamp and server name.  It is common to most (all?) filters.
  3. The green part identifies the daemon or command.
  4. The blue part matches the process' PID shown in square brackets and followed by a :
  5. The rest is specific to the particular message.

Writing a filter

When writing a filter, you can almost certainly use parts 1 to 4 above from a filter for the same package and make up your own part 5.  The filter must be a POSIX extended regular expression matching the message you want to filter out.  The idea is to match only the messages you want to filter. 

If your regex is too general it may filter messages that show there's an issue to be resolved.  For example ...

    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ (openvpn|ovpn-[._[:alnum:]-]+)\[[[:digit:]]+\]:.*$

... would filter out all openvpn log messages.

If your regex is too specific you may need several filters to address closely related messages.  For example ... [TODO: add example]

If you are not familiar with extended regular expressions, it may be worth studying some existing logcheck filters alongside samples of the messages they filter to see how they work.

Extended regular expression references:

Testing and debugging

Regexes are tested by seeing if they match the message in the log file.  The first step is to find which log file the message came from.

The rest of this article assumes the message came from /var/log/syslog.

A convenient technique is to have the filter regexp in a file and to discard the output from egrep:

    egrep --quiet --file /tmp/my_filter /var/log/syslog && echo found || echo not found

Having the regexp in a file allows it to be easily changed and avoids the shell modifying it on the command line before it is passed to egrep.

If the regex does not work these can help:

  1. Progressively right-truncate the regex until it does match.  Then the last part discarded contains the error.  Having the regex in an editor with undo/redo helps.
  2. Drop egrep's --quiet option and add the --only-matching (-o) option.  This shows what the regex is actually matching.

Further information (includes tips on writing and testing filters):

zcat /usr/share/doc/logcheck-database/README.logcheck-database.gz | less

Notes on README.logcheck-database:

  • If using sort to order filter files as suggested, sort's --version-sort (-V) option may be required.
  • Multiple local-<package name> files have the advantage of being installable and removable with the associated package.

Installing the filter

Assuming you have just successfully tested a filter for package foo ...

If the local filters file for foo, /etc/logcheck/ignore.d.server/local-foo, already exists, optionally make a backup.  logcheck ignores files with a . in the name so local-foo.bak would be OK.

Add the new filter to local-foo on a line of its own.

Optionally sort local-foo.  This will help identify duplicate and similar filters.

Assuming your newly tested filter is in /tmp/my_regex, this could all be done with something like:

package=foo; my_new_filter=/tmp/my_filter; cd /etc/logcheck/ignore.d.server \
    && mv --force local-$package local-$package.bak \
    && cat local-$package.bak $my_new_filter | sort --version-sort > local-$package

Gotchas

If a valid filter does not pass testing:

  • Are there any trailing spaces on the message in the log?  logcheck automatically removes these before looking for filter matches.  This behaviour can be mimicked during testing by:
        sed -e 's/[[:space:]]*$//' /var/log/<log name> | ...

If a tested filter does not work:

  • Does the message match logcheck's "SECURITY ALERTS" (defined in /etc/logcheck/cracking.d/* files) or "SECURITY EVENTS" (defined in /etc/logcheck/violations.d/* files)?  If so, the solution is described in README.logcheck-database.

XXX

  • No labels