rsyslog: Intermediate Local Logging

Please see Learning rsyslog for the introduction and index to this series of blog posts about rsyslog.

Let's start with the result of a basic logger command:

$ logger "GO log test 43"

As root (/var/log/messages isn't usually visible to regular users) we look at the output in the log:

Oct 24 18:17:33 sli7-f giles[535019]: GO log test 43

Notice that it's tagged with the date and time, the hostname ("sli7-f"), the user who sent the message ("giles"), and the process ID in square brackets. This is from Fedora; on Debian, it's essentially the same minus the PID. Let's start trying to control the "facility," "priority," and "tagging:"

$ logger -p local0.notice -t MYTAG "GO logging test 6"

The resulting output looks like this:

Oct 23 17:26:56 toshi7fed MYTAG[191307]: GO logging test 6

Notice the facility and priority don't get recorded here - and that "tagging" defaults to the user name but can be over-ridden by providing a word as a tag.

Before we start making changes to /etc/rsyslog.conf, it might be a good time to install etckeeper - which is available as a package on both Debian and Fedora. Read the docs enough to get it running! It's a great service to have on any machine that's not living under Infrastructure-as-Code control.

Now let's modify the /etc/rsyslog.conf file to handle the filtering of logging we're sending:

# order matters: this goes before the /var/log/messages line:
local0.*  /var/log/local0.log
*.info;mail.none;authpriv.none;cron.none;local0.none    /var/log/messages

You'll need to restart the rsyslog service to re-read the configuration. I've done it so often - and so often wrong - that I've taken to using systemctl restart rsyslog ; systemctl status rsyslog | cat - this allows you to immediately see if the service is unhappy with your changes.

The above configuration tells it that anything that's marked as facility "local0" goes into the /var/log/local0.log file - and I've added ;local0.none on the line feeding /var/log/messages so that my local0 mesages appear only in the log file I've chosen. There are (inevitably!) other ways of doing this, but we're now in a position to declare that facility "local0" is associated with whatever logging project we're working on, and we've arranged that its output will all land in one specific file.

This is a possible solution and not too bad, but it seems to me it would be better to filter on tagging. So let's try something else.

# added to /etc/rsyslog.conf - before lines dealing with /var/log/messages
# Look for "Filter Conditions" in the rsyslog documentation
:syslogtag, startswith, "MEMORY" /var/log/memfree.log
:syslogtag, startswith, "MEMORY" stop

Restart the service. This says "if you find a 'syslogtag'" (the tagging that can be added with the logger -t ...) "then send it to /var/log/memfree.log." Remembering that messages continue to propagate (any "MEMORY" tagged message will also end up in /var/log/messages), I added another line with the "stop" command - it amounts to "don't send this message anywhere else."

[CAVEATS: "MEMORY" has to be in quotes, it's otherwise a keyword and does weird things. And I called the log file "memfree" because on Fedora there's already a /var/log/memory.log file.]

To test, we run a script like this:

tmpfile=$(mktemp /tmp/memXXXXXX)
free > $tmpfile
logger -t MEMORY -f $tmpfile
rm $tmpfile

This is a good approximation of the script I would use on a cron job to write memory information. I've thrown in one new logger trick, which is the use of the -f switch that reads in a file as the log message. And with the above set-up, the output lands in /var/log/memfree.log.

'Learning rsyslog' Index