Logfile watcher

What is it?

Logfile watcher is a simple unix daemon monitoring various logfiles and issuing commands based on configurable delays and regular expressions. You can use it to have a logfile diff mailed to you.

For each logfile you want monitored, you set up the following.

• Idletime required for a command to be run
• Command to run when the idle time has passed
• uid and gid to execute the command with
• Regular expression logfile lines must (not) match

How to set it up

logfilewatcher is currently in development and only available via git. Just do

git clone http://www.hcesperer.org/logfilewatcher.git

to check the repository out

chdir into the extracted directory and issue 'make'. If it compiles successfully, copy the watch binary to /usr/bin or whereever your binaries lie.

If you also need the prepend utility, chdir to prepend/, issue 'make' and copy the prepend binary to /usr/bin.

Create a config directory, for example /etc/filestolog or /etc/loggees. Chdir into that directory and declare it as a logfilewatcher directory by issuing

echo logfilewatcher > .watch

Create a subdirectory for each logfile you wish to monitor.

The directory for each service must contain the following files; each of them containing exactly one line:

For each service, only contain or notcontain may exist, not both.

After you've created all subdirectories, cd into the main directory, in our example /etc/filestolog, and start watch.

Writing a cmd script

cmd is the command that is invoked every time a logfile diff is to be handled. cmd receives the logfile diff through stdin. It is responsible for correctly handling the data read at stdin; if they could be successfully processed, it must return 0. Otherwise, it must return a non-zero value so it is invoked again until the action succeeds.

Sending emails

A typical scenario is to have your logfile diffs sent to you or to some ML via a mail agent. To do that, just call your mail injection program in cmd, for example /usr/sbin/sendmail. Now, to be able to specify some headers, you'd need to fiddle with stdin. To simplify matters, a small tool called prepend comes with logfilewatcher. prepend first acts like echo; it writes all command line arguments to stdout. After that, it acts like a cat invoked without arguments; it simple reads stdin until it gets an EOF and writes it to stdout.

Suppose your email address is president@whitehouse.gov, and you want to send yourself the current date via email, you could issue the following command:

date | prepend 'Subject: The current time\n\n' | sendmail president@whitehouse.gov

Example

Real world example for an irc channel I use (note that the irc-logging is done by supybot, not logfilewatcher)

The configuration lies in /etc/service/logfilewatcher/watchees/bpThese files are used:

[^ ]+  \*\*\*

#!/bin/sh
/usr/bin/prepend 'Subject: Bachelorpraktikumschannelprotokoll per Email\n\n' | /usr/sbin/sendmail -f hc@hcesperer.org -F "HCs LogfileWatcher" bp@y1p.de

Why was it created?

I am monitoring several irc channels used for orga discussions. A supybot (quite bloated but great python irc bot) sits in each of these channels, logging them (amongst other things). These logs are monitored by the logfilewatcher, and after a certain inactivity, the channellog is posted to mailinglists.

Originally, I had written a logfilewatcher in java to do the job, and it used more real memory than the python bot, and a hundred megs of virtual memory! In response, I rewrote the logfilewatcher in C, making it a little more generic. It now uses 450kB of real memory on my machine.