[hobbit] Large Log Files

James,

You might need to write your own log checker to generate something that Hobbit can catch. Hobbit was not really designed to handle large log file volumes. The MAX variable, if I remember correctly, tells how much data the Hobbit client should sift through. Once this limit is reached, it just pumps what's left -- but I may have that wrong.

One way to do it might be to "tail" the log file and pipe the data through grep on the way to another log file, which you can then have Hobbit watch. The problem you will have is keeping up with the log files as they rotate. Perhaps someone else on the list can pipe up with how that can be done -- maybe by monitoring the file name associated with the inode? Of if there is no data for x amount of time, switching? All this may be more than a simple shell script can handle, so you may need to write something in Perl or C or whatever.

My two cents worth -- you can have a receipt if you need one...

GLH

From: James Wade [mailto:jkwade at futurefrontiers.com] 
Sent: Wednesday, November 29, 2006 11:50 AM
To: hobbit at hswn.dk
Subject: [hobbit] Large Log Files



I'm having problems monitoring a large LOG file and can

use some assistance. I've tried several entries in the

client-local.cfg

file, but the results have not been great.

 

Basically, the developers want to monitor a web server log file

which is

in debug mode. The file fills up to 10mb, then rotates to a

date/time stamp.

They want to monitor for specific strings. Example: J2CA0056I

 

The problem is that if Hobbit is checking every 5 minutes, or

even every 1 or

2 minutes, the error can feed by to fast for Hobbit to detect.

 

I tried using the trigger function, (See Below), but it didn't

seem to work.

We entered the string into the file manually, and Hobbit gave no

errors.

One question I have is the MAX size variable. How does this

relate to the file size?

As you can see below, I increased the MAX size variable.

 

I've put my client-local.cfg entry below.

 

Any suggestions would be appreciated.

 

[server10]

log:/var/adm/messages:10240

log:/logs/server10/SystemOut.log:40960

trigger J2CA0056I

log:/logs/server10/SystemErr.log:40960

log:`ls /logs/server10/SystemErr_*`:40960

log:`ls /logs/server10/SystemOut_*`:40960

 

The last two entries, I'm trying to monitor the rollup files as

well in case the log rolls

over before Hobbit has a chance to see the error. However, this

didn't seem to work to effectively

either.

 

Thanks.....James