On Sat, Dec 17, 2005 at 08:43:31AM, Henrik Stoerner wrote:

On Fri, Dec 16, 2005 at 10:11:18PM -0500, Asif Iqbal wrote:

...

Hi All

I am running solaris 9. My hobbit is running off of /export/home/hobbit dir. The df -i shows /dev/md/dsk/d30 1005696 1005005 691 100% /export/home [snip] Either I think of using these resources, like may be creating some RAM disks (Henrik had a tip on that, cant find it) or get rid of hist' folder which has about 4000 files in it. I really like to keep the hist' files for report and going back to check for old events.

Keep the "hist" folder - there's only one file per status you have in Hobbit, and there won't be more.

What you should have a look at is the "histlogs" folder. This holds a copy of each status log whenever a status change occurred - and it gets *really* large over time. These historical status logs are only used when you look at the history of a status, and click on the red button off the "Jun 16 2003 14:21:58" event to see the detailed status log.

What I do on my production systems is to archive the history logs once a month, and then deleting those history logs that are older than 3 months. So you can get the full, detailed status of any event that happened within the past 3 months; if you need to go further back than that, it is still visible in the history view, but you'll need to extract the detailed status log from the tar file to see all of the details.

Feel free to use this snippet from my monthly cron job - it requires the GNU date utility. Right now, you will of course need to clean up your histlogs directory by hand, the snippet below only processes the histlog-files one month at a time.

cd /var/lib/hobbit/data

Archive histlogs generated in the past month

FNPATTERN=date --date="today -1 month" +"???_%b_*_%Y" ARCFN=date --date="today -1 month" +"histlogs_%Y_%m.cpio.gz" find histlogs -name "$FNPATTERN" | cpio -o --format=newc | gzip >archive/$ARCFN

Delete histlogs from 4 months ago (so we always have the last 3 months online)

FNPATTERN=date --date="today -4 months" +"???_%b_*_%Y" find histlogs -name "$FNPATTERN" | xargs rm

Excellent. I am slowly moving old logs. I wonder if that will skew my monthly and yearly reports.

snippet of the yearly report that gets called by bbcmd.

YEAR=$1 DATE=/usr/local/bin/date MKDIR=/usr/local/bin/mkdir YEAR=${YEAR:=$DATE +%Y} START=$DATE +%s --date="01 Jan $YEAR 00:00:00" END=$DATE +%s --date="31 Dec $YEAR 23:59:59" $MKDIR -p /usr/local/apache/htdocs/reports/yearly/$YEAR BBWEB=/reports/yearly/$YEAR
$BBHOME/bin/bbgen --reportopts=$START:$END:1:crit
--ignorecolumns=ssh,bbgen,bbtest,hobbitd,info,trends,msgs,scanmsg,http --subpagecolumns=2
/usr/local/apache/htdocs/reports/yearly/$YEAR

I noticed I have about 70000 files under histlogs. Wow!!. I guess would be nice to use database instead to store histlogs and make report still work. I know there are suggestion out there regarding mysql use to store logs.

(hobbit)@bigbrother:~/data$ find histlogs -name "???_Nov_*_2005" | wc -l 67912

Do you have any plan to have a DB plugin for that in the future? Or may be a suggestion, on storing log in database, that you think best work with hobbit?

Thanks

Regards, Henrik

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu "..there are two kinds of people: those who work and those who take the credit...try to be in the first group;...less competition there." - Indira Gandhi