monitor a series of files for absolute size (alert & trending)
greetings,
I need to monitor some os and db backup files for their sizes for alert based on absolute sizes as well as rrd trending. Does Hobbit do this now? I am running 4.2-RC-20060712 on CentOS 4.3/i386.
A twist is the file names are timestamped (os-backup-YYYYMMDD-HHmm.star.gz). any suggestions/tricks? I thought of making a copy to a fixed name for monitoring, but it is kinda expensive due to the size of the backups themselves.
you can probably write a custom script that will do what is needed, and have the Hobbit client run it wherever you need it. Then you can write another script on the server side to "catch" the data and set up the RRD calls. There are several moving parts, but Henrik has done a nice job explaining what needs to be done, and provides a helpful example.
I use this function for a number of custom measurements, and it ticks like a watch. Took me a few hours to get the first one working, and the rest were pretty easy after that.
GLH
From: Jerry Yu [mailto:jjj863 at gmail.com]
Sent: Thursday, August 03, 2006 8:04 AM
To: hobbit at hswn.dk
Subject: [hobbit] monitor a series of files for absolute size(alert & trending)
greetings,
I need to monitor some os and db backup files for their sizesfor alert based on absolute sizes as well as rrd trending. Does Hobbit do this now? I am running 4.2-RC-20060712 on CentOS 4.3/i386.
A twist is the file names are timestamped(os-backup-YYYYMMDD-HHmm.star.gz). any suggestions/tricks? I thought of making a copy to a fixed name for monitoring, but it is kinda expensive due to the size of the backups themselves.
On 8/3/06, Jerry Yu <jjj863 at gmail.com> wrote:
greetings,
I need to monitor some os and db backup files for their sizes for alert based on absolute sizes as well as rrd trending. Does Hobbit do this now? I am running 4.2-RC-20060712 on CentOS 4.3/i386.
A twist is the file names are timestamped (os-backup-YYYYMMDD-HHmm.star.gz). any suggestions/tricks? I thought of making a copy to a fixed name for monitoring, but it is kinda expensive due to the size of the backups themselves.
Why not make a hardlink to each file and change it at midnight with a cron entry?? That gives you the fixed names without the overhead of copying the files.
Ralph Mitchell
On Thu, Aug 03, 2006 at 09:04:23AM -0400, Jerry Yu wrote:
I need to monitor some os and db backup files for their sizes for alert based on absolute sizes as well as rrd trending. Does Hobbit do this now? I am running 4.2-RC-20060712 on CentOS 4.3/i386.
A twist is the file names are timestamped (os-backup-YYYYMMDD-HHmm.star.gz). any suggestions/tricks? I thought of making a copy to a fixed name for monitoring, but it is kinda expensive due to the size of the backups themselves.
First thing is to get a "file:" entry in client-local.cfg to grab the data for the latest file. Something like:
file:`ls -t -1 /backup(os-backup-*.tar.gz|head -1`This runs the "ls -t...." command to determine the filename. Since it uses a time-sort and grabs only the first line, it should give you the name of the latest file.
Next you want to track the size of it. In hobbit-clients.cfg define a FILE entry to track this - it needs to use a regex to match the filename, and an explicit RRD id to make it always use a specific RRD file. Perhaps you want to alert if they get bigger than 1 GB. So:
FILE %^/backup/os-backup.*.tar.gz SIZE<1G TRACK=osbackupOther interesting options for the FILE entry might be "MTIME<86400" to check that the latest backup file is at most 24 hours old.
[10 minutes later]
OK, I've learnt to test things before sending mails like this. The TRACK setting for files and directories currently ignores the ID you may pass to it, and uses the current filename when deciding on the RRD filename. So to use this, you'll need to grab either the current snapshot and build that, or the current "all-in-one" patch from http://www.hswn.dk/hobbitsw/betapatches/
Regards, Henrik
thanks for verifying, Henrik. I'll wait for 4.2 release to rebuild my RPMs. Right now, I am still settling in with my first Hobbit installation. used to work with bb-1.3/5/9.
On 8/3/06, Henrik Stoerner <henrik at hswn.dk> wrote:
On Thu, Aug 03, 2006 at 09:04:23AM -0400, Jerry Yu wrote:
I need to monitor some os and db backup files for their sizes for alert based on absolute sizes as well as rrd trending. Does Hobbit do this now? I am running 4.2-RC-20060712 on CentOS 4.3/i386.
A twist is the file names are timestamped ( os-backup-YYYYMMDD-HHmm.star.gz). any suggestions/tricks? I thought of making a copy to a fixed name for monitoring, but it is kinda expensive due to the size of the backups themselves.
First thing is to get a "file:" entry in client-local.cfg to grab the data for the latest file. Something like:
file:`ls -t -1 /backup(os-backup-*.tar.gz|head -1`This runs the "ls -t...." command to determine the filename. Since it uses a time-sort and grabs only the first line, it should give you the name of the latest file.
Next you want to track the size of it. In hobbit-clients.cfg define a FILE entry to track this - it needs to use a regex to match the filename, and an explicit RRD id to make it always use a specific RRD file. Perhaps you want to alert if they get bigger than 1 GB. So:
FILE %^/backup/os-backup.*.tar.gz SIZE<1G TRACK=osbackupOther interesting options for the FILE entry might be "MTIME<86400" to check that the latest backup file is at most 24 hours old.
[10 minutes later]
OK, I've learnt to test things before sending mails like this. The TRACK setting for files and directories currently ignores the ID you may pass to it, and uses the current filename when deciding on the RRD filename. So to use this, you'll need to grab either the current snapshot and build that, or the current "all-in-one" patch from http://www.hswn.dk/hobbitsw/betapatches/
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Henrik, I set it up per the steps above. the correct files now show up under the 'files' column. however, no rrd of any files.*.rrd gets created under the server:/var/lib/hobbit/DB09p/rrd. I do understand if the rrd name doesn't use the track id under 4.2-RC-20060712, then it is not useful anyway. FILE %^/backup/*trans*cp size<500M mtime<3600 track=transDbDump FILE %^/backup/*full*dp size<50G track=fullDbDump
On 8/3/06, Jerry Yu <jjj863 at gmail.com> wrote:
thanks for verifying, Henrik. I'll wait for 4.2 release to rebuild my RPMs. Right now, I am still settling in with my first Hobbit installation. used to work with bb-1.3/5/9.
On 8/3/06, Henrik Stoerner <henrik at hswn.dk> wrote:
On Thu, Aug 03, 2006 at 09:04:23AM -0400, Jerry Yu wrote:
I need to monitor some os and db backup files for their sizes for alert based on absolute sizes as well as rrd trending. Does Hobbit do this now? I am running 4.2-RC-20060712 on CentOS 4.3/i386.
A twist is the file names are timestamped ( os-backup-YYYYMMDD-HHmm.star.gz). any suggestions/tricks? I thought of making a copy to a fixed name for
monitoring, but it is kinda expensive due to the size of the backups themselves.
First thing is to get a "file:" entry in client-local.cfg to grab the data for the latest file. Something like:
file:`ls -t -1 /backup(os-backup-*.tar.gz|head -1`This runs the "ls -t...." command to determine the filename. Since it uses a time-sort and grabs only the first line, it should give you the name of the latest file.
Next you want to track the size of it. In hobbit-clients.cfg define a FILE entry to track this - it needs to use a regex to match the filename, and an explicit RRD id to make it always use a specific RRD file. Perhaps you want to alert if they get bigger than 1 GB. So:
FILE %^/backup/os-backup.*.tar.gz SIZE<1G TRACK=osbackupOther interesting options for the FILE entry might be "MTIME<86400" to check that the latest backup file is at most 24 hours old.
[10 minutes later]
OK, I've learnt to test things before sending mails like this. The TRACK setting for files and directories currently ignores the ID you may pass to it, and uses the current filename when deciding on the RRD filename. So to use this, you'll need to grab either the current snapshot and build that, or the current "all-in-one" patch from http://www.hswn.dk/hobbitsw/betapatches/
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
eh, neither does the size check generate alerts either. I changed it to size<1G to coerce it to fail as the file is around 20G
FILE %^/backup/*full*cmp* size<1G mtime<86400 track=fullDbDump
Click on the file as listed under 'files' gives the following: [file:/backup/db_full.cmp_080406.101245] type:100000 (file) mode:644 (-rw-r-----) linkcount:1 owner:506 (sql) group:506 (dbas) size:22155416434 clock:1154705700 (2006/08/04-11:35:00) atime:1154702075 (2006/08/04-10:34:35) ctime:1154701823 (2006/08/04-10:30:23) mtime:1154701823 (2006/08/04-10:30:23)
On 8/4/06, Jerry Yu <jjj863 at gmail.com> wrote:
Henrik, I set it up per the steps above. the correct files now show up under the 'files' column. however, no rrd of any files.*.rrd gets created under the server:/var/lib/hobbit/DB09p/rrd. I do understand if the rrd name doesn't use the track id under 4.2-RC-20060712, then it is not useful anyway. FILE %^/backup/*trans*cp size<500M mtime<3600 track=transDbDump FILE %^/backup/*full*dp size<50G track=fullDbDump
On 8/3/06, Jerry Yu <jjj863 at gmail.com> wrote:
thanks for verifying, Henrik. I'll wait for 4.2 release to rebuild my RPMs. Right now, I am still settling in with my first Hobbit installation. used to work with bb-1.3/5/9.
On 8/3/06, Henrik Stoerner < henrik at hswn.dk> wrote:
On Thu, Aug 03, 2006 at 09:04:23AM -0400, Jerry Yu wrote:
I need to monitor some os and db backup files for their sizes for alert based on absolute sizes as well as rrd trending. Does Hobbit do this now? I am running 4.2-RC-20060712 on CentOS 4.3/i386.
A twist is the file names are timestamped ( os-backup-YYYYMMDD-HHmm.star.gz). any suggestions/tricks? I thought of making a copy to a fixed name for monitoring, but it is kinda expensive due to the size of the backups themselves.
First thing is to get a "file:" entry in client-local.cfg to grab the data for the latest file. Something like:
file:`ls -t -1 /backup(os-backup-*.tar.gz|head -1`This runs the "ls -t...." command to determine the filename. Since it uses a time-sort and grabs only the first line, it should give you the
name of the latest file.
Next you want to track the size of it. In hobbit-clients.cfg define a FILE entry to track this - it needs to use a regex to match the filename, and an explicit RRD id to make it always use a specific RRD file. Perhaps you want to alert if they get bigger than 1 GB. So:
FILE %^/backup/os-backup.*.tar.gz SIZE<1G TRACK=osbackupOther interesting options for the FILE entry might be "MTIME<86400" to
check that the latest backup file is at most 24 hours old.
[10 minutes later]
OK, I've learnt to test things before sending mails like this. The TRACK setting for files and directories currently ignores the ID you may pass to it, and uses the current filename when deciding on the RRD filename. So to use this, you'll need to grab either the current snapshot and build that, or the current "all-in-one" patch from http://www.hswn.dk/hobbitsw/betapatches/
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Fri, Aug 04, 2006 at 11:44:17AM -0400, Jerry Yu wrote:
eh, neither does the size check generate alerts either. I changed it to size<1G to coerce it to fail as the file is around 20G
FILE %^/backup/*full*cmp* size<1G mtime<86400 track=fullDbDump
Your regex is wrong. "*" by itself doesn't do what you want; it must be ".*". So it should be
FILE %^/backup/.*full.*cmp.* size<1G mtime<86400 track=fullDbDump
Regards, Henrik
hmm, I suspected that. however, the file is now listed under 'files'. I guess that's not enough?
On 8/4/06, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Aug 04, 2006 at 11:44:17AM -0400, Jerry Yu wrote:
eh, neither does the size check generate alerts either. I changed it to size<1G to coerce it to fail as the file is around 20G
FILE %^/backup/*full*cmp* size<1G mtime<86400 track=fullDbDump
Your regex is wrong. "*" by itself doesn't do what you want; it must be ".*". So it should be
FILE %^/backup/.*full.*cmp.* size<1G mtime<86400 track=fullDbDump
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Fri, Aug 04, 2006 at 11:58:12AM -0400, Jerry Yu wrote:
Your regex is wrong. "*" by itself doesn't do what you want; it must be ".*". So it should be
FILE %^/backup/.*full.*cmp.* size<1G mtime<86400 track=fullDbDump hmm, I suspected that. however, the file is now listed under 'files'. I
guess that's not enough?
It goes on the "files" display as soon as the client reports anything about it. However, to apply any kind of action to the file data it must be recognized by hobbit-clients.cfg. That includes size checks and tracking, which is why your size check doesn't trigger and the RRD file doesn't exist.
Regards, Henrik
you are absolutely correct, Henrik. corrected the RE and the alerts were triggered. can't wait to have track rrd working as well..
On 8/4/06, Henrik Stoerner <henrik at hswn.dk> wrote:
On Fri, Aug 04, 2006 at 11:58:12AM -0400, Jerry Yu wrote:
Your regex is wrong. "*" by itself doesn't do what you want; it must be ".*". So it should be
FILE %^/backup/.*full.*cmp.* size<1G mtime<86400 track=fullDbDump hmm, I suspected that. however, the file is now listed under 'files'. I
guess that's not enough?
It goes on the "files" display as soon as the client reports anything about it. However, to apply any kind of action to the file data it must be recognized by hobbit-clients.cfg. That includes size checks and tracking, which is why your size check doesn't trigger and the RRD file doesn't exist.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
At work we use strict iptables rules, so what exceptions need to be made to get hobbit working? Do I need to allow traffic on port 1984 from the server to the client, client to the server, or both? Also do I need to allow both TCP and UDP traffic on that connection?
Thanks, Jordan
Jordan Mendler wrote:
At work we use strict iptables rules, so what exceptions need to be made to get hobbit working? Do I need to allow traffic on port 1984 from the server to the client, client to the server, or both? Also do I need to allow both TCP and UDP traffic on that connection?
In "normal" mode, the hobbit clients send information TO the server on TCP port 1984. If you use hobbitfetch then you need to allow outbound TCP port 1984 to the clients. Hobbit does not use UDP for anything.
-Charles
participants (6)
-
greg.hubbard＠eds.com
-
henrik＠hswn.dk
-
jjj863＠gmail.com
-
jmendler＠ucla.edu
-
jonescr＠cisco.com
-
ralphmitchell＠gmail.com