- Xymon - Xymon mailman web

[hobbit] Message to long
by Vernon.Everett＠dpi.wa.gov.au 07 Aug '06

07 Aug '06

On Solaris 10? ________________________________ From: Colin Spargo [mailto:cspargo2 at csc.com] Sent: Monday, 7 August 2006 4:48 PM To: hobbit at hswn.dk Subject: RE: [hobbit] Message to long MAXMSG_CLIENT seems to need to allocate a shared memory segment of equal size. If your kernel shared memory settings are set too low (eg. if you are running defaults on solaris) then you may not be able to allocate the segment. If that happens, things like you just described seem to happen... you could possibly have dangling segments left over that were not cleaned up when you restarted hobbit, or possibly another app. Try checking what is allocated with "ipcs -a", and increasing the parameter "shmmax". ------------------------------------------------------------------------ ------------------------------------------------------------------------ -------------------------------- This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind CSC to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. ------------------------------------------------------------------------ ------------------------------------------------------------------------ --------------------------------

2 1

[hobbit] Message to long
by Vernon.Everett＠dpi.wa.gov.au 07 Aug '06

07 Aug '06

Hi Henrik Here's an interesting bug. Or at least I think it's a bug. I set the MAXMSG_CLIENT to 1024, and it fixed the problem I was having. (See below) Later on, I had to restart the server, and it crapped out. I remember that sometimes deleting ~hobbit/server/tmp/* before a startup can fix things, so I gave it a try. Still no go. I then restarted it with --debug, and the only log file getting written to was hobbitlaunch.log (See extract below) I then reversed the only thing that had changed - removed the MAXMSG_CLIENT entry, and it all started up fine. A bug? Or a PEBKAC? Regards Vernon --- snip --- 2006-08-07 16:23:54 Loading hostnames 2006-08-07 16:23:54 Opening file /export/home/hobbit//server/etc/bb-hosts 2006-08-07 16:23:54 Opening file /export/home/hobbit//server/etc/client-local.cfg 2006-08-07 16:23:54 Loading saved state 2006-08-07 16:23:54 Cannot access checkpoint file /export/home/hobbit//server/tmp/hobbitd.chk for restore 2006-08-07 16:23:54 Setting up network listener on 0.0.0.0:1984 2006-08-07 16:23:54 Setting up signal handlers 2006-08-07 16:23:54 Setting up hobbitd channels 2006-08-07 16:23:54 Setting up status channel (id=1) 2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',1) 2006-08-07 16:23:54 ftok() returns: 0x100748F 2006-08-07 16:23:54 shmget() returns: 0x1200001E 2006-08-07 16:23:54 Setting up stachg channel (id=2) 2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',2) 2006-08-07 16:23:54 ftok() returns: 0x200748F 2006-08-07 16:23:54 shmget() returns: 0x1200001F 2006-08-07 16:23:54 Setting up page channel (id=3) 2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',3) 2006-08-07 16:23:54 ftok() returns: 0x300748F 2006-08-07 16:23:54 shmget() returns: 0x12000020 2006-08-07 16:23:54 Setting up data channel (id=4) 2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',4) 2006-08-07 16:23:54 ftok() returns: 0x400748F 2006-08-07 16:23:54 shmget() returns: 0x12000021 2006-08-07 16:23:54 Setting up notes channel (id=5) 2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',5) 2006-08-07 16:23:54 ftok() returns: 0x500748F 2006-08-07 16:23:54 shmget() returns: 0x12000022 2006-08-07 16:23:54 Setting up enadis channel (id=6) 2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',6) 2006-08-07 16:23:54 ftok() returns: 0x600748F 2006-08-07 16:23:54 shmget() returns: 0x12000023 2006-08-07 16:23:54 Setting up client channel (id=7) 2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',7) 2006-08-07 16:23:54 ftok() returns: 0x700748F 2006-08-07 16:23:54 shmget() returns: 0x12000024 2006-08-07 16:23:54 Setting up clichg channel (id=8) 2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',8) 2006-08-07 16:23:54 ftok() returns: 0x800748F 2006-08-07 16:23:54 Could not get shm of size 1048576: Invalid argument 2006-08-07 16:23:54 Cannot setup clichg channel --- snip --- -----Original Message----- From: Everett, Vernon [mailto:Vernon.Everett at dpi.wa.gov.au] Sent: Monday, 7 August 2006 12:17 PM To: hobbit at hswn.dk Subject: RE: [hobbit] Message to long Yep, that was it. Thanks Lars Cheers Vernon -----Original Message----- From: lars ebeling [mailto:lars.ebeling at leopg9.no-ip.org] Sent: Monday, 7 August 2006 11:53 AM To: hobbit at hswn.dk Subject: Re: [hobbit] Message to long Have a look at hobbitserver.cfg manpage. MAXMSG_CLIENT Regards Lars ----- Original Message ----- From: "Everett, Vernon" <Vernon.Everett at dpi.wa.gov.au> To: <hobbit at hswn.dk> Sent: Monday, August 07, 2006 3:42 AM Subject: [hobbit] Message to long Hi all Despite my best efforts, my procs page remains purple. I found this in the hobbitd page - which also remains yellow. (Server name changed to pluto) --- snip --- Latest errormessages: Oversize data/client msg from 10.1.5.242 truncated (n=949733, limit 524288) First line: pluto|sunos|sunos Oversize stachg msg from 10.1.5.242 for pluto:ports truncated (n=508447, limit=262144) Oversize page/ack/notify msg from 10.1.5.242 for pluto:ports truncated (n=508456, limit=262144) Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508452, limit=262144) Oversize data/client msg from 10.1.5.242 truncated (n=977049, limit 524288) First line: pluto|sunos|sunos Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508448, limit=262144) --- snip --- I am assuming that the output from the ports and procs test is too large. How do we make Hobbit happy with the larger message? This machine is a proxy server, so we can expect many procs and a long ports output. Regards Vernon To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

2 1

page.log
by Dominique.Frise＠unil.ch 07 Aug '06

07 Aug '06

Hi, What does these line means: 2006-08-07 08:51:02 Semaphore wait aborted: Interrupted system call 2006-08-07 08:51:20 Setting up page channel (id=3) 2006-08-07 08:51:20 calling ftok('/soft/pub/BB/hobbit/server',3) 2006-08-07 08:51:20 ftok() returns: 0x3000A01 2006-08-07 08:51:20 shmget() returns: 0x25B Dominique UNIL - University of Lausanne

2 2

[hobbit] Message to long
by Vernon.Everett＠dpi.wa.gov.au 07 Aug '06

07 Aug '06

Yep, that was it. Thanks Lars Cheers Vernon -----Original Message----- From: lars ebeling [mailto:lars.ebeling at leopg9.no-ip.org] Sent: Monday, 7 August 2006 11:53 AM To: hobbit at hswn.dk Subject: Re: [hobbit] Message to long Have a look at hobbitserver.cfg manpage. MAXMSG_CLIENT Regards Lars ----- Original Message ----- From: "Everett, Vernon" <Vernon.Everett at dpi.wa.gov.au> To: <hobbit at hswn.dk> Sent: Monday, August 07, 2006 3:42 AM Subject: [hobbit] Message to long Hi all Despite my best efforts, my procs page remains purple. I found this in the hobbitd page - which also remains yellow. (Server name changed to pluto) --- snip --- Latest errormessages: Oversize data/client msg from 10.1.5.242 truncated (n=949733, limit 524288) First line: pluto|sunos|sunos Oversize stachg msg from 10.1.5.242 for pluto:ports truncated (n=508447, limit=262144) Oversize page/ack/notify msg from 10.1.5.242 for pluto:ports truncated (n=508456, limit=262144) Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508452, limit=262144) Oversize data/client msg from 10.1.5.242 truncated (n=977049, limit 524288) First line: pluto|sunos|sunos Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508448, limit=262144) --- snip --- I am assuming that the output from the ports and procs test is too large. How do we make Hobbit happy with the larger message? This machine is a proxy server, so we can expect many procs and a long ports output. Regards Vernon To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

1 0

Message to long
by Vernon.Everett＠dpi.wa.gov.au 07 Aug '06

07 Aug '06

Hi all Despite my best efforts, my procs page remains purple. I found this in the hobbitd page - which also remains yellow. (Server name changed to pluto) --- snip --- Latest errormessages: Oversize data/client msg from 10.1.5.242 truncated (n=949733, limit 524288) First line: pluto|sunos|sunos Oversize stachg msg from 10.1.5.242 for pluto:ports truncated (n=508447, limit=262144) Oversize page/ack/notify msg from 10.1.5.242 for pluto:ports truncated (n=508456, limit=262144) Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508452, limit=262144) Oversize data/client msg from 10.1.5.242 truncated (n=977049, limit 524288) First line: pluto|sunos|sunos Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508448, limit=262144) --- snip --- I am assuming that the output from the ports and procs test is too large. How do we make Hobbit happy with the larger message? This machine is a proxy server, so we can expect many procs and a long ports output. Regards Vernon

2 1

check once an hour or day for a given check for a given host
by jjj863＠gmail.com 06 Aug '06

06 Aug '06

how can I specify check should be run once a day or hour (or any time period other than the default 5 minutes), at the check level, say for only one FILE or PROC entry out of several FILE/PROC entries ? I know I can change the frequency of checking for all.

3 2

[hobbit] Some more clear RRD duplicating-causing samples
by brodie＠mcw.edu 06 Aug '06

06 Aug '06

Buchan- you should really capture the incoming data segment while simultaneously watching the rrd-data.log file; that way you can 100% let Henrik know which data chunks are causing RRD issues. In my case, I only had to "watch" for 5 minutes to capture all the instances of duplicztes happening.... --------------------------------------------------------- Kent C. Brodie - brodie at phys.mcw.edu Department of Physiology Medical College of Wisconsin (414) 456-8590 -----Original Message----- From: Buchan Milne [mailto:bgmilne at staff.telkomsa.net] Sent: Thursday, August 03, 2006 9:46 AM To: hobbit at hswn.dk Subject: Re: [hobbit] Some more clear RRD duplicating-causing samples On Thursday 03 August 2006 16:28, Brodie, Kent wrote: > Henrik, here you go: Two more netstat data chunks that cause the > Duplication error (parsing issue?) in rrd/hobbit. > > My simple observation is the one thing these two hosts have in common is > virtual network interfaces, so instead of simpler items like eth0, eth1 > and so on, you have eth0, eth0:0, and so forth. Linux in this case, > but that would apply to solaris and so on as well. Just a > hunch................. > > @@data#98185|1154614736.024527|192.168.224.202||dunn.hmgc.mcw.edu|ifstat > data dunn,hmgc,mcw,edu.ifstat > linux > eth0 Link encap:Ethernet HWaddr 00:09:3D:13:DC:AB > inet addr:192.168.224.105 Bcast:192.168.224.255 > Mask:255.255.255.0 > inet6 addr: fe80::209:3dff:fe13:dcab/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:40318794 errors:0 dropped:0 overruns:0 frame:0 > TX packets:15352953 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:2942042461 (2.7 GiB) TX bytes:21803054108 (20.3 GiB) > Interrupt:185 > > eth0:0 Link encap:Ethernet HWaddr 00:09:3D:13:DC:AB > inet addr:192.168.224.107 Bcast:192.168.224.255 > Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > Interrupt:185 > > eth0:1 Link encap:Ethernet HWaddr 00:09:3D:13:DC:AB > inet addr:192.168.224.160 Bcast:192.168.224.255 > Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > Interrupt:185 > > lo Link encap:Local Loopback > inet addr:127.0.0.1 Mask:255.0.0.0 > inet6 addr: ::1/128 Scope:Host > UP LOOPBACK RUNNING MTU:16436 Metric:1 > RX packets:37737 errors:0 dropped:0 overruns:0 frame:0 > TX packets:37737 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:6042388 (5.7 MiB) TX bytes:6042388 (5.7 MiB) > > @@ > > > > > > > @@data#98366|1154614859.203728|192.168.224.202||clifford.hmgc.mcw.edu|if > stat > data clifford,hmgc,mcw,edu.ifstat > linux > eth0 Link encap:Ethernet HWaddr 00:09:3D:13:C8:78 > inet addr:192.168.224.104 Bcast:192.168.224.255 > Mask:255.255.255.0 > inet6 addr: fe80::209:3dff:fe13:c878/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:146220628 errors:0 dropped:0 overruns:0 frame:0 > TX packets:42544430 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:14048692258 (13.0 GiB) TX bytes:57506442876 (53.5 > GiB) > Interrupt:185 > > eth0:0 Link encap:Ethernet HWaddr 00:09:3D:13:C8:78 > inet addr:192.168.224.106 Bcast:192.168.224.255 > Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > Interrupt:185 > > eth0:1 Link encap:Ethernet HWaddr 00:09:3D:13:C8:78 > inet addr:192.168.224.159 Bcast:192.168.224.255 > Mask:255.255.255.0 > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > Interrupt:185 > > lo Link encap:Local Loopback > inet addr:127.0.0.1 Mask:255.0.0.0 > inet6 addr: ::1/128 Scope:Host > UP LOOPBACK RUNNING MTU:16436 Metric:1 > RX packets:1189289 errors:0 dropped:0 overruns:0 frame:0 > TX packets:1189289 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:347179798 (331.0 MiB) TX bytes:347179798 (331.0 MiB) > > @@ Hmm, now that I think about this more carefully, this issue could be affecting me as well, resulting in some gaps in various rrd files (which I first thought might be affected by running everything via bbproxy and a rather complex setup). In our environment we typically use bonding/teaming (eg with eth0 connected to one switch, eth1 connected to another, with the two interfaces set up in passive failover, appearing as one "bond0" device). Then, to ensure access to the right networks, we have VLANs (eg bond0.4, bond0.54, bond0106) on top of the bonded interface. In some cases (HA setups), there are virtual interfaces (eg bond0.4:0)on top of the bonded interface. I've included sample ifstat contents (but, not necessarily at the time I've seen gaps in the rrds): [ifstat] bond0 Link encap:Ethernet HWaddr 00:16:35:7E:8D:59 inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:1353358109 errors:0 dropped:0 overruns:0 frame:0 TX packets:1414161412 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:3466938915 (3.2 GiB) TX bytes:1361756811 (1.2 GiB) bond0.4 Link encap:Ethernet HWaddr 00:16:35:7E:8D:59 inet addr:192.168.211.41 Bcast:192.168.211.255 Mask:255.255.255.0 inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:1353172280 errors:0 dropped:0 overruns:0 frame:0 TX packets:1414125729 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:3747147076 (3.4 GiB) TX bytes:2930351274 (2.7 GiB) bond0.4:0 Link encap:Ethernet HWaddr 00:16:35:7E:8D:59 inet addr:192.168.211.203 Bcast:192.168.211.255 Mask:255.255.255.0 UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 bond0.54 Link encap:Ethernet HWaddr 00:16:35:7E:8D:59 inet addr:192.168.12.18 Bcast:192.168.15.255 Mask:255.255.252.0 inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:15562 errors:0 dropped:0 overruns:0 frame:0 TX packets:220 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:952935 (930.6 KiB) TX bytes:9408 (9.1 KiB) bond0.106 Link encap:Ethernet HWaddr 00:16:35:7E:8D:59 inet addr:192.168.106.68 Bcast:192.168.106.255 Mask:255.255.255.0 inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:173127 errors:0 dropped:0 overruns:0 frame:0 TX packets:38287 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:10379598 (9.8 MiB) TX bytes:3224568 (3.0 MiB) eth0 Link encap:Ethernet HWaddr 00:16:35:7E:8D:59 inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:1353358109 errors:0 dropped:0 overruns:0 frame:0 TX packets:1414161412 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3466938915 (3.2 GiB) TX bytes:1361756811 (1.2 GiB) Interrupt:201 eth1 Link encap:Ethernet HWaddr 00:16:35:7E:8D:59 inet6 addr: fe80::216:35ff:fe7e:8d59/64 Scope:Link UP BROADCAST NOARP SLAVE MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:209 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:1668785 errors:0 dropped:0 overruns:0 frame:0 TX packets:1668785 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:720576454 (687.1 MiB) TX bytes:720576454 (687.1 MiB) Regards, Buchan -- Buchan Milne ISP Systems Specialist B.Eng,RHCE(803004789010797),LPIC-2(LPI000074592)

3 2

[hobbit] hobbit-*.cfg syntax check
by brodie＠mcw.edu 06 Aug '06

06 Aug '06

Speaking of config files--- the "STOP" keyword in hobbit-alerts.cfg does not appear to be recognized, even though it's referenced in the documentation?!?!?!? --------------------------------------------------------- Kent C. Brodie - brodie at phys.mcw.edu Department of Physiology Medical College of Wisconsin (414) 456-8590 -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Friday, August 04, 2006 10:09 AM To: hobbit at hswn.dk Subject: Re: [hobbit] hobbit-*.cfg syntax check On Fri, Aug 04, 2006 at 10:40:12AM -0400, Gary B. wrote: > > Is there a syntax checker for the hobbit-*.cfg configuration files > (such as hobbit-clients.cfg)? I'm having an issue where some of the > rules I set up are apparently being ignored, or otherwise not working > as expected. I have a feeling it may be due to a syntax error, but > I'm not sure. The utilities that read the configuration file does complain if they find something they cannot understand. Look in the clientdata.log and page.log files (for hobbit-clients.cfg and hobbit-alerts.cfg, respectively). There are also "--dump-config" options for these two utilities, which will dump the configuration as they understand it. And complain about any errors. > I'm also curious, how does Hobbit deal with syntax errors in the > configuration files? Does it just skip over bad syntax, or stop > completely when it encounters an error? It tries to cope, ignoring the error and continuing with whatever bits of the config file it understands. Regards, Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

2 1

[hobbit] BUG: Glibc 'free(): invalid next size' in bbcmd on FC5
by henrik＠hswn.dk 05 Aug '06

05 Aug '06

On Fri, Aug 04, 2006 at 12:33:37PM -0700, Japheth J.C. Cleaver wrote: > > Running "bbcmd" after a clean install (default configuration, current > allinone patch) results in a glibc free() error. This is on Fedora > Core 5 with Glibc2.4-8. Running as "MALLOC_CHECK_=0 ./bbcmd" is a > temporary workaround. Thanks, this appears to be a 1-byte miscalculation of a buffer needed for one of the Hobbit environment variables. This patch should solve it. Regards, Henrik

1 0

monitor a series of files for absolute size (alert & trending)
by jjj863＠gmail.com 04 Aug '06

04 Aug '06

greetings, I need to monitor some os and db backup files for their sizes for alert based on absolute sizes as well as rrd trending. Does Hobbit do this now? I am running 4.2-RC-20060712 on CentOS 4.3/i386. A twist is the file names are timestamped (os-backup-YYYYMMDD-HHmm.star.gz). any suggestions/tricks? I thought of making a copy to a fixed name for monitoring, but it is kinda expensive due to the size of the backups themselves.

6 12