July 2008 - Xymon - Xymon mailman web

[hobbit] RECOVERED alerts red->yellow
by Martin.Ward＠colt.net 23 Jul '08

23 Jul '08

Hi Alan, I think there is some confusion because in your first email you said: > I've read the sections on ALERTCOLORS and OKCOLORS. It does work better > when I remove yellow from the list. But I need to sometimes alert on > yellow status, so this is not really an option. But here you state: > red->yellow->green. If I use only the COLOR=red,yellow, I'm > goind to get alerted on a yellow > disk (undesired). So we need to understand more clearly when you want to be alerted on red and yellow and when you don't. |\/|artin -- > -----Original Message----- > From: Alan Sparks [mailto:asparks at doublesparks.net] > Sent: 09 July 2008 06:53 > To: hobbit at hswn.dk > Subject: Re: [hobbit] RECOVERED alerts red->yellow > > > OK, I've tried: > SERVICE=disk COLOR=red > MAIL asparks at gobucket.com COLOR=red,yellow RECOVERED > > And even: > SERVICE=disk COLOR=red,yellow > MAIL asparks at gobucket.com COLOR=red RECOVERED > > And neither sends a recovery page when disk goes > red->yellow->green. If I use only the COLOR=red,yellow, I'm > goind to get alerted on a yellow > disk (undesired). > Did I misunderstand your suggestion? Thanks. > -Alan > > > Tim McCloskey wrote: > > Try the following in the specific line in hobbitalerts.cfg: > > COLOR=red,yellow > > > > > > Alan Sparks wrote: > >> I have the following rule in my hobbit-alerts.cfg: SERVICE=disk > >> COLOR=red > >> MAIL asparks at photobucket.com RECOVERED > >> > > > > > > To unsubscribe from the hobbit list, send an e-mail to > > hobbit-unsubscribe at hswn.dk > > > > > > > > > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk > > > ************************************************************************************* The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing security at colt.net and delete the message and any attachments without retaining any copies. Internet communications are not secure and COLT does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of COLT Telecommunications, its subsidiaries or affiliates ("COLT") and any other party by email Communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900.

4 6

[hobbit] Help! bbtest-net gets http test timing wrong
by asparks＠doublesparks.net 23 Jul '08

23 Jul '08

So I've been trying for a onth to get this working, to no avail, and have pretty much exhausted everything to figure our why Hobbit randomly gets 3-second return times on HTTP tests. I'm even willing to call it a kernel problem, a problem with select() -- but it happens in multiple reasonably-contemporary kernels. I've tried: * CentOS 4.6 my standard build, CentOS 4.6 out-of-box, CentOS 5.1 out of box -- all have same problem. * Disabled ARES. Checked my DNS servers, they are answering fast. Besides, the tests are against IP addresses, not host names. * Removed all sysctl settings, let them default. No change. * Experimented with concurrency settings on [bbnet]. Doesn't help. * Ran tcpdumps between the web servers and Hobbit server. The tcpdumps indicate the web server is always answering immediately and sending the response... but the FIN packet (when Hobbit completes the test) is delayed 3 seconds. This issue tends to move around from host to host. It seems to affect Web servers that are sending static HTML pages, and all of them less than 4000 bytes (many about 155 bytes). As before, testing with other tools on the box show no issues with network connectivity or responses against the same servers. The /only/ thing that has come close to helping is adding a setsockopt() to bbtest-net, to set the receive bufer to 1024 bytes. This seems to override something that helps the select() call return better somehow. It's not a reliable or even sensible solution. It also does not work on the 2.6.18 kernel on CentOS 5.1... I'm really stumped and at the end of the rope on this. Has anyone had anything that looks like this? -Alan Alan Sparks wrote: > Does exactly the same thing on a fresh install of CentOS 5, x86_64. > All built by hand. > -Alan > > Alan Sparks wrote: >> I see where the problem seems to be occurring. But for my life I >> can't understand why. >> >> Packet traces from the Hobbit server and the Web servers showing the >> 3-second delays show that Hobbit connects, and gets an imediate >> answer from the server (milliseconds). But the servers show that >> Hobbit does not close the connection (a FIN packets sends/acks) for 3 >> seconds. >> >> Looking at the bb-network debugging logging, I see that the select() >> call sleeps for 3 seconds before returning in these cases. So the >> only conclusion I can arrive at is that select() doesn't return with >> the active file descriptors on schedule for some bizarre reason. >> >> For a desperation test, I forced the receive buffer on the sockets to >> a small number (1024 bytes): >> if (sockok) { >> int size = 1024; >> res = setsockopt(nextinqueue->fd, >> SOL_SOCKET, SO_RCVBUF, &size, >> sizeof(size)); >> >> This sortof works. the select() no longer hangs, and the HTTP tests >> start returning "normal"-ish results, i.e. numbers that match curl >> and wget statistics. >> >> But, it messes with numbers for other Web servers, the ones that >> return a page significantly larger than 1024 bytes. >> >> Like I said, I just can't get it. Hobbit or CentOS? There's nothing >> odd about this build, a generic CentOS 4.6 x86_64 build, Hobbit 4.2 >> with allinone patch, build for x86_64. >> >> Any suggestions at all? If this isn't the right place to ask, where >> would be? I can't get my hands around why the only thing that I >> can't get to work here is Hobbit... >> >> Thanks for your indulgence. I really wish I could fix this. >> -Alan >> >> >> Alan Sparks wrote: >>> After some Googling, I have added "AcceptFilter http none" >>> directives to the Apache 2.2 servers, which hasn't really helped >>> anything... >>> >>> Perhaps I should ask: Can anyone verify Hobbit works correctly on a >>> 64-bit system? Not should, but does, on a Centos 4 or RHEL 4 x86_64 >>> install? >>> >>> I see a lot of debugging trace stuff (dbgprint calls) in the contest >>> and httptest code. Can anyone tell me how to enable it to trace >>> what Hobbit is doing? >>> >>> Am really at a loss. This can't be rocket science to get it to >>> probe HTTP correctly. But a week later, I still cannot get it to >>> match any other monitoring tool's results. >>> -Alan >>> >>> Alan Sparks wrote: >>>> tcpdumps show a couple of interesting points. >>>> >>>> 1) There are definitely no DNS lookups occurring as a consequence >>>> of the Hobbit probes. No port 53 traffic out. >>>> >>>> 2) The packets from the Hobbit server, and the incoming packets to >>>> the Apache server, sometimes look like: >>>> >>>> 15:20:01.160095 IP (tos 0x0, ttl 62, id 31129, offset 0, flags >>>> [DF], proto 6, length: 60) hobbit.45116 > target.http: S [tcp sum >>>> ok] 265769416:265769416(0) win 17520 <mss 8760,sackOK,timestamp >>>> 143665233 0,nop,wscale 2> >>>> >>>> 15:20:04.159715 IP (tos 0x0, ttl 62, id 31131, offset 0, flags >>>> [DF], proto 6, length: 60) hobbit.45116 > target.http: S [tcp sum >>>> ok] 265769416:265769416(0) win 17520 <mss 8760,sackOK,timestamp >>>> 143668233 0,nop,wscale 2> >>>> >>>> 15:20:04.160223 IP (tos 0x0, ttl 62, id 31133, offset 0, flags >>>> [DF], proto 6, length: 40) hobbit.45116 > target.http: . [tcp sum >>>> ok] 265769417:265769417(0) ack 1051782089 win 17520 >>>> >>>> So that accounts for three seconds... it appears there are 2 SYN >>>> packets, but the first isn't getting processed and there's a >>>> 3-second delay to the next SYN (which gets ACKed). I don't know >>>> why this happens only with the Hobbit connections... and I don't >>>> know why the first SYN seems to be getting ignored. Server is not >>>> at all busy. >>>> >>>> -Alan >>>> Tim McCloskey wrote: >>>>> I get that wget/curl always work. Not sure what resolver settings >>>>> may be implemented differently for hobbit. >>>>> >>>>> Still thinking this may be unrelated to hobbit (even though >>>>> wget/curl work fine for you). We have many apache boxes spanning >>>>> multiple networks running httpd versions 1.3, 2.0 and 2.2 that >>>>> hobbit(4.2 with allinone patch) likes just fine and reports >>>>> accurate times (Seconds: 0.nn). We also have fairly proper >>>>> forward and reverse DNS records for the systems involved. >>>>> >>>>> I can't imagine hobbit parsing the wrong response times, but if >>>>> that is the case I wonder what external libraries are used (not >>>>> hobbit provided libs, as ours parse fine and are likely the same >>>>> as yours). >>>>> >>>>> Anyway, good luck with the tcpdump. >>>>> >>>>> Regards, >>>>> >>>>> Tim >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Alan Sparks wrote: >>>>>> UseCanonicalName is off, and HostNameLookup is off, on every >>>>>> server, regardless of version. >>>>>> -Alan >>>>>> >>>>>> Tim McCloskey wrote: >>>>>>> What do you have for >>>>>>> UseCanonicalName >>>>>>> in the apache 2.0 boxes? >>>>>>> >>>>> >>>>> >>>>> >>>>> >>>>> To unsubscribe from the hobbit list, send an e-mail to >>>>> hobbit-unsubscribe at hswn.dk >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> To unsubscribe from the hobbit list, send an e-mail to >>>> hobbit-unsubscribe at hswn.dk >>>> >>>> >>>> >>> >>> >>> >>> To unsubscribe from the hobbit list, send an e-mail to >>> hobbit-unsubscribe at hswn.dk >>> >>> >>> >> >> >> >> To unsubscribe from the hobbit list, send an e-mail to >> hobbit-unsubscribe at hswn.dk >> >> >> > > > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk > > >

1 0

FROM Address in emails
by Rafal.Roginela＠AmeriCashLoans.net 23 Jul '08

23 Jul '08

Hi All, I'm sorry if I sound dumb but I just want to change the "from address" or the "reply to" on the mails going out from my hobbit server. I've searched through the mailing list and docs and can't seem to see it (I'm sure it is simple). Please help. Thank you, Rafal Roginela Network Engineer AmeriCash Loans, LLC 880 Lee Street, Suite 302 Des Plaines, IL 60016 Office (847) 827-9740 x109 Fax (847) 493-8031 rroginela at americashloans.net <mailto:rroginela at americashloans.net> This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

5 14

"Info" column slow on latest snapshot (20080422)
by Craig.Cook＠gpi.com 23 Jul '08

23 Jul '08

I have similar symptoms. The info column on the hobbit server does not display, apache error log gives me this: [Wed Jul 23 08:26:28 2008] [error] [client 10.10.10.2] Premature end of script headers: bb-hostsvc.sh, referer: http://hobbitmon.example.com/hobbit/ It seems to have something to do with hosts with a client that reports full details, ie, disk, cpu, memory etc. I have some hosts doing ping only tests, info column is fine. Info column on Windows and Solaris clients that report in fail with a similar apache error eg. [Wed Jul 23 08:34:18 2008] [error] [client 10.10.10.2] Premature end of script headers: bb-hostsvc.sh, referer: http://hobbitmon.example.com/hobbit/Solaris/Solaris.html Stop/start of hobbit has no affect. I'm also using the Snapshot release. I use FQDN on all hosts. Craig

2 1

Weird conn test ping output
by josh＠imaginenetworksllc.com 23 Jul '08

23 Jul '08

Today I have a conn test that is saying this: x.x.105.219 is alive (54.7 ms) [<- x.x.98.226] The history has always said this: x.x.105.219 What's with the square bracket IP??? Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer

5 6

[hobbit] "Info" column slow on latest snapshot (20080422)
by cncook001＠yahoo.com 23 Jul '08

23 Jul '08

I have similar symptoms. The info column on the hobbit server does not display, apache error log gives me this: [Wed Jul 23 08:26:28 2008] [error] [client 10.10.10.2] Premature end of script headers: bb-hostsvc.sh, referer: http://hobbitmon.example.com/hobbit/ The info column on all other hosts seem to be fine though. Stop/start of hobbit has no affect. I'm also using the Snapshot release. I use FQDN on all hosts. Craig

1 0

[hobbit] "Info" column slow on latest snapshot (20080422)
by gumby3203＠gmail.com 23 Jul '08

23 Jul '08

Is anyone else experiencing a slow "info" page load with the 2008-04-22 snapshot? It takes about a minute to load. Also, I get this: Alerting: Alert configuration unavailable Admittedly, I just did an in-line install over the 4.2.0 release, so I don't know if it's just a matter of some configuration file weirdness.

4 11

[hobbit] UPDATE - some RRD graphs not drawn
by bb4＠richter-it.net 22 Jul '08

22 Jul '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, nobody has any thoughts on this? I have to say that the file system(s) not being shown on the disk's page graph is a cluster file system that can be mounted on any of both machines. Regards Torsten Torsten Richter wrote: | Hi, | | I've just checked the trends column. There all graphs are displayed | correctly. So only disk is affected. | | Regards | Torsten | | Torsten Richter wrote: | | Hi there, | | I'm experiencing a problem with the rrd graphs of the disk space. | | I'm running Hobbit 4.2.0 with the allinone patch on Debian Etch as | server and the clients are BB 1.9c. | | I have 2 Linux servers running with Veritas Cluster and reporting to | the Hobbit server. | | When I click on the disk icon of the first client everything is | displayed correctly. I have all the mount points on top of the page and | below them all mount points are displayed as graphs. | | On the second machine some of these graphs are missing although they | are displayed in the list on top and the RRD files are generated correctly. | | | | Regards | | Torsten | | | | To unsubscribe from the hobbit list, send an e-mail to | | hobbit-unsubscribe at hswn.dk | | | | | | | To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk - -- +---------------------------------------------------------+ | Richter IT-Consulting | | E-mail : bb4 [at] richter-it.net | | | | Homepage: http://www.richter-it.net/ | +---------------------------------------------------------+ Download my public key from: http://www.keyserver.net:11371/pks/lookup?op=get&search=0x899093AC -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3-nr1 (Windows XP) iD8DBQFIfkiq7DlmxomQk6wRAg+2AJ91sqUPJfE5CHDxy70PKnOMMMvxfQCfaCLJ F5RpBopcFIX7bHIXvlm/Kxo= =TAjS -----END PGP SIGNATURE-----

2 4

FILE monitor and mtime<600 troubleshooting
by denver＠fni-stl.com 22 Jul '08

22 Jul '08

Hi, I'm running Hobbit 4.2.0 with the allinone patch on hpux and can't get the FILE monitor to work. I've added the required file: entry in the client-local.cfg for the host, then put the FILE directive in the hobbit-clients.cfg file. After adding the monitor, the BBDISPLAY picks up the new file and shows it as 'green'. When testing the monitor by changing the mtime of my file, hobbit doesn't alert on the change. Here's my hobbit-client.cfg entry: CLASS=classname FILE /etc/sudoers yellow mtime<600 The client-local.cfg entry: [classname] file:/etc/sudoers Has anyone else seen this before or have any advice on how to troubleshoot? Thanks, Denver Osborn

4 10

[hobbit] Changing the front page layout
by Russell.Cook＠synetrix.co.uk 22 Jul '08

22 Jul '08

Thanks - problem solved -----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: 22 July 2008 13:39 To: hobbit at hswn.dk Subject: Re: [hobbit] Changing the front page layout On Tue, Jul 22, 2008 at 10:18:24AM +0100, Russell.Cook at synetrix.co.uk wrote: > I have been asked to change the way the front page of one of our Hobbit > looks. > > There are 3 page definitions below the title and I would like them to > appear in a list instead of appearing two to a line, which bunches > everything below the title and looks a bit messy. Wouldn't running bbgen with "--subpagecolumns=1" do what You want ? Either in hobbitserver.cfg (BBGENOPTS) or in hobbitlaunch.cfg which runs the bbgen command. Henrik To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk ---------------------------------------------------------------------------------------------------------- Synetrix Holdings Limited Tel: +44 (0)1252 405 600 www.synetrix.co.uk Synetrix (Holdings) Limited is a limited company registered in England and Wales. Registered number: 0349 1956. VAT number: GB776 1259 07. Registered office: Synetrix House, 49-51 Victoria Road, Farnborough, Hampshire, GU14 7PA. IMPORTANT NOTICE: This message is intended solely for the use of the Individual or organisation to whom it is addressed. It may contain privileged or confidential information. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you should not use, copy, alter, or disclose the contents of this message. All information or opinions expressed in this message and/or any attachments are those of the author and are not necessarily those of Synetrix Holdings Limited. Synetrix Holdings Limited accepts no responsibility for loss or damage arising from its use, including damage from virus.

1 0