March 2009 - Xymon - Xymon mailman web

Xymon Client dies unexpectedly?
by jbl＠jbldata.com 29 Mar '09

29 Mar '09

Having an odd problem that I'm not quite sure how to troubleshoot it. I'm using Xymon 4.2.3 on Debian (kernel 2.4.27-2-386). I have one client which was was installed as user 'xymon' (configure/make/make install). I run the client with 'runclient.sh start' and I see the services go great on the web interface. All looks great. After about 30 minutes however, the client seems to stop sending reports to the server, and all services go purple. Here's what I see when looking at the process listing on the client when things are working correctly: xymon at JBLDMZWEB01:~/client$ ps -ef | grep hobbit xymon 25702 1 0 05:23 ? 00:00:00 /home/xymon/client/bin/hobbitlaunch --config=/home/xymon/client/etc/clientlaunch.cfg --log=/home/xymon/client/logs/clientlaunch.log --pidfile=/home/xymon/client/logs/clientlaunch.JBLDMZWEB01.pid xymon 25724 1 0 05:23 ? 00:00:00 sh -c vmstat 300 2 1>/home/xymon/client/tmp/hobbit_vmstat.JBLDMZWEB01.25707 2>&1; mv /home/xymon/client/tmp/hobbit_vmstat.JBLDMZWEB01.25707 /home/xymon/client/tmp/hobbit_vmstat.JBLDMZWEB01 And then when the client stops reporting to the server, the process listing looks like this: xymon at JBLDMZWEB01:~/client$ ps -ef | grep hobbit xymon 25702 1 0 05:23 ? 00:00:00 /home/xymon/client/bin/hobbitlaunch --config=/home/xymon/client/etc/clientlaunch.cfg --log=/home/xymon/client/logs/clientlaunch.log --pidfile=/home/xymon/client/logs/clientlaunch.JBLDMZWEB01.pid So other than the virtual memory stats missing from the process list, it seems like it should still be sending reports. I've checked the log files in 'logs/' and 'tmp/' but found nothing useful. This is a pretty fresh install of the Xymon server/client, so there isn't really much that hsould be affecting it. Should I be running the client as root? Does it provide more logs that way? Any thoughts? -- J. Bobby Lopez Web: http://jbldata.com/ Twitter: http://www.twitter.com/jbobbylopez

2 4

memory leak on hobbitd_channel ?
by nico＠crysto.org 28 Mar '09

28 Mar '09

Hi, On my hobbit server, i got lot of memory used by hobbitd_channel (4.4g = 74% of all the memory ). Tasks: 177 total, 14 running, 162 sleeping, 0 stopped, 1 zombie Cpu(s): 20.6% us, 3.2% sy, 0.0% ni, 32.9% id, 42.9% wa, 0.2% hi, 0.3% si Mem: 6107804k total, 6059264k used, 48540k free, 195772k buffers Swap: 4194288k total, 4194252k used, 36k free, 209976k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 24938 hobbit 15 0 6702m 4.4g 1400 S 0.0 74.7 0:00.76 hobbitd_channel 16808 hobbit 15 0 304m 253m 3600 S 0.3 4.3 26:55.67 hobbitd 16886 hobbit 15 0 145m 114m 444 S 0.0 1.9 0:02.41 hobbitd_channel 16902 hobbit 15 0 36312 31m 524 S 1.0 0.5 3:54.60 hobbitd_alert 24838 hobbit 17 0 36336 31m 344 R 0.3 0.5 0:00.01 hobbitd_alert 24829 hobbit 15 0 36312 31m 332 S 0.0 0.5 0:00.00 hobbitd_alert 3028 root 16 0 358m 21m 1556 S 0.0 0.4 64:04.23 mxdtf 16930 hobbit 17 0 87048 13m 1588 R 5.0 0.2 30:05.96 devmon 16897 hobbit 16 0 26128 12m 636 S 0.0 0.2 19:29.34 hobbitd_client My questions: - is it normal that hobbitd_channel takes so much memory (4.4Gb). - Is it possible to decrease it ? (kill ? ) - Is it possible to know why it's taking so much memory (strace ?) With strace on it, i only see this things: write(4, "@@status#902860|1238227571.36674"..., 486) = -1 EAGAIN (Resource temporarily unavailable) nanosleep({0, 2500000}, NULL) = 0 semop(9666576, 0x7fbfffe460, 1) = 0 rt_sigaction(SIGALRM, {0x402140, [ALRM], SA_RESTORER|SA_RESTART, 0x2a9569a300}, {SIG_IGN}, 8) = 0 alarm(2) = 2 semop(9666576, 0x7fbfffe460, 1) = 0 rt_sigaction(SIGALRM, {SIG_IGN}, {0x402140, [ALRM], SA_RESTORER|SA_RESTART, 0x2a9569a300}, 8) = 0 semop(9666576, 0x7fbfffe460, 1) = 0 write(4, "@@status#902860|1238227571.36674"..., 486) = -1 EAGAIN (Resource temporarily unavailable) nanosleep({0, 2500000}, NULL) = 0 semop(9666576, 0x7fbfffe460, 1) = 0 rt_sigaction(SIGALRM, {0x402140, [ALRM], SA_RESTORER|SA_RESTART, 0x2a9569a300}, {SIG_IGN}, 8) = 0 alarm(2) = 2 semop(9666576, 0x7fbfffe460, 1) = 0 rt_sigaction(SIGALRM, {SIG_IGN}, {0x402140, [ALRM], SA_RESTORER|SA_RESTART, 0x2a9569a300}, 8) = 0 semop(9666576, 0x7fbfffe460, 1) = 0 write(4, "@@status#902860|1238227571.36674"..., 486) = -1 EAGAIN (Resource temporarily unavailable) nanosleep({0, 2500000}, ----> it s writing status. (?) Here the status statistic (bbgen): bbgen for Hobbit version 4.2.0 Statistics: Hosts : 8910 Status messages : 70987 Purple messages : 0 Pages : 29 Any idea about memory is welcome ! Thanks, Regards, nico

1 0

How to get reports of acks??
by Harald.Husemann＠materna.de 27 Mar '09

27 Mar '09

Hi, in our datacenter, the operators are responsible for acknowledging every alert after they've informed the second level staff about it. To track this, we'd like to have the acks in the event log report, together with the other events - the best solution would be to see any transition from one status to another in the event view, i. e.: - Ack's - Changes in the icon from the smileys to the others (e.g. exclamation mark for warnings), to signal long-lasting, unacknowledged alarms The first should not be very complicated, so, I hope someone did it before, :-). The second point is a bit more tricky, I think... Thanks, hh -- Harald Husemann Netzwerk- und Systemadministrator Operation Management Center (OMC) MATERNA GmbH Information & Communications Westfalendamm 98 44141 Dortmund Geschäftsführer: Dr. Winfried Materna, Helmut an de Meulen, Ralph Hartwig Amtsgericht Dortmund HRB 5839 Tel: +49 231 9505 222 Fax: +49 231 9505 100 www.annyway.com <http://www.annyway.com/> www.materna.com <http://www.materna.com/>

1 0

performance measurement with collectl
by martin.flemming＠desy.de 27 Mar '09

27 Mar '09

Hi ! I want to aks, if somebody use this great performance-meassure-tool "Collectl" for linux for/with xymon or anyhow ... http://collectl.sourceforge.net/ I think if collectl could use for xymon it's adjusted to fit almost any meassurepoint we want ... :-) cheers, Martin

1 0

NCV trouble
by SKreien＠la-z-boy.com 27 Mar '09

27 Mar '09

Hi everyone, This is my first attempt at getting the NCV module working and I'm having trouble getting the graph to work. The broken image shows up in the test page and the trends page for my custom test and the RRD file exists and is recording values, so I think it's my graph definition. But I can't figure out what about it is wrong. Here's the top part of the RRD from the dump: rrdtool dump /home/hobbit/data/rrd/retv6000.na.lzb.hq/lowmem.rrd <rrd> <version> 0003 </version> <step> 300 </step>  <lastupdate> 1238122513 </lastupdate>  <ds> <name> lowmem </name> <type> GAUGE </type> <minimal_heartbeat> 600 </minimal_heartbeat> <min> NaN </min> <max> NaN </max>  <last_ds> 686268 </last_ds> <value> 8.9214840000e+06 </value> <unknown_sec> 0 </unknown_sec> </ds> Here's the section from hobbitgraph.cfg: [lowmem] TITLE Low memory usage YAXIS kb DEF:memory=lowmem.rrd:lowmem:AVERAGE LINE2:memory#00CCC:Memory Usage I'd appreciate some help in getting this working. Thanks, Scot Kreienkamp La-Z-Boy Inc.

2 2

Error building xymon 4.4.0
by eric.j.boyd＠gmail.com 26 Mar '09

26 Mar '09

Attemping to build xymon 4.4.0 on Mac OS 10.5.6 (client) and got the following error during make: mkdir .libs /usr/bin/ar cru .libs/libcares.a ares_fds.o ares_getsock.o ares_process.o ares_free_hostent.o ares_query.o ares__close_sockets.o ares_free_string.o ares_search.o ares__get_hostent.o ares_gethostbyaddr.o ares_send.o ares__read_line.o ares_gethostbyname.o ares_strerror.o ares_cancel.o ares_init.o ares_timeout.o ares_destroy.o ares_mkquery.o ares_version.o ares_expand_name.o ares_parse_a_reply.o windows_port.o ares_strdup.o ares_expand_string.o ares_parse_ptr_reply.o ares_parse_aaaa_reply.o ares_getnameinfo.o inet_net_pton.o bitncmp.o inet_ntop.o ares_writev.o ares_parse_ns_reply.o ares_llist.o ares__timeval.o ares_strcasecmp.o ranlib: file: .libs/libcares.a(windows_port.o) has no symbols ranlib: file: .libs/libcares.a(ares_strdup.o) has no symbols ranlib: file: .libs/libcares.a(inet_net_pton.o) has no symbols ranlib: file: .libs/libcares.a(inet_ntop.o) has no symbols ranlib: file: .libs/libcares.a(ares_writev.o) has no symbols ranlib: file: .libs/libcares.a(ares_strcasecmp.o) has no symbols ranlib .libs/libcares.a ranlib: file: .libs/libcares.a(windows_port.o) has no symbols ranlib: file: .libs/libcares.a(ares_strdup.o) has no symbols ranlib: file: .libs/libcares.a(inet_net_pton.o) has no symbols ranlib: file: .libs/libcares.a(inet_ntop.o) has no symbols ranlib: file: .libs/libcares.a(ares_writev.o) has no symbols ranlib: file: .libs/libcares.a(ares_strcasecmp.o) has no symbols creating libcares.la (cd .libs && rm -f libcares.la && ln -s ../libcares.la libcares.la) if gcc -DHAVE_CONFIG_H -I. -I. -O -Wall -Wno-unused -D_REENTRANT -DBIND_8_COMPAT=1 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DDarwin -isystem . -isystem /Users/coreadmin/dev/src/xymon-4.4.0/include -isystem /usr/local/include -g0 -Wno-system-headers -MT ahost.o -MD -MP -MF ".deps/ahost.Tpo" -c -o ahost.o ahost.c; \ then mv -f ".deps/ahost.Tpo" ".deps/ahost.Po"; else rm -f ".deps/ahost.Tpo"; exit 1; fi if gcc -DHAVE_CONFIG_H -I. -I. -O -Wall -Wno-unused -D_REENTRANT -DBIND_8_COMPAT=1 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DDarwin -isystem . -isystem /Users/coreadmin/dev/src/xymon-4.4.0/include -isystem /usr/local/include -g0 -Wno-system-headers -MT ares_getopt.o -MD -MP -MF ".deps/ares_getopt.Tpo" -c -o ares_getopt.o ares_getopt.c; \ then mv -f ".deps/ares_getopt.Tpo" ".deps/ares_getopt.Po"; else rm -f ".deps/ares_getopt.Tpo"; exit 1; fi /bin/sh ./libtool --tag=CC --mode=link gcc -O -Wall -Wno-unused -D_REENTRANT -DBIND_8_COMPAT=1 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DDarwin -isystem . -isystem /Users/coreadmin/dev/src/xymon-4.4.0/include -isystem /usr/local/include -g0 -Wno-system-headers -o ahost ahost.o ares_getopt.o ./libcares.la gcc -O -Wall -Wno-unused -D_REENTRANT -DBIND_8_COMPAT=1 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DDarwin -isystem . -isystem /Users/coreadmin/dev/src/xymon-4.4.0/include -isystem /usr/local/include -g0 -Wno-system-headers -o ahost ahost.o ares_getopt.o ./.libs/libcares.a if gcc -DHAVE_CONFIG_H -I. -I. -O -Wall -Wno-unused -D_REENTRANT -DBIND_8_COMPAT=1 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DDarwin -isystem . -isystem /Users/coreadmin/dev/src/xymon-4.4.0/include -isystem /usr/local/include -g0 -Wno-system-headers -MT adig.o -MD -MP -MF ".deps/adig.Tpo" -c -o adig.o adig.c; \ then mv -f ".deps/adig.Tpo" ".deps/adig.Po"; else rm -f ".deps/adig.Tpo"; exit 1; fi adig.c:144: error: ‘T_NAPTR’ undeclared here (not in a function) adig.c: In function ‘display_rr’: adig.c:648: warning: case label value exceeds maximum value for type make[3]: *** [adig.o] Error 1 make[2]: *** [all] Error 2 make[1]: *** [c-ares/.libs/libcares.a] Error 2 make: *** [bbnet-build] Error 2 any ideas? Eric

1 0

hobbit] Report Availability not html format into Internet Explorer
by GODARTHE＠e-i.com 26 Mar '09

26 Mar '09

Hi, In xymon 4.2.2, When i start a Report Availability or Snapshot Report. Into my Internet Explorer, the page is text Into my Firefox, the page is html i don't understand. Can i help me ? Thank you --bbrep-23533-1238088364 Content-type: text/html <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML> <HEAD> <META HTTP-EQUIV="Set-Cookie" CONTENT="pagepath=; path=/"> <META HTTP-EQUIV="Set-Cookie" CONTENT="host=; path=/"> <TITLE>Xymon Availability Report : Mar 25 2009</TITLE>  <link rel="stylesheet" type="text/css" href="/bb/menu/menu.css">  <link rel="shortcut icon" href="/bb/gifs/favicon-blue.ico"> </HEAD> <BODY BGCOLOR="blue" BACKGROUND="/bb/gifs/bkg-blue.gif" TEXT="#D8D8BF" LINK="#00FFAA" VLINK="#FFFF44"> <TABLE SUMMARY="Topline" WIDTH="100%"> <TR><TD HEIGHT=16> </TD></TR>  <TR> <TD VALIGN=MIDDLE ALIGN=LEFT WIDTH="30%"> Xymon </TD> <TD VALIGN=MIDDLE ALIGN=CENTER WIDTH="40%"> <CENTER>Availability Report</CENTER> </TD> <TD VALIGN=MIDDLE ALIGN=RIGHT WIDTH="30%"> Mar 25 2009 </TD> Hervé GODART }}} Euro Information Pôle Technologie 36, rue de messines 59237 Verlinghem Tel: +33 3.20.13.52.15 Fax: +33 3.20.40.58.10 e-mail: godarthe at e-i.com<mailto:godarthe at e-i.com> <mailto:godarthe at e-i.com> *********************************************************************************************** Ce message et toutes les pièces jointes sont confidentiels et établis à l'intention exclusive de son ou ses destinataires. Si vous avez reçu ce message par erreur, merci d'en avertir immédiatement l'émetteur et de détruire le message. Toute modification, édition, utilisation ou diffusion non autorisée est interdite. L'émetteur décline toute responsabilité au titre de ce message s'il a été modifié, déformé, falsifié, infecté par un virus ou encore édité ou diffusé sans autorisation. This message and any attachments are confidential and intended for the named addressee(s) only. If you have received this message in error, please notify immediately the sender, then delete the message. Any unauthorized modification, edition, use or dissemination is prohibited. The sender does not be liable for this message if it has been modified, altered, falsified, infected by a virus or even edited or disseminated without authorization. ***********************************************************************************************

1 0

PATCH: ignore .rpm* debris in directories included in config files
by jcleaver＠soe.sony.com 26 Mar '09

26 Mar '09

This patch alters hobbit-wide "directory" include reading logic to skip files that have been replaced or left over by an RPM install. Specifically, we're splitting hobbitlaunch.cfg into a directory of files, but we don't want to read in things tweaked by a package upgrade; and there's no way to exclude those in the syntax... Regards, Japheth Cleaver

2 2

Drop test columns
by icicimov＠gmail.com 26 Mar '09

26 Mar '09

Hi all, I have Hobbit 4.2.0 running on Debian box and was trying numerous times to drop the "apt" and "libs" columns from the columns display for couple of client hosts with no avail. The reason I would like to drop them is they go yellow after system upgrades and started to be annoing now after some time. I have tried doing it according to hobbit documentation: - drop the test using the bb command - use NOCOLUMNS statement in the bb-hosts file for each of the hosts to hide the columns but the columns keep coming back after couple of minutes. So none of the above approaches work really. Any idea why this is not working and what else can be done to remove the columns permanently? I have searched the forum but couldn't find appropriate answer apart from one stating config file hacking which I don't take as appropriate solution since this should be working functionality in Hobbit. Thanks in advance for any answers and ones again thanks to everybody involved in this project for the great monitoring tool they have created :) Igor

4 4

acknowledge past disruptions
by wouter＠schoot.org 26 Mar '09

26 Mar '09

Dear list, I'm now using hobbit (4.2) for a few months and I really like it's possibilities, thanks for writing it! However, one thing I've not found and answer for yet, is how to handle downtime which is in the past. The ISP where my monitoring box is located suffered a DDoS quite recently, which rendered the monitoring quite useless. I'm now looking for a way to tell hobbit "all the downtime and alerts between <timestamp> and <timestamp> can be safely ignored and should not be taken into account when generating an availability report". Is this possible? Thanks in advance! Wouter

1 0