January 2008 - Xymon - Xymon mailman web

New Monitors Added
by gjohnson＠trantor.org 29 Jan '08

29 Jan '08

New monitors recently added to The Shire: * urlplus.pl - http://www.trantor.org/theshire/doku.php/monitors:urlplus --- The Shire

2 1

Monitor HP-UX network log
by robert.herron＠gmail.com 28 Jan '08

28 Jan '08

HP-UX stores its network log in a binary file (/var/adm/nettl.LOG000) that you view with the netfmt command. Before I start working on my own, does anyone have an EXT script to monitor it? If so, could I have a copy? Thanks, Robert Herron

2 1

urlplus.pl http/https monitoring extension script
by gumby3203＠gmail.com 28 Jan '08

28 Jan '08

We are in the process of replacing several sitescope monitors with Hobbit, and I have found some of the built-in Hobbit http checks inadequate. As a result, I have been working on an extension script which at its core is essentially a wrapper for curl. However, it provides for more robust user-customization than is easily possible with Hobbit's built-in tests. It is still a work-in-progress, but I believe it has matured enough to be worth posting on The Shire. I wrote it using perl, but I have a feeling that some point in the future it may need to be converted to C (mostly for the speed). However, it seems to work pretty good for me now. The main feature I required from this script was the addition of a per-host customizable timeout period. Some of the hosts I'm testing take up to a minute to return back any data, and this is considered normal. However, I didn't want to bump up the timeout for _all_ hosts. Admittedly, this script uses an additional configuration file and doesn't completely integrate with the bb-hosts file, but it works for me and helps keep the bb-hosts file clean. I'd be interested in what other people think of this script, which is the main reason I'm contributing it now. I have attached it to this post (I haven't yet subscribed to theshire-developers, so I'm not able to upload it myself yet).

3 6

hobbit-client: segfault on several systems
by hobbit＠alexkeller.de 28 Jan '08

28 Jan '08

Hi all, I'm currently testing hobbit und I'm quite satisfied with it. Great monitoring tool! Hobbit will definitely replace our BigBrother installation. But yesterday a somehow strange error occured. About 15 hobbit clients suddenly died (all at the same time! +/- 5min) They all wrote a core file. Some clients reported a segfault: hobbitlaunch[15432]: segfault at 00007fff75d42a48 rip 00000000004047ad rsp 00007fff75d42a50 error 6 I compiled the clients from snapshot version 20071026. The server is already version 20080115. Client-Systems are Debian Linux Sarge/Etch and Solaris. gdb-Output (gdb ./client/bin/hobbitlaunch core) : GNU gdb 6.4.90-debian Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i486-linux-gnu"...Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1". Reading symbols from /lib/tls/i686/cmov/libc.so.6...done. Loaded symbols for /lib/tls/i686/cmov/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Core was generated by `/opt/hobbit/client/bin/hobbitlaunch --config=/opt/hobbit/client/etc/clientlaunc'. Program terminated with signal 11, Segmentation fault. #0 errprintf (fmt=0x805507c "Time warp detected: Adjusting returned clock by %d seconds\n") at errormsg.c:42 42 errormsg.c: Datei oder Verzeichnis nicht gefunden. in errormsg.c (gdb) (gdb) bt #0 errprintf (fmt=0x805507c "Time warp detected: Adjusting returned clock by %d seconds\n") at errormsg.c:42 #1 0x08050c8e in getcurrenttime (retparm=0x0) at timefunc.c:55 #2 0x0804c10b in errprintf (fmt=0x805507c "Time warp detected: Adjusting returned clock by %d seconds\n") at errormsg.c:42 #3 0x08050c8e in getcurrenttime (retparm=0x0) at timefunc.c:55 #4 0x0804c10b in errprintf (fmt=0x805507c "Time warp detected: Adjusting returned clock by %d seconds\n") at errormsg.c:42 #5 0x08050c8e in getcurrenttime (retparm=0x0) at timefunc.c:55 #6 0x0804c10b in errprintf (fmt=0x805507c "Time warp detected: Adjusting returned clock by %d seconds\n") at errormsg.c:42 #7 0x08050c8e in getcurrenttime (retparm=0x0) at timefunc.c:55 [cut] #3966 0x0804c10b in errprintf (fmt=0x805507c "Time warp detected: Adjusting returned clock by %d seconds\n") at errormsg.c:42 #3967 0x08050c8e in getcurrenttime (retparm=0x0) at timefunc.c:55 #3968 0x0804ac2d in main (argc=4, argv=0x0) at hobbitlaunch.c:536 Are there any suggestions? Any ideas? Thank you for your help! Regards, Alexander

3 4

Purple results on server's tests
by josh＠imaginenetworksllc.com 28 Jan '08

28 Jan '08

I rebooted the hobbit server for power related reasons and since it came up the following tests are purple on the Hobbit server: bbgen/bbtest/cpu/disk/files/hobbitd/memory/msgs/ports/procs In bb-hosts the server looks like... 74.x.x.x serversname.imaginenetworksllc.com # bbd http://serversname.imaginenetworksllc.com/ The server was powered on at Sun Jan 27 05:17 but the purple reports are saying Mon Jan 28 10:11:15 2008. The time is reporting as... -> date Mon Jan 28 10:32:38 EST 2008 Anyone know how I broke it? =) Thanks in advance! -- Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer

3 5

Hobbit DDOS Attack Detection (submission)
by jonescr＠cisco.com 28 Jan '08

28 Jan '08

I had a hobbit-monitored site become the victim of a DDOS syn-flood attack. To help detect this in the futrue and also aid information-gathering, I did the following, which I am sharing: 1. Made the following additions to hobbit-clients.cfg # Detect more than 100 half-open connections (possible syn-flood attack) PORT STATE=SYN_RECV MIN=0 MAX=100 COLOR=red TRACK="SYN" # Detect more than 2000 established connections (possible simple DDOS http get attack) PORT STATE=ESTABLISHED MIN=0 MAX=2000 COLOR=red 2. Made a small modification to hobbit-linux.sh, specifically to the [ports] section. I crammed it all into a single line. echo "[ports]" echo "SYN_REC Quick Stats:";SYNs=`netstat -pant 2>/dev/null | grep SYN | awk -F: '{print $2;}' |awk {'print $2"\t"$1'}| sort | uniq -c |sort -n`;if [ -n "$SYNs" ]; then echo " # Address Port";printf "$SYNs\n";echo =============================;else echo "No SYNs Found";echo =============================;fi # Bug in RedHat's netstat spews annoying error messages. netstat -ant 2>/dev/null #1 allows Hobbit to detect and alert for 2 common DDOS attack signatures (syn floods and plain old http overloading), as well as creating graphs via the "track" feature. #2 prefixes the netstat output you see in the "ports" column with a table of IPs that have half-open connections. It shows the address, IP, and what port they are "attacking". Here is an example (IPs and ports masked of course): SYN_REC Quick Stats: # Address Port 1 289.122.3.20 80 1 213.102.135.60 80 1 200.120.152.6 8080 1 201.192.9.130 443 1 174.231.84.4 443 2 191.136.92.135 80 2 216.122.32.240 8080 ============================= The prefixing of this data in the client message has no effect on the Hobbit server, as it is only looking for the netstat output in the ports section, and so ignores that extra data (but still displays it). So if the site gets a syn flood, you can check the ports column of your web host(s) and easily see the offending IP(s) that are performing the attack. Note that most "floodbots" are coded such that they spoof their IP addresses, so the information you gather may not actually help you determine who is attacking, but is still useful for temporarily firewalling or whatever other steps you need to take to block it. Note: this works on RHEL, so should work on RedHat/CentOS/Fedora. I havn't checked to see if debian/ubuntu has the same netstat parameters I used, so YMMV :) -Charles

3 4

Feature request: OS support for GNU/kFreeBSD
by beckert＠phys.ethz.ch 28 Jan '08

28 Jan '08

Hi, I tried to get hobbit compiling and running on Debian GNU/kFreeBSD[1]. Getting it to compile was quite easy, getting it run, too, but getting it work isn't[2]. [1] http://www.debian.org/ports/kfreebsd-gnu/ [2] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=458417 Getting the client run needed some more files to exist. Now I've got an ugly but running client sending data to my hobbit server. Unfortunately the server drops it with the message 2008-01-25 22:41:09 No client backend for OS 'gnu/kfreebsd' sent by [...] In noticed that the reason for this is an OS switch() in hobbitd/hobbitd_client.c line 1763 ff. Back to the client: Adding a new OS in general isn't difficult with hobbit on the client side, you just copy some shell script, perhaps edit it and it works. With GNU/kFreeBSD it looks a little bit different: 14/0/0 root at c-metisse:pts/ttyp3 22:49:51 [~] # uname -o GNU/kFreeBSD 15/0/0 root at c-metisse:pts/ttyp3 22:49:54 [~] # There is a slash in the OS name. So all my workarounds are quite ugly since they all involve the creation of directories to allow filenames like hobbitclient-gnu/kfreebsd.sh Not nice, but works. The solution suggested by the GNU/kFreeBSD developers for this problem is to replace all slashes in the output of "uname -o" with underscores (written in Perlish: s(/)(_)g) so that the above example would look like this: hobbitclient-gnu_kfreebsd.sh Hernik: How do you think that problem is solved best from your view as hobbit developer? You probably have more an idea which code may need to be patch. I've lost the overview at the moment, since hobbit seems to use the output of "uname -o" in a lot of places. Do you think, you can generate a patch I can test? If not: with a few pointers to most of the places in the source, I probably also can try to create a patch on my own... Alternatively I should be able to get you an account on a Test machine running Debian GNU/kFreeBSD. (I'm currently testing inside a virtual machine using QEMU/KVM.) P.S.: No, it's not for the job this time, it's for my own boxes and those of friends. :-) Kind regards, Axel Beckert -- Axel Beckert <beckert at phys.ethz.ch> support: +41 44 633 2668 IT Support Group, HPR E 86.1 voice: +41 44 633 4189 Departement Physik, ETH Zurich fax: +41 44 633 1239 CH-8093 Zurich, Switzerland http://nic.phys.ethz.ch/

3 4

[hobbit] Turn off full system logs
by Chris.Morris＠RWEsystemsUK.com 28 Jan '08

28 Jan '08

Henrik, Basically I just want to reduce the amount of data that is displayed on the web page and in email alerts. Detailed log files can be viewed in other ways. Regards, Chris > -----Original Message----- > From: Henrik Stoerner [SMTP:henrik at hswn.dk] > Sent: Monday, January 28, 2008 12:13 PM > To: hobbit at hswn.dk > Subject: Re: [hobbit] Turn off full system logs > > On Mon, Jan 28, 2008 at 12:07:17PM -0000, Morris, Chris (SS-IS) wrote: > > > > Is there a simple way to turn off the full system logs displayed in the > web > > page and sent in email alerts? > > Sorry, no - this isn't possible. > > > I see you can use --no-ps-listing and --no-ports-listing, it would be > useful > > to have a --no-full-msgs-listing as some log files can contain a large > > amount of data, which you could access using the links on the msgs page. > > You can also access it via the "Client data" link on the same page, so > what exactly is it that you want to accomplish ? > > > Regards, > Henrik > > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe at hswn.dk > ********************************************************************** This e-mail including any attachments is provided for general information purposes only and does not constitute advice except to the addressee if expressly stated in this email. For the avoidance of doubt the contents of this email and any attachments are subject to contract and are not intended to and will not constitute a legally binding contract or any part of a legally binding contract. The information contained in this email and any attachments is confidential. It is intended only for the use of the intended recipient at the email address to which it has been addressed. If the reader of this message is not an intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution or copying of the message or associated attachments is strictly prohibited. If you have received this e-mail in error, please notify postmaster at RWEsystemsUK.com and permanently delete the email and any attachments immediately from your system. You should not retain, copy or use this email or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you for your co-operation. Neither RWE Systems UK Ltd nor any of the other companies in the RWE Group from whom this e-mail originates accept any responsibility for losses or damage as a result of any viruses and it is your responsibility to check attachments (if any) for viruses. RWE Systems UK Ltd Registered office: Mistral, Westlea Campus, Chelmsford Road, Swindon, Wiltshire, SN5 7EZ. Registered in England and Wales: company number 6052966. *******************************************

1 0

bbwin 0.10 available
by etienne.grignon＠gmail.com 28 Jan '08

28 Jan '08

Hello, After more than a year without new release, I publish today BBWin 0.10. This is the new stuffs I put in it: BBWin 0.10 Preview version - December 30th 2007 ------------------- +New Features ** Centralized mode for agents cpu, disk, memory, msgs, netstat, ports, procs, timediff, uptime ** Centralized mode graphing network traffic ** See the bbWin webpage for instructions http://bbwin.sourceforge.net/ * Msgs agent reports parts of description events when the message file contening the static string can't be found. It will reports as the Windows Event Viewer, just like this : "The description for Event ID ( XXX ) in Source ( XXX ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: XXXXX, XXXXX." * You can use environment variables in the bbwinupdate agent. Example : <setting name="filename" value="bbwin\%COMPUTERNAME%.cfg" /> * svcs services rules can be used with comments * disk agent can monitor free space from XX-rom drive and not only if a media is inserted. (feature requested to monitor dvd-ram burning state) * disk agent can monitor volume mount points * Proxy access : BBWin can access to your hobbit server through a proxy (http proxy) You can specify a user/password if you use http proxy basic authentication. * disk agent will alert you if a disk monitored with a custum rule disappear from the Server. (Example : external usb harddrive turned off) * uptime agent : add max delay option to get alerts when servers are up for a too long time * who agent : report current connected users on the servers Here the link for the download : http://sourceforge.net/projects/bbwin About the centralized mode, to handle it with bbwin, you have to use a patch for hobbit 4.2 or for the future 4.3, it is available on http://bbwin.sourceforge.net/ The centralized mode for BBWin in hobbit sources has been developed by François Lacroix, one of the new developper of devmon. Big thanks to him. Even if we tested on our servers, it may contains bugs so you should test it not on productions servers for the moment. You can report bugs on the bbwin or hobbit mailing list so we can improve the solution. We hope to have something enough good to be included in hobbit 4.3 and will continue to update the patch for hobbit 4.2. The client-local.cfg is not supported for the moment. It is the next step (files, dir, logs, clientupdate). I have also some general questions about bbwin : Actually, svcs only works as a local agent because there is no option for it in hobbit-clients.cfg, it is very specific to Windows world. What do you prefer ? - we keep svcs agent with local configuration. BBWin can handle centralized mode with agents which will run with local configuration - we create a new option for the hobbit-clients.cfg which will be only used for Windows client - May be you have other good ideas about that problem. At this time, it is only about svcs agent but for advanced features in the future, we will have other choices to make. With the centralized mode, you will see that you get graphs for each of your networks connections. However, for Windows, the names are too long compared to the unix name (eth0, eri0..). So, at this time, I use the mac address of the network card for the graph name. Would you prefer to use the name by making a very short name ? the ip address ? or is the mac address fine ? There is a new agent 'who' used to report current connected users on the windows servers. There is no alert for it at this time. Which relevant type of alerts could be useful ? there is no alert options on Unix side. I was thinking of a rule like PROC but called USER with a minimum logged number and a maximum logged number for a username. For 2008, I will do my best to create more releases. 2007 was not a good year :-) The first goal will be to finalize the centralized mode with all the unix options. I now use the sourceforge subversion repository so you will always get the lastest sources of BBWin. As usual, let me know if you get troubles with this new version. Happy new year for all the hobbit community :-) Etienne.

6 6

Notification logs analysis
by emarganne＠be.tiauto.com 28 Jan '08

28 Jan '08

Hello, I downloaded some files which are about the log analyis but I can not tell if they need other files. Also I do not really know how to compile properly the C code to "add" it to an already installed Hobbit version. If someone could explain me how I could make that feature work, I am very interested. Best regards, Etienne. ________________________________ From: Marganne, Etienne Sent: Thursday, January 03, 2008 4:23 PM To: hobbit at hswn.dk Subject: Notification logs analysis Hello, Could you please tell me which part of the code of the futur 4.3 release of Hobbit performs the notification log analysis? Please also let me know if it is independent from all the other new features of that release. I am very enthusiast about that feature and I would like to test and if it works according to my desire put into production. Also of course if you think for some reason that there could be bugs in that feature let me know :-). Thank you, Marganne Etienne. DISCLAIMER: The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are Not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

1 0