- Xymon - Xymon mailman web

[hobbit] Current development plans
by craig＠cookitservices.com 10 Jun '05

10 Jun '05

Okay, here is some more info from the author... ---- http://madstop.com/svn/libmetrics/ At this point it's just about pulled directly out of ganglia, although I've added a bit so it actually acts like a library interface -- your code doesn't need to know as much about the internals. It should be straightforward to build a client around this code. > This post just came up on the hobbitmon mailing list: > > <snip> > So my idea currently is to design a new type of client. It won't generate > "status" > messages, it will just collect data. Imagine a client that just sends Hobbit a > "client data" package, like [...] This shouldn't be difficult to do, but it's just about exactly what ganglia already does. Unfortunately, ganglia is relatively special-purpose and thus won't scale to doing other tasks like sending alerts, but it already sends all of its data to a central node, and you can easily configure aggregation nodes that then send the rest of the data further upstream. It's a pretty slick system for getting graphs of data, and because it's all XML you can easily parse the data and do whatever else you want with it. > I think it is the same concept at the metric thing I am thinking about. > It may be a development shortcut... I highly recommend spending an hour or so getting ganglia working, just so you understand it and know how much it can already do. I don't know how much you're hoping to accomplish on the client, but ganglia might suffice. In general, I agree that for many cases it makes sense for clients to push performance data to the server, especially for info that can only be collected locally like disks, as long as you have some process on the server that causes an alert if it hasn't heard from the client recently. The reason I split the libmetrics stuff out (and no one else is actually using it at this point, although the ganglia people have expressed interest in coding to the library instead of what they have) is because I'm planning on using it eventually in puppet, so that puppet could actually collect these stats as one of its tasks and then make them available via SOAP or something. At some point your config mgmt tools will have to integrate with your monitoring tools, so that you can do things like respond to high or low loads, so I'm just thinking a bit ahead on that. ---- Craig Cook -- Systems Monitoring Consulting and Support Services http://www.cookitservices.com

1 0

[hobbit] Current development plans
by craig＠cookitservices.com 10 Jun '05

10 Jun '05

That client style sounded familiar... This may be what you are looking for: http://sourceforge.net/projects/libmetrics It is a metrics library that has been extracted from a larger tool http://ganglia.sourceforge.net/ Does not have any files though, I'm checking on the status now. May save a lot of work... Craig Cook -- Systems Monitoring Consulting and Support Services http://www.cookitservices.com

1 0

[hobbit] Current development plans
by roland.graeub＠rtc.ch 10 Jun '05

10 Jun '05

Hello Henrik I'd also like to see something for the acknowledgement enhancement before you break off on server development and start concentrating on the client. Nice Summer to all Hobbitians Roland

1 0

BBTEST PROBLEM : WARNING: Runtime 421 longer than time limit (300)
by brianlynch＠gmail.com 09 Jun '05

09 Jun '05

Has anyone seen the following problem with bbtest where the total processing exceeds the 300 second limit? Note that I do not have an excessive # of hosts/tests. I've also included the hobbitlaunch.cfg entry at the bottom of the email. Thanks, Bria bbtest-net version 4.0.3 SSL library : OpenSSL 0.9.7a Feb 19 2003 LDAP library: OpenLDAP 20224 Statistics: Hosts total : 279 Hosts with no tests : 126 Total test count : 278 Status messages : 285 Alert status msgs : 0 Transmissions : 4 DNS statistics: # hostnames resolved : 154 # succesful : 153 # failed : 1 # calls to dnsresolve : 190 TCP test statistics: # TCP tests total : 118 # HTTP tests : 37 # Simple TCP tests : 81 # Connection attempts : 118 # bytes written : 5458 # bytes read : 156561 Error output: WARNING: Runtime 421 longer than time limit (300) TIME SPENT Event Starttime Duration bbtest-net startup 1118290228.818694 - Service definitions loaded 1118290228.819663 0.000969 Tests loaded 1118290228.833090 0.013427 DNS lookups completed 1118290233.835342 5.002252 Test engine setup completed 1118290233.837863 0.002521 TCP tests completed 1118290243.999609 10.161746 PING test completed (148 hosts) 1118290244.000476 0.000867 PING test results sent 1118290648.938097 404.937621 Test result collection completed 1118290648.938682 0.000585 LDAP test engine setup completed 1118290648.938793 0.000111 LDAP tests executed 1118290649.044316 0.105523 LDAP tests result collection completed 1118290649.044325 0.000009 NSLOOKUP tests executed 1118290649.045273 0.000948 Test results transmitted 1118290649.049315 0.004042 bbtest-net completed 1118290649.051183 0.001868 TIME TOTAL 420.232489 ************ # "bbnet" runs the bbtest-net tool to perform the network based tests - i.e. http, smtp, ssh, dns and # all of the various network protocols we need to test. [bbnet] ENVFILE /opt/hobbit/server/etc/hobbitserver.cfg NEEDS hobbitd CMD bbtest-net --report --ping --checkresponse --debug LOGFILE $BBSERVERLOGS/bb-network.log INTERVAL 5m

2 2

[hobbit] Current development plans
by KauffmanT＠nibco.com 09 Jun '05

09 Jun '05

Henrik -- First of all, you deserve the time off, so enjoy! I'd like to see something for the acknowledgement enhancement before you break off on server development and start concentrating on the client. On the client side -- SNMP integration *might* be nice -- but it may not do as much good for us on AIX environments as others. I haven't checked lately, but back under AIX 4.3.3 and earlier the out-of-the box SNMP support provided network statistics. To get disk, cpu, and other info via SNMP required installing additional MIBs and data collectors -- that you could only get by licensing Netview 6000 (IBM's rebrand of HP Openview). FWIW, one of my problems with the BB client is the way most of the 'native' tests are now run under one script. AIX has no support in df to just use local drives -- so a hard nfs mount from a server that is not responding causes df to hang and ALL the local tests go purple in 30 minutes. One of these days I'll try installing the open source df and see if it works. . . Tom

1 0

[hobbit] Current development plans
by craig＠cookitservices.com 09 Jun '05

09 Jun '05

>From what I can see, there is no "one right answer" to this problem... Using cfengine/NFS/AFS are all good methods to deploy updates to remote clients. In these environments you also monitor cfengine/NFS/AFS since they are also critical in your infrastructure. Probably as critical as keeping DNS working. Some sites do not allow NFS to be used though. Some sites do not have AFS/cfengine installed either. Using ssh/scripting does work, but quickly becomes non-trivial to maintain. Building SNMP support into core hobbit would be a good idea, it is also non trivial. Take a look at OpenNMS. From what I have seen it does the discovery and auto config very well. I have never looked at it's code though. It may be possible to rip a large chunk and put into hobbit, since they are both GPL. Nagios does SNMP in a different fashion. It makes you test using the OID string. It seems to work well for nagios users. Instead of writing a hobbit client from scratch, it may be worth looking at modifying the Nagios client. The unix version has been ported to run on windows. It would be "very cool" if hobbit accepted clients from both BB and Nagios... (Can you say "scope creep"?) Nagios clients sit on the local client, but do not do anything until contacted by the server and asked for information. A variation on both BB and bb-central. bb-xsnmp.pl is a good tool. It does have some problems, but it certainly has potential. As for next development effort, my vote is also for a Hobbit client (that in some way sits locally on each client). Craig Cook -- Systems Monitoring Consulting and Support Services http://www.cookitservices.com

1 0

[hobbit] Current development plans
by rdeal＠tigr.org 09 Jun '05

09 Jun '05

I monitor over 340 hosts of various different architectures etc with no problems. bb/bin and bb/tmp are local but bb/etc and bb/ext are nfs mounted on clients. Yes I have a different set for each arch. "are relying on your network + NFS for the monitoring system" BB and Hobbit kind of need the network Besides I monitor the NFS servers as well. IF they go down in any way we are alerted from their tests. Besides if the NFS servers go down the clients will not be able to do very much at all. I use primarily one script for most of my SNMP testing, bb-xsnmp.pl. It is not perfect but does test quite a bit of different hardware, routers, switches, Netapps, APC UPS. As I mentioned I don't have any problem with authentication/encryption as long as it does not make scripts more difficult and is optional. Many sites would have no reason to encrypt BB data as it is generally non critical information and travels over secure networks at most locations. Authentication is more widely useful but should not be set up so that it is overly burdensome to use. Timing is very critical as well, most people do not want to purchase a large machine just to be able to decrypt/authenticate their monitoring messages in time. -----Original Message----- From: Adam Goryachev [mailto:adam at websitemanagers.com.au] Sent: Thursday, June 09, 2005 8:43 AM To: hobbit at hswn.dk Subject: RE: [hobbit] Current development plans On Thu, 2005-06-09 at 08:26 -0400, Deal, Richard wrote: > I would really like to have the option of a client on the unix world. > Having a central machine poll all of the clients constantly a far less > distributed system than running remote clients. I prefer a locally run > client from an NFS mounted area were I can centrally control each with > the config files. This can work if all your machines are the same architecture and library versions etc, but it soon falls apart. Also you are relying on your network + NFS for the monitoring system to actually work, which isn't a good presumtion. For little extra work, you can simply push out the config files/etc via some ssh/scripting.... > I agree, on the snmp situation, BB/Hobbits strengths are in scripts and > ext's snmp should be an ext to Hobbit. Well, why couldn't this be core to hobbit, just as http network tests, or fping etc... You need a fairly complex/flexible config file to be able to specify enough SNMP values to monitor, and which values are good/ok/bad, etc, but I wouldn't see the need for writing multiple scripts for each and every snmp device you want to monitor like we have at the moment (apc UPS/cisco routers/etc...) > My only concern about changing the BB protocols would be to make it > optional and not to make adding EXT scripts more difficult. Actually, thinking about this for a few seconds, it should only require changes to the bb (client exe file) and bbd (server exe file) and all other scripts would stay the same. bbd would have some config variable which says either ignore un-encrypted msgs, accept both, or accept only un-encrypted. Then the client would be configured for either send encrypted/send plain. Finally, the server would 'detect' wether the client is sending encrypted message or plain text, and if encrypted, would decrypt it, and pass it to itself as per normal for processing (or discard it if it is invalid/not permitted). One drawback I see in the BB protocol is that there is no client/server method to see any 'version' of each other, so they can't negotiate these things very well. Just my 0.02c worth Regards, Adam To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

1 0

[hobbit] Current development plans
by pwilliamson＠mandtbank.com 09 Jun '05

09 Jun '05

>>> adam at websitemanagers.com.au 06/09/05 8:43 AM >>> On Thu, 2005-06-09 at 08:26 -0400, Deal, Richard wrote: > > I would really like to have the option of a client on the unix world. > > Having a central machine poll all of the clients constantly a far less > > distributed system than running remote clients. I prefer a locally run > > client from an NFS mounted area were I can centrally control each with > > the config files. > > > This can work if all your machines are the same architecture and library > versions etc, but it soon falls apart. Also you are relying on your > network + NFS for the monitoring system to actually work, which isn't a > good presumtion. > > For little extra work, you can simply push out the config files/etc via > some ssh/scripting.... cfengine works perfectly for me to do this... Paul

1 0

[hobbit] Current development plans
by rdeal＠tigr.org 09 Jun '05

09 Jun '05

I would really like to have the option of a client on the unix world. Having a central machine poll all of the clients constantly a far less distributed system than running remote clients. I prefer a locally run client from an NFS mounted area were I can centrally control each with the config files. I agree, on the snmp situation, BB/Hobbits strengths are in scripts and ext's snmp should be an ext to Hobbit. My only concern about changing the BB protocols would be to make it optional and not to make adding EXT scripts more difficult. -----Original Message----- From: Daniel J McDonald [mailto:dan.mcdonald at austinenergy.com] Sent: Thursday, June 09, 2005 8:18 AM To: Hobbit List Subject: Re: [hobbit] Current development plans On Thu, 2005-06-09 at 15:58 +1000, Adam Goryachev wrote: > On Thu, 2005-06-09 at 07:41 +0200, Henrik Stoerner wrote: > > Personally, I'd most like to see a 'free' client (ie, GPL, without the > BB license issue), Ditto, but I'd really like the bb-central approach. Most of the status information can be grabbed from non-privileged accounts on all unix-like platforms. I concede that a client is necessary in the windows world. > and I'd also like to see *much* better SNMP support. > ie, point it at a router, and it will automatically (or some tool) setup > the various values to monitor (interfaces, byte counter thresholds, cpu, > temperature, etc) or a switch, or firewall, or UPS, or whatever > thingamabob you have lying around. Although I'd love to see a "better mrtg", I'd hate to re-invent the wheel on that one. It would be nice if the mrtg folks would add snmp-v3, but that's not in the offing today. But I have a set of cfgmaker templates that I use to do pretty much what you are asking. It's got a few complex scripts wrapped around it. By policy every device has a unique snmp string, so I have a huge database with hostnames, community strings, and device-types that allows me to generate my mrtg and hobbit configurations on the fly. > Finally, what about some sort of compression/encryption protocol, If we are building an extended protocol, we should support authentication as well. That's been a serious hole in bb for a long time - any hacker who sees that you are trusting bb for monitoring can simply send spoofed status messages to either distract you from the real mischief or hide it from obvious view. -- Daniel J McDonald, CCIE # 2495, CNX Austin Energy dan.mcdonald at austinenergy.com To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

2 1

[hobbit] Current development plans
by nfigaro＠effigie.fr 09 Jun '05

09 Jun '05

Hi, Just some ideas : - the portability is very important (so the future hobbit client should be as simple as possible and should compile on most platforms). I think developping the hobbit client is the most urgent. (so the hobbit project won't depend anymore on bigbrother client and BTF licence) - crypting the infos can be quite useful (but perhaps it can be good to allow crypted and unencrypted datas to be sent). - snmp integration can take a lot of time (I tried to modified mrtg cfgmaker script to integrate brocade switches and decided afterwards to create a new one). - offer the ability to generate pages using another language (I can work on the french translation if you need). Not urgent at all (but french people like softwares that can speak french, I guess danish people like software that can speak danish). - I was very interested with the bb-central extension provided with bigbrother (ability to collect infos through an ssh connection). Not urgent at all two I hope the temp will rise enough so you won't stay at home to write hobbit client this summer :) :) Nicolas Figaro -----Message d'origine----- De : Henrik Stoerner [mailto:henrik at hswn.dk] Envoyé : jeudi 9 juin 2005 07:42 À : hobbit at hswn.dk Objet : [hobbit] Current development plans It seems version 4.0 has reached some stability - there are still a few odd bug reports, but nothing that looks like major problems. So I thought it would be worthwhile to let you know what my plans are. First, I'm currently pretty busy with other stuff so questions raised here are not being answered as quickly as I'd like to. Rest assured that they have not been forgotten, and I will look at all of the reports and questions - but probably not very much for the next week or so. I'm currently working on a 4.0.5 release which will be a performance- improvement release. The current Hobbit code has a flaw in the alert- module, that makes it use much more CPU time than it should - especially if you have a large number of statuses that are in an alert state, but which do not have any recipients defined in hobbit-alerts.cfg. This will be released sometime in June. But summer is upon us, and that means less development activity - so things will slow down as the temperature rises. So don't expect a lot of activity during the summer. When I pick up speed again, I haven't yet decided if I should go for the alert/acknowledge improvements that I've talked about for some time, or if I should tackle the issue of a Hobbit client and get that into a reasonable shape for the more common platforms. I have some ideas for a rather different client architecture than the current one, and the client is also *the* missing piece of the whole Hobbit puzzle - so I could be tempted to get that done. I'd like some feedback on what you find most urgent. Regards, Henrik

1 0