Current development plans
It seems version 4.0 has reached some stability - there are still a few odd bug reports, but nothing that looks like major problems. So I thought it would be worthwhile to let you know what my plans are.
First, I'm currently pretty busy with other stuff so questions raised here are not being answered as quickly as I'd like to. Rest assured that they have not been forgotten, and I will look at all of the reports and questions - but probably not very much for the next week or so.
I'm currently working on a 4.0.5 release which will be a performance- improvement release. The current Hobbit code has a flaw in the alert- module, that makes it use much more CPU time than it should - especially if you have a large number of statuses that are in an alert state, but which do not have any recipients defined in hobbit-alerts.cfg. This will be released sometime in June.
But summer is upon us, and that means less development activity - so things will slow down as the temperature rises. So don't expect a lot of activity during the summer.
When I pick up speed again, I haven't yet decided if I should go for the alert/acknowledge improvements that I've talked about for some time, or if I should tackle the issue of a Hobbit client and get that into a reasonable shape for the more common platforms. I have some ideas for a rather different client architecture than the current one, and the client is also *the* missing piece of the whole Hobbit puzzle - so I could be tempted to get that done. I'd like some feedback on what you find most urgent.
Regards, Henrik
On Thu, 2005-06-09 at 07:41 +0200, Henrik Stoerner wrote:
When I pick up speed again, I haven't yet decided if I should go for the alert/acknowledge improvements that I've talked about for some time, or if I should tackle the issue of a Hobbit client and get that into a reasonable shape for the more common platforms. I have some ideas for a rather different client architecture than the current one, and the client is also *the* missing piece of the whole Hobbit puzzle - so I could be tempted to get that done. I'd like some feedback on what you find most urgent.
Personally, I'd most like to see a 'free' client (ie, GPL, without the BB license issue), and I'd also like to see *much* better SNMP support. ie, point it at a router, and it will automatically (or some tool) setup the various values to monitor (interfaces, byte counter thresholds, cpu, temperature, etc) or a switch, or firewall, or UPS, or whatever thingamabob you have lying around.
Beyond that, I would then go for more of the admin tools, like the bb-hosts editor recently being developed/released, and some other auto-discovery type tools.... This makes things easier for initial deployment/users who don't understand unix so much... But overall, if you make the right API, then other people can step in and make their own GUI tools...
Finally, what about some sort of compression/encryption protocol, so that it is possible to do more frequent test/report without using so much bandwidth? I know very little about these things, but the overhead does concern me somewhat, especially if you wanted to test every 30 seconds or something...
Regards, Adam
On Thu, 2005-06-09 at 15:58 +1000, Adam Goryachev wrote:
On Thu, 2005-06-09 at 07:41 +0200, Henrik Stoerner wrote:
Personally, I'd most like to see a 'free' client (ie, GPL, without the BB license issue),
Ditto, but I'd really like the bb-central approach. Most of the status information can be grabbed from non-privileged accounts on all unix-like platforms. I concede that a client is necessary in the windows world.
and I'd also like to see *much* better SNMP support. ie, point it at a router, and it will automatically (or some tool) setup the various values to monitor (interfaces, byte counter thresholds, cpu, temperature, etc) or a switch, or firewall, or UPS, or whatever thingamabob you have lying around.
Although I'd love to see a "better mrtg", I'd hate to re-invent the wheel on that one. It would be nice if the mrtg folks would add snmp-v3, but that's not in the offing today.
But I have a set of cfgmaker templates that I use to do pretty much what you are asking. It's got a few complex scripts wrapped around it. By policy every device has a unique snmp string, so I have a huge database with hostnames, community strings, and device-types that allows me to generate my mrtg and hobbit configurations on the fly.
Finally, what about some sort of compression/encryption protocol,
If we are building an extended protocol, we should support authentication as well. That's been a serious hole in bb for a long time - any hacker who sees that you are trusting bb for monitoring can simply send spoofed status messages to either distract you from the real mischief or hide it from obvious view.
-- Daniel J McDonald, CCIE # 2495, CNX Austin Energy
dan.mcdonald at austinenergy.com
On Thu, 2005-06-09 at 07:18 -0500, Daniel J McDonald wrote:
On Thu, 2005-06-09 at 15:58 +1000, Adam Goryachev wrote:
On Thu, 2005-06-09 at 07:41 +0200, Henrik Stoerner wrote:
Personally, I'd most like to see a 'free' client (ie, GPL, without the BB license issue),
Ditto, but I'd really like the bb-central approach. Most of the status information can be grabbed from non-privileged accounts on all unix-like platforms. I concede that a client is necessary in the windows world.
While I can see that some people might like this approach, I would think that the overhead of this method is significantly higher than the bbclient method... Also, you probably don't want to be 'giving' a user level account to people (ie, if they manage to hack your BB central box, then they get free access to your entire network.... As opposed to being able to screw-over your monitoring setup but not really affect much else...
and I'd also like to see *much* better SNMP support. ie, point it at a router, and it will automatically (or some tool) setup the various values to monitor (interfaces, byte counter thresholds, cpu, temperature, etc) or a switch, or firewall, or UPS, or whatever thingamabob you have lying around.
Although I'd love to see a "better mrtg", I'd hate to re-invent the wheel on that one. It would be nice if the mrtg folks would add snmp-v3, but that's not in the offing today.
Well, I don't really know what is needed, but since hobbit includes rrd, really all you need is to integrate some snmp library which can handle retreiving the snmp values for you. The hobbit can do it's normal alerting/trending the same as it does for everything else. Finally, once the various config file formats for this are done, then someone can create a nice search/discover/configure tool to create the config files.
Finally, what about some sort of compression/encryption protocol,
If we are building an extended protocol, we should support authentication as well. That's been a serious hole in bb for a long time - any hacker who sees that you are trusting bb for monitoring can simply send spoofed status messages to either distract you from the real mischief or hide it from obvious view.
Yes, authentication should be included as well, but perhaps it should be server-side as opposed to client-side.
eg, xyz IP address can send reports for abc hostname + a and b status, etc...
This simple option would solve most of the security issues, once they hack the machine, all bets are off anyway (ie, they can see/find the username/password you have configured, use the standard bb tools to send the status/etc...)
Regards, Adam
--
Adam Goryachev Website Managers Ph: +61 2 9345 4395 adam at websitemanagers.com.au Fax: +61 2 9345 4396 www.websitemanagers.com.au
Henrik , enjoy the summer !
Friends,
Who was the man that developed the first client for windows?
Has anyone have a start point source code , so we can develop our version, or personally someone have a start code and we can together work on this client.
Personally, I would like to contribute and particpate in the development.
God bless you all.
Best Regards,
Mario.
Henrik,
This might not be quite the right thread for this, but anyway...
There appears to be a bug in bbproxy which should probably be fixed sometime. When a message is to be sent to several servers, and the transfer to one sender fails, then the transfer to the other servers isn't attempted. (The state machine jumps straight to the exit state rather than looping back to send the message to the next server.) I first noticed this in an older version of bbproxy, but a quick glance at the code suggests it might still be a problem.
Also in bbproxy, I'd like to see an option where the proxy reports can be sent to a different list of servers from where ordinary status reports are sent. In my case for example, I am using bbproxy mainly for load balancing on the server where bbd (not yet hobbitd) runs, so it only send messages to the local server. However I would like the reports sent to both our servers. I think I sent you some code back in January which attempted to do this, but it probably got lost in the ether somewhere. I could resurrect it if you like.
Regarding the hobbit client, one of my main complaints against various other monitoring/management tools as that they run large complex processes as root with links deep into the OS. I wouldn't like to see the hobbit client go down that path.
Kevin.
Since the bb-client is running under this BTF-license, I would very much want a Hobbitclient.
I wish you all (that live where the summer is coming) a very nice summer and hope that the weather will be like it is just now here. Clear blue sky and not to hot.
Regards Lars
----- Original Message ----- From: "Henrik Stoerner" <henrik at hswn.dk> To: <hobbit at hswn.dk> Sent: Thursday, June 09, 2005 7:41 AM Subject: [hobbit] Current development plans
It seems version 4.0 has reached some stability - there are still a few odd bug reports, but nothing that looks like major problems. So I thought it would be worthwhile to let you know what my plans are.
First, I'm currently pretty busy with other stuff so questions raised here are not being answered as quickly as I'd like to. Rest assured that they have not been forgotten, and I will look at all of the reports and questions - but probably not very much for the next week or so.
I'm currently working on a 4.0.5 release which will be a performance- improvement release. The current Hobbit code has a flaw in the alert- module, that makes it use much more CPU time than it should - especially if you have a large number of statuses that are in an alert state, but which do not have any recipients defined in hobbit-alerts.cfg. This will be released sometime in June.
But summer is upon us, and that means less development activity - so things will slow down as the temperature rises. So don't expect a lot of activity during the summer.
When I pick up speed again, I haven't yet decided if I should go for the alert/acknowledge improvements that I've talked about for some time, or if I should tackle the issue of a Hobbit client and get that into a reasonable shape for the more common platforms. I have some ideas for a rather different client architecture than the current one, and the client is also *the* missing piece of the whole Hobbit puzzle - so I could be tempted to get that done. I'd like some feedback on what you find most urgent.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Henrik Stoerner schrieb:
When I pick up speed again, I haven't yet decided if I should go for the alert/acknowledge improvements that I've talked about for some time, or if I should tackle the issue of a Hobbit client and get that into a reasonable shape for the more common platforms. I have some ideas for a rather different client architecture than the current one, and the client is also *the* missing piece of the whole Hobbit puzzle - so I could be tempted to get that done. I'd like some feedback on what you find most urgent.
Performance is not an issue, here. So I really would like to see a real hobbit client, first.
Best wishes, Dirk
As to improvements to there server, well Do they ever end ! <smile> As to the hobbit client, yes that is the missing piece! However it may be very tough to do! First NO external libraries such as PCRE, RRDtool, libpng, OpenSSL, OpenLDAP , perl if used should be perl 4! Why , portability! Ive bbclient for hpux 9.0 ,10.x 11.0 and Solaris 2.5.1. Its one thing to find a relatively modern box to run the hobbit server on but I must support all sorts of client and may not be able to install external libraries. (if I remember right bb 19c actually compiles with the c compiler used for kernel gens on hpux!)
I am thinking your aiming to do ALL it in C rather than C plus shell scripts like bb?
Henrik Stoerner wrote:
It seems version 4.0 has reached some stability - there are still a few odd bug reports, but nothing that looks like major problems. So I thought it would be worthwhile to let you know what my plans are.
First, I'm currently pretty busy with other stuff so questions raised here are not being answered as quickly as I'd like to. Rest assured that they have not been forgotten, and I will look at all of the reports and questions - but probably not very much for the next week or so.
I'm currently working on a 4.0.5 release which will be a performance- improvement release. The current Hobbit code has a flaw in the alert- module, that makes it use much more CPU time than it should - especially if you have a large number of statuses that are in an alert state, but which do not have any recipients defined in hobbit-alerts.cfg. This will be released sometime in June.
But summer is upon us, and that means less development activity - so things will slow down as the temperature rises. So don't expect a lot of activity during the summer.
When I pick up speed again, I haven't yet decided if I should go for the alert/acknowledge improvements that I've talked about for some time, or if I should tackle the issue of a Hobbit client and get that into a reasonable shape for the more common platforms. I have some ideas for a rather different client architecture than the current one, and the client is also *the* missing piece of the whole Hobbit puzzle - so I could be tempted to get that done. I'd like some feedback on what you find most urgent.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
--
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
| _p_ Mike Nemeth
| ___| |_____ email(w) michael.nemeth at lmco.com Work: 856 359-1425
|><___________) | Home Page:http://www.geocities.com/mjnemeth/
| Work Page:http://faraday.motown.lmco.com:3000/~nemethm/
| Work Page:http://ortsweb/~mnemeth/
|++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Henrik,
Whatever break you take over the summer is certainly well deserved. You've provided the world with a wonderful tool and this reader is very appreciative of all of your hard work.
I must concur that a Hobbit client should take precedence over other development activities (except perhaps bug fixes).
I tend to like the SNMP idea myself. Building a client around that arthitecture could solve a large number of problems, perhaps the biggest is support of a diverse number of operating systems.
Henrik Stoerner wrote:
It seems version 4.0 has reached some stability - there are still a few odd bug reports, but nothing that looks like major problems. So I thought it would be worthwhile to let you know what my plans are.
First, I'm currently pretty busy with other stuff so questions raised here are not being answered as quickly as I'd like to. Rest assured that they have not been forgotten, and I will look at all of the reports and questions - but probably not very much for the next week or so.
I'm currently working on a 4.0.5 release which will be a performance- improvement release. The current Hobbit code has a flaw in the alert- module, that makes it use much more CPU time than it should - especially if you have a large number of statuses that are in an alert state, but which do not have any recipients defined in hobbit-alerts.cfg. This will be released sometime in June.
But summer is upon us, and that means less development activity - so things will slow down as the temperature rises. So don't expect a lot of activity during the summer.
When I pick up speed again, I haven't yet decided if I should go for the alert/acknowledge improvements that I've talked about for some time, or if I should tackle the issue of a Hobbit client and get that into a reasonable shape for the more common platforms. I have some ideas for a rather different client architecture than the current one, and the client is also *the* missing piece of the whole Hobbit puzzle - so I could be tempted to get that done. I'd like some feedback on what you find most urgent.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Rich Smrcina VM Assist, Inc. Main: (262)392-2026 Cell: (414)491-6001 Ans Service: (360)715-2467 rich.smrcina at vmassist.com
Catch the WAVV! http://www.wavv.org WAVV 2006 - Chattanooga, TN - April 7-11, 2006
No in my experience using SNMP CAUSES a large number or problems:
slow! bugs(memory leaks) and security.
And IM not sure how portable it really is !
Rich Smrcina wrote:
Henrik,
Whatever break you take over the summer is certainly well deserved. You've provided the world with a wonderful tool and this reader is very appreciative of all of your hard work.
I must concur that a Hobbit client should take precedence over other development activities (except perhaps bug fixes).
I tend to like the SNMP idea myself. Building a client around that arthitecture could solve a large number of problems, perhaps the biggest is support of a diverse number of operating systems.
Henrik Stoerner wrote:
It seems version 4.0 has reached some stability - there are still a few odd bug reports, but nothing that looks like major problems. So I thought it would be worthwhile to let you know what my plans are.
First, I'm currently pretty busy with other stuff so questions raised here are not being answered as quickly as I'd like to. Rest assured that they have not been forgotten, and I will look at all of the reports and questions - but probably not very much for the next week or so.
I'm currently working on a 4.0.5 release which will be a performance- improvement release. The current Hobbit code has a flaw in the alert- module, that makes it use much more CPU time than it should - especially if you have a large number of statuses that are in an alert state, but which do not have any recipients defined in hobbit-alerts.cfg. This will be released sometime in June.
But summer is upon us, and that means less development activity - so things will slow down as the temperature rises. So don't expect a lot of activity during the summer.
When I pick up speed again, I haven't yet decided if I should go for the alert/acknowledge improvements that I've talked about for some time, or if I should tackle the issue of a Hobbit client and get that into a reasonable shape for the more common platforms. I have some ideas for a rather different client architecture than the current one, and the client is also *the* missing piece of the whole Hobbit puzzle - so I could be tempted to get that done. I'd like some feedback on what you find most urgent.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
--
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
| _p_ Mike Nemeth
| ___| |_____ email(w) michael.nemeth at lmco.com Work: 856 359-1425
|><___________) | Home Page:http://www.geocities.com/mjnemeth/
| Work Page:http://faraday.motown.lmco.com:3000/~nemethm/
| Work Page:http://ortsweb/~mnemeth/
|++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
On Thu, Jun 09, 2005 at 07:41:42AM +0200, Henrik Stoerner wrote:
It seems version 4.0 has reached some stability - there are still a few odd bug reports, but nothing that looks like major problems. So I thought it would be worthwhile to let you know what my plans are.
Well, lots of responses so I'll try to pick out some trends and respond to them in one place.
The Hobbit client
Adam Goryachev :
Personally, I'd most like to see a 'free' client (ie, GPL, without the BB license issue)
Right now the vote seems to be in favour of working on the client. It certainly will be GPL.
Craig Cook :
Instead of writing a hobbit client from scratch, it may be worth looking at modifying the Nagios client.
That might be an idea, yes. I haven't spent much time with Nagios, so I am not really familiar with their architecture. I had the impression it was more SNMP focused than BB/Hobbit.
SNMP - or "How to collect data"
Adam Goryachev :
I'd also like to see *much* better SNMP support.
Craig Cook :
Building SNMP support into core hobbit would be a good idea, it is also non trivial.
Daniel J McDonald :
I'd really like the bb-central approach. Most of the status information can be grabbed from non-privileged accounts on all unix-like platforms. I concede that a client is necessary in the windows world.
There's no doubt that some sort of support in Hobbit for collecting data via SNMP would be very useful. However, I believe that's better implemented as a stand-alone tool, somewhat like the network tester. It would obviously rely on some library like Net-SNMP 5 for the dirty stuff of talking SNMP (meaning it would support SNMP v1, v2c and v3 automatically - although I have at one point implemented SNMP daemons and MIB instrumentation from scratch, I'd rather not repeat that excercise :-))
However, I don't want to base a Hobbit client on SNMP or any other central polling-style method of data collection. There are at least two reasons for that:
It doesn't scale well. My main setup has over 2000 boxes to monitor. Doing that from one central server would mean polling 7 systems per second - that just won't work. There are always some servers that are down causing timeouts... whether it happens via SNMP, ssh or some other protocol really doesn't matter. It probably works fine for a setup with 50 or even 100 systems, but not for me.
The central server needs to know about all kinds of systems If my central polling server runs an "ssh hobbit at clientIP uptime;df;ps" - then the central server must know how to interpret the output. That's one of the major design problems in Hobbit currently - everytime Redhat^Wsomeone comes up with a new layout for the "vmstat" output, Hobbit needs to be modified to recognize these data.
So my idea currently is to design a new type of client. It won't generate "status" messages, it will just collect data. Imagine a client that just sends Hobbit a "client data" package, like
os: Linux osver: 2.6.11 osid: Debian/Sarge i386 uptime: 173201 seconds loadaverage: 0.4 filesystem /: 26102 MB, 71% used inodes /: 10291029 total, 21% used filesystem /var: 102400 MB, 50% used inodes /var: 40182910 total, 7% used
This is one well-defined format that Hobbit needs to recognize, and based on these data it can match e.g. filesystem utilisation against a configuration file on the Hobbit server and generate the necessary status messages - so the end result will look exactly like what you have today, but with much less complexity in how e.g. the RRD handlers need to know about the types of systems that report into Hobbit.
The only drawback is that the client becomes slightly more advanced but not much; it's really just formatting the information differently before sending it off to the Hobbit server.
Another very nice thing about this is that you can easily (well, relatively) write Hobbit modules that handle new kinds of information.
And it can be done without breaking compatibility with the existing clients, so you can run a mix of BB and Hobbit clients without any problems.
Encryption/authentication and compression of status messages
Adam Goryachev :
Finally, what about some sort of compression/encryption protocol, so that it is possible to do more frequent test/report without using so much bandwidth?
Daniel J McDonald :
If we are building an extended protocol, we should support authentication as well.
There's already some IP-based access controls built into Hobbit; see the hobbitd man-page for the --status-senders, --maint-senders, --www-senders, --admin-senders options. The first one should be sufficient to block most attempts at sending fake status messages - an attacker would need to break into your network test server and send the fake messages from there.
However, authentication could be nice. I am tempted to handle both of these problems with one solution - and just implement an SSL-encrypted protocol where you can then use client-side certificates for authentication. That will be significant overhead on the processing side, but the good thing is that you can offload SSL to hardware devices fairly easy (and OpenSSL does support that kind of hardware).
Compression ... is it necessary ? All of the status messages in my setup combined are about 6 MB for 2000 servers - ie. 3 KB/server which gets updated every 300 secs (on average). So that's 10 bytes/second per server. So a rough bandwidth estimate for Hobbit would be 100 bps per server monitored. For a LAN, that's peanuts.
Well, thanks for the feedback - it's really good to learn what ideas and problems are the important ones.
Regards, Henrik
The primary reason I switch to hobbit over bigbrother and nagios is performance. I am monitoring 300 plus server's with an average of 7 services per server with a total of more than 2000 host.service.
I had been using bigbrother for almost 4 years and nagios for a year or so. I am never going back. I also do not like to see bbcentral since same reason that Henrik said, way too much overload for my server. Plus ssh communication w/ rsa key depends on rsa key and dns. So if the hosts rsa key changed and/or dns for the hosts local domain stop working ssh will timeout. I definitely like all systems working together and push/pull method and instead of just pull using bbcentral.
I guess the ack part of hobbitd can wait while a hobbit client grows. I like to test a hobbit client when ready.
Henrik, please enjoy summer while it lasts :-). Also thanks again for a monitoring tool that really performs well.
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu "..there are two kinds of people: those who work and those who take the credit...try to be in the first group;...less competition there." - Indira Gandhi
participants (10)
-
dan.mcdonald@austinenergy.com
-
Dirk.Kastens@uni-osnabrueck.de
-
henrik@hswn.dk
-
iqbala-hobbit@qwestip.net
-
kevin.pye@gmail.com
-
lars.ebeling@leopg9.no-ip.org
-
mailinglists@websitemanagers.com.au
-
michael.nemeth@lmco.com
-
rower.master@gmail.com
-
rsmrcina@wi.rr.com