I agree completely with Henrik. Adding 'intelligence' into a script is rather difficult. But I do have a rather 'painful' IIS ASP application. And frequently IIS will die, hang, stop processing ASP. So I wrote a script that runs on each IIS server, that queries the HTTP status from the monitoring server and if the status is red all dllhost & inetinfo processes are killed & the IIS is started. My fear was that this restart script could be stuck in a loop, i.e. possibility that IIS can not be started. So I added the intelligence into my script that logs the date & time of each time the script restarts IIS. It will only restart, if it has not done it more than 3 times in the last hour. Lately I have been thinking of adding in more logic to check if a restart occured in the last 5 minutes (poling period). Becuase I have seen the restart script do it's job and fix IIS, but the monitoring server has not checked yet. Thus the restart script bounces IIS again...
-----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Friday, July 21, 2006 11:07 AM To: hobbit at hswn.dk Subject: Re: [hobbit] bbfix functionality in hobbit
On Fri, Jul 21, 2006 at 10:21:40AM -0400, Gary B. wrote:
Is there any method with hobbit to have the client automatically restart services, like bbfix does for BB? We would like to be able to restart services that tend to fail often (such as SSH tunnels) automatically through Hobbit. Without writing a custom external script, I can't seem to find any information about doing this.
You will need to do some scripting, no doubt about that.
Whether it's a good idea or not ... it depends. From the Hobbit "design" perspective (whoa - that sounds expensive) I have a very firm belief that Hobbit should *monitor* things, not *fix* them. I have seen far too many "intelligent" systems get in the way of real problem-fixing because "intelligent" systems are usually pretty dumb, and cannot handle anything out of the ordinary. When they try, they often fail in spectacular ways.
And having things happen behind your back - because you forgot about that little automatic script someone else setup 2 years ago - is just plain frustrating.
With that little sermon as introduction, here's what you can do. On the host(s) where you want to restart these services, write a script to query the Hobbit server for the status of the service. If it's red, do the restart. You can use the Hobbit "query" command to tell what status the service has. E.g. if you want to reset the SSH tunnels when the "tunnels" status goes red, then this little script run from the Hobbit client's clientlaunch.cfg would do it:
#!/bin/sh
TUNNELSTATUS=
$BB $BBDISP "query $MACHINE.tunnels"|awk '{print $1}'if test "$TUNNELSTATUS" = "red"; then sudo /etc/init.d/sshtunnels stop sleep 5 sudo /etc/init.d/sshtunnels start echo "date: SSH tunnels restarted" fiexit 0
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk