turning off the alert module so hobbit does everything except mail alerts
We will be having a site-wide shutdown in the near future and I would like to find a way to prevent a deluge of emails/pages as we recover the systems.
Usually, we'd just enable/disable the systems/tests that we don't want to hear about, but that puts every test in a blue color and doesn't give us an easy way to see what each system is doing by simply looking at the non-green view. We have to click on each status to see the "true" color of the test (green, red, yellow, or purple), and that won't be very efficient when we are dealing with hundreds of servers.
Is it possible to turn off just the hobbit alert module, so that hobbit will do everything like it normally does EXCEPT mail the alerts? I thought about using a temporary hobbit-alerts.cfg that doesn't contain email addresses or killing the hobbitd_alert process (bad idea I'm sure), but wasn't sure if there was a better way to deal with this. I didn't see anything in the hobbitd_alert manpage that addresses this kind of scenario, so I thought I'd ask.
BTW, we're using hobbit 4.0.4.
Thanks,
Tom
On Tue, Sep 27, 2005 at 03:38:17PM -0400, Tom Georgoulias wrote:
We will be having a site-wide shutdown in the near future and I would like to find a way to prevent a deluge of emails/pages as we recover the systems. [snip] Is it possible to turn off just the hobbit alert module, so that hobbit will do everything like it normally does EXCEPT mail the alerts?
Yep, that should work just fine. Just edit hobbitlaunch.cfg and add DISABLE for the hobbitd_alert module, and it will shut down and stop sending out alerts. When your systems are back up, you can re-enable it and alerts should start flowing normally.
(At work, I'm actually running two Hobbit servers in parallel - one is the main production system, the other is a hot standby located at our backup datacenter. The users do not fancy getting all alerts twice, so the backup system is running with the hobbitd_alert module permanently disabled).
Regards, Henrik
Greetings (again),
I recently put the HPUX client onto 5 11.11 boxes and the clients seem to be running, but the server (running the latest snapshot) is not reporting that it is getting any information from these servers. I have tried looking in the logs, and there doesn't seem to be anything out of place... Any ideas??
thanks, Adam
On Tue, Sep 27, 2005 at 03:16:24PM -0500, Adam Scheblein wrote:
I recently put the HPUX client onto 5 11.11 boxes and the clients seem to be running, but the server (running the latest snapshot) is not reporting that it is getting any information from these servers. I have tried looking in the logs, and there doesn't seem to be anything out of place... Any ideas??
The most common cause of this is that the hostname used by the clients doesn't match what's in your bb-hosts file on the server.
The clients pick up the hostname reported by "uname -n" - and often, that is without the domain part. So your clients are reporting as host "foo", but the Hobbit server expects it to be "foo.bar.com".
Either run the client with "--hostname=foo.bar.com", or put a "CLIENT:foo" tag on the host in the bb-hosts file on the Hobbit server. (Note that the --hostname commandline option goes on the runclient.sh command that starts the client, and it is only available with the client version from the snapshots - not in the 4.1.1 clients).
Regards, Henrik
CLIENT:foo worked -- thanks for the fqdn info :)
Also, did you ever have any ideas on what might be my problem with the rrd graphs (see http://www.hswn.dk/hobbiton/2005/09/msg00228.html)
Thanks again for your help, Adam
Henrik Stoerner wrote:
On Tue, Sep 27, 2005 at 03:16:24PM -0500, Adam Scheblein wrote:
I recently put the HPUX client onto 5 11.11 boxes and the clients seem to be running, but the server (running the latest snapshot) is not reporting that it is getting any information from these servers. I have tried looking in the logs, and there doesn't seem to be anything out of place... Any ideas??
The most common cause of this is that the hostname used by the clients doesn't match what's in your bb-hosts file on the server.
The clients pick up the hostname reported by "uname -n" - and often, that is without the domain part. So your clients are reporting as host "foo", but the Hobbit server expects it to be "foo.bar.com".
Either run the client with "--hostname=foo.bar.com", or put a "CLIENT:foo" tag on the host in the bb-hosts file on the Hobbit server. (Note that the --hostname commandline option goes on the runclient.sh command that starts the client, and it is only available with the client version from the snapshots - not in the 4.1.1 clients).
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Tue, Sep 27, 2005 at 04:01:23PM -0500, Adam Scheblein wrote:
CLIENT:foo worked -- thanks for the fqdn info :)
Also, did you ever have any ideas on what might be my problem with the rrd graphs (see http://www.hswn.dk/hobbiton/2005/09/msg00228.html)
I just answered that one in the original thread.
Regards, Henrik
Henrik Stoerner wrote:
Is it possible to turn off just the hobbit alert module, so that hobbit will do everything like it normally does EXCEPT mail the alerts?
Yep, that should work just fine. Just edit hobbitlaunch.cfg and add DISABLE for the hobbitd_alert module, and it will shut down and stop sending out alerts. When your systems are back up, you can re-enable it and alerts should start flowing normally.
Cool. I see this in my hobbitlaunch.cfg:
[bbpage] ENVFILE /home/bb/server/etc/hobbitserver.cfg NEEDS hobbitd CMD hobbitd_channel --channel=page --log=$BBSERVERLOGS/page.log hobbitd_alert
I would just add "DISABLED" to that section, right? Like this?:
[bbpage] DISABLED ENVFILE /home/bb/server/etc/hobbitserver.cfg NEEDS hobbitd CMD hobbitd_channel --channel=page --log=$BBSERVERLOGS/page.log hobbitd_alert
Tom
On Tue, Sep 27, 2005 at 04:27:15PM -0400, Tom Georgoulias wrote:
Cool. I see this in my hobbitlaunch.cfg:
[bbpage] ENVFILE /home/bb/server/etc/hobbitserver.cfg NEEDS hobbitd CMD hobbitd_channel --channel=page --log=$BBSERVERLOGS/page.log hobbitd_alert
I would just add "DISABLED" to that section, right? Like this?:
[bbpage] DISABLED ENVFILE /home/bb/server/etc/hobbitserver.cfg NEEDS hobbitd CMD hobbitd_channel --channel=page --log=$BBSERVERLOGS/page.log hobbitd_alert
Exactly.
Henrik
participants (3)
-
adam.scheblein＠marquette.edu
-
henrik＠hswn.dk
-
tomg＠mcclatchyinteractive.com