On 11/7/07, Paul Ehrenreich <paulehr at gmail.com> wrote:
Hi everyone,
Long time listener, first time caller (or maybe second time ;) ). I seem to be having a weird issue with hobbit not sending out email alerts. I have the following rules set up (emails changed to protect the innocent). I did have macros setup for this and I thought it was part of the problem, but it was not.
PAGE=oracle MAIL email1 at test.com SERVICE=conn,oracle,disk COLOR=red REPEAT=15m RECOVERED MAIL email2 at test.com SERVICE=conn,oracle,disk COLOR=red REPEAT=15m RECOVERED
Looks okay. This won't fix the problem, but you can shorten the individual MAIL lines by removing "RECOVERED" and adding it to the end of "PAGE=oracle". Same with "SERVICE" and "COLOR"
"REPEAT=15m" will have to be on each MAIL line.
You can also do "MAIL email1 at test.com,email2 at test.com".
So in the end, you get: PAGE=oracle SERVICE=conn,oracle,disk RECOVERED MAIL email1 at test.com,email2 at test.com REPEAT=15m
I sent out a test alert using bbcmd hobbitd_alerts --test atgtier2app1
oracle which is a host on my oracle page and it shows that the rules match and that it sends on the emails
2007-11-07 15:42:52 Using default environment file /usr/lib/hobbit/server/etc/hobbitserver.cfg 00030883 2007-11-07 15:42:52 send_alert atgtier2app1:oracle state Paging 00030883 2007-11-07 15:42:52 Matching host:service:page 'atgtier2app1:oracle:oracle' against rule line 131 00030883 2007-11-07 15:42:52 *** Match with 'PAGE=oracle' *** 00030883 2007-11-07 15:42:52 Matching host:service:page 'atgtier2app1:oracle:oracle' against rule line 132 00030883 2007-11-07 15:42:52 *** Match with 'MAIL email1 at test.com SERVICE=conn,oracle,disk COLOR=red REPEAT=15m RECOVERED' *** 00030883 2007-11-07 15:42:52 Mail alert with command 'mailx -s "Hobbit [12345] atgtier2app1:oracle CRITICAL (RED)" email1 at test.com' 00030883 2007-11-07 15:42:52 Matching host:service:page 'atgtier2app1:oracle:oracle' against rule line 133 00030883 2007-11-07 15:42:52 *** Match with 'MAIL email2 at test.comSERVICE=conn,oracle,disk COLOR=red REPEAT=15m RECOVERED' *** 00030883 2007-11-07 15:42:52 Mail alert with command 'mailx -s "Hobbit [12345] atgtier2app1:oracle CRITICAL (RED)" email2 at test.com'
I verified that my server can send mail out by sending a test message using mailx to myself and I did receive it. I even took my troubleshooting a step further and ran tcpdump on the network interface looking for anything going to the remote mail server and saw SMTP traffic when i sent my test message, but not my hobbitd_alert --test message. At this point I am scratching my head. Is there is anything in hobbit to debug why messages are not being sent out? I tried looking at some of the hobbit and postfix logs, but nothing really indicated any problems.
"hobbitd_alert --test" won't actually execute the alert rules. It will just run through all of the alert rules and show you what matches and doesn't.
Check the notifications.log file in your Hobbit log directory. If Hobbit is doing what it should, you will see messages that say emails were sent out. If you see this and you aren't getting the emails, then it wouldn't be a problem with Hobbit that is causing it, and the problem is probably somewhere else.
The hobbit server is running Linux (ubuntu 7.04 server) and hobbit 4.2.0
Thanks for the help!
-Paul