Intelligent alerting across multiple servers?
Hello everyone,
We have 3 Xymon servers communicating together and I have noticed they are clever enough to communicate enable/disables across the board. For example if I was to disable a check on the gui from one xymon server it will communicate this with the other 2 and blue them out automatically.
I was wondering if there is something similar for the alerts? We have 3 xymons for redundancy however as far as I can tell if I set up the alerts config on all 3 then we are going to get 3x identical alerts. Is there any way they can talk to each other and only send out one?
Many thanks James
Date: Thu, 30 Jul 2009 11:15:14 +0100 From: j.sansford at ntlworld.com To: hobbit at hswn.dk Subject: [hobbit] Intelligent alerting across multiple servers?
Hello everyone,
We have 3 Xymon servers communicating together and I have noticed they are clever enough to communicate enable/disables across the board. For example if I was to disable a check on the gui from one xymon server it will communicate this with the other 2 and blue them out automatically.
I have a two servers setting for redunancy.
This is a new feature to me as a hobbit 4.2.0 user, which version are you using ?
for blue(maintenance mode) records cloning from one server to the other, we need to dump the blue records on primary and rsync the blue records to secondary and finally reimport blue records into secondary.
I was wondering if there is something similar for the alerts? We have 3 xymons for redundancy however as far as I can tell if I set up the alerts config on all 3 then we are going to get 3x identical alerts. Is there any way they can talk to each other and only send out one?
To avoid alert duplication from secondary servers with hobbit 4.2.0 , we need to write a perl script to check primary from secondary. if primary has outage of network connection or hobbit server is down then we enable alert module of secondary hobbbit server. at any given time only one hobbit server is allowed to send out alerts.
So are you configuring all your hobbit clients to send message to all three servers ?
This not a concern for a LAN setting but it may become a problem if the deployment is at the scale of thousands of clients and sending message across WAN the has smaller wan pipe.
tj
Many thanks James
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Bing™ brings you maps, menus, and reviews organized in one place. Try it now. http://www.bing.com/search?q=restaurants&form=MLOGEN&publ=WLHMTAG&crea=TXT_M...
Wouldn't it be better on the secondary server(s) to put a conditional statement in the alert rules that if the primary server is online not to alert?
I'm pretty sure the rules can handle that.
On Thu, Jul 30, 2009 at 8:32 AM, T.J. Yang<tj_yang at hotmail.com> wrote:
Date: Thu, 30 Jul 2009 11:15:14 +0100 From: j.sansford at ntlworld.com To: hobbit at hswn.dk Subject: [hobbit] Intelligent alerting across multiple servers?
Hello everyone,
We have 3 Xymon servers communicating together and I have noticed they are clever enough to communicate enable/disables across the board. For example if I was to disable a check on the gui from one xymon server it will communicate this with the other 2 and blue them out automatically.
I have a two servers setting for redunancy. This is a new feature to me as a hobbit 4.2.0 user, which version are you using ?
for blue(maintenance mode) records cloning from one server to the other, we need to dump the blue records on primary and rsync the blue records to secondary and finally reimport blue records into secondary.
I was wondering if there is something similar for the alerts? We have 3 xymons for redundancy however as far as I can tell if I set up the alerts config on all 3 then we are going to get 3x identical alerts. Is there any way they can talk to each other and only send out one?
To avoid alert duplication from secondary servers with hobbit 4.2.0 , we need to write a perl script to check primary from secondary. if primary has outage of network connection or hobbit server is down then we enable alert module of secondary hobbbit server. at any given time only one hobbit server is allowed to send out alerts.
So are you configuring all your hobbit clients to send message to all three servers ? This not a concern for a LAN setting but it may become a problem if the deployment is at the scale of thousands of clients and sending message across WAN the has smaller wan pipe.
tj
Many thanks James
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Bing™ brings you maps, menus, and reviews organized in one place. Try it now.
TJ, we have 3 data centres and each data centre contains a xymon server. All clients in a data centre only report to their local xymon server. However the xymon servers can communicate with each other using BBDISPLAYS (its a little more complicated than that as we utilise a bbproxy in each DC to take the messages and spray them to all 3 xymons).
---- Patrick Nixon <pnixon at gmail.com> wrote:
Wouldn't it be better on the secondary server(s) to put a conditional statement in the alert rules that if the primary server is online not to alert?
I'm pretty sure the rules can handle that.
On Thu, Jul 30, 2009 at 8:32 AM, T.J. Yang<tj_yang at hotmail.com> wrote:
Date: Thu, 30 Jul 2009 11:15:14 +0100 From: j.sansford at ntlworld.com To: hobbit at hswn.dk Subject: [hobbit] Intelligent alerting across multiple servers?
Hello everyone,
We have 3 Xymon servers communicating together and I have noticed they are clever enough to communicate enable/disables across the board. For example if I was to disable a check on the gui from one xymon server it will communicate this with the other 2 and blue them out automatically.
I have a two servers setting for redunancy. This is a new feature to me as a hobbit 4.2.0 user, which version are you using ?
for blue(maintenance mode) records cloning from one server to the other, we need to dump the blue records on primary and rsync the blue records to secondary and finally reimport blue records into secondary.
I was wondering if there is something similar for the alerts? We have 3 xymons for redundancy however as far as I can tell if I set up the alerts config on all 3 then we are going to get 3x identical alerts. Is there any way they can talk to each other and only send out one?
To avoid alert duplication from secondary servers with hobbit 4.2.0 , we need to write a perl script to check primary from secondary. if primary has outage of network connection or hobbit server is down then we enable alert module of secondary hobbbit server. at any given time only one hobbit server is allowed to send out alerts.
So are you configuring all your hobbit clients to send message to all three servers ? This not a concern for a LAN setting but it may become a problem if the deployment is at the scale of thousands of clients and sending message across WAN the has smaller wan pipe.
tj
Many thanks James
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Bing™ brings you maps, menus, and reviews organized in one place. Try it now.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Date: Thu, 30 Jul 2009 14:57:06 +0100 From: j.sansford at ntlworld.com To: hobbit at hswn.dk CC: pnixon at gmail.com Subject: Re: [hobbit] Intelligent alerting across multiple servers?
TJ, we have 3 data centres and each data centre contains a xymon server. All clients in a data centre only report to their local xymon server. However the xymon servers can communicate with each other using BBDISPLAYS (its a little more complicated than that as we utilise a bbproxy in each DC to take the messages and spray them to all 3 xymons).
This is new architecture that I was not aware of thus the inclusion your notes here.
http://en.wikibooks.org/wiki/System_Monitoring_with_Xymon/Administration_Gui...
Please explain more in details on the wiki page if you like to.
Thanks for the information
T.J. Yang
---- Patrick Nixon wrote:
Wouldn't it be better on the secondary server(s) to put a conditional statement in the alert rules that if the primary server is online not to alert?
I'm pretty sure the rules can handle that.
On Thu, Jul 30, 2009 at 8:32 AM, T.J. Yang wrote:
Date: Thu, 30 Jul 2009 11:15:14 +0100 From: j.sansford at ntlworld.com To: hobbit at hswn.dk Subject: [hobbit] Intelligent alerting across multiple servers?
Hello everyone,
We have 3 Xymon servers communicating together and I have noticed they are clever enough to communicate enable/disables across the board. For example if I was to disable a check on the gui from one xymon server it will communicate this with the other 2 and blue them out automatically.
I have a two servers setting for redunancy. This is a new feature to me as a hobbit 4.2.0 user, which version are you using ?
for blue(maintenance mode) records cloning from one server to the other, we need to dump the blue records on primary and rsync the blue records to secondary and finally reimport blue records into secondary.
I was wondering if there is something similar for the alerts? We have 3 xymons for redundancy however as far as I can tell if I set up the alerts config on all 3 then we are going to get 3x identical alerts. Is there any way they can talk to each other and only send out one?
To avoid alert duplication from secondary servers with hobbit 4.2.0 , we need to write a perl script to check primary from secondary. if primary has outage of network connection or hobbit server is down then we enable alert module of secondary hobbbit server. at any given time only one hobbit server is allowed to send out alerts.
So are you configuring all your hobbit clients to send message to all three servers ? This not a concern for a LAN setting but it may become a problem if the deployment is at the scale of thousands of clients and sending message across WAN the has smaller wan pipe.
tj
Many thanks James
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Bing™ brings you maps, menus, and reviews organized in one place. Try it now.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
NEW mobile Hotmail. Optimized for YOUR phone. Click here. http://windowslive.com/Mobile?ocid=TXT_TAGLM_WL_CS_MB_new_hotmail_072009
Date: Thu, 30 Jul 2009 09:02:45 -0400 From: pnixon at gmail.com To: hobbit at hswn.dk Subject: Re: [hobbit] Intelligent alerting across multiple servers?
Wouldn't it be better on the secondary server(s) to put a conditional statement in the alert rules that if the primary server is online not to alert?
I'm pretty sure the rules can handle that.
This is new to me, would you mind to elaborate more on how to do this kind of alert configuration file in hobbit-alerts.cfg ?
tj
On Thu, Jul 30, 2009 at 8:32 AM, T.J. Yang wrote:
Date: Thu, 30 Jul 2009 11:15:14 +0100 From: j.sansford at ntlworld.com To: hobbit at hswn.dk Subject: [hobbit] Intelligent alerting across multiple servers?
Hello everyone,
We have 3 Xymon servers communicating together and I have noticed they are clever enough to communicate enable/disables across the board. For example if I was to disable a check on the gui from one xymon server it will communicate this with the other 2 and blue them out automatically.
I have a two servers setting for redunancy. This is a new feature to me as a hobbit 4.2.0 user, which version are you using ?
for blue(maintenance mode) records cloning from one server to the other, we need to dump the blue records on primary and rsync the blue records to secondary and finally reimport blue records into secondary.
I was wondering if there is something similar for the alerts? We have 3 xymons for redundancy however as far as I can tell if I set up the alerts config on all 3 then we are going to get 3x identical alerts. Is there any way they can talk to each other and only send out one?
To avoid alert duplication from secondary servers with hobbit 4.2.0 , we need to write a perl script to check primary from secondary. if primary has outage of network connection or hobbit server is down then we enable alert module of secondary hobbbit server. at any given time only one hobbit server is allowed to send out alerts.
So are you configuring all your hobbit clients to send message to all three servers ? This not a concern for a LAN setting but it may become a problem if the deployment is at the scale of thousands of clients and sending message across WAN the has smaller wan pipe.
tj
Many thanks James
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Bing™ brings you maps, menus, and reviews organized in one place. Try it now.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
NEW mobile Hotmail. Optimized for YOUR phone. Click here. http://windowslive.com/Mobile?ocid=TXT_TAGLM_WL_CS_MB_new_hotmail_072009
Oh wow, I must have been in space when I thought about that.
In the standard hobbit-alerts, you are not able to do what I just mentioned.
However, using an external paging script, you can use the 'bb query HOSTNAME.TESTNAME' command, get the status from secondary server2 for primary server1 and send the alert to it's destination if server1 is down. Otherwise discard the alert.
Sorry for the bad lead.
On Thu, Jul 30, 2009 at 12:48 PM, T.J. Yang<tj_yang at hotmail.com> wrote:
Date: Thu, 30 Jul 2009 09:02:45 -0400 From: pnixon at gmail.com To: hobbit at hswn.dk Subject: Re: [hobbit] Intelligent alerting across multiple servers?
Wouldn't it be better on the secondary server(s) to put a conditional statement in the alert rules that if the primary server is online not to alert?
I'm pretty sure the rules can handle that.
This is new to me, would you mind to elaborate more on how to do this kind of alert configuration file in hobbit-alerts.cfg ?
tj
On Thu, Jul 30, 2009 at 8:32 AM, T.J. Yang wrote:
Date: Thu, 30 Jul 2009 11:15:14 +0100 From: j.sansford at ntlworld.com To: hobbit at hswn.dk Subject: [hobbit] Intelligent alerting across multiple servers?
Hello everyone,
We have 3 Xymon servers communicating together and I have noticed they are clever enough to communicate enable/disables across the board. For example if I was to disable a check on the gui from one xymon server it will communicate this with the other 2 and blue them out automatically.
I have a two servers setting for redunancy. This is a new feature to me as a hobbit 4.2.0 user, which version are you using ?
for blue(maintenance mode) records cloning from one server to the other, we need to dump the blue records on primary and rsync the blue records to secondary and finally reimport blue records into secondary.
I was wondering if there is something similar for the alerts? We have 3 xymons for redundancy however as far as I can tell if I set up the alerts config on all 3 then we are going to get 3x identical alerts. Is there any way they can talk to each other and only send out one?
To avoid alert duplication from secondary servers with hobbit 4.2.0 , we need to write a perl script to check primary from secondary. if primary has outage of network connection or hobbit server is down then we enable alert module of secondary hobbbit server. at any given time only one hobbit server is allowed to send out alerts.
So are you configuring all your hobbit clients to send message to all three servers ? This not a concern for a LAN setting but it may become a problem if the deployment is at the scale of thousands of clients and sending message across WAN the has smaller wan pipe.
tj
Many thanks James
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Bing™ brings you maps, menus, and reviews organized in one place. Try it now.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
NEW mobile Hotmail. Optimized for YOUR phone. Click here. http://windowslive.com/Mobile?ocid=TXT_TAGLM_WL_CS_MB_new_hotmail_072009 To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Date: Thu, 30 Jul 2009 21:14:19 -0400 From: pnixon at gmail.com To: hobbit at hswn.dk Subject: Re: [hobbit] Intelligent alerting across multiple servers?
Oh wow, I must have been in space when I thought about that.
In the standard hobbit-alerts, you are not able to do what I just mentioned.
However, using an external paging script, you can use the 'bb query HOSTNAME.TESTNAME' command, get the status from secondary server2 for primary server1 and send the alert to it's destination if server1 is down. Otherwise discard the alert.
This is a very creative approach !!
Hopefully, Henrik can implement this idea in C source code layer.
Before that happened customized external script in perl or shell script can do the work too.
Sorry for the bad lead.
np problem, thanks for the pointer.
tj
On Thu, Jul 30, 2009 at 12:48 PM, T.J. Yang<tj_yang at hotmail.com> wrote:
Date: Thu, 30 Jul 2009 09:02:45 -0400 From: pnixon at gmail.com To: hobbit at hswn.dk Subject: Re: [hobbit] Intelligent alerting across multiple servers?
Wouldn't it be better on the secondary server(s) to put a conditional statement in the alert rules that if the primary server is online not to alert?
I'm pretty sure the rules can handle that.
This is new to me, would you mind to elaborate more on how to do this kind of alert configuration file in hobbit-alerts.cfg ?
tj
On Thu, Jul 30, 2009 at 8:32 AM, T.J. Yang wrote:
Date: Thu, 30 Jul 2009 11:15:14 +0100 From: j.sansford at ntlworld.com To: hobbit at hswn.dk Subject: [hobbit] Intelligent alerting across multiple servers?
Hello everyone,
We have 3 Xymon servers communicating together and I have noticed they are clever enough to communicate enable/disables across the board. For example if I was to disable a check on the gui from one xymon server it will communicate this with the other 2 and blue them out automatically.
I have a two servers setting for redunancy. This is a new feature to me as a hobbit 4.2.0 user, which version are you using ?
for blue(maintenance mode) records cloning from one server to the other, we need to dump the blue records on primary and rsync the blue records to secondary and finally reimport blue records into secondary.
I was wondering if there is something similar for the alerts? We have 3 xymons for redundancy however as far as I can tell if I set up the alerts config on all 3 then we are going to get 3x identical alerts. Is there any way they can talk to each other and only send out one?
To avoid alert duplication from secondary servers with hobbit 4.2.0 , we need to write a perl script to check primary from secondary. if primary has outage of network connection or hobbit server is down then we enable alert module of secondary hobbbit server. at any given time only one hobbit server is allowed to send out alerts.
So are you configuring all your hobbit clients to send message to all three servers ? This not a concern for a LAN setting but it may become a problem if the deployment is at the scale of thousands of clients and sending message across WAN the has smaller wan pipe.
tj
Many thanks James
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Bing™ brings you maps, menus, and reviews organized in one place. Try it now.
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
NEW mobile Hotmail. Optimized for YOUR phone. Click here. http://windowslive.com/Mobile?ocid=TXT_TAGLM_WL_CS_MB_new_hotmail_072009 To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Windows Live™ Hotmail®: Celebrate the moment with your favorite sports pics. Check it out. http://www.windowslive.com/Online/Hotmail/Campaign/QuickAdd?ocid=TXT_TAGLM_W...
participants (3)
-
j.sansford＠ntlworld.com
-
pnixon＠gmail.com
-
tj_yang＠hotmail.com