Where to find the latest hostdata (or: How to monitor one process on two servers)?
Hi all,
I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this.
I can't have something that runs on the client since I won't know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple!
This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.
I know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it?
Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server?
|\/|
[Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing abuse at colt.net and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900
You can get this using the xymon command e.g...
xymon $XYMSERVERS "clientlog hostname section=ps"
That will dump the [ps] section from the current client data, you can then grep through that to look for the required process, repeat for the second host and then hopefully you should know which one it's running on.
Steve
On 12 January 2012 16:05, Ward, Martin <Martin.Ward at colt.net> wrote:
Hi all,****
I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this.****
I can’t have something that runs on the client since I won’t know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple!****
This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.*
I know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it?****
Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server?****
|\/|**** [Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing abuse at colt.net and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
My thanks to Steven, Bruce and Iqbal for your suggestions, with your help I finally managed to get a nice, simple script that runs on the server, checks the process list from both clients and creates a Xymon message for a dummy host that sets the status of the process.
In this instance it was simpler and cleaner to have a single script running on the server than to have client-side scripts plus combo.cfg entries, ymmv.
|\/|artin
From: Steven Carr [mailto:sjcarr at gmail.com] Sent: 12 January 2012 17:09 To: Ward, Martin Cc: xymon at xymon.com Subject: Re: [Xymon] Where to find the latest hostdata (or: How to monitor one process on two servers)?
You can get this using the xymon command e.g...
xymon $XYMSERVERS "clientlog hostname section=ps"
That will dump the [ps] section from the current client data, you can then grep through that to look for the required process, repeat for the second host and then hopefully you should know which one it's running on.
Steve
On 12 January 2012 16:05, Ward, Martin <Martin.Ward at colt.net<mailto:Martin.Ward at colt.net>> wrote: Hi all,
I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this.
I can’t have something that runs on the client since I won’t know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple!
This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.
I know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it?
Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server?
|\/| [Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing abuse at colt.net<mailto:abuse at colt.net> and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net<http://www.colt.net> or contact us on +44(0)20 7390 3900<tel:%2B44%280%2920%207390%203900>
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon
On Thu, Jan 12, 2012 at 11:05 AM, Ward, Martin <Martin.Ward at colt.net> wrote:
Hi all,
I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this.
I can’t have something that runs on the client since I won’t know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple!
This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.
xymon localhost "clientlog client.example.com section=ps" | grep pr[o]c
will let you know if the process is running on client.example.com
create a new column for that process called proc for all your hosts with an ext script on the server
Then use combo.cfg to generate alert like this
allhosts.proc = ( client1.proc + client2.gis ) == 1
as long as the result is 1 allhosts.proc won't be red
HTH
I know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it?
Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server?
|\/|
[Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing abuse at colt.net and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
On Thu, Jan 12, 2012 at 12:13 PM, Asif Iqbal <vadud3 at gmail.com> wrote:
On Thu, Jan 12, 2012 at 11:05 AM, Ward, Martin <Martin.Ward at colt.net> wrote:
Hi all,
I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this.
I can’t have something that runs on the client since I won’t know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple!
This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.
xymon localhost "clientlog client.example.com section=ps" | grep pr[o]c
will let you know if the process is running on client.example.com
create a new column for that process called proc for all your hosts with an ext script on the server
Then use combo.cfg to generate alert like this
allhosts.proc = ( client1.proc + client2.gis ) == 1
correction:
allhosts.proc = (client1.proc + client2.proc) == 1
as long as the result is 1 allhosts.proc won't be red
HTH
I know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it?
Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server?
|\/|
[Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing abuse at colt.net and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
Actually, I have multiple HP Service Guard clusters which have the same basic issue you listed for your VCS cluster. I solved the issue by copying the Xymon client directory on the cluster node and naming it something which matched the "package" name (or Service Group in VCS), editing down the xymonclient-hp-ux.sh to just date, df, mount, and ps sections. I added the stop/start of this new client process into the Service Guard Package start-up script (VCS service Group stop/start scripts). With careful ignores in the analysis.cfg (was hobbit-clients.cfg) file, I control what disk space is monitored by which client process. I then added an entry in the hosts.cfg (was bb-hosts) file with the IP associated with the Package (Service Group). No matter where the Package (SG) is running on the cluster, the info is passed back to Xymon. If a Package is not running, then the client stops. Of course if the Package is down for any time, Xymon could turn the "conn" dot red and send out alerts. I can also monitor at the new package client level any processes particular to that package and not worry what server is actually running the package. If they stop, the package alerts, not the node in the cluster. I also added a custom script to monitor the output of the cmviewcl command (cluster status similar to the hastatus -summary command in VCS). If the output indicates a failure (package or node) then this test (running on each node in the cluster) goes red and I get an alert that something on the cluster is not good.
Hope that helps, Bruce
Bruce White Senior Enterprise Systems Engineer | Phone: 1-630-671-5169 | Fax: 630-893-1648 | bewhite at fellowes.com | http://www.fellowes.com/ Disclaimer: The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. Fellowes, Inc. -----Original Message----- From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Asif Iqbal Sent: Thursday, January 12, 2012 11:14 AM To: Ward, Martin Cc: xymon at xymon.com Subject: Re: [Xymon] Where to find the latest hostdata (or: How to monitor one process on two servers)?
On Thu, Jan 12, 2012 at 12:13 PM, Asif Iqbal <vadud3 at gmail.com> wrote:
On Thu, Jan 12, 2012 at 11:05 AM, Ward, Martin <Martin.Ward at colt.net> wrote:
Hi all,
I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this.
I can't have something that runs on the client since I won't know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple!
This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.
xymon localhost "clientlog client.example.com section=ps" | grep pr[o]c
will let you know if the process is running on client.example.com
create a new column for that process called proc for all your hosts with an ext script on the server
Then use combo.cfg to generate alert like this
allhosts.proc = ( client1.proc + client2.gis ) == 1
correction:
allhosts.proc = (client1.proc + client2.proc) == 1
as long as the result is 1 allhosts.proc won't be red
HTH
I know that the latest host data is stored somewhere because I can
see it in
the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it?
Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server?
|\/|
[Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing abuse at colt.net and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
On Thu, Jan 12, 2012 at 12:14 PM, Asif Iqbal <vadud3 at gmail.com> wrote:
On Thu, Jan 12, 2012 at 12:13 PM, Asif Iqbal <vadud3 at gmail.com> wrote:
On Thu, Jan 12, 2012 at 11:05 AM, Ward, Martin <Martin.Ward at colt.net> wrote:
Hi all,
I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this.
I can’t have something that runs on the client since I won’t know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple!
This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.
xymon localhost "clientlog client.example.com section=ps" | grep pr[o]c
will let you know if the process is running on client.example.com
create a new column for that process called proc for all your hosts with an ext script on the server
Then use combo.cfg to generate alert like this
allhosts.proc = ( client1.proc + client2.gis ) == 1
correction:
allhosts.proc = (client1.proc + client2.proc) == 1
forgot to mention the last part.
you will also need to have an entry on hosts.cfg, something like this
page allhosts All Hosts proc service group-compress <H3><I>All Hosts proc service</I></H3> group-only proc|info 0.0.0.0 allhosts # noconn NAME:"Combo test for proc on all hosts"
as long as the result is 1 allhosts.proc won't be red
HTH
I know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it?
Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server?
|\/|
[Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing abuse at colt.net and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
participants (4)
-
bewhite＠fellowes.com
-
Martin.Ward＠colt.net
-
sjcarr＠gmail.com
-
vadud3＠gmail.com