New server causing issues with CONN test
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today). I did not see a fix to my issue in the 4.3.4 change log though, so figured I'd post here.
In doing my update, I install the brand new server from scratch. Basically install CentOS 6 as a web server install, and then add in all the bits xymon needs (pcre, openssl, openldap, rrdtool, etc).. Then I compile and install xymon to /usr/lib/xymon. Next I copy over the bb-hosts file to the hosts.cfg, and follow the "migration" steps to get the data and configuration files over. Then I turn on xymon on the new server.
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
I'm completely at a loss here. Does anyone know what may be causing these issues where the only difference is the OS being used (the distro, that is)?
I just want to get our monitoring server upgraded to a stable OS, with updates, and get xymon up to date as well.
Thanks, -Ben
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
Are the two servers using the same IP? Tied to one another in any way? I would start a ping and turn the other server on and see when it goes down. On Aug 15, 2011 5:03 PM, "Poppy, Ben" <poppy.ben at marshfieldclinic.org> wrote:
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today). I did not see a fix to my issue in the 4.3.4 change log though, so figured I'd post here.
In doing my update, I install the brand new server from scratch. Basically install CentOS 6 as a web server install, and then add in all the bits xymon needs (pcre, openssl, openldap, rrdtool, etc).. Then I compile and install xymon to /usr/lib/xymon. Next I copy over the bb-hosts file to the hosts.cfg, and follow the "migration" steps to get the data and configuration files over. Then I turn on xymon on the new server.
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
I'm completely at a loss here. Does anyone know what may be causing these issues where the only difference is the OS being used (the distro, that is)?
I just want to get our monitoring server upgraded to a stable OS, with updates, and get xymon up to date as well.
Thanks, -Ben
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
The 2 servers are not using the same IP. They are tied to each other in the hobbit configs in that they point to each other.
My existing hobbit servers, hobbit1 and hobbit2, are the fail-over for each other. So they have the exact same configuration, and report data to each other in their client settings. All servers with hobbit or bbwin clients send data to both servers.
When I bring the new replacement server online, I shutdown hobbit2, and have the new server assume it's IP.
I will try the ping test to see exactly when it stops responding..
The strangest thing, like I said, is it's the exact same 4 hosts that show red for CONN. This was with every combination of centos/xymon/hobbit below.. I even cloned one of my existing centos 5 32-bit servers running hobbit 4.2 in another environment (our perimeter network that is firewalled off), such that the only thing that was different was the linux distro, and that also caused the same 4 servers to show red..
From: Josh Luthman [mailto:josh at imaginenetworksllc.com] Sent: Monday, August 15, 2011 4:07 PM To: Poppy, Ben Cc: xymon at xymon.com Subject: Re: [Xymon] New server causing issues with CONN test
Are the two servers using the same IP? Tied to one another in any way? I would start a ping and turn the other server on and see when it goes down. On Aug 15, 2011 5:03 PM, "Poppy, Ben" <poppy.ben at marshfieldclinic.org<mailto:poppy.ben at marshfieldclinic.org>> wrote:
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today). I did not see a fix to my issue in the 4.3.4 change log though, so figured I'd post here.
In doing my update, I install the brand new server from scratch. Basically install CentOS 6 as a web server install, and then add in all the bits xymon needs (pcre, openssl, openldap, rrdtool, etc).. Then I compile and install xymon to /usr/lib/xymon. Next I copy over the bb-hosts file to the hosts.cfg, and follow the "migration" steps to get the data and configuration files over. Then I turn on xymon on the new server.
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
I'm completely at a loss here. Does anyone know what may be causing these issues where the only difference is the OS being used (the distro, that is)?
I just want to get our monitoring server upgraded to a stable OS, with updates, and get xymon up to date as well.
Thanks, -Ben
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
CONN is done by the server, so it is best to look from the server's perspective. Without knowing your network and details of the server it's tough to know where to start, but I would start by seeing when the server fails to ping the host (see if you get ARP, route to it, etc).
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Mon, Aug 15, 2011 at 5:11 PM, Poppy, Ben <poppy.ben at marshfieldclinic.org>wrote:
The 2 servers are not using the same IP. They are tied to each other in the hobbit configs in that they point to each other.****
My existing hobbit servers, hobbit1 and hobbit2, are the fail-over for each other. So they have the exact same configuration, and report data to each other in their client settings. All servers with hobbit or bbwin clients send data to both servers.****
When I bring the new replacement server online, I shutdown hobbit2, and have the new server assume it's IP.****
I will try the ping test to see exactly when it stops responding..****
The strangest thing, like I said, is it's the exact same 4 hosts that show red for CONN. This was with every combination of centos/xymon/hobbit below.. I even cloned one of my existing centos 5 32-bit servers running hobbit 4.2 in another environment (our perimeter network that is firewalled off), such that the only thing that was different was the linux distro, and that also caused the same 4 servers to show red..****
*From:* Josh Luthman [mailto:josh at imaginenetworksllc.com] *Sent:* Monday, August 15, 2011 4:07 PM *To:* Poppy, Ben *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] New server causing issues with CONN test****
Are the two servers using the same IP? Tied to one another in any way? I would start a ping and turn the other server on and see when it goes down.
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today). I did not see a fix to my issue in the 4.3.4 change log though, so figured I'd post here.
In doing my update, I install the brand new server from scratch. Basically install CentOS 6 as a web server install, and then add in all the bits xymon needs (pcre, openssl, openldap, rrdtool, etc).. Then I compile and install xymon to /usr/lib/xymon. Next I copy over the bb-hosts file to the hosts.cfg, and follow the "migration" steps to get the data and configuration files over. Then I turn on xymon on the new server.
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
I'm completely at a loss here. Does anyone know what may be causing these issues where the only difference is the OS being used (the distro, that is)?
I just want to get our monitoring server upgraded to a stable OS, with updates, and get xymon up to date as well.
Thanks, -Ben
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.***
On Aug 15, 2011 5:03 PM, "Poppy, Ben" <poppy.ben at marshfieldclinic.org> wrote: *
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
Josh is correct. Clear the arp cache, everywhere. (client and switch)
"When I bring the new replacement server online, I shutdown hobbit2, and have the new server assume it's IP."
From: xymon-bounces at xymon.com [xymon-bounces at xymon.com] On Behalf Of Josh Luthman [josh at imaginenetworksllc.com] Sent: Monday, August 15, 2011 2:14 PM To: Poppy, Ben Cc: xymon at xymon.com Subject: Re: [Xymon] New server causing issues with CONN test
CONN is done by the server, so it is best to look from the server's perspective. Without knowing your network and details of the server it's tough to know where to start, but I would start by seeing when the server fails to ping the host (see if you get ARP, route to it, etc).
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Mon, Aug 15, 2011 at 5:11 PM, Poppy, Ben <poppy.ben at marshfieldclinic.org<mailto:poppy.ben at marshfieldclinic.org>> wrote: The 2 servers are not using the same IP. They are tied to each other in the hobbit configs in that they point to each other.
My existing hobbit servers, hobbit1 and hobbit2, are the fail-over for each other. So they have the exact same configuration, and report data to each other in their client settings. All servers with hobbit or bbwin clients send data to both servers.
When I bring the new replacement server online, I shutdown hobbit2, and have the new server assume it's IP.
I will try the ping test to see exactly when it stops responding..
The strangest thing, like I said, is it's the exact same 4 hosts that show red for CONN. This was with every combination of centos/xymon/hobbit below.. I even cloned one of my existing centos 5 32-bit servers running hobbit 4.2 in another environment (our perimeter network that is firewalled off), such that the only thing that was different was the linux distro, and that also caused the same 4 servers to show red..
From: Josh Luthman [mailto:josh at imaginenetworksllc.com<mailto:josh at imaginenetworksllc.com>] Sent: Monday, August 15, 2011 4:07 PM To: Poppy, Ben Cc: xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] New server causing issues with CONN test
Are the two servers using the same IP? Tied to one another in any way? I would start a ping and turn the other server on and see when it goes down. On Aug 15, 2011 5:03 PM, "Poppy, Ben" <poppy.ben at marshfieldclinic.org<mailto:poppy.ben at marshfieldclinic.org>> wrote:
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today). I did not see a fix to my issue in the 4.3.4 change log though, so figured I'd post here.
In doing my update, I install the brand new server from scratch. Basically install CentOS 6 as a web server install, and then add in all the bits xymon needs (pcre, openssl, openldap, rrdtool, etc).. Then I compile and install xymon to /usr/lib/xymon. Next I copy over the bb-hosts file to the hosts.cfg, and follow the "migration" steps to get the data and configuration files over. Then I turn on xymon on the new server.
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
I'm completely at a loss here. Does anyone know what may be causing these issues where the only difference is the OS being used (the distro, that is)?
I just want to get our monitoring server upgraded to a stable OS, with updates, and get xymon up to date as well.
Thanks, -Ben
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
I'll give this a try as well during my next testing phase.
-----Original Message----- From: Tim McCloskey [mailto:tm at freedom.com] Sent: Monday, August 15, 2011 4:18 PM To: Josh Luthman; Poppy, Ben Cc: xymon at xymon.com Subject: RE: [Xymon] New server causing issues with CONN test
Josh is correct. Clear the arp cache, everywhere. (client and switch)
"When I bring the new replacement server online, I shutdown hobbit2, and have the new server assume it's IP."
From: xymon-bounces at xymon.com [xymon-bounces at xymon.com] On Behalf Of Josh Luthman [josh at imaginenetworksllc.com] Sent: Monday, August 15, 2011 2:14 PM To: Poppy, Ben Cc: xymon at xymon.com Subject: Re: [Xymon] New server causing issues with CONN test
CONN is done by the server, so it is best to look from the server's perspective. Without knowing your network and details of the server it's tough to know where to start, but I would start by seeing when the server fails to ping the host (see if you get ARP, route to it, etc).
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Mon, Aug 15, 2011 at 5:11 PM, Poppy, Ben <poppy.ben at marshfieldclinic.org<mailto:poppy.ben at marshfieldclinic.org>> wrote: The 2 servers are not using the same IP. They are tied to each other in the hobbit configs in that they point to each other.
My existing hobbit servers, hobbit1 and hobbit2, are the fail-over for each other. So they have the exact same configuration, and report data to each other in their client settings. All servers with hobbit or bbwin clients send data to both servers.
When I bring the new replacement server online, I shutdown hobbit2, and have the new server assume it's IP.
I will try the ping test to see exactly when it stops responding..
The strangest thing, like I said, is it's the exact same 4 hosts that show red for CONN. This was with every combination of centos/xymon/hobbit below.. I even cloned one of my existing centos 5 32-bit servers running hobbit 4.2 in another environment (our perimeter network that is firewalled off), such that the only thing that was different was the linux distro, and that also caused the same 4 servers to show red..
From: Josh Luthman [mailto:josh at imaginenetworksllc.com<mailto:josh at imaginenetworksllc.com>] Sent: Monday, August 15, 2011 4:07 PM To: Poppy, Ben Cc: xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] New server causing issues with CONN test
Are the two servers using the same IP? Tied to one another in any way? I would start a ping and turn the other server on and see when it goes down. On Aug 15, 2011 5:03 PM, "Poppy, Ben" <poppy.ben at marshfieldclinic.org<mailto:poppy.ben at marshfieldclinic.org>> wrote:
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today). I did not see a fix to my issue in the 4.3.4 change log though, so figured I'd post here.
In doing my update, I install the brand new server from scratch. Basically install CentOS 6 as a web server install, and then add in all the bits xymon needs (pcre, openssl, openldap, rrdtool, etc).. Then I compile and install xymon to /usr/lib/xymon. Next I copy over the bb-hosts file to the hosts.cfg, and follow the "migration" steps to get the data and configuration files over. Then I turn on xymon on the new server.
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
I'm completely at a loss here. Does anyone know what may be causing these issues where the only difference is the OS being used (the distro, that is)?
I just want to get our monitoring server upgraded to a stable OS, with updates, and get xymon up to date as well.
Thanks, -Ben
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
I got it figured out, turns out the systems were in multiple domains DNS wise, and I had my /etc/resolve.conf entries out of order a bit from the existing hobbit servers.. But they were both pointing to the same 2 DNS servers. So what was happening was one server would get the "wrong" IP, and cache it on the DNS servers, then get the right IP and cache it, and so on. And this would cause the flapping every 5-10 minutes..
Once I got it sync'd up, everything started working. one or 2 quirks, but I'll start a new thread for that.. thanks for your help!
-----Original Message----- From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Poppy, Ben Sent: Monday, August 15, 2011 4:27 PM To: Tim McCloskey; Josh Luthman Cc: xymon at xymon.com Subject: Re: [Xymon] New server causing issues with CONN test
I'll give this a try as well during my next testing phase.
-----Original Message----- From: Tim McCloskey [mailto:tm at freedom.com] Sent: Monday, August 15, 2011 4:18 PM To: Josh Luthman; Poppy, Ben Cc: xymon at xymon.com Subject: RE: [Xymon] New server causing issues with CONN test
Josh is correct. Clear the arp cache, everywhere. (client and switch)
"When I bring the new replacement server online, I shutdown hobbit2, and have the new server assume it's IP."
From: xymon-bounces at xymon.com [xymon-bounces at xymon.com] On Behalf Of Josh Luthman [josh at imaginenetworksllc.com] Sent: Monday, August 15, 2011 2:14 PM To: Poppy, Ben Cc: xymon at xymon.com Subject: Re: [Xymon] New server causing issues with CONN test
CONN is done by the server, so it is best to look from the server's perspective. Without knowing your network and details of the server it's tough to know where to start, but I would start by seeing when the server fails to ping the host (see if you get ARP, route to it, etc).
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Mon, Aug 15, 2011 at 5:11 PM, Poppy, Ben <poppy.ben at marshfieldclinic.org<mailto:poppy.ben at marshfieldclinic.org>> wrote: The 2 servers are not using the same IP. They are tied to each other in the hobbit configs in that they point to each other.
My existing hobbit servers, hobbit1 and hobbit2, are the fail-over for each other. So they have the exact same configuration, and report data to each other in their client settings. All servers with hobbit or bbwin clients send data to both servers.
When I bring the new replacement server online, I shutdown hobbit2, and have the new server assume it's IP.
I will try the ping test to see exactly when it stops responding..
The strangest thing, like I said, is it's the exact same 4 hosts that show red for CONN. This was with every combination of centos/xymon/hobbit below.. I even cloned one of my existing centos 5 32-bit servers running hobbit 4.2 in another environment (our perimeter network that is firewalled off), such that the only thing that was different was the linux distro, and that also caused the same 4 servers to show red..
From: Josh Luthman [mailto:josh at imaginenetworksllc.com<mailto:josh at imaginenetworksllc.com>] Sent: Monday, August 15, 2011 4:07 PM To: Poppy, Ben Cc: xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] New server causing issues with CONN test
Are the two servers using the same IP? Tied to one another in any way? I would start a ping and turn the other server on and see when it goes down. On Aug 15, 2011 5:03 PM, "Poppy, Ben" <poppy.ben at marshfieldclinic.org<mailto:poppy.ben at marshfieldclinic.org>> wrote:
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today). I did not see a fix to my issue in the 4.3.4 change log though, so figured I'd post here.
In doing my update, I install the brand new server from scratch. Basically install CentOS 6 as a web server install, and then add in all the bits xymon needs (pcre, openssl, openldap, rrdtool, etc).. Then I compile and install xymon to /usr/lib/xymon. Next I copy over the bb-hosts file to the hosts.cfg, and follow the "migration" steps to get the data and configuration files over. Then I turn on xymon on the new server.
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
I'm completely at a loss here. Does anyone know what may be causing these issues where the only difference is the OS being used (the distro, that is)?
I just want to get our monitoring server upgraded to a stable OS, with updates, and get xymon up to date as well.
Thanks, -Ben
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
On 15-08-2011 22:46, Poppy, Ben wrote:
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today).
[installs and starts 4.3 version]
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
As I understand, you were running both versions simultaneously. Did those servers also go red on the new Xymon version, or only on the old one? If they were red also on the new server, did you try stopping network tests on the old server and did that make a difference ?
Which ping-tool are you using - xymonping or fping ?
I haven't heard of anything like this before, but I suspect it may be an issue with the way "ping" works. When routing traffic, most systems will pass ping-traffic with a low priority, so it is quite easy for ping-requests and -responses to be dropped. Since xymonping and fping pump out a lot of ping-traffic rather quickly, maybe the new server just happened to be more "lucky" with its data than the old one - perhaps due to the switch port it is on, or the speed of the network interface and so on.
It might be worthwhile to make sure that the old and the new system does not run the network tests at the same time - keep an eye (with "ps" on when the network test runs on the old system, and don't start Xymon on the new system until about 30 secs after the old system completes the network tests. (Assuming your network tests don't take more than a couple of minutes, so there is time for both systems to run their tests within the default 5 minute interval).
Regards, Henrik
The new server went into a "flapping" state.
During my next test, I'll try stopping the tests on the new server and see what happens..
-----Original Message----- From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Henrik Størner Sent: Monday, August 15, 2011 4:17 PM To: xymon at xymon.com Subject: Re: [Xymon] New server causing issues with CONN test
On 15-08-2011 22:46, Poppy, Ben wrote:
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today).
[installs and starts 4.3 version]
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
As I understand, you were running both versions simultaneously. Did those servers also go red on the new Xymon version, or only on the old one? If they were red also on the new server, did you try stopping network tests on the old server and did that make a difference ?
Which ping-tool are you using - xymonping or fping ?
I haven't heard of anything like this before, but I suspect it may be an issue with the way "ping" works. When routing traffic, most systems will pass ping-traffic with a low priority, so it is quite easy for ping-requests and -responses to be dropped. Since xymonping and fping pump out a lot of ping-traffic rather quickly, maybe the new server just happened to be more "lucky" with its data than the old one - perhaps due to the switch port it is on, or the speed of the network interface and so on.
It might be worthwhile to make sure that the old and the new system does not run the network tests at the same time - keep an eye (with "ps" on when the network test runs on the old system, and don't start Xymon on the new system until about 30 secs after the old system completes the network tests. (Assuming your network tests don't take more than a couple of minutes, so there is time for both systems to run their tests within the default 5 minute interval).
Regards, Henrik
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
Also to note, the original servers were using hobbitping, the new servers were using fping at one point, and then xymonping at another point (thinking it was the ping tool being different)..
-----Original Message----- From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Henrik Størner Sent: Monday, August 15, 2011 4:17 PM To: xymon at xymon.com Subject: Re: [Xymon] New server causing issues with CONN test
On 15-08-2011 22:46, Poppy, Ben wrote:
I'm having a pretty strange issue. We have our existing hobbit servers running on Fedora servers running hobbit 4.2.0. I'm working on installing brand new servers that will be running CentOS 6 64-bit and the latest version of xymon (4.3.3 before I saw 4.3.4 today).
[installs and starts 4.3 version]
Within a few minutes, 4 servers turn to red alerts on CONN on the existing Fedora based Hobbit servers. They begin flapping on and off of red alert until I shutdown the new CentOS xymon server. Within a few minutes of the new server being shut down, the alerts go away for good.
I have tried going to Centos 5 32-bit, 64-bit, even trying xymon 4.2.3, or all the way back to hobbit 4.2.0 all with the same result, and the exact same 4 servers each time.
As I understand, you were running both versions simultaneously. Did those servers also go red on the new Xymon version, or only on the old one? If they were red also on the new server, did you try stopping network tests on the old server and did that make a difference ?
Which ping-tool are you using - xymonping or fping ?
I haven't heard of anything like this before, but I suspect it may be an issue with the way "ping" works. When routing traffic, most systems will pass ping-traffic with a low priority, so it is quite easy for ping-requests and -responses to be dropped. Since xymonping and fping pump out a lot of ping-traffic rather quickly, maybe the new server just happened to be more "lucky" with its data than the old one - perhaps due to the switch port it is on, or the speed of the network interface and so on.
It might be worthwhile to make sure that the old and the new system does not run the network tests at the same time - keep an eye (with "ps" on when the network test runs on the old system, and don't start Xymon on the new system until about 30 secs after the old system completes the network tests. (Assuming your network tests don't take more than a couple of minutes, so there is time for both systems to run their tests within the default 5 minute interval).
Regards, Henrik
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
The contents of this message may contain private, protected and/or privileged information. If you received this message in error, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained within. Please contact the sender and advise of the erroneous delivery by return e-mail or telephone. Thank you for your cooperation.
participants (4)
-
henrik＠hswn.dk
-
josh＠imaginenetworksllc.com
-
poppy.ben＠marshfieldclinic.org
-
tm＠freedom.com