Frequent purple alerts
In my xymonclient_config.xml file I specify the logs to be generated in C:\Logs <clientlogfile>c:\Logs\xymonclient.log</clientlogfile>. Yours may be in folder with script. Look at the xymonclient.log file which contains the transmission data. Look for the size of the Sent file. If your buffers are set too low on Xymon server, data file gets truncated, usually causing White, not purple.
2019-07-10 12:28:43.087 Sending to server 2019-07-10 12:28:43.087 Using "original" ASCII encoding 2019-07-10 12:28:43.087 Connecting to host 128.172.5.33 2019-07-10 12:28:43.103 Sent 54761 bytes to server 2019-07-10 12:28:43.309 Received 436 bytes from server
Tim Williams
On Wed, Jul 10, 2019 at 12:23 PM Jaime Kikpole via Xymon <xymon at xymon.com> wrote:
---------- Forwarded message ---------- From: Jaime Kikpole <jkikpole at cairodurham.org> To: "Root, Paul T" <Paul.Root at centurylink.com> Cc: "xymon at xymon.com" <xymon at xymon.com> Bcc: Date: Wed, 10 Jul 2019 12:21:42 -0400 Subject: Re: Frequent purple alerts On Wed, Jul 10, 2019 at 11:44 AM Root, Paul T <Paul.Root at centurylink.com> wrote:
This could be the network is too congested.
That seems unlikely. The Xymon server and clients are all on one of two VM clusters, so there aren't a lot of "hops" and the bandwidth is very high.
The Xymon client is one of two AD controllers, named dir1 and dir2. The one with purple alerts is dir2. On dir1, we also have .1x authentication, but they're otherwise pretty much the same. Their biggest difference is RAM: 8GB for dir1 and 4GB for dir2. Do you think I should increase the RAM? Other than leaving Task Manager open at all times and hoping to catch it as soon as the purple alert occurs, I'm not sure how to check if it's running out of RAM.
Or the computer is overloaded. Or even that it is sending messages that are too big (a frequent Windows problem in my experience – log files get so big).
Any way to check on the log size?
Does it go purple for 1 minute, 5 minutes, 30 minutes?
I just looked at the log. The purple alert durations are roughly 40 minutes each during the last three occasions.
If it helps, this server is an Active Directory domain controller, but it is one of two and the other one (a) has more stuff running on it, such as .1x authentication for our wifi and (b) isn't having this issue.
Thanks for the advice!
Jaime Kikpole
Director of Technology & Innovations Cairo-Durham Central School District (518) 622-8543, x59500 cairodurham.org <http://www.cairodurham.org>
Technical Support: help at cairodurham.org go.cairodurham.org/techtips
[image: Google Certified Educator, Level 1] [image: Google Certified Educator, Level 2]
This electronic message and any attachment(s) may contain confidential or legally privileged information protected by law from further disclosure and is intended only for the individual or entity identified above as the addressee. If you are not the addressee (or the employee or agency responsible to deliver it to the addressee), or if this message has been addressed to you in error, you are hereby notified that you may not copy, forward, disclose or use any part of this message or any attachment(s). Please notify the sender immediately by return email or telephone and permanently delete this message and attachment(s) from your system.
---------- Forwarded message ---------- From: Jaime Kikpole via Xymon <xymon at xymon.com> To: "Root, Paul T" <Paul.Root at centurylink.com> Cc: "xymon at xymon.com" <xymon at xymon.com> Bcc: Date: Wed, 10 Jul 2019 12:21:42 -0400 Subject: Re: [Xymon] Frequent purple alerts
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Sorry to resurrect this old thread, but I finally was able to grab the logs from the Xymon client during a purple alert. Usually, it would go back to green before I would notice, could switch gears, and began working on it.
Thanks, Timoth Williams, for pointing out the file uploading parts of the logs. Based on that, I found these lines in the xymonclient.log file: 2019-07-31 15:25:38 Connecting to host 163.153.163.90 2019-07-31 15:25:59 ERROR: Cannot connect to host monitor1.cairodurham.org (163.153.163.90) : System.Management.Automation.MethodInvocationException: Exception calling "Connect" with "2" argument(s): "A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 163.153.163.90:1984" ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 163.153.163.90:1984
It looks like it was somehow resolving the FQDN (monitor1.cairodurham.org) to its external IP address instead of its internal IP address. I'm not sure why. I just checked the DNS settings and they're the same as another Windows 2012R2 server that isn't having this issue.
I changed the FQDN to the internal IP address and restarted the service. Everything went green almost immediately.
Any idea how it could resolve to the public IP address 2 - 4 each day but only for a few hours total each day?
Jaime Kikpole
Director of Technology & Innovations Cairo-Durham Central School District (518) 622-8543, x59500 cairodurham.org <http://www.cairodurham.org>
Technical Support: help at cairodurham.org go.cairodurham.org/techtips
[image: Google Certified Educator, Level 1][image: Google Certified Educator, Level 2] <https://www.credential.net/d24m9rrp>
-- This electronic message and any attachment(s) may contain confidential or legally privileged information protected by law from further disclosure and is intended only for the individual or entity identified above as the addressee. If you are not the addressee (or the employee or agency responsible to deliver it to the addressee), or if this message has been addressed to you in error, you are hereby notified that you may not copy, forward, disclose or use any part of this message or any attachment(s). Please notify the sender immediately by return email or telephone and permanently delete this message and attachment(s) from your system.
I don't know about the DNS switching around, unless it is due to some DC synchronizing stuff, and one has a manual entry the other doesn't? Two ways to circumvent that is to use the IP in the Xymon Settings file <servers> tag ( I think that is what you said you did), or add the internal IP to the server HOSTS file; both of which requires future editing if the IP of the hostname gets changed.
I should have mentioned that I use the tag <clientlogretain>4</clientlogretain> in my xymonclient_config.xml file to save multiple versions of the logs to give me some time to look at them and track changes from one file to another when I make a change.
Glad you are able to get it stable.
Tim Williams VCU Computer Center
On Wed, Jul 31, 2019 at 4:33 PM Jaime Kikpole <jkikpole at cairodurham.org> wrote:
Sorry to resurrect this old thread, but I finally was able to grab the logs from the Xymon client during a purple alert. Usually, it would go back to green before I would notice, could switch gears, and began working on it.
Thanks, Timoth Williams, for pointing out the file uploading parts of the logs. Based on that, I found these lines in the xymonclient.log file: 2019-07-31 15:25:38 Connecting to host 163.153.163.90 2019-07-31 15:25:59 ERROR: Cannot connect to host monitor1.cairodurham.org (163.153.163.90) : System.Management.Automation.MethodInvocationException: Exception calling "Connect" with "2" argument(s): "A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 163.153.163.90:1984" ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 163.153.163.90:1984
It looks like it was somehow resolving the FQDN (monitor1.cairodurham.org) to its external IP address instead of its internal IP address. I'm not sure why. I just checked the DNS settings and they're the same as another Windows 2012R2 server that isn't having this issue.
I changed the FQDN to the internal IP address and restarted the service. Everything went green almost immediately.
Any idea how it could resolve to the public IP address 2 - 4 each day but only for a few hours total each day?
Jaime Kikpole
Director of Technology & Innovations Cairo-Durham Central School District (518) 622-8543, x59500 cairodurham.org <http://www.cairodurham.org>
Technical Support: help at cairodurham.org go.cairodurham.org/techtips
[image: Google Certified Educator, Level 1][image: Google Certified Educator, Level 2] <https://www.credential.net/d24m9rrp>
This electronic message and any attachment(s) may contain confidential or legally privileged information protected by law from further disclosure and is intended only for the individual or entity identified above as the addressee. If you are not the addressee (or the employee or agency responsible to deliver it to the addressee), or if this message has been addressed to you in error, you are hereby notified that you may not copy, forward, disclose or use any part of this message or any attachment(s). Please notify the sender immediately by return email or telephone and permanently delete this message and attachment(s) from your system.
You can tell xymon to use the IP address in the hosts.cfg file. Put testip in the comment section of that host. See the hosts.cfg man page.
From: Timothy Williams <tlwilliams4 at vcu.edu> Sent: Thursday, August 01, 2019 10:02 AM To: Jaime Kikpole <jkikpole at cairodurham.org> Cc: Root, Paul T <Paul.Root at CenturyLink.com>; xymon at xymon.com Subject: Re: [Xymon] Frequent purple alerts
I don't know about the DNS switching around, unless it is due to some DC synchronizing stuff, and one has a manual entry the other doesn't? Two ways to circumvent that is to use the IP in the Xymon Settings file <servers> tag ( I think that is what you said you did), or add the internal IP to the server HOSTS file; both of which requires future editing if the IP of the hostname gets changed.
I should have mentioned that I use the tag <clientlogretain>4</clientlogretain> in my xymonclient_config.xml file to save multiple versions of the logs to give me some time to look at them and track changes from one file to another when I make a change.
Glad you are able to get it stable.
Tim Williams VCU Computer Center
On Wed, Jul 31, 2019 at 4:33 PM Jaime Kikpole <jkikpole at cairodurham.org<mailto:jkikpole at cairodurham.org>> wrote: Sorry to resurrect this old thread, but I finally was able to grab the logs from the Xymon client during a purple alert. Usually, it would go back to green before I would notice, could switch gears, and began working on it.
Thanks, Timoth Williams, for pointing out the file uploading parts of the logs. Based on that, I found these lines in the xymonclient.log file: 2019-07-31 15:25:38 Connecting to host 163.153.163.90 2019-07-31 15:25:59 ERROR: Cannot connect to host monitor1.cairodurham.org<http://monitor1.cairodurham.org> (163.153.163.90) : System.Management.Automation.MethodInvocationException: Exception calling "Connect" with "2" argument(s): "A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 163.153.163.90:1984<http://163.153.163.90:1984>" ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 163.153.163.90:1984<http://163.153.163.90:1984>
It looks like it was somehow resolving the FQDN (monitor1.cairodurham.org<http://monitor1.cairodurham.org>) to its external IP address instead of its internal IP address. I'm not sure why. I just checked the DNS settings and they're the same as another Windows 2012R2 server that isn't having this issue.
I changed the FQDN to the internal IP address and restarted the service. Everything went green almost immediately.
Any idea how it could resolve to the public IP address 2 - 4 each day but only for a few hours total each day?
[https://s3.amazonaws.com/htmlsig-assets/spacer.gif]
Jaime Kikpole
Director of Technology & Innovations Cairo-Durham Central School District (518) 622-8543, x59500 cairodurham.org<http://www.cairodurham.org>
Technical Support: help at cairodurham.org<mailto:help at cairodurham.org> go.cairodurham.org/techtips<http://go.cairodurham.org/techtips>
[Google Certified Educator, Level 1][Google Certified Educator, Level 2][https://api.accredible.com/v1/frontend/credential_website_embed_image/badge/13415328]<https://www.credential.net/d24m9rrp>
This electronic message and any attachment(s) may contain confidential or legally privileged information protected by law from further disclosure and is intended only for the individual or entity identified above as the addressee. If you are not the addressee (or the employee or agency responsible to deliver it to the addressee), or if this message has been addressed to you in error, you are hereby notified that you may not copy, forward, disclose or use any part of this message or any attachment(s). Please notify the sender immediately by return email or telephone and permanently delete this message and attachment(s) from your system. This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
participants (3)
-
jkikpole＠cairodurham.org
-
Paul.Root＠CenturyLink.com
-
tlwilliams4＠vcu.edu