December 2015 - Xymon - Xymon mailman web

deleted test purple status
by wlrutherford＠alaska.edu 09 Dec '15

09 Dec '15

Hello, I ended up with two ssh tests in my xymon main page: ssh and ssh2. This isn't a big problem except it's redundant. What I *really* wanted was to test that ssh2 is running but ssh1 is not. I think that's accomplished with a combo test but I haven't configured one of those yet. But in the meantime when I got rid of ssh from the hosts.cfg file and ran: <PATH>/xymon 127.0.0.1 "drop HOSTNAME ssh" I got a full purple ssh column. I found the hist and histlogs directories with many ssh subdirectories (which I thought the 'drop' should've cleared) and recursively removed all of the ssh directories. Still as purple as Barney. I know I've handled this problem a lifetime ago while using BigBrother, and what I did today feels familiar, but I'm missing something. I finally had to put ssh back into the hosts.cfg file when management noticed all the purple indicators and got a bit anxious. What did I do wrong? Walter R.

1 0

Alerts.cfg doesn't wait before sending alert, help?
by Eric.Wedaa＠marist.edu 08 Dec '15

08 Dec '15

My Google-fu failed me, maybe somebody can point me in the right direction. I have an alerts.cfg file with the following contents: page=webservers2 mail eric.wedaa at marist.edu COLOR=red duration>5m repeat=4h mail eric.wedaa at marist.edu COLOR=!red recovered What is happening is it sends a yellow alert right away when the webserver goes down, and 15 minutes later sends a red alert. I THOUGHT that this would not send a yellow alert, and then send a RED alert after 5 minutes. What am I doing wrong please? >>>Ericw

2 1

layout of terabithia xymon client
by Paul.Root＠CenturyLink.com 08 Dec '15

08 Dec '15

I've recently inherited supporting a xymon network where the old admin use the terabithia rpms. I've spent most of the day looking for a particular test for a particular client. I finally found the script in /usr/share/xymon-client/sections directory. I can't find where that gets called from? Is this a normal terabithia layout? Where does it get invoked? Thanks, Paul. Paul Root Lead Engineer CenturyLink Network Reliability Operations Center 390 Commerce Dr Woodbury, MN 55125 Direct: (651)312-5207 Paul.Root at centurylink.com This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.

2 2

Alarm Output to Text File?
by Foster.Patch＠accuweather.com 08 Dec '15

08 Dec '15

Hi, Pretty random question here- I have had the severe frustration of working with CPU monitoring on Windows 2000. I was wondering if it is possible to write a text file if an alarm like CPU goes red. I would have a persistent script running in the background that would check to see if that file exists, and commit and action based on its existence. I am still very new to Xymon and was just wondering if this was possible. Thank you, Foster Patch

2 1

Layout
by dewelker＠gmail.com 08 Dec '15

08 Dec '15

I'm already familiar with "group"-ing things in almost every way possible (sorted, compressed, etc.), but was wondering if there is a way to indent or put different icons in the front of entries in the same group? For example: col1 col2 col3 col4 col5 col6 ------ ------ ------ ------ ------ ------ host 1 3 3 3 switch 1 1 1 1 switch 2 2 2 2 host 2 4 4 4 switch 1 1 1 1 switch 2 2 2 2 I was hoping changing fonts would work, but adding different header types, for example, simply made Xymon see that as part of the hostname for each entry, even when adding non-breaking spaces in front of an entry in hosts.cfg, Thanks, David

2 3

"Discarding timed-out partial msg" Error Messages
by matt1299＠gmail.com 08 Dec '15

08 Dec '15

Hi J.C., Thanks for the e-mail and advice! A couple of questions: What's the default --lqueue value that Xymon uses? (Is there a way to see what it's using?) What exactly is your definition of "tons of simultaneous connections" here? Can you give me a number or range that you think would warrant increasing the --lqueue value? Could it be from clients/senders with longer than usual process listings? Or other clientlog statistics? (But still under the max client message value.) How would I be able to tell if there are long messages being sent in if the long messages are being discarded? The clients/senders are all different and there doesn't seem to be a pattern that I can see. Some hosts are showing up more then once though. All the connections to these machines SHOULD be coming in at the same connection speed.... I'm not seeing any network issues over any of our switches. And I'm not seeing any significant or unusual network traffic on the machines in question around the time of the time-out error messages (although it could be a very brief spike in traffic that isn't being seen since the status message is being discarded). I'll definitely look into the possibility of doing some TCP tuning on the Xymon server machine! Thanks again! -- Matt Vander Werf On Tue, Nov 24, 2015 at 3:17 PM, J.C. Cleaver <cleaver at terabithia.org> wrote: > > > On Tue, November 24, 2015 6:29 am, Matt Vander Werf wrote: > > Hey all, > > > > Lately, I've been seeing quite a few error messages show up in xymond > > indicating that it was discarding a timed-out partial message from some > > machine. > > > > i.e. > > > > Latest error messages: > > Discarding timed-out partial msg from X.X.X.X > > > > They seem to be happening sporadically but more often than usual as of > > late. Maybe one or two every couple of days or so. They don't seem to be > > coming from the same machine/machines either. > > > > Is this something I should be worried about? Are there any side-effects > > from this happening too much? > > > > What are the causes of this happening? Any way to make it not happen as > > much? > > > > > > Any ideas or advice is greatly appreciated! > > > > Thanks!! > > > Broadly speaking, this is a result of the entire message not making it in > in the time allotted by xymond, which is 10s by default. It could be the > result of network congestion issues or packet loss, slow sender > performance, or slow xymon server performance. > > A quick fix might be to increase the --timeout= option to xymond to > something like 15 or 20s. > > If a netstat shows tons of simultaneous connections, you could also > increase --lqueue= to 768 or 1024. > > Are there any patterns on the clients/senders that are affected? Unusually > huge messages being sent over slow connections? > > If there isn't a network issue per se, and there are no local network > errors (or you're seeing the reports about messages from all over the > place), then it's time to look at network performance tuning on the xymon > box. Consider the various tcp* options via sysctl (recycle and reuse in > particular). If xymonnet is running on the same system (and you're doing) > high concurrency testing, be sure to increase your ip_local_port_range for > outbound connections. > > http://www.lognormal.com/blog/2012/09/27/linux-tcpip-tuning/ is a nice > resource for that. > > > HTH, > > -jc > >

2 1

Xymon 4.3.22-beta Release Download
by tgm＠fastmail.com 08 Dec '15

08 Dec '15

Now that Xymon 4.3.24 is out, I'm still seeing the same compile warning previously noted in this thread: xymonping.c: In function ‘show_results’: xymonping.c:363:5: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 2 has type ‘long unsigned int’ [-Wformat=] printf(" (%u usec)\n", rtt_usecs); I'm not sure what the impact is. Thanks, Tim

2 1

Idea about GUI
by pretorius.riaan＠gmail.com 04 Dec '15

04 Dec '15

Good Day, I have a suggestion for a XYMON UI update. Please let me know what you think of the idea? Riaan

3 3

double proc test results
by wlrutherford＠alaska.edu 03 Dec '15

03 Dec '15

I just noticed that several of my systems are reporting the same PROC multiple times. I've checked the analysis.d/libit.cfg file and the processes are defined only once for the machines as far as I can see - I even removed a wildcard host I thought might've been matching too much. Thu Dec 3 09:56:02 AKST 2015 - Processes ok [image: green] mysqld (found 2, req. 2 or more)[image: green] shibd (found 1, req. 1 or more)[image: green] salt-master (found 11, req. 1 or more)[image: green] munin-node (found 1, req. between 1 and 3)[image: green] shibd (found 1, req. 1 or more)[image: green] salt-master (found 11, req. 1 or more)[image: green] munin-node (found 1, req. between 1 and 3) It's not wrong, I still get the correct results on the main page, but it's distracting. Has anyone else seen this and what is/was your cure? WLR

2 1

Possible Bug in Xymon Related to Purple Statuses
by matt1299＠gmail.com 03 Dec '15

03 Dec '15

Hello, I am having an issue with Xymon where instead of tests going purple when the client stops reporting, the tests are going clear. I noticed this with a host that had all it's tests go clear instead of purple. Turns out the network interface on the machine had completely died and this had happened a week ago! We never noticed because instead of going purple the tests for the machine went clear! This seems to only be an issue with a certain group of machines. For this group of machines, we have the ping test disabled by using the 'noping' option on all of them. This is because they are all behind a firewall with private IP addresses so they are unable to be contacted by the Xymon server. But they can still send client data out to the Xymon server. Turns out, ever since we started using the 'noping' option for all of them, none of the machines have ever gone purple... I tested this by stopping the xymon-client service on one of the machines in question, and sure enough, after the STATUSLIFETIME time limit, all the tests for that host went clear, instead of going purple. I looked through the different logs (I already had most set in debug mode for a different reason), and I didn't see much that would explain this (but I could have missed something). I did notice in the xymond log file that, according to xymond, they should have been going purple and not clear. Here's an excerpt from that log file (this is the machine which I stopped the service on): 9680 2015-12-02 09:57:48.040111 -> check_purple_status 9680 2015-12-02 09:57:48.047630 Purple log from <HOST> memory 9680 2015-12-02 09:57:48.047674 ->handle_status 9680 2015-12-02 09:57:48.047676 modifyonly = 0, changed = 0 9680 2015-12-02 09:57:48.047680 - sum: 0, synced: 0, oldcolor: 0, newcolor: 1, modifychanged: 0 9680 2015-12-02 09:57:48.047682 posting to stachg channel: host=<HOST>, test=memory 9680 2015-12-02 09:57:48.047684 -> posttochannel 9680 2015-12-02 09:57:48.047697 Posting message 14359 to 1 readers 9680 2015-12-02 09:57:48.047703 <- posttochannel 9680 2015-12-02 09:57:48.047705 posting to status channel 9680 2015-12-02 09:57:48.047706 -> posttochannel 9680 2015-12-02 09:57:48.047712 Posting message 72429 to 2 readers 9680 2015-12-02 09:57:48.047726 <- posttochannel 9680 2015-12-02 09:57:48.047727 <-handle_status Basically this showed up for all the different tests for this machine. And here's the event log for the same machine: Wed Dec 2 09:57:48 2015 <HOST> cpu [image: green] [image: From -> To] <https://mon.crc.nd.edu/xymon-cgi/historylog.sh?HOST=jim.vectorbase.org&SERV…>[image: clear] Wed Dec 2 09:57:48 2015 <HOST> disk [image: green] [image: From -> To] <https://mon.crc.nd.edu/xymon-cgi/historylog.sh?HOST=jim.vectorbase.org&SERV…>[image: clear] Wed Dec 2 09:57:48 2015 <HOST> inode [image: green] [image: From -> To] <https://mon.crc.nd.edu/xymon-cgi/historylog.sh?HOST=jim.vectorbase.org&SERV…>[image: clear] Wed Dec 2 09:57:48 2015 <HOST> memory [image: green] [image: From -> To] [image: clear] Any thoughts as to what's going on? Looks like a bug to me... Thanks!! -- Matt Vander Werf

2 3