October 2012 - Xymon - Xymon mailman web

Brainstorming - open xymon tickets on *acknowledgement*? (and Xymon talking to Jira and ticket systems) (and a bit of Nagios)
by betsy.schwartz＠gmail.com 19 Oct '12

19 Oct '12

We're using Jira for ticketing, and Xymon for monitoring. Jira allows opening tickets via email, so we have a one-way communication. I've been asked (by someone coming from the world of Nagios) to investigate opening tickets on *ACKNOWLEDGE*. His reasoning is that if a ticket is opened on ACK, it is clear that every acked alert has a ticket and it is clear what ticket goes with the ack, and all tickets will have owners. If we open a ticket via email it might or might not have an owner right away, and there's no way to match the ticket with the dot on the screen. Also if we open tickets via email, we get multiple tickets when a service *flaps*, such as a disk that is cyclically going above the limits. Has anyone else worked with these issues? Thoughts I have so far: 1) process. Make sure that everyone who takes a ticket also acks the alert manually with the ticket number. This is clunky. 2) hook into the email ack (not currently enabled). When procmail parses the ack, trigger the ticket open email. Difficulty: alerts.cfg has logic to control which group gets which tickets; with procmail I am starting from scratch. 3) switch to Nagios (Not a short term project but it's on the table) 4) use a custom alert script that both sends the ticket email and sends an email ack. (difficulty: multiple tickets for a flapping service would do ... what?) (I think this is the most promising idea so far...) Interested in any ideas, thoughts, and ramblings on the subject(s)

2 1

Customising message subject in email alerts
by abs＠shadymint.com 19 Oct '12

19 Oct '12

Hi, This has been talked about a few times before and I am aware that we have the SCRIPT mechanism available for this purpose. However, what I really need to be able to do is annotate the subject line with the page where the object is located. The environment variables do not currently provide this information and I have been playing a little with findhost.cgi to see if it can be used within a script to extract the page. This has not been successful and anyway is flawed because the object may appear on multiple pages. I have even experimented with fetching hosts.cfg and parsing the pages which kindof works, but there must surely be a better way? Anyone had similar problems or got any other suggestions? Thanks -- Andy

2 1

Quick urgent question - *IS* there an issue with combo tests in 4.3.10?
by betsy.schwartz＠gmail.com 19 Oct '12

19 Oct '12

I'm scheduled to upgrade to 4.3.10 this weekend so if there's an issue with combo tests I need to verify that it won't affect us What are people seeing? Working ok for you? Not OK? sign/symptom/test case? thanks so much!!

2 1

Xymon Wiki Pages
by josh＠imaginenetworksllc.com 19 Oct '12

19 Oct '12

Who is responsible for this? http://en.wikibooks.org/wiki/System_Monitoring_with_Xymon I've been trying to make changes to the compiling guide for CentOS and since January I've not gotten my changes reviewed. Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373

2 3

diskstat.sh/RRD oddity
by sholmes42＠mac.com 18 Oct '12

18 Oct '12

This is just a comment on an oddity with respect to diskstat.sh and RRD. We make pretty heavy use of the diskstat.sh script, which I believe I downloaded from xymonton. When I installed it I used the standard clientlaunch.cfg stanza for the configuration and everything worked great. I was called to task today because we have been having some disk io issues on the RHEL VMs and someone was looking at the trend graphs for some servers to see if there was anything they could learn and they noticed that beginning at about 4pm local time on Monday the graphs for the number of sectors written per second on a couple of file systems on several VMs jumped from the 10 to 20 range to the 300 to 340 range and stayed there. The graph for number of disk writes per second had a corresponding jump up to about 40 or 50 from close to zero. In analyzing the data I discovered that the file system that was displaying this behavior is the same file system to which the diskstat.sh script is writing its temp files. It appears that for some reason, starting at 4pm on Monday the 5 minute test interval and the 5 minute average for RRD got in sync and all it was seeing was the data point that corresponded to its own writing activity and RRD was using it for the entire 5 minute average (of course, that's what RRD does). I 'fixed' it by changing the test interval to something less than 5 minutes. I tried 2, 3, and 4 minutes and they all had the effect of reducing the data in the plot back to the expected level, i.e. to the level it was before 4pm on Monday. The mystery remains why it suddenly started seeing and using its own disk activity at the same time on several different servers. Steve Holmes ITaP/Purdue University -- If they give you ruled paper, write the other way. -Juan Ramon Jimenez, poet, Nobel Prize in literature (1881-1958) I prayed for freedom for twenty years, but received no answer until I prayed with my legs. -Frederick Douglass, Former slave, abolitionist, editor, and orator (1817-1895)

5 10

Metrics Reports
by rblumenthal＠telmar.com 17 Oct '12

17 Oct '12

When trying to run a metrics report with multiple servers it cannot display the graph, however it works when choosing 1 server. Any insight on where I should check? Thanks

1 0

xymon failure this am
by betsy.schwartz＠gmail.com 17 Oct '12

17 Oct '12

Still on 4.3.7, on RHEL5. Had a custom test fail and hang last night, a perl script called from tasks.cfg . Not sure if this is related. But six hours later xymon started failing wtih 2012-10-17 06:51:00 Whoops ! Failed to send message (Connection failed) 2012-10-17 06:51:00 -> Could not connect to Xymon daemon at 10.100.5.42:1984 (Connection refused) 2012-10-17 06:51:00 -> Recipient '10.100.5.42', timeout 15 drilling into detail pages was also failing. Resetting fixed it and nobody thought to run the self test. Betsy PS the self test I created was a file monitor that looks for /home/xymon/TESTOK . If that file exists it sends email to whatever addresses are in /home/xymon/testalert.txt . All of our support team has privilege to touch those two files, and the rule is the last one in alerts.cfg . Created this after munging up alerts.cfg so some pages were not going out.

1 0

Plagued by purple
by David.Mills＠ngc.com 16 Oct '12

16 Oct '12

> Josh Luthman josh at imaginenetworksllc.com <mailto:xymon%40xymon.com?Subject=Re%3A%20%5BXymon%5D%20Plagued%20by%20purple&In-Reply-To=%3CCAN9qwJ-iF_r5AsQx5HFnpgK%3DPDnr0qtt1m%3DxnGjKdnVAFGwzSA%40mail.gmail.com%3E> > >Either the server isn't gathering it or the clients aren't sending the data. > >The raid script that I have is sent by the client. Thanks, Josh, but can you point out where I can check to see which of those options is my likely culprit? Also, don't quite follow your comment about your "raid script". What raid script? Sorry! David >On Oct 16, 2012 2:08 AM, "Mills, David (IS)" <David.Mills at ngc.com> wrote: > >> All . >> >> Am fairly new to Xymon and apologize if this is documented somewhere and I >> missed it. >> >> I have inherited a Xymon server (4.3.3 on Solaris 10) which shows itself >> as having about half its tests as .purple., the other half as green. As far >> as I can tell, everything.s running just fine. It seems to be a simple data >> refresh problem of some sort, but haven.t been able prove my hunch. >> >> The problem started after a reboot of the server. Everything came up OK, >> except for the following list of .missing. tests: >> >> Purple Tests: >> iostat iostat2 mem2 meta nfs prtdiag raid sar tm zones >> >> Green Tests: >> bbd conn cpu disk http info memory msgs >> ntp ports procs ssh trends >> >> I.ve restarted the server: (.$xymon_home/xymon.sh restart.) and tried to >> find something in the logs, but am not making ground on this fast enough. >> >> Help? >> >> Thanks! >> >> david ~~~~~~~~~~~~~~~~~~~ David Mills Systems Administrator Northrop Grumman 512-595-1238 david.mills at ngc.com

5 5

Xymon status does not get updated
by y.goudetsidis＠mail.cryst.bbk.ac.uk 16 Oct '12

16 Oct '12

Hi John | Do you have anything like '--status-senders' set for xymond in your | tasks.cfg file? If so, is this IP address listed? No, I do not have anything like that in tasks.cfg. I haven't touched this file. | Are you using multiple Xymon servers or should this ('0.0.0.0') be | whatever XYMONSERVERIP is set to in xymonserver.cfg? There is no secondary xymon server but there is a big brother running. My task at the moment is to gradually retire big brother and move everything to Xymon but I still have a LONG way to go. | When I had accidentally restricted access to our Xymon server, I | received errors such as 'connection refused'. In your case you get a | timeout, which tends to imply it is not connecting. If iptables and | selinux have been stopped, then I would check the settings in the | 'server/etc' files to ensure that your Xymon server is configured | correctly. I haven't done anything like that in my case. After running: $killall -9 xymond The main xymon page gets updated but the xymond column is yellow. (???????) I am full of questionmarks.

3 8

Xymon status does not get updated
by y.goudetsidis＠mail.cryst.bbk.ac.uk 16 Oct '12

16 Oct '12

Hi everyone 2 days ago I updated xymon to the latest version:4.3.10 and for some reason unknown to me xymond stopped updating the hosts.cfg file. Please see xymongen.log below: 2012-10-10 16:53:58 -> Recipient '193.61.54.144', timeout 15 2012-10-10 16:53:58 -> 1st line: 'xymondboard fields=hostname,testname,color,flags,lastchange,logtime,validtime,acktime,disabletime,sender,cookie,line1,acklist' 2012-10-10 16:53:58 xymond status-board not available, code 7 2012-10-10 16:53:58 Failed to load current Xymon status, aborting page-update Could somebody please help me? Many thanks Yanni

3 7