December 2014 - Xymon - Xymon mailman web

Mutiple Xymon Servers
by everett.vernon＠gmail.com 17 Dec '14

17 Dec '14

Hi all We have Xymon up and running, but want a second server in our other DC. How do we set up a 2nd server, and (hopefully) keep them in sync? Is there a document on how to set this up anywhere? Thanks Vernon -- "Accept the challenges so that you can feel the exhilaration of victory" - General George Patton

4 3

memswap group alerting doesn't work
by Dan.Kessenich＠landsend.com 17 Dec '14

17 Dec '14

Xymon Mailing List, I can't seem to get any alerts by group on memory tests. It works fine for DISK tests but not MEMSWAP. My only thought is that memswap is part of the memory service and not a service of its own, so maybe this messes up the alerting. Is this by design or am I missing something? [xymon at leusxy02 clients]$ /opt/xymon/server/bin/xymoncmd --version Xymon version 4.3.10 etc/alerts/etl.cfg GROUP=etl_team MAIL Administrators at mycompany.com REPEAT=60 SCRIPT /opt/xymon/server/ext/smsplus.sh pageetl REPEAT=15 COLOR=red etc/clients/etl.cfg HOST=leuset03 MEMSWAP 90 95 GROUP=etl_team INODE * 80 90 GROUP=etl_team DISK / 90 95 GROUP=etl_team DISK /tmp 90 95 GROUP=etl_team DISK /home 90 95 GROUP=etl_team DISK /var 90 95 GROUP=etl_tea I've run some tests and it all looks good. /opt/xymon/server/bin/xymoncmd xymond_alert --test leuset03 memory --group=etl_team --color=yellow ... 00029871 2014-12-17 12:51:30 *** Match with 'GROUP=etl_team' *** 00029871 2014-12-17 12:51:30 Matching host:service:dgroup:page 'leuset03:memory:NONE:ETL' against rule line 364 00029871 2014-12-17 12:51:30 *** Match with 'GROUP=etl_team' *** 00029871 2014-12-17 12:51:30 Mail alert with command 'mail -s "Xymon memory:leuset03 warning (YELLOW) [12345]" Administrators at mycompany.com<mailto:Administrators at mycompany.com>' ... It catches the etl_team in the alerts file [xymon at leusxy02 alerts]$ /opt/xymon/server/bin/xymoncmd xymond_client --test 2014-12-17 12:47:55 Using default environment file /opt/xymon/server/etc/xymonserver.cfg Hostname (.=end, ?=dump, !=reload) []: leuset03 Hosttype []: Test (cpu, mem, disk, proc, log, port): mem Phys: Yellow at 100, red at 101 Swap: Yellow at 90, red at 95 Act.: Yellow at 90, red at 97 As you can see by the default of 80/90, the xymond_client test is picking up the thresholds of 90/95 from my etc/clients/etl.cfg file. DEFAULT UP 30m LOAD 5.0 10.0 DISK * 90 95 INODE * 70 90 MEMPHYS 100 101 MEMSWAP 80 90 MEMACT 90 97 Thanks for any help. Dan

1 0

RDP - unexpected service response - looking for alternative definitions for RDP in protocols.cfg
by glauber.ribeiro＠experian.com 17 Dec '14

17 Dec '14

Hello, I have a couple of servers in my xymon configuration that are responding with something unexpected to the RDP test. All the others work fine. RDP is ok on these servers, so it’s probably a version or protocol version problem. I don’t have a way to “sniff” the traffic to find out what’s wrong. These happen to be Windows 2008 R2 VMs, but I have other similar systems that work fine. Does anyone here have alternative sequences for RDP, for protocols.cfg, that I could try? Thanks! glauber

1 0

xymon on a large architecture
by leo＠minas.fr 17 Dec '14

17 Dec '14

hello what kind of hardware do you use ? 300msg/s is not that much. One of my cluster is monitoring 12 000 devices but only network test. I will check one with 3000 devices to see. oau <div>-------- Message d'origine --------</div><div>De : fmaillard.ext at orange.com </div><div>Date :17/12/2014 17:04 (GMT+01:00) </div><div>À : xymon at xymon.com </div><div>Objet : [Xymon] xymon on a large architecture </div><div> </div>Hello, We’re running a quite large xymon setup, and have been dealing with performance issue for quite a while. Here are some stats to give an idea about the setup: - We have 2 xymon servers per datacenter, on 3 datacenter (all messages are sent to both servers for a given site) - Each xymon server receives on average between 200msg/s and 250msg/s. We’re getting peaks at 400msg/sec. - Each site hosts about 3000 hosts / 30 000 services We’ve been suspecting for a long time that we might be losing messages… and I think I finally tracked it down to xymond not fetching the messages quickly enough so that the kernel’s buffer fill up and messages get discarded (by the kernel). On one of our servers, even though I have already increased net.ipv4.tcp_rmem and net.ipv4.tcp_wmem I got the following output from “netstat -s”: 148909 packets pruned from receive queue because of socket buffer overrun 4453143 packets collapsed in receive queue due to low socket buffer And here I come to the question I’m having: 1/ Is 250msg/s too much for a single xymond instance? Is anyone running instances with a lot more traffic than that? 2/ I’m starting to look into running several instances of xymond on the same machine, by binding them to different ports. Another option is to set up new machines, but that would mean migrating history files (several million files), sorting out the firewalling issues (our xymon interfaces are deeply connected to our information system) so I’d rather like avoiding this option. Are there any guidelines on how to do this? 3/ Are there any settings and best practice that could improve performance? For instance, should we move to a massive use of combo statuses in order to lessen the number of messages received? Best regards, Francois Maillard Pilote des plateformes Supervision, DNS & FTP - Sysadmin Infrastructure Altran Méditerranée pour Orange/OF/DTSI/DSI/DFY/HBX Sophia Antipolis tél. 04 97 12 87 53 fmaillard.ext at orange.com _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

1 0

xymon on a large architecture
by fmaillard.ext＠orange.com 17 Dec '14

17 Dec '14

Hello, We're running a quite large xymon setup, and have been dealing with performance issue for quite a while. Here are some stats to give an idea about the setup: - We have 2 xymon servers per datacenter, on 3 datacenter (all messages are sent to both servers for a given site) - Each xymon server receives on average between 200msg/s and 250msg/s. We're getting peaks at 400msg/sec. - Each site hosts about 3000 hosts / 30 000 services We've been suspecting for a long time that we might be losing messages... and I think I finally tracked it down to xymond not fetching the messages quickly enough so that the kernel's buffer fill up and messages get discarded (by the kernel). On one of our servers, even though I have already increased net.ipv4.tcp_rmem and net.ipv4.tcp_wmem I got the following output from "netstat -s": 148909 packets pruned from receive queue because of socket buffer overrun 4453143 packets collapsed in receive queue due to low socket buffer And here I come to the question I'm having: 1/ Is 250msg/s too much for a single xymond instance? Is anyone running instances with a lot more traffic than that? 2/ I'm starting to look into running several instances of xymond on the same machine, by binding them to different ports. Another option is to set up new machines, but that would mean migrating history files (several million files), sorting out the firewalling issues (our xymon interfaces are deeply connected to our information system) so I'd rather like avoiding this option. Are there any guidelines on how to do this? 3/ Are there any settings and best practice that could improve performance? For instance, should we move to a massive use of combo statuses in order to lessen the number of messages received? Best regards, Francois Maillard Pilote des plateformes Supervision, DNS & FTP - Sysadmin Infrastructure Altran Méditerranée pour Orange/OF/DTSI/DSI/DFY/HBX Sophia Antipolis tél. 04 97 12 87 53 fmaillard.ext at orange.com<mailto:fmaillard.ext at orange.com> _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

1 0

Sending SNMP information
by everett.vernon＠gmail.com 17 Dec '14

17 Dec '14

Hi all At $CLIENT, an "executive decision" was made to do all monitoring using $OTHER_TOOL. $OTHER_TOOL uses only SNMP. Although the results in Wintendo-land are acceptable, what we get in Solaris-land is not. As a compromise, it was resolved to keep Xymon for Solaris-land for admin team monitoring and diagnostics, but get it to send critical elerts to $OTHER_TOOL in the form of SNMP traps. This might not even make sense, but somehow, probably using the SCRIPT directive in alerts.cfg, I need to notify $OTHER_TOOL of critical events. Any tips on how? Regards Vernon -- "Accept the challenges so that you can feel the exhilaration of victory" - General George Patton

2 2

html_mail.pl colour
by bdale＠kitchen-net.com.au 17 Dec '14

17 Dec '14

Just gave that a try changed the paths to use the URL and also tried the full local path on the server without using those BB/xymon variables in the perl script still getting the same problem. I must be missing something really obvious somewhere. Regards, Brandon From: Galen Johnson [mailto:Galen.Johnson at sas.com] Sent: Wednesday, 17 December 2014 11:16 AM To: Brandon Dale; xymon at xymon.com Subject: RE: html_mail.pl colour You probably need to tweak the perl code to use the full url for your Xymon server...something like <body background="http://example.com/xymon/gifs/bkg-red.gif"> Of course, the script says it's embedding it in the message so I'm likely off base here. =G= ________________________________ From: Xymon <xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>> on behalf of Brandon Dale <bdale at kitchen-net.com.au<mailto:bdale at kitchen-net.com.au>> Sent: Tuesday, December 16, 2014 6:42 PM To: xymon at xymon.com<mailto:xymon at xymon.com> Subject: [Xymon] html_mail.pl colour I am trying to setup HTML email alerts, I found this script on xymonton which appears to be exactly what I need https://wiki.xymonton.org/doku.php/alerts:html_mail I followed the instructions and added it to my alerts.cfg. It successfully sends HTML emails, however it doesn't seem to contain a colour to match the alert which I think it should? I assumed I would get a line down the left side similar to what you see on the xymon webpages. This is what I get emailed: [cid:image001.png at 01D019EC.8E747F00] If I look at the HTML source for the email I think based on this I can assume the scripts logic is successfully matching the alert colour to the correct .gif it needs: <body background="cid:bkg-red.gif"> I checked on the xymon server and these files appear to exist where they need to: usr/lib/xymon/server/www/gifs# ls arrow.gif bkg-red.gif clear.gif favicon-purple.ico green-recent.gif red-ack.gif xymonbody.css bkg-blue.gif bkg-yellow.gif clear-recent.gif favicon-red.ico purple-ack.gif red.gif yellow-ack.gif bkg-clear.gif blue-ack.gif favicon-blue.ico favicon-unknown.ico purple.gif red-recent.gif yellow.gif bkg-green.gif blue.gif favicon-clear.ico favicon-yellow.ico purple-recent.gif unknown.gif yellow-recent.gif bkg-purple.gif blue-recent.gif favicon-green.ico green.gif README unknown-recent.gif zoom.gif Am I doing something obivously wrong? And how do I go about troubleshooting this Note: - This example is from a custom test, but I don't think that could be the cause of the problem as the script is determing the colour of the alert as expected. - Xymon 4.3.17 running on Debian 6 Brandon

2 2

Host Name Links
by everett.vernon＠gmail.com 15 Dec '14

15 Dec '14

Hi all Some time back, there was a feature, hwereby the host name could become a link, perhaps to an as-built, or a CMDB entry for the host. I used it once before at another customer. I can't recall how to do this, and I can't seem to find the notes on how. Can anybody refresh my memory? Regards Vernon -- "Accept the challenges so that you can feel the exhilaration of victory" - General George Patton

5 8

NAME tag
by jlaidman＠rebel-it.com.au 12 Dec '14

12 Dec '14

I don't know if it will fix the problem, but it might provide a suitable alternative to use the "COMMENT:" or "DESCR:" tags. On 9 December 2014 at 18:51, DNS via Xymon <xymon at xymon.com> wrote: > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon > > > ---------- Forwarded message ---------- > From: DNS <dns1407 at yahoo.com> > To: Xymon Mailinglist <xymon at xymon.com> > Cc: > Date: Tue, 9 Dec 2014 07:46:01 +0000 (UTC) > Subject: NAME tag > Hi All, > > I'm using xymon 4.3.17 and has a issue with NAME tag in hosts.cfg. > When defining one host multiple times in the hosts.cfg I want to give them > different names. I use the NAME tag, but this doesn't change the display > name on the webpage. Does anyone has familiar problem? Or is there a > solution for this? > > It looks like this in the hosts.cfg: > > group-except info ORG > 10.1.1.1 hostnameorg > > group-only info ALIAS > 0.0.0.0 hostnameorg # noping NAME:"NAMEIWANT" > > On the webpage it still displaying "hostnameorg" for my second entry. > > Greeting.... > > DNS > > > > > > >

3 2

SIGPIPE on xymonproxy
by sean.clark＠twcable.com 11 Dec '14

11 Dec '14

I am running the latest version of xymon, with J.C. Cleaver's patches. I have a xymonproxy that sits dual homed, forwarding messages from one network segment to the main xymon display. It's roughly 4,000 hosts, about half using bbwin as their client, the other half are linux xymon clients. The proxy is mostly working, but is dropping/retransmitting messages as well – the log indicates a problem with getting SIGPIPE like this: 2014-12-11 10:29:28.204598 conn 8182371, socket 238: error writing to socket: Broken pipe 2014-12-11 10:29:28.204608 conn 8182371, socket 238: attempting recovery from write error to server 0 (192.168.0.100:1984): Broken pipe 2014-12-11 10:29:28 Got SIGPIPE... 2014-12-11 10:29:28.208113 conn 8179105, socket 131: error writing to socket: Broken pipe 2014-12-11 10:29:28.208118 conn 8179105, socket 131: attempting recovery from write error to server 0 (192.168.0.100:1984): Broken pipe I've played around with lqueue settings, and timeout settings, to no avail – it seems to constantly be posting those messages in log no matter what. Ulimit for open files for xymon is 65535, and I don't see it going above 200-300 open files The report lists this under resources: (also looks like there is a spelling error for resource in xymonproxy.c, which is duplicated in the man pages) Proxy ressources - Connection table size : 134 - Buffer space : 6504124 kByte Timeout/failure details - reading from client : 24 - connecting to server : 0 - sending to bfq : 0 - bfq recovered : 0 - sending to server : 593820 - recovered : 53130 - reading from server : 24 - sending to client : 0 On the receiving server, it tells me in xymond.log --> Discarding timed-out partial msg from X.X.X.X Wondering what the next steps I can take for troubleshooting/fixing — any thoughts? Tweaks on receiving server? Changes to xymonproxy? Network pathing issues? -Sean ________________________________ This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

1 0