June 2018 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

[Patch] xymond_rrd crash when parsing Linux ifconfig data
by steve＠opendium.com 22 Feb '19

22 Feb '19

This patch addresses two problems: 1. For Linux clients, do_ifstat_rrd parses the ifconfig output using the regex: ([a-z0-9]+(_[0-9]+)?:*|lo:?)\s pickdata() is called like this: pickdata(bol, ifstat_linux_pcres[0], 1, &ifname) With a network interface name such as "foo1", the regex produces 2 results: 0: "foo1 " 1: "foo1" pickdata() discards result 0 and assigns result 1 ("foo1") to the ifname parameter as expected. With a network interface name such as "foo_1", the regex produces 3 results: 0: "foo_1 " 1: "foo_1" 2: "_1" pickdata() discards result 0, assigns result 1 ("foo_1") to the ifname parameter and then tries to assign result 2 ("_1") to the next parameter in the variable argument list. Since there it no such parameter, this results in a segmentation fault and xymond_rrd crashes. The attached patch adds a "num_args" parameter to pickdata() to tell it how many arguments are in the variable argument list and ensure it never rolls off the end of that list. 2. I have some clients with network interface names in the form "foo.5", "foo_bar" and "foo-bar" ("eth0.5" is especially common when eth0 is carrying an 802.1q trunk, for example). Xymon does not collect statistics on those network interfaces because they don't match the regex. This patch relaxes the regex a bit so that it matches these network interfaces. -- - Steve Hill Technical Director Opendium Online Safety / Web Filtering http://www.opendium.com Enquiries: sales at opendium.com +44-1792-824568 Support: support at opendium.com +44-1792-825748 Opendium Limited is a company registered in England and Wales. Company No. 5465437 Highfield House, 1 Brue Close, Bruton, Somerset, BA10 0HY, England.

2 1

Project: Xymon dashboard
by herzog＠phys.ethz.ch 08 Nov '18

08 Nov '18

Hi all, for the past 3 weeks we've been working on a new Xymon web frontend that better fits our needs of managing our somewhat extensive (~ 900 hosts) setup. It's now in a state of being ready for daily use, so we would like to announce it in case somebody else is interested: https://github.com/daduke/Xymondash Let us know if you have comments or even pull requests! cheers, -Christian -- Dr. Christian Herzog <herzog at phys.ethz.ch> support: +41 44 633 26 68 IT Services Group, HPT H 8 voice: +41 44 633 39 50 Department of Physics, ETH Zurich 8093 Zurich, Switzerland http://nic.phys.ethz.ch/

6 14

Setting thresholds in analysis.cfg
by chris.seip＠hpe.com 21 Aug '18

21 Aug '18

I could a hand getting the basics of analysis.cfg worked out, please. Here's mine: # egrep -v '^#' analysis.cfg HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 DISK /disk/data 50 55 DISK * 90 95 MEMSWAP 80 90 HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 DISK /disk/data 92 96 DISK * 90 95 HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 DISK /BACKUP 98 99 DISK * 90 95 DEFAULT # Ignore some usually uninteresting tmpfs mounts. DISK /dev IGNORE DISK /dev/shm IGNORE DISK /lib/init/rw IGNORE DISK /run IGNORE # These are the built-in defaults. You should only modify these # lines, not add new ones (no PROC, DISK, LOG ... lines). UP 1h LOAD 5.0 10.0 DISK * 90 95 INODE * 70 90 MEMPHYS 100 101 MEMSWAP 50 80 MEMACT 90 97 Three issues with this: 1. Swap consumption in the first host, swnfs06, has been steady at 74%, so I was trying to hush the alerts with the MEMSWAP line. This change hasn't had any effect; I am still getting a memory low yellow-warning for swap/page usage on swnfs06. 2. On the same swnfs06 host, its /disk/data partition is 56% full, so my "DISK..50 60" line was an attempt to trigger a yellow alert. I was testing my understanding of the analysis.cfg file, but the filesystems test remains green. 3. And my 96% full /BACKUP drive on hpnsvr18 is issuing a red alert for being over the panic level of 95%, where I was trying to set the panic level at 99%. After wrestling with the man page and many experiments, I'm tossing this to the group for help. Seems very basic, but it's just not working for me. What'm I missing? I tried switching to the "threshold hostname" format, like this: # egrep -v '^#' analysis.cfg | head -11 DISK /disk/data 50 55 HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 DISK * 90 95 HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 MEMSWAP 80 90 HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 DISK /disk/data 92 96 HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 DISK * 90 95 HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 DISK /BACKUP 98 99 HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 DISK * 90 95 HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 This produced no change in behavior. I am stopping and starting the Xymon server software and waiting for new html pages to generate after every change in the analysis.cfg. In my configuration report, I can see that every server configured for local memory tests has acquired the 80%/90% threshold setting, not just swnfs06. And my "DISK /disk/data 50 55" is having no effect at all on any host: The strings "50%" or "60%" appear nowhere in my configuration report. # egrep 'swnfs0[67]' hosts.cfg 16.93.247.204 swnfs06.rose.rdlabs.hpecorp.net # rpc=mount,nlockmgr,nfs,ypbind ssh 16.93.247.205 swnfs07.rose.rdlabs.hpecorp.net # NOCOLUMNS=files rpc=mount,nlockmgr,nfs,ypbind ssh 16.93.247.204 swnfs06.rose.rdlabs.hpecorp.net # xymoncmd xymond_client --dump-config DISK /disk/data 50% 55% 0 -1 red HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 (line: 351) DISK * 90% 95% 0 -1 red HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 (line: 352) MEMSWAP 80 90 HOST=%swnfs06.rose.rdlabs.hpecorp.net,%swnfs06 (line: 353) DISK /disk/data 92% 96% 0 -1 red HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 (line: 355) DISK * 90% 95% 0 -1 red HOST=%swnfs07.rose.rdlabs.hpecorp.net,%swnfs07 (line: 356) DISK /BACKUP 98% 99% 0 -1 red HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 (line: 360) DISK * 90% 95% 0 -1 red HOST=%hpnsvr18.rose.rdlabs.hpecorp.net,%hpnsvr18 (line: 361) DISK /dev IGNORE (line: 371) DISK /dev/shm IGNORE (line: 372) DISK /lib/init/rw IGNORE (line: 373) DISK /run IGNORE (line: 374) UP 3600 -1 (line: 377) LOAD 5.00 10.00 (line: 378) DISK * 90% 95% 0 -1 red (line: 379) INODE * 70% 90% 0 -1 red (line: 380) MEMREAL 100 101 (line: 381) MEMSWAP 50 80 (line: 382) MEMACT 90 97 (line: 383) Thanks for any insights you can provide. Feels like I'm making a wrong assumption about how analysis.cfg works. Best thing I can figure to do would be to switch to local configuration of my Xymon clients, but I'd rather manage custom thresholds centrally. - Chris

2 2

Problems with Content Security Policy in Safari, Chrome, and IE
by jtrott＠dancrai.com 17 Jul '18

17 Jul '18

Xymon 4.3.28-1.el7.terabithia with Safari 11 on High Sierra and Safari on iOS 11. Problem occurs on the trends page. https://xymon.domain.com.au/xymon-cgi/svcstatus.sh?HOST=host.com.au&SERVICE… If you click on any of the time based buttons, 48hrs for example, the requested page doesn't load. Safari on macOS look like it's loading a page but doesn't get anywhere. Safari on iOS does nothing at all when you tap the button. The console in Safari reveals the following error: Refused to load https://xymon.domain.com.au/xymon-cgi/svcstatus.sh?HOST=host.com.au&SERVICE… because it does not appear in the form-action directive of the Content Security Policy. Checking the headers shows this content security policy: Content-Security-Policy: script-src 'self' 'unsafe-inline'; connect-src 'self'; form-action 'self'; sandbox allow-forms allow-scripts; I'm not that well versed in the CSP stuff, but I note that it also fails with the same error in the latest Chrome 62.0.3202.89, and in Internet Explorer 11.0.9600.18817 (no error logged), but works in the latest Firefox 56.0.2. Has anyone else run into this issue, or has any more information on how I can modify the CSP headers to test? I tried using Header set Content-Security-Policy in apache but that seems to add an improperly formatted addition to the rules rather than overwriting them. Thanks, JT

7 11

Need help with upgrade from 4.3.17 to 4.3.20
by vadud3＠gmail.com 17 Jul '18

17 Jul '18

I am not sure what I need to after the upgrade for the columns to show data instead of something like this ELF �� >� �� @��@�� Þ ��@�8� �@�%�"� �� @��@�@��@�@��ø ��ø �� 8 ��8 @��8 @�� @��@��ää��ää�� I removed FollowSymLinks from xymon-apache.conf file under /xymon-cgi/ and /xymon-seccgi/ and restarted. I am using ubuntu -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

9 29

Interval for the combostatus task
by john.thurston＠alaska.gov 29 Jun '18

29 Jun '18

I use some combo-tests, and the delay in their reaction is sometimes a hassle. I just looked in the tasks.cfg and see I have my combostatus task set to run every 5 mins. 5 mins is the frequency of my xymonnet task, but many of my combo tests don't rely on xymonnet. I'm thinking I should shorten the interval on combostatus to 2 minutes, and thereby reduce the latency of those test results. Can anyone relate to me horror stories or good reasons I should not shorten this interval? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Department of Administration State of Alaska

2 1

xymondXboard syntax limitation - XMH tags?
by john.thurston＠alaska.gov 27 Jun '18

27 Jun '18

The xymondboard message type is shown in the docs with the following syntax: > xymondboard [CRITERIA] [fields=FIELDLIST] The xymondXboard message type is shown in the docs with the following: > Retrieves an XML string with the summary of all status logs as for the "xymondboard" command. but it looks to me like Xboard does not support the 'fields=' capabilities of board. I always get the same fields in my XML, regardless of how I craft my message. My quick look in source-code makes me think the values returned by Xboard are limited and hard-coded. I'd like to use Xboard, but I'd also like to have it return some XMH tags (like XMH_DGNAME, XMH_DISPLAYNAME). Can anyone suggest a way to do this? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Department of Administration State of Alaska

1 0

Project: Xymon dashboard
by herzog＠phys.ethz.ch 27 Jun '18

27 Jun '18

> First thing I've found is the xymon2json wants python3. I only had python2 installed. I added python3.4 but that did nothing. ok I'll add this requirement in the README. > As it stands, it doesn't work at all in Internet Explorer. it sure does, my colleagues are using it. > I do get the menu, reload and eye in the corner using firefox. The menu comes up but no servers or tests actually come up. is xymon2json producing the JSON? Open its URL in the browser and see if you get the output. Once this works, the JS interface will either show output or an error message. good luck, -Christian

5 17

Chrome cookie deprecations (fix/workaround)
by EDSchminke＠Hormel.com 19 Jun '18

19 Jun '18

Hi All, I wanted to share my fix/workaround for the "Chrome cookie deprecation". Put the following code above the closing body tag in the $XYMONHOME/web/stdnormal_footer template file. -------------------------------------------------------------------------------- <script type="text/javascript"> var metas = document.getElementsByTagName('meta'); for (var i=0; i<metas.length; i++) { var type = metas[i].getAttribute("HTTP-EQUIV"); if (type == "Set-Cookie") { var prop = metas[i].getAttribute("CONTENT"); document.cookie = prop; } } </script> -------------------------------------------------------------------------------- Chrome has disabled, as a security measure, the ability to set cookies using HTML tags. Xymon (as of v4.3.27) relies on HTML cookies to allow a user to go from a particular page of monitored hosts to either the "Acknowledge alert" or "Enable/Disable" pages and, based on the origin, filter the hosts/alerts for that page. I've noticed it, as have a few others in the Xymon community. This should get us by until a permanent fix can be implemented in a release version. Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com

4 4