June 2017 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

Need help with upgrade from 4.3.17 to 4.3.20
by vadud3＠gmail.com 17 Jul '18

17 Jul '18

I am not sure what I need to after the upgrade for the columns to show data instead of something like this ELF �� >� �� @��@�� Þ ��@�8� �@�%�"� �� @��@�@��@�@��ø ��ø �� 8 ��8 @��8 @�� @��@��ää��ää�� I removed FollowSymLinks from xymon-apache.conf file under /xymon-cgi/ and /xymon-seccgi/ and restarted. I am using ubuntu -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

9 29

Recovery notifications not generated when test goes red->yellow->green?
by hobbit＠elyograg.org 13 Dec '17

13 Dec '17

It appears that if a test goes from the red state, to the yellow state, and then to the green state, no recovery notification is sent. If it goes directly from red to green, then we get a notification. Initial notifications are only sent on RED and PURPLE. We have recovery notifications configured for most tests. Xymon server is 4.3.21, built using makerpm.sh and installed with yum on CentOS 6. I do not know whether our previous version (4.3.14) behaved similarly. It was almost an accident that I discovered the problem at all. Is this expected behavior? If it is, can I change it? Thanks, Shawn

4 3

CLASS not working as expected
by steve＠opendium.com 11 Dec '17

11 Dec '17

Under Xymon 4.3.25, I'm trying to use the class attribute to group hosts together by what monitoring and alerting they need. So for each host I have a hosts.cfg line like: 192.0.2.1 somehost.example.com # CLASS:Foo trace ssh Then in analysis.cfg I have: CLASS=Foo PROC this PROC that And alerts.cfg has: CLASS=Foo SCRIPT=blah However, the tests in analysis.cfg are not being applied. If I replace CLASS=Foo with HOST=somehost.example.com they work as expected. Also, CLASS=linux seems to work, which implies that the CLASS: directive in the hosts.cfg is being ignored. As far as I can tell, the messages on xymon's "client" channel don't contain the class name that was specified in hosts.cfg, so xymond_client is using the class that the client sent instead. Other channels, such as "status" do include the class name that was specified in hosts.cfg. Am I misunderstanding how the class is supposed to be used, or is this a bug? -- - Steve Hill Technical Director Opendium Limited http://www.opendium.com Direct contacts: Instant messager: xmpp:steve at opendium.com Email: steve at opendium.com Phone: sip:steve at opendium.com Sales / enquiries contacts: Email: sales at opendium.com Phone: +44-1792-824568 / sip:sales at opendium.com Support contacts: Email: support at opendium.com Phone: +44-1792-825748 / sip:support at opendium.com

7 9

Alert transition yellow -> red with repeat problem
by john.r.rothlisberger＠accenture.com 27 Nov '17

27 Nov '17

This problem contributed to an outage last night - something is wrong. Last night we had a disk that was in the warning state - warning email sent. That disk then went into an alert state and an alert email was triggered (right away as the DURATION value took into account the time it was yellow) and then again 15 minutes later as designed. THEN - another disk on that server went yellow. It did NOT trigger any emails (as expected we should continue to focus on the alerts) but it somehow interfered with the REPEAT time of the alerts and those STOPPED. There is a bug somewhere. Thanks, John Upcoming PTO: _____________________________________________________________________ John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture 312.693.3136 office _____________________________________________________________________ From: Rothlisberger, John R. Sent: Tuesday, April 11, 2017 7:49 AM To: 'xymon >> xymon at xymon.com' <xymon at xymon.com> Subject: Alert transition yellow -> red with repeat problem This is a problem I have seen for a long long time and have actually brought it up on the list before. Xymon 4.3.21 Ubuntu 14.04LTS The problem I have (in this instance) a warning which is to be repeated daily suddenly goes red and triggers a single alert but doesn't repeat again until the repeat time of the warning has passed. >From the notification.log: Tue Mar 28 05:31:28 2017 ServerA.disk (IP) disk_warn 1490697082 100 <- warning sets a repeat time of 1 day Tue Mar 28 05:53:32 2017 ServerA.disk (IP) disk_alert 1490698405 100 <- minutes later it goes red (red repeat time is 15 minutes but no further alerts are generated) Next alert comes out 1 day after the above warning: Wed Mar 29 05:31:31 2017 ServerA.disk (IP) disk_alert 1490783486 100 <- 1 day after previous warning. This should have been repeated every 15 minutes. Wed Mar 29 05:46:40 2017 ServerA.disk (IP) disk_alert 1490784394 100 <- now, the repeat time is 15 minutes ... <- and continues every 15 minutes. This alert went a full 24 hours with only a single notification. :( I have seen this before (not always) where the repeat time in a warning overrides a follow up alert until the warning repeat time has expired. Alert rules: SCRIPT /home/xymon/server/ext/pg/exwarn_SCRIPT disk_warn DURATION>30 REPEAT=1d COLOR=yellow SERVICE=disk FORMAT=TEXT UNMATCHED SCRIPT /home/xymon/server/ext/pg/exalert_SCRIPT disk_alert DURATION>20 REPEAT=15 COLOR=red SERVICE=disk FORMAT=TEXT UNMATCHED Ideas/thoughts? Thanks, John Upcoming PTO: 4/3 _____________________________________________________________________ John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture 312.693.3136 office _____________________________________________________________________ ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com

3 5

Windows Powershell client v2.21
by zak.beck＠accenture.com 20 Oct '17

20 Oct '17

Hi v2.21 of XymonPSClient has been committed. This has a couple of new minor features and a minor fix, as follows: * Added reporting data to the server using UTF8 encoding. This is optional and the default remains to use ASCII. Use at your own risk - there's a possibility that not all server compiles will support receiving UTF8. (thanks Luboš Pavlíček) * Added the ability to include 'local' data in the core client message. This replicates functionality found in the Linux client - basically you can have external scripts write data to a file under the local folder (location specified by setting localdatalocation) and it will get written to a section in the client message like [local:<filename>]. You can then use a command like this to retrieve it: xymon 0 "clientlog <client host> section=local:<filename>" (thanks David Boyer for the inspiration!) * Fixed the Powershell provider appearing in file, log, dirsize and other tests - you would have only seen this if you were using UNC paths, it would have appeared like this: [file:Microsoft.PowerShell.Core\FileSystem::\\server\share\filename.txt], it should now appear like this: [file:\\server\share\filename.txt] I have also updated the documentation to include the new features. I have also updated nssm.exe in the repository. If you are running the client on Windows 10 Creator's Edition, you will need to use this new updated nssm.exe. Thanks Zak ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com

2 5

DNS failures causing "runtime longer than time limit"
by jlaidman＠rebel-it.com.au 11 Sep '17

11 Sep '17

Hi I'm running Xymon v4.3.10 on Linux, and I'm quite sure it's compiled with c-ares support. I have 12 new DNS servers that were added to Xymon about one month ago. All of my server entries in hosts.cfg have "testip". The tasks.cfg runs xymonet with "--dns-timeout=3". The hosts entries look like so: 10.10.10.1 dnshost1.example.com # testip dns=NS:example.com,SOA: example.com About a week ago, connectivity to all of these servers failed, and at the same time, the xymonnet run time jumped from less than 15 seconds to about 330 seconds, so about 315 seconds extra. The xymonnet page says 295 seconds is taken up by DNS tests. If the increase in time taken is about 315 and is entirely due to the 12 servers failing, then each failed server is adding about 26 seconds to the total run time. I don't think this should be happening like this. With two DNS checks per server, the DNS checks should be taking 6 seconds each to time-out, not 26. If I run xymonnet with "--timing --no-update" and specify only one hostname, I can view the results and the timing. This shows that the ping check gets reported after about 3 seconds, and then the DNS tests are executed and take 26 seconds total. My naiive assumption was that when a server failed a ping (and didn't have "noclear" defined in hosts.cfg) then the network checks would be skipped. On re-reading the man page for hosts.cfg, it dawned on me that a failed ping simply suppresses failed test /results/, but doesn't stop the tests from being run. So the real problem is that the "--dns-timeout=3" is not being taken into consideration by xymonnet. If I run xymonnet with "--debug" it tells me: 1900 2015-06-03 12:02:20 ares_search: tlookup='example.com', class=1, type=2 1900 2015-06-03 12:02:20 ares_search: tlookup='example.com', class=1, type=6 1900 2015-06-03 12:02:20 Processing 0 DNS lookups with ARES 1900 2015-06-03 12:02:46 Finished ARES queue after loop 423 This is peculiar. Why would it say "processing 0 DNS lookups" when there are two lookups to test? Could this be because xymonnet hasn't actually been built with ARES support and I didn't know it? Is there a good way to tell? If I add "--no-ares" I get the same results perhaps suggesting a lack of ARES support. On the other hand, if I add "timeout:3" and "attempts:1" into resolv.conf, I also get the same results. If I run "nm /path/to/xymonnet | grep gethostby" it returns "ares_gethostbyname". Just for fun, I compiled Xymon v4.3.21 and ran the xymonnet binary from there, with no change in behaviour. I also tried removing the "--dns-timeout" option so that it defaults to 30 seconds, but still no change - 26 seconds for two DNS tests. So, I'm not really sure what the problem is, but xymonnet certainly isn't behaving as I would expect. Cheers Jeremy

2 3

Powershell external scripts
by sgb＠pics.com 04 Jul '17

04 Jul '17

Hi, What format to external scripts <externalscriptlocation> need to be? Can they be windows bat files or do they need to be powershell .ps1 scripts or something else? Thanks Steve

3 6

False SSL cert alerts
by zforray＠vcu.edu 29 Jun '17

29 Jun '17

We are constantly having issues with sslcert alerts going non-green eventhough it says the cert is fine. Related to this is there being an issue getting to the https page from the Xymon server yet I can access it just fine from my browser. The latest one is an http only check - no Xymon client. The host file shows: 128.172.23.196 https:quikfm.vcu.edu # testip https://quikfm.vcu.edu Page is fully accessible. sslcert page says: Mon Jun 26 06:06:48 2017 [image: green] SSL certificate for https://quikfm.vcu.edu/ expires in 315 days Server certificate: subject:/C=US/postalCode=23284/ST=VA/L=Richmond/street=box 843059/street=701 W Broad Street/O=Virginia Commonwealth University/OU=Business Applications support/CN=quikfm.vcu.edu start date: 2015-05-08 00:00:00 GMT expire date:2018-05-07 23:59:59 GMT key size:2048 issuer:/C=US/ST=MI/L=Ann Arbor/O=Internet2/OU=InCommon/CN=InCommon RSA Server CA To make matters even more confusing, I flushed out all history and now the sslcert info isn't there and it still says there is an SSL error. How can I figure out why I get SSL ERROR ? Any help would be appreciated in figuring this out. Xymon server is at 4.3.19. -- *Zoltan Forray* Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator Xymon Monitor Administrator VMware Administrator Virginia Commonwealth University UCC/Office of Technology Services www.ucc.vcu.edu zforray at vcu.edu - 804-828-4807 <(804)%20828-4807> Don't be a phishing victim - VCU and other reputable organizations will never use email to request that you reply with your password, social security number or confidential personal information. For more details visit http://infosecurity.vcu.edu/phishing.html

5 10

xymon-powershell.ps1
by sgb＠pics.com 29 Jun '17

29 Jun '17

Hi Is there a way to send an html table through the xymon-powershell.ps1 script? I currently use bb-central where I have a Linux script that runs on the xymon server. I'm able to send the html table with the following OBJFILE="<iframe name=\"locktbl\" src=\"/hobbit/dba/${RHOST}/${MNAME}-lock.html\" frameborder=\"no\" height=\"600\" width=\"1200\"></iframe>" LINE="status $MACHINE.$TEST $COLOR ${OBJFILE} " $BB $BBDISP "$LINE" #SEND IT TO BBDISPLAY The status line is very similar to the xymon-powershell.ps1 input powershell -file xymon-powershell.ps1 %XYMSRV% "status %COMPUTERNAME%.mytest green %DATE% %TIME%" "The status of 'mytest' is 'satisfactory' at %DATE% %TIME%" "" "here's the last line" I basically want to replace the "The status of 'mytest' is 'satisfactory' at %DATE% %TIME%" with an html table. Thank you for your help Steve

3 9