May 2020 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

4.3.30 RPMs released
by cleaver＠terabithia.org 23 May '22

23 May '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 1

random errors with imaps/pop3s servers offering TLS 1.3
by lolo＠troll.free.org 10 Aug '21

10 Aug '21

Hi, I have found an issue when trying to monitor imaps and pop3s server offering TLS1.3 . My xymon server configuration: # freebsd-version -u 12.0-RELEASE-p5 # openssl version OpenSSL 1.1.1a-freebsd 20 Nov 2018 standard freebsd xymon-server pkg : # pkg info xymon-server-4.3.28 xymon-server-4.3.28 Name : xymon-server Version : 4.3.28 Installed on : Fri Feb 22 14:19:11 2019 CET Origin : net-mgmt/xymon-server Architecture : FreeBSD:12:amd64 Prefix : /usr/local Categories : net-mgmt www Licenses : GPLv2 Maintainer : feld at FreeBSD.org WWW : http://xymon.sourceforge.net/ Comment : System for monitoring servers and networks Options : DEBUG : off LDAP : off NETSNMP : off Shared Libs required: libcares.so.2 libpng16.so.16 libpcre.so.1 librrd.so.8 Annotations : FreeBSD_version: 1200086 cpe : cpe:2.3:a:xymon:xymon:4.3.28:::::freebsd12:x64 repo_type : binary repository : FreeBSD Flat size : 26.5MiB ... # ldd /usr/local/www/xymon/server/bin/xymonnet /usr/local/www/xymon/server/bin/xymonnet: libcares.so.2 => /usr/local/lib/libcares.so.2 (0x80027c000) libssl.so.111 => /usr/lib/libssl.so.111 (0x800297000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x80032c000) libpcre.so.1 => /usr/local/lib/libpcre.so.1 (0x800619000) libc.so.7 => /lib/libc.so.7 (0x8006bd000) libthr.so.3 => /lib/libthr.so.3 (0x800ab0000) When trying to monitor a pop3s or imaps server offering tls 1.3, I got random errors: WARNING: Flapping status Service imaps on xxx is not OK : Unexpected service response If I monitor the same services on an other server not offering tls 1.3, all is fine. Both servers where running dovecot. I wrote an extension in perl using IO::Socket::SSL (and the same local openssl) to monitor the server offerng tls 1.3 and all is fine with it. For some strange reason, I can monitor a https web servers (running nginx or apache) offering TLS 1.3 without this issue but not imaps/pop3s. It looks like the https test is different from other ssl/tls tests and does not have the TLS 1.3 issue Regards, -- Laurent Frigault | Free.org - BookMyName.com - ONLINE SAS - Registar ID 74

2 1

Too much data for svcstatus.sh?
by mburger＠bubbanfriends.org 12 Oct '20

12 Oct '20

Good morning. Running Xymon 4.3.30, compiled from source tarball, on a RHEL 6.10 server. This morning, I encountered an interesting issue. I have a monitored system that has 197 discrete filesystems/mounts. When I attempt to enter this server's "Trends" section, I receive an internal server error (500). My httpd error log shows this when I attempt to access the trends link for the server in question: [Thu May 21 11:29:41 2020] [error] [client 10.40.24.55] Premature end of script headers: svcstatus.sh, referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] *** glibc detected *** /apps/xymon/server/bin/svcstatus.cgi: realloc(): invalid next size: 0x0000000000de7dd0 ***, referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] /apps/xymon/server/bin/svcstatus.cgi[0x408a4f], referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] /apps/xymon/server/bin/svcstatus.cgi[0x403453], referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] /apps/xymon/server/bin/svcstatus.cgi[0x404bb7], referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] /apps/xymon/server/bin/svcstatus.cgi[0x4022c9], referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] 00400000-0043c000 r-xp 00000000 fd:08 24906 /apps/xymon/server/bin/svcstatus.cgi, referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] 0063c000-0063e000 rw-p 0003c000 fd:08 24906 /apps/xymon/server/bin/svcstatus.cgi, referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html This does not happen for any other monitored system in my environment. This system is one of the more critical servers in my environment, so I'd like to figure out what's going on and remediate it if possible. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1

4 4

Xymon 4.3.30 Released
by cleaver＠terabithia.org 31 Jul '20

31 Jul '20

Xymon 4.3.30 has been released and is now available for download. 4.3.30 is mostly a bug-fix release, quashing issues stemming from the security fixes in 4.3.29, including improperly-tight restrictions on allowed characters in hostname for browsing along with several other parsing errors. Thanks in particular to Tom Schmidt for his assistance in tracking these down. Xymon should also now be more easily buildable on older GCC versions without the diagnostics pragma available. Xymon 4.3.30 is available from the Xymon SourceForge page at https://sourceforge.net/projects/xymon/ As always, thank you to all who have contributed code, ideas, and features to the project! Regards, Japheth "J.C." Cleaver

2 3

Sorry to bother you ... But I was wondering if your could possibly Help us ...
by eric.prevost＠volvo.com 30 May '20

30 May '20

(2nd Sending as I had forgotten a Screenshoot for the Value I changed to 10000 in the Powershell File, so sorry about this) Hello ... Sorry to bother you ... But I was wondering if you could possibly Help us ... with a weird Issue we started having this Week () If so here it is: At some point for the past couple of night, we started receiving the following ALERT: &red XymonPSClient: No matching service - want started/automatic But when I look on the Windows Server, under the (Windows mmc) Service window, it is shown as Started ... But Xymon doesn't see it as such :( Now what is weird is that the ClientLog does list it, but under the SVCS button ... I don't see all of the Services almost all but the last portion is missing (depending on how many there are ... the more there are I guess the more I would be missing) ... So I was thinking that it might be related to some sort of a VARIABLE Type or Buffer Size (Limitation) that is just not big enough ... but when I looked at the "xymonclient.ps1", there are way too many lines for me to understand all of it an try to figure out what I would need to change :$ Also I only fiddle a Little bit in PowerShell so my skills are at best at the Beginner Level ... Thus this is why I am taking a Chance sending you this Email Here are a few Screenshot to try to show you the Issue and what I saw there leaning me towards the Incorrect Variable Type or Buffer Size [cid:image001.jpg at 01D635D0.096C71E0] [cid:image002.jpg at 01D635D0.096C71E0] As you can see here, the last Line seems to be Incomplet (cuts in the middle) ... That is Compared to the Content of the Client Log screen to the right ... thus making me believe that the Data is bigger than what it can Hold/Process ... thus it disregards anything coming after that ! [cid:image003.jpg at 01D635D0.096C71E0] I am hoping that this is something that you guys might have seen before and know of the Way to Fix it Now I did try to modify the following Value from 5000 to 10000 (a shot in the Dark), but it did not seem to Help resolve the Issue: [cid:image005.jpg at 01D635D2.DF81D340] FYI here is the size of the Client Data File (when the Issue was present): [cid:image004.png at 01D635D1.F28C0D30] Again I was thinking of a possible 512kb limitation that could possibly Explain this Issue Anyways I will appreciate any Assistance or Insight you could provide a newbie to Xymon Thanks in Avance for your Help Have a Great Day & Week-End Best Regards Eric Pr?vost AVERTISSEMENT | WARNING Ce courriel peut ?tre de nature l?galement privil?gi?e et confidentielle. Si vous n'?tes pas le destinataire vis?, tout usage, copie ou distribution de ce message est strictement interdit. | This email could be legally privileged and confidential. If you are not the intended recipient, any use, copying or reproduction of this document is strictly forbidden. This email message (including its attachments) is confidential and may contain privileged information and is intended solely for the use of the individual and/or entity to whom it is addressed. If you are not the intended recipient of this e-mail you may not disseminate, distribute or copy this e-mail (including its attachments), or any part thereof. If this e-mail is received in error, please notify the sender immediately by return e-mail and make sure that this e-mail (including its attachments), and all copies thereof, are immediately deleted from your system. Please further note that when you communicate with us via email or visit our website we process your personal data. See our privacy policy for more information about how we process it: https://www.volvogroup.com/en-en/privacy.html

4 3

Configuring a red alert for: Machine recently rebooted
by m.mueller＠theearlytimes.de 29 May '20

29 May '20

I tried to configure an alert that should stay red for one hour after a reboot. The toolonglimit should be infinite. My analysis.conf contains: HOST=hal ??????? UP 1h -1? red This leads to a red cpu status: "Machine has been up more than 0 days" Obviously the toolonglimit of -1 isn't working as I expected it. dump-config shows 0 in thetoolong-column: bbrother at xymon:~/server $ bin/xymond_client --dump-config | grep UP UP 3600 0 HOST=hal (line: 358) ?tested under 4.3.28 (the dump-config test showed the same result under 4.3.30) Is this the expected behaviour? I worked around it by specifying a lot of weeks for toolonglimit.

3 3

Test http : when firewall between xymon server and host
by diffusion＠bulot-fr.com 26 May '20

26 May '20

Hello, I do often things like that (on hosts.d/mytest.cfg) 172.16.y.xxx host31 \ #http://www.toto.com=/172.16.y.xxx but recently on new hosts, this test is red but if i do that (on the host to check) curl -sIL -H "Host: www.toto.com" "http://localhost" all is ok So does the test is start from the server ? 1/ If yes, is there a way to execute the same test in localhost ? It's because the only open ports from Xymon Serveur to host are * tcp/22 * icmp And there's no way to open more. 2/ if no what's wrong -- -- Mail non conforme ? la netiquette (https://fr.wikipedia.org/wiki/Netiquette) Cordialement, Gr?gory Bulot Coaxis Hosting 0825 82 82 57

2 1

Still purple
by rltexas1＠gmail.com 22 May '20

22 May '20

I have updated the STATUSLIFETIME=540 and still getting purple for conn test. Then I updated the LIFE status+8h .. for green and still getting purpple. Im just trying to do a simple conn every 8hours to a few servers and see green or red.

1 0

2 Xymon the second have some data not send to it
by diffusion＠bulot-fr.com 22 May '20

22 May '20

Hello, I have 2 xymon server 1st one (xymon01=172.16.2.108) : all clients send data do it (*toto* + *titi*) 2nd one (back01=172.16.1.198) : only few clients (only with name like *toto*) send data to it For exemple back01 https://back01/xymon-cgi/confreport.sh i see all hosts (titi02 for exemple) that only send data to xymon01 only ===> how is it possible ? client toto04 : grep "^XYMONSERVERS" /etc/default/xymon-client XYMONSERVERS="172.16.2.108 172.16.1.198" client titi02: grep "^XYMONSERVERS" /etc/default/xymon-client XYMONSERVERS="172.16.2.108" I do grep -r titi02 /etc/xymon /usr/lib/xymon /var/lib/xymon on back01, but nothing is reporting -- Greg

2 3