January 2020 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

4.3.30 RPMs released
by cleaver＠terabithia.org 23 May '22

23 May '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 1

random errors with imaps/pop3s servers offering TLS 1.3
by lolo＠troll.free.org 10 Aug '21

10 Aug '21

Hi, I have found an issue when trying to monitor imaps and pop3s server offering TLS1.3 . My xymon server configuration: # freebsd-version -u 12.0-RELEASE-p5 # openssl version OpenSSL 1.1.1a-freebsd 20 Nov 2018 standard freebsd xymon-server pkg : # pkg info xymon-server-4.3.28 xymon-server-4.3.28 Name : xymon-server Version : 4.3.28 Installed on : Fri Feb 22 14:19:11 2019 CET Origin : net-mgmt/xymon-server Architecture : FreeBSD:12:amd64 Prefix : /usr/local Categories : net-mgmt www Licenses : GPLv2 Maintainer : feld at FreeBSD.org WWW : http://xymon.sourceforge.net/ Comment : System for monitoring servers and networks Options : DEBUG : off LDAP : off NETSNMP : off Shared Libs required: libcares.so.2 libpng16.so.16 libpcre.so.1 librrd.so.8 Annotations : FreeBSD_version: 1200086 cpe : cpe:2.3:a:xymon:xymon:4.3.28:::::freebsd12:x64 repo_type : binary repository : FreeBSD Flat size : 26.5MiB ... # ldd /usr/local/www/xymon/server/bin/xymonnet /usr/local/www/xymon/server/bin/xymonnet: libcares.so.2 => /usr/local/lib/libcares.so.2 (0x80027c000) libssl.so.111 => /usr/lib/libssl.so.111 (0x800297000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x80032c000) libpcre.so.1 => /usr/local/lib/libpcre.so.1 (0x800619000) libc.so.7 => /lib/libc.so.7 (0x8006bd000) libthr.so.3 => /lib/libthr.so.3 (0x800ab0000) When trying to monitor a pop3s or imaps server offering tls 1.3, I got random errors: WARNING: Flapping status Service imaps on xxx is not OK : Unexpected service response If I monitor the same services on an other server not offering tls 1.3, all is fine. Both servers where running dovecot. I wrote an extension in perl using IO::Socket::SSL (and the same local openssl) to monitor the server offerng tls 1.3 and all is fine with it. For some strange reason, I can monitor a https web servers (running nginx or apache) offering TLS 1.3 without this issue but not imaps/pop3s. It looks like the https test is different from other ssl/tls tests and does not have the TLS 1.3 issue Regards, -- Laurent Frigault | Free.org - BookMyName.com - ONLINE SAS - Registar ID 74

2 1

Xymon 4.3.30 Released
by cleaver＠terabithia.org 31 Jul '20

31 Jul '20

Xymon 4.3.30 has been released and is now available for download. 4.3.30 is mostly a bug-fix release, quashing issues stemming from the security fixes in 4.3.29, including improperly-tight restrictions on allowed characters in hostname for browsing along with several other parsing errors. Thanks in particular to Tom Schmidt for his assistance in tracking these down. Xymon should also now be more easily buildable on older GCC versions without the diagnostics pragma available. Xymon 4.3.30 is available from the Xymon SourceForge page at https://sourceforge.net/projects/xymon/ As always, thank you to all who have contributed code, ideas, and features to the project! Regards, Japheth "J.C." Cleaver

2 3

Monitoring AWS
by damien＠makelofine.org 29 Jan '20

29 Jan '20

Hi all, I'm starting to work on a set of Xymon client extensions to monitor Amazon Web Services (AWS) resources. As my own use case is not very representative, could you let me know what is missing in this list ? -EC2 instances checks (only the infrastructure layer, as the OS can easily run genuine xymon client) -RDS instances -CloudWatch metrics/logs (most important, as it can be used as a central point) -CloudTrail events -S3 usage -IAM users/roles/policies (to ensure nothing changed) -CloudFormation (check stacks are not drifting from template) Feel free to tell me what service are missing, and explain the use case, the important things to check. This could be a long work, so do not expect to have something delivered before Q2.

2 1

[solved] Xymon say "No port checks defined", but it configure to check in analysis and the same for procs
by diffusion＠bulot-fr.com 28 Jan '20

28 Jan '20

Hello, Le Mon, 27 Jan 2020 17:33:02 +0100, Torsten Richter <bb4 at richter-it.net> a ?crit : > 1. You have a second XYmon server which has no checks defined for that > host and intermittently sends this erroneous data to your primary > server. You'll be probably right for this point. I've forgotten this 2nd one because is "outside " :-/ Surprised by this conflict, but after reflexion, the answer, "flapping" is logic. I've been waiting for 20 minutes after sync analysis, flapping still goes (probably, i have to wait more), but checks are ok inside procs and ports !! -- Mail non conforme ? la netiquette (https://fr.wikipedia.org/wiki/Netiquette)

1 0

Xymon say "No port checks defined", but it configure to check in analysis and the same for procs
by diffusion＠bulot-fr.com 28 Jan '20

28 Jan '20

Le Mon, 27 Jan 2020 17:33:02 +0100, Torsten Richter <bb4 at richter-it.net> a ?crit : > 2. The client message is too big and does not contain ALL data. The name of the test is clientlog for me. the test is clear, and it seems ok : ... [clientversion] Xymon version 4.3.28 [clock] epoch: 1580219164.418498 local: 2020-01-28 14:46:04 CET UTC: 2020-01-28 13:46:04 GMT -- -- Mail non conforme ? la netiquette (https://fr.wikipedia.org/wiki/Netiquette) Cordialement, Gr?gory Bulot Coaxis Hosting 0825 82 82 57

1 0

hosts.cfg validity check
by john.thurston＠alaska.gov 28 Jan '20

28 Jan '20

What's a simple way to programmatically confirm a hosts.cfg is syntactically valid? With apache web server, one can do apachectl -t to parse the config file without launching the daemon. What are my options with xymon? I see I can do xymond --hosts=hosts.cfg but that looks like it is going to launch the daemon It looks like xymonnet offers the --no-update flag, but doesn't let me explicitly specify a hosts.cfg I thought the xymoncfg utility would do it. But it seems to always end with exit-code == 0 regardless of what I feed it. I thought maybe I could use xymoncmd with an explicit hosts.cfg to call xymonnet, but it appears I'd need to create a whole new environment file rather than just passing in the name of a new hosts.cfg How can I confirm a hosts.cfg isn't going to crash my xymon instance before I feed it to my actual instance? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Department of Administration State of Alaska

1 0

Xymon say "No port checks defined", but it configure to check in analysis and the same for procs
by bb4＠richter-it.net 27 Jan '20

27 Jan '20

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Resending to list On 27.01.2020 17:33, Torsten Richter wrote: > > - From my POV there are 2 opportunities why this happens: > > 1. You have a second XYmon server which has no checks defined for that > host and intermittently sends this erroneous data to your primary server. > > 2. The client message is too big and does not contain ALL data. > This can happen e.g. if there is a very long process listing sent with > the client data. > When the checks turn to clear just click on the "Client data" link > available on each check page of a server OR on the "clientlog" check for > that server and check if the message is complete. > A complete message would start with the line: > > [collector:] > > and end with 4 lines with time information like: > > [clock] > epoch: 1580142177.579044 > local: 2020-01-27 17:22:57 CET > UTC: 2020-01-27 16:22:57 GMT > > If those 4 lines are missing then your message is too big to be > transferred and processed and you'll need to tweak some settings in > xymonserver.cfg like MAXMSG or MAXLINE. > > Also, if you have the check "xymond" for your XYmon server check if that > check changed the color when the other checks were flapping to clear. > > Regards, > Torsten > > > On 27.01.2020 13:41, diffusion at bulot-fr.com wrote: >> Le Fri, 24 Jan 2020 12:48:06 +0000, >> John Horne <john.horne at plymouth.ac.uk> a ?crit : > > >>> If Xymon is processing the client data netstat output, then I suspect >>> the '$' in your regex above is causing the problem. The netstat >>> output includes other info after the port number, but your regex is >>> saying it should be the end of the line. > >> I've already try a simple check, but nothing better, this one not more : > > >> PORT LOCAL=%[\.:]3306 STATE=LISTEN min=1 TEXT=mysql3306 > >> By the way, i'll try procs also, and it's the same trouble : > >> PROC mysqld 1 1 >> WARNING: Flapping status >> "clear" No process checks defined > >> strange ! > >> _______________________________________________ >> Xymon mailing list >> Xymon at xymon.com >> http://lists.xymon.com/mailman/listinfo/xymon > > > - -- +---------------------------------------------------------+ | E-mail : bb4 [at] richter-it.net | | | | Homepage: http://www.richter-it.net/ | +---------------------------------------------------------+ Download my public key from: http://keys.gnupg.net/pks/lookup?search=0x899093AC&op=get -----BEGIN PGP SIGNATURE----- iF0EARECAB0WIQQJ3zkARmUgZ5/ufuPsOWbGiZCTrAUCXi8UuQAKCRDsOWbGiZCT rMEUAKDb3cqv2Ib/7Nyf9lfwo9ewBeHsAgCgraI3q48cFzJdj41DjDZrlTIeRqE= =gLZK -----END PGP SIGNATURE-----

1 0

Xymon say "No port checks defined", but it configure to check in analysis
by diffusion＠bulot-fr.com 27 Jan '20

27 Jan '20

Hello On my x-www-browser i have xymon-cgi/svcstatus.sh?HOST=totomysql02&SERVICE=ports says : "No port checks defined" and "WARNING: Flapping status" On xymon server's analysis.cfg HOST=totomysql02 PROC /usr/sbin/mysqld 1 1 PORT "LOCAL=%([.:]3306)$" state=LISTEN TEXT=mysql3306$ My conf seem ok (still in xymon server) /usr/lib/xymon/server/bin/xymond_client --dump-config | grep vdmztoptmysql02 | grep 3306 PORT local=%([.:]3306)$ state=LISTEN min=1 color=red HOST=totomysql02 TEXT=mysql3306 (line: 390) on xymon server's /var/log/xymon/xymond.log 2020-01-23 15:45:59.822981 Flapping detected for totomysql02:ports - 5 changes in 600 seconds 1/ why "Flapping detected" 2/ why still "No port checks defined" -- Mail non conforme ? la netiquette (https://fr.wikipedia.org/wiki/Netiquette) Cordialement, Gr?gory Bulot Coaxis Hosting 0825 82 82 57

4 4