June 2022 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

trying to monitor non-standard logfile with server client-local.cfg and failing
by tp6891＠att.com 25 Jun '22

25 Jun '22

Hi, I am trying to monitor non-standard logfile with server client-local.cfg and failing. I am using Xymon version 4.3.30 for both the server and client. The Server is on Redhat and 1 client on redhat and the other on solaris-10 I never see the file attempts in the clientlog on the xymon server for either host. I have checked for errors in the server/tmp directory and also on the client in client/tmp an client/logs and to not any attempts to read the file. Here is what I tried so far in the client-local.cfg file [host=hostname] file:/var/adm/6800.log and [hostname] file:/var/adm/6800.log >From the xymon/help/manpages/man5/client-local.cfg.5.html I cannot see anything that I am missing. Any guidance is appreciated . Tim

2 3

4.3.30 RPMs released
by cleaver＠terabithia.org 19 Jun '22

19 Jun '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 2

xymond xymongen xymonnet status purple after restart
by Berry.vanSleeuwen＠atos.net 16 Jun '22

16 Jun '22

Hi All, Today I have restarted the Xymon service to activate an update in xymonserver.cfg. I have updated XYMONSERVERHOSTNAME and XYMONSERVERWWWNAME to reflect the actual FQDN hostname. I restarted xymon with server/bin/xymon.sh. All monitored servers are still monitored, the trend data for them is appended. But I noticed no client data was reported for the Xymon machine itself, as it turned out the client wasn?t started. So I started the client as well. All reporting is now correct, except for the status of xymond, xymongen and xymonnet. They remain purple, the trends graphic is not updated for them as well. It looks like these services are not started. How can I get these services to be started? Met vriendelijke groet/With kind regards/Mit freundlichen Gr??en, Berry van Sleeuwen Flight Forum 3000 5657 EW Eindhoven ? +31 (0)6 22564276 [cid:image001.png at 01D88190.37905830] [cid:image002.png at 01D88190.37905830] This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, Atos? liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. On all offers and agreements under which Atos Nederland B.V. supplies goods and/or services of whatever nature, the Terms of Delivery from Atos Nederland B.V. exclusively apply. The Terms of Delivery shall be promptly submitted to you on your request.

1 1

Terabithia Xymon EL 9 Builds
by phaleintx＠gmail.com 15 Jun '22

15 Jun '22

Does anyone on the list know how we can request EL9 builds of the Terabithia packages? ? Thanks, Phil

1 0

Bogus Dovecot upgraded libs warnings
by jwinius＠umrk.nl 07 Jun '22

07 Jun '22

Hi folks, Christoph was recently so kind as to describe for me the solution below. However, the question I'd like to ask now is, how does one go about figuring out that kind of solution? Thanks, Jaap Quoting Jaap Winius via Xymon <xymon at xymon.com>: > Hi Christoph, > > Your solution works! I added /etc/xymon/libs.local.yaml to the > client configuration with the following content: > > # local config file for the libs test > --- > whitelist: > *: > - 'dovecot(\.index|-uidlist|\.list\.index)' > > > This is different from the existing line in /etc/xymon/libs.yaml: > > - 'dovecot(\.index|-uidlist)' > > Thank you very much. My Xymon page is green again! > > Cheers, > > Jaap > > > Quoting Christoph Zechner <zechner at vrvis.at>: > >> ... >> I think we had the same problems (also Debian) and I guess, this >> line in our libs.yaml stems from there: >> >> - 'dovecot(\.index|-uidlist|\.list\.index)' >> >> I just compared it to our other servers and this line seems to be >> unique in our xymon hosts, so I'm pretty sure it was added to fix >> this problem. >> ...

1 0

Bogus Dovecot upgraded libs warnings
by zechner＠vrvis.at 03 Jun '22

03 Jun '22

Hi, On 02/06/2022 18:32, Jaap Winius via Xymon wrote: > > Bogus Dovecot upgraded libs warnings.eml > > Subject: > Bogus Dovecot upgraded libs warnings > From: > Jaap Winius <jwinius at umrk.nl> > Date: > 02/06/2022, 18:32 > > To: > xymon at xymon.com > > > > Hi folks, > > Recently I upgraded a Dovecot IMAP host and now Xymon regularly warns > about linked libs that have been upgraded for many of the "dovecot/imap" > processes. When the users log out and the processes disappear, the > warning disappears as well. Obviously, these warning are bogus. Would > anyone happen to know what to add to /etc/xymon/libs.local.yaml, > presumably, to prevent this warning from occurring? > > Old host: > ? * Debian 10 (buster) > ? * Dovecot 2.3.4.1 > ? * Xymon client: 4.3.28 > > New host: > ? * Debian 11 (bullseye) > ? * Dovecot 2.3.13 > ? * Xymon client: 4.3.30 > > Xymon host (in both cases): > ? * Debian 11 (bullseye) > ? * Xymon server: 4.3.30 > > > Cheers, I think we had the same problems (also Debian) and I guess, this line in our libs.yaml stems from there: - 'dovecot(\.index|-uidlist|\.list\.index)' I just compared it to our other servers and this line seems to be unique in our xymon hosts, so I'm pretty sure it was added to fix this problem. Cheers Christoph > > Jaap

2 1

automated svcs listing with xymon ps client and analysis.d
by norkrie＠gmail.com 03 Jun '22

03 Jun '22

Hi, I want to automatically identify and configure test for all services of win servers which has the startup setting ?automatic? Getting the analysis.cfg entries is quite easy: for servername in $LIST do echo ?HOST=$servername? > $servername.cfg xymon localhost "xymondlog $servername.svcs"|awk '/ automatic /{printf(" SVC %-30s startup=automatic\n",$1)}' >> $servername.cfg done This will create an individual cfg file for every server just for the SVC check (with ~60 entries each). I thought I can just create these cfg files in $XYMONHOME/etc/analysis.d and it will be used similar to hosts.d or tasks.d directories. But obviously this does not work (and also there is no mention of analysis.d in the manuals). Of course I can add all these lines directly into analysis.cfg, but this would create a very huge file. Is there a way to use a subdir for those settings? Any other ideas to combine the normal analysis.cfg with let?s say 100 extra server cfg files? Xymon 4.3.30 self compiled and latest xymon PS client. Any ideas welcome regards Norbert

3 4

Bogus Dovecot upgraded libs warnings
by alt36＠cam.ac.uk 03 Jun '22

03 Jun '22

On 02/06/2022 17:32, Jaap Winius via Xymon wrote: > Hi folks, > > Recently I upgraded a Dovecot IMAP host and now Xymon regularly warns > about linked libs that have been upgraded for many of the "dovecot/imap" > processes. When the users log out and the processes disappear, the > warning disappears as well. Obviously, these warning are bogus. Would > anyone happen to know what to add to /etc/xymon/libs.local.yaml, > presumably, to prevent this warning from occurring? The libs report includes the process ID(s) of the relevant processes in brackets. The libs test pretty much just runs the lsof command and looks for files which are either 'deleted' or 'DEL', so you can do this check for yourself for a particular pid.. lsof -p 12345 | | awk '$4 ~ "DEL|deleted" {print $NF} ..will print a list of filenames you might want to add libs.local.yml Adam

1 0

Bogus Dovecot upgraded libs warnings
by jwinius＠umrk.nl 02 Jun '22

02 Jun '22

Hi folks, Recently I upgraded a Dovecot IMAP host and now Xymon regularly warns about linked libs that have been upgraded for many of the "dovecot/imap" processes. When the users log out and the processes disappear, the warning disappears as well. Obviously, these warning are bogus. Would anyone happen to know what to add to /etc/xymon/libs.local.yaml, presumably, to prevent this warning from occurring? Old host: * Debian 10 (buster) * Dovecot 2.3.4.1 * Xymon client: 4.3.28 New host: * Debian 11 (bullseye) * Dovecot 2.3.13 * Xymon client: 4.3.30 Xymon host (in both cases): * Debian 11 (bullseye) * Xymon server: 4.3.30 Cheers, Jaap

1 0