June 2022 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

trying to monitor non-standard logfile with server client-local.cfg and failing
by jeremy＠laidman.org 25 Jun '22

25 Jun '22

Glad to see you have it working now. Yeah, two Xymon servers means the second one's client-local.cfg is used because it's the one that the client contacts last of all. I've been bitten by that several times in the past. J On Sat, 25 June 2022, 01:42 POLYAK, TIM P, <tp6891 at att.com> wrote: > Hi, > > > > What I meant by non standard log file was just not the default > /var/adm/messages. > > Looking at the contents of a different file example /var/adm/6800.log. > > > > Server client-local.cfg file > > [Fully-Qualified-Hostname] > > log:/var/adm/messages:10240 > > file:/var/adm/6800.log > > log:/var/adm/6800.log:10240 > > > > > > Update I did find my trouble. I was updating 1 of 2 xymon servers > correctly the test/backup server but I found the primary xymon server was > over writing my changes in the tmp/logfetch.Fully-Qualified-Hostname.cfg > after the test/backup server made the changes. I was in a loop where 1 > xymon server made the change and the other xymon server changed it back. > > > > Thank you for pointing out the xymon server updates the client > tmp/logfetch.Fully-Qualified-Hostname.cfg when you make changes to the > servers etc/client-local.cfg > > > > Thanks > > Tim > > > > > > > > *From:* Jeremy Laidman <jeremy at laidman.org> > *Sent:* Wednesday, June 22, 2022 6:53 PM > *To:* POLYAK, TIM P <tp6891 at att.com> > *Cc:* xymon at xymon.com > *Subject:* Re: [Xymon] trying to monitor non-standard logfile with server > client-local.cfg and failing > > > > Hi Tim > > > > On Thu, 23 Jun 2022 at 02:24, POLYAK, TIM P <tp6891 at att.com> wrote: > > I am trying to monitor non-standard logfile with server client-local.cfg > and failing. > > > > Can you explain how it's non-standard? Are you trying to monitor the > contents or the attributes of the file? Would a "log:/var/adm/6800.log" > entry be more suitable for a logfile? > > > > Here is what I tried so far in the client-local.cfg file > > [host=hostname] > > file:/var/adm/6800.log > and > > [hostname] > > file:/var/adm/6800.log > > > > The correct format is the latter. > > > > Firstly, note that it can take something like 10 minutes for log/file > monitoring to start reporting due to delays in propagating the config to > the clients, and then for the clients to report back again based on the > config. > > What I would do is look in the $XYMONTMP directory on the client (defined > in xymonclient.cfg, often /tmp, /var/tmp or /dev/shm) for the temporary > files created by the client. They will be called logfetch.<hostname>.cfg > and logfetch.<hostname>.status. > > The cfg file is essentially just the [hostname] section from the > client-local.cfg, retrieved when the client last reported to the server. > > The status file has a line for every "file:" entry in the client-local.cfg > file, and is used to track the logfile position on each run. If you don't > have any "file:" entries in your client-local.cfg file for a server, it > might not have a status file. > > So, first see if the cfg file is present. If not, your update to > client-local.cfg might not have been detected by the xymond process on the > Xymon server. Usually it picks this up by itself, but if not, you could > restart xymond and see if that helps (again, wait 10 minutes to see > progress). If the cfg file is present, see if its contents make sense (that > is, match what's in the client-local.cfg file on the server). > > J >

1 0

trying to monitor non-standard logfile with server client-local.cfg and failing
by tp6891＠att.com 24 Jun '22

24 Jun '22

Hi, I am trying to monitor non-standard logfile with server client-local.cfg and failing. I am using Xymon version 4.3.30 for both the server and client. The Server is on Redhat and 1 client on redhat and the other on solaris-10 I never see the file attempts in the clientlog on the xymon server for either host. I have checked for errors in the server/tmp directory and also on the client in client/tmp an client/logs and to not any attempts to read the file. Here is what I tried so far in the client-local.cfg file [host=hostname] file:/var/adm/6800.log and [hostname] file:/var/adm/6800.log >From the xymon/help/manpages/man5/client-local.cfg.5.html I cannot see anything that I am missing. Any guidance is appreciated . Tim

2 2

ntpq compatibility with chronyd
by bruno.manzoni＠ubi-network.ch 19 Jun '22

19 Jun '22

Hello all, I update the ntpq script to work with chronyd (as I only use only chrony) - Expect some bug with NTPd (As I did not test the ntpd part) -> Do not hesitate to report me some problems https://github.com/bonomani/ntpq And let me know if you found some optimisation or if you like it! Bonomani

1 0

xymond xymongen xymonnet status purple after restart
by Berry.vanSleeuwen＠atos.net 16 Jun '22

16 Jun '22

Hi All, Today I have restarted the Xymon service to activate an update in xymonserver.cfg. I have updated XYMONSERVERHOSTNAME and XYMONSERVERWWWNAME to reflect the actual FQDN hostname. I restarted xymon with server/bin/xymon.sh. All monitored servers are still monitored, the trend data for them is appended. But I noticed no client data was reported for the Xymon machine itself, as it turned out the client wasn?t started. So I started the client as well. All reporting is now correct, except for the status of xymond, xymongen and xymonnet. They remain purple, the trends graphic is not updated for them as well. It looks like these services are not started. How can I get these services to be started? Met vriendelijke groet/With kind regards/Mit freundlichen Gr??en, Berry van Sleeuwen Flight Forum 3000 5657 EW Eindhoven ? +31 (0)6 22564276 [cid:image001.png at 01D88190.37905830] [cid:image002.png at 01D88190.37905830] This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, Atos? liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. On all offers and agreements under which Atos Nederland B.V. supplies goods and/or services of whatever nature, the Terms of Delivery from Atos Nederland B.V. exclusively apply. The Terms of Delivery shall be promptly submitted to you on your request.

1 1

Terabithia Xymon EL 9 Builds
by phaleintx＠gmail.com 15 Jun '22

15 Jun '22

Does anyone on the list know how we can request EL9 builds of the Terabithia packages? ? Thanks, Phil

1 0

Bogus Dovecot upgraded libs warnings
by jwinius＠umrk.nl 07 Jun '22

07 Jun '22

Hi folks, Christoph was recently so kind as to describe for me the solution below. However, the question I'd like to ask now is, how does one go about figuring out that kind of solution? Thanks, Jaap Quoting Jaap Winius via Xymon <xymon at xymon.com>: > Hi Christoph, > > Your solution works! I added /etc/xymon/libs.local.yaml to the > client configuration with the following content: > > # local config file for the libs test > --- > whitelist: > *: > - 'dovecot(\.index|-uidlist|\.list\.index)' > > > This is different from the existing line in /etc/xymon/libs.yaml: > > - 'dovecot(\.index|-uidlist)' > > Thank you very much. My Xymon page is green again! > > Cheers, > > Jaap > > > Quoting Christoph Zechner <zechner at vrvis.at>: > >> ... >> I think we had the same problems (also Debian) and I guess, this >> line in our libs.yaml stems from there: >> >> - 'dovecot(\.index|-uidlist|\.list\.index)' >> >> I just compared it to our other servers and this line seems to be >> unique in our xymon hosts, so I'm pretty sure it was added to fix >> this problem. >> ...

1 0

Bogus Dovecot upgraded libs warnings
by zechner＠vrvis.at 03 Jun '22

03 Jun '22

Hi, On 02/06/2022 18:32, Jaap Winius via Xymon wrote: > > Bogus Dovecot upgraded libs warnings.eml > > Subject: > Bogus Dovecot upgraded libs warnings > From: > Jaap Winius <jwinius at umrk.nl> > Date: > 02/06/2022, 18:32 > > To: > xymon at xymon.com > > > > Hi folks, > > Recently I upgraded a Dovecot IMAP host and now Xymon regularly warns > about linked libs that have been upgraded for many of the "dovecot/imap" > processes. When the users log out and the processes disappear, the > warning disappears as well. Obviously, these warning are bogus. Would > anyone happen to know what to add to /etc/xymon/libs.local.yaml, > presumably, to prevent this warning from occurring? > > Old host: > ? * Debian 10 (buster) > ? * Dovecot 2.3.4.1 > ? * Xymon client: 4.3.28 > > New host: > ? * Debian 11 (bullseye) > ? * Dovecot 2.3.13 > ? * Xymon client: 4.3.30 > > Xymon host (in both cases): > ? * Debian 11 (bullseye) > ? * Xymon server: 4.3.30 > > > Cheers, I think we had the same problems (also Debian) and I guess, this line in our libs.yaml stems from there: - 'dovecot(\.index|-uidlist|\.list\.index)' I just compared it to our other servers and this line seems to be unique in our xymon hosts, so I'm pretty sure it was added to fix this problem. Cheers Christoph > > Jaap

2 1

automated svcs listing with xymon ps client and analysis.d
by norkrie＠gmail.com 03 Jun '22

03 Jun '22

Hi, I want to automatically identify and configure test for all services of win servers which has the startup setting ?automatic? Getting the analysis.cfg entries is quite easy: for servername in $LIST do echo ?HOST=$servername? > $servername.cfg xymon localhost "xymondlog $servername.svcs"|awk '/ automatic /{printf(" SVC %-30s startup=automatic\n",$1)}' >> $servername.cfg done This will create an individual cfg file for every server just for the SVC check (with ~60 entries each). I thought I can just create these cfg files in $XYMONHOME/etc/analysis.d and it will be used similar to hosts.d or tasks.d directories. But obviously this does not work (and also there is no mention of analysis.d in the manuals). Of course I can add all these lines directly into analysis.cfg, but this would create a very huge file. Is there a way to use a subdir for those settings? Any other ideas to combine the normal analysis.cfg with let?s say 100 extra server cfg files? Xymon 4.3.30 self compiled and latest xymon PS client. Any ideas welcome regards Norbert

3 4

Bogus Dovecot upgraded libs warnings
by alt36＠cam.ac.uk 03 Jun '22

03 Jun '22

On 02/06/2022 17:32, Jaap Winius via Xymon wrote: > Hi folks, > > Recently I upgraded a Dovecot IMAP host and now Xymon regularly warns > about linked libs that have been upgraded for many of the "dovecot/imap" > processes. When the users log out and the processes disappear, the > warning disappears as well. Obviously, these warning are bogus. Would > anyone happen to know what to add to /etc/xymon/libs.local.yaml, > presumably, to prevent this warning from occurring? The libs report includes the process ID(s) of the relevant processes in brackets. The libs test pretty much just runs the lsof command and looks for files which are either 'deleted' or 'DEL', so you can do this check for yourself for a particular pid.. lsof -p 12345 | | awk '$4 ~ "DEL|deleted" {print $NF} ..will print a list of filenames you might want to add libs.local.yml Adam

1 0