October 2019 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

4.3.30 RPMs released
by cleaver＠terabithia.org 19 Jun '22

19 Jun '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 2

random errors with imaps/pop3s servers offering TLS 1.3
by lolo＠troll.free.org 10 Aug '21

10 Aug '21

Hi, I have found an issue when trying to monitor imaps and pop3s server offering TLS1.3 . My xymon server configuration: # freebsd-version -u 12.0-RELEASE-p5 # openssl version OpenSSL 1.1.1a-freebsd 20 Nov 2018 standard freebsd xymon-server pkg : # pkg info xymon-server-4.3.28 xymon-server-4.3.28 Name : xymon-server Version : 4.3.28 Installed on : Fri Feb 22 14:19:11 2019 CET Origin : net-mgmt/xymon-server Architecture : FreeBSD:12:amd64 Prefix : /usr/local Categories : net-mgmt www Licenses : GPLv2 Maintainer : feld at FreeBSD.org WWW : http://xymon.sourceforge.net/ Comment : System for monitoring servers and networks Options : DEBUG : off LDAP : off NETSNMP : off Shared Libs required: libcares.so.2 libpng16.so.16 libpcre.so.1 librrd.so.8 Annotations : FreeBSD_version: 1200086 cpe : cpe:2.3:a:xymon:xymon:4.3.28:::::freebsd12:x64 repo_type : binary repository : FreeBSD Flat size : 26.5MiB ... # ldd /usr/local/www/xymon/server/bin/xymonnet /usr/local/www/xymon/server/bin/xymonnet: libcares.so.2 => /usr/local/lib/libcares.so.2 (0x80027c000) libssl.so.111 => /usr/lib/libssl.so.111 (0x800297000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x80032c000) libpcre.so.1 => /usr/local/lib/libpcre.so.1 (0x800619000) libc.so.7 => /lib/libc.so.7 (0x8006bd000) libthr.so.3 => /lib/libthr.so.3 (0x800ab0000) When trying to monitor a pop3s or imaps server offering tls 1.3, I got random errors: WARNING: Flapping status Service imaps on xxx is not OK : Unexpected service response If I monitor the same services on an other server not offering tls 1.3, all is fine. Both servers where running dovecot. I wrote an extension in perl using IO::Socket::SSL (and the same local openssl) to monitor the server offerng tls 1.3 and all is fine with it. For some strange reason, I can monitor a https web servers (running nginx or apache) offering TLS 1.3 without this issue but not imaps/pop3s. It looks like the https test is different from other ssl/tls tests and does not have the TLS 1.3 issue Regards, -- Laurent Frigault | Free.org - BookMyName.com - ONLINE SAS - Registar ID 74

2 1

Xymon 4.3.30 Released
by cleaver＠terabithia.org 18 Aug '20

18 Aug '20

Xymon 4.3.30 has been released and is now available for download. 4.3.30 is mostly a bug-fix release, quashing issues stemming from the security fixes in 4.3.29, including improperly-tight restrictions on allowed characters in hostname for browsing along with several other parsing errors. Thanks in particular to Tom Schmidt for his assistance in tracking these down. Xymon should also now be more easily buildable on older GCC versions without the diagnostics pragma available. Xymon 4.3.30 is available from the Xymon SourceForge page at https://sourceforge.net/projects/xymon/ As always, thank you to all who have contributed code, ideas, and features to the project! Regards, Japheth "J.C." Cleaver

5 9

Devmon template for Cisco ISR4000
by colin.coe＠gmail.com 19 Dec '19

19 Dec '19

Hi Just wondering if anyone already has a Devmon template for the ISR4000 series that they'd be willing to share? Thanks

4 5

[PATCH] - client data link on graphs
by john.horne＠plymouth.ac.uk 24 Oct '19

24 Oct '19

Hello, We are using the Terabithia 4.3.30 RPMs. For most of the test web pages, a link to the data sent by the client is shown on the page. This is seen in the 'Client data available' message on the page. This is something we do not want. The graphs can be seen by our users, but we do not want details of the clients (servers) being made available. (We have previously just commented out the relevant bit of code, but this requires rebuilding the RPMs at each release.) This patch simply provides a 'xymonserver.cfg' variable ('SHOWCLIENTLINK') which be can set to true or false. The default value is true, which will cause the link to be displayed (as before). The patches are all very small, typically just one or two lines. The patches do not affect the collecting of the client data in any way. Instead, just before the link would be displayed, a simple check of the variable is performed to see if the link should be displayed or not. Note - it was noticed that the html pages and man pages included the 'CLIENTSVCS' xymonserver.cfg variable. However, looking at the source code, this variable is not used anywhere. It should be removed from the documentation. John. -- John Horne | Senior Operations Analyst | Technology and Information Services University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

1 0

[PATCH] - xymonserver - all ok message - part 2
by john.horne＠plymouth.ac.uk 24 Oct '19

24 Oct '19

Hello, This is the second of two sets of patches, both of which are based on the Terabithia 4.3.30 RPMs. Currently when all the clients/hosts report green a message on the 'All non- green view' page is displayed saying 'All Monitored Systems OK'. I was asked 'what' is all green - that is, how many things are you testing? These patches allow the above message to include two variables which will be converted into numbers. (Management love to see numbers.) One variable (#XMHOSTS) reports the number of clients/hosts configured (taken from the xymongen.c 'hostcount' variable), the second variable (#XMTESTS) shows the number of tests being performed (taken from the xymongen.c 'statuscount' variable). So, in our case we have the following defined in our xymonserver.cfg file: ====== XYMONALLOKTEXT=" All Monitored Systems OK (Monitoring #XMHOSTS devices;  performing #XMTESTS tests) " ====== So when everything is green, then on the 'All non-green view' web page we see the following message (centered on the page): ===== All Monitored Systems OK (Monitoring 62 devices; performing 1165 tests) ===== The first set of patches created a second xymonserver.cfg configuration variable (XYMONALLOKCRIT) to be used on the 'Critical systems' web page. The reason was that the number of clients/hosts and tests may be misinterpreted if displayed on that page. For example, is the number of hosts the total number of hosts or just the number of configured critical hosts? To avoid any misinterpretation, XYMONALLOKCRIT does not expand any included variables. Included in these patches is a change to the 'loaddata.c' file. This produces the statistics shown on the xymongen web page. It excludes the count of 'info' and 'trends' messages, since these are generated by Xymon and not real tests. By default they always show a green colour, although the colour is configurable in the xymonserver.cfg file, and so would bias the green percentage. The patch excludes the 'clientlog' entries as well. These too are always shown as green by default. Although they are not generated as such, the column just shows data already received from the client. As such they are not an actual test, and should not be included in the statistics. By doing this, the 'statuscount' value now matches the number of actual test results seen in the '/var/lib/xymon/hist' directory. Again, the patches are relatively small in size. John. -- John Horne | Senior Operations Analyst | Technology and Information Services University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

2 4

Developer question - number of tests?
by john.horne＠plymouth.ac.uk 24 Oct '19

24 Oct '19

Hello, The xymonnet and xymongen web pages include an entry for the total number of hosts being tested. What I was also looking for though was if the total number of tests was recorded too? Tests such as 'procs', which may test for several processes being present, should be counted as one test. I'll carry on looking through the (Terabithia 4.3.30 RPM) code, but if anyone knows of a variable already being used to record the number of tests, then I would be grateful to know what it is. Thanks, John. -- John Horne | Senior Operations Analyst | Technology and Information Services University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

2 3

xymon-graphs
by yyyadvendra＠gmail.com 24 Oct '19

24 Oct '19

Hi Guys I have installed xymon x.30, it appears graphs are not being generated. where should I be looking at? I am new to xymon. Thanks Yadvendra

2 1

Graphs are not working.
by damien＠makelofine.org 24 Oct '19

24 Oct '19

Hello Mukesh, Did you check the following : -xymonserver.cfg file is set to store the data -RRD files for the expected tests are updated on a regular basis -Graphs definition graphs.cfg file -Content of /var/log/xymon/rrd-data.log Le 23/10/2019 ? 16:02, Semwal, Mukesh via Xymon a ?crit?: > Sujet?: > Graphs are not working. > De?: > "Semwal, Mukesh" <mukesh.semwal at merck.com> > Date?: > 23/10/2019 ? 16:02 > > Pour?: > "xymon at xymon.com" <xymon at xymon.com> > > > *Proprietary* > > Hi, > > I have installed new XYMON server xymon-4.3.30, I can see all the > clients data but there is no graph populating anywhere Please help. > > ps -ef|grep -i rrd > > xymon??? 16088 16053 97 09:36 ? 00:23:55 xymond_channel > --channel=status --log=/var/log/xymon/rrd-status.log xymond_rrd > --rrddir=/usr/lib/xymon/data/rrd > > xymon??? 16089 16053 97 09:36 ? 00:23:54 xymond_channel --channel=data > --log=/var/log/xymon/rrd-data.log xymond_rrd > --rrddir=/usr/lib/xymon/data/rrd > > xymon??? 26775 14376? 0 10:00 pts/1 00:00:00 grep --color=auto -i rrd > > -bash-4.2$ > > */_Mukesh Semwal._/* > > *High Performance Computing*** > > *Computational Platforms, MRL-IT Engineering*** > > *Click to Request HPC Support > <https://servicerequests.merck.com/ux/rest/share/OJSXG33VOJRWKSLEHU4DEMBGOJS…> > **Or call 1-866-637-2543 (Prompts 1-8) to open case for Provider Group > SciComp_HPC* > > *HPC Portal (Beta) <https://go.merck.com/hpc>* > > *SciComp_HPC* >

2 2