July 2020 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

4.3.30 RPMs released
by cleaver＠terabithia.org 23 May '22

23 May '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 1

Terabithia or compile
by ralphmitchell＠gmail.com 16 Mar '22

16 Mar '22

I downloaded the source rpm and installed it. The spec file for building the RPMs contains this: %if %{without trunk} %if 0%{?rhel} <= 7 Requires: ntp %endif %endif So, yeah, it builds into the binary RPMs the requirement to pull in ntp. I'm not sure if that can be made into a conditional, to require ntp OR chrony. Ralph Mitchell On Thu, Jun 18, 2020 at 5:05 PM Root, Paul T via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: "Root, Paul T" <Paul.Root at centurylink.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: RE: [Xymon] Terabithia or compile > That's odd. I did not do that to me. > > -----Original Message----- > From: Xymon <xymon-bounces at xymon.com> On Behalf Of John Thurston > Sent: Thursday, June 18, 2020 2:24 PM > To: xymon at xymon.com > Subject: Re: [Xymon] Terabithia or compile > > On 6/17/2020 4:38 PM, John Horne wrote: > > I used to build from source, but for our CentOS 7 and 8 servers I have > been > > using the Terabithia RPMs with no problems. The downside is that, like > Malcolm > > Hunter mentioned, I have subsequently applied some local patches - sent > to the > > list - so that meant rebuilding the RPMs. > > I'm now looking at the Terabithia package for the first time in years. > My first reaction is, wth? > > Installing the xymon package on my server has disabled chronyd and > installed (but not started) ntpd. why?! If I hadn't read through the > list of packages it was going to grab, and understand the significance > of this one, I'd be going blindly forward with an un-synchronized system > clock. > > Now I wonder how many other of my services it has disabled or replaced. > I feel like I've just walked into a mine field :( > > > -- > Do things because you should, not just because you can. > > John Thurston 907-465-8591 > John.Thurston at alaska.gov > Department of Administration > State of Alaska > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > > https://imss91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f… > This communication is the property of CenturyLink and may contain > confidential or privileged information. Unauthorized use of this > communication is strictly prohibited and may be unlawful. If you have > received this communication in error, please immediately notify the sender > by reply e-mail and destroy all copies of the communication and any > attachments. > > > > ---------- Forwarded message ---------- > From: "Root, Paul T via Xymon" <xymon at xymon.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: Re: [Xymon] Terabithia or compile > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

5 5

random errors with imaps/pop3s servers offering TLS 1.3
by lolo＠troll.free.org 10 Aug '21

10 Aug '21

Hi, I have found an issue when trying to monitor imaps and pop3s server offering TLS1.3 . My xymon server configuration: # freebsd-version -u 12.0-RELEASE-p5 # openssl version OpenSSL 1.1.1a-freebsd 20 Nov 2018 standard freebsd xymon-server pkg : # pkg info xymon-server-4.3.28 xymon-server-4.3.28 Name : xymon-server Version : 4.3.28 Installed on : Fri Feb 22 14:19:11 2019 CET Origin : net-mgmt/xymon-server Architecture : FreeBSD:12:amd64 Prefix : /usr/local Categories : net-mgmt www Licenses : GPLv2 Maintainer : feld at FreeBSD.org WWW : http://xymon.sourceforge.net/ Comment : System for monitoring servers and networks Options : DEBUG : off LDAP : off NETSNMP : off Shared Libs required: libcares.so.2 libpng16.so.16 libpcre.so.1 librrd.so.8 Annotations : FreeBSD_version: 1200086 cpe : cpe:2.3:a:xymon:xymon:4.3.28:::::freebsd12:x64 repo_type : binary repository : FreeBSD Flat size : 26.5MiB ... # ldd /usr/local/www/xymon/server/bin/xymonnet /usr/local/www/xymon/server/bin/xymonnet: libcares.so.2 => /usr/local/lib/libcares.so.2 (0x80027c000) libssl.so.111 => /usr/lib/libssl.so.111 (0x800297000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x80032c000) libpcre.so.1 => /usr/local/lib/libpcre.so.1 (0x800619000) libc.so.7 => /lib/libc.so.7 (0x8006bd000) libthr.so.3 => /lib/libthr.so.3 (0x800ab0000) When trying to monitor a pop3s or imaps server offering tls 1.3, I got random errors: WARNING: Flapping status Service imaps on xxx is not OK : Unexpected service response If I monitor the same services on an other server not offering tls 1.3, all is fine. Both servers where running dovecot. I wrote an extension in perl using IO::Socket::SSL (and the same local openssl) to monitor the server offerng tls 1.3 and all is fine with it. For some strange reason, I can monitor a https web servers (running nginx or apache) offering TLS 1.3 without this issue but not imaps/pop3s. It looks like the https test is different from other ssl/tls tests and does not have the TLS 1.3 issue Regards, -- Laurent Frigault | Free.org - BookMyName.com - ONLINE SAS - Registar ID 74

2 1

Too much data for svcstatus.sh?
by mburger＠bubbanfriends.org 12 Oct '20

12 Oct '20

Good morning. Running Xymon 4.3.30, compiled from source tarball, on a RHEL 6.10 server. This morning, I encountered an interesting issue. I have a monitored system that has 197 discrete filesystems/mounts. When I attempt to enter this server's "Trends" section, I receive an internal server error (500). My httpd error log shows this when I attempt to access the trends link for the server in question: [Thu May 21 11:29:41 2020] [error] [client 10.40.24.55] Premature end of script headers: svcstatus.sh, referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] *** glibc detected *** /apps/xymon/server/bin/svcstatus.cgi: realloc(): invalid next size: 0x0000000000de7dd0 ***, referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] /apps/xymon/server/bin/svcstatus.cgi[0x408a4f], referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] /apps/xymon/server/bin/svcstatus.cgi[0x403453], referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] /apps/xymon/server/bin/svcstatus.cgi[0x404bb7], referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] /apps/xymon/server/bin/svcstatus.cgi[0x4022c9], referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] 00400000-0043c000 r-xp 00000000 fd:08 24906 /apps/xymon/server/bin/svcstatus.cgi, referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html [Thu May 21 11:29:59 2020] [error] [client 10.40.24.55] 0063c000-0063e000 rw-p 0003c000 fd:08 24906 /apps/xymon/server/bin/svcstatus.cgi, referer: http://sysmon.fhmc.local/xymon/edmprod/edmprod.html This does not happen for any other monitored system in my environment. This system is one of the more critical servers in my environment, so I'd like to figure out what's going on and remediate it if possible. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1

4 4

Xymon 4.3.30 Released
by cleaver＠terabithia.org 31 Jul '20

31 Jul '20

Xymon 4.3.30 has been released and is now available for download. 4.3.30 is mostly a bug-fix release, quashing issues stemming from the security fixes in 4.3.29, including improperly-tight restrictions on allowed characters in hostname for browsing along with several other parsing errors. Thanks in particular to Tom Schmidt for his assistance in tracking these down. Xymon should also now be more easily buildable on older GCC versions without the diagnostics pragma available. Xymon 4.3.30 is available from the Xymon SourceForge page at https://sourceforge.net/projects/xymon/ As always, thank you to all who have contributed code, ideas, and features to the project! Regards, Japheth "J.C." Cleaver

2 3

xymondboard error
by dl1025＠att.com 30 Jul '20

30 Jul '20

I recently started seeing these error messages in my log file periodically. When these messages appeared, I noticed that some of the tests turned red and white status but not all. Could you point me in the right direction of how I can trouble shoot this error messages? 2020-07-29 12:30:13.241732 Whoops ! Failed to send message (timeout) 2020-07-29 12:30:13.456322 -> 2020-07-29 12:30:13.456362 -> Recipient x.x.x.x, timeout 15 2020-07-29 12:30:13.456375 -> 1st line: 'xymondboard fields=hostname,testname,color,flags,lastchange,logtime,validtime,acktime,disabletime,sender,cookie,line1,acklist ' 2020-07-29 12:30:13.456394 xymond status-board not available, code 7 2020-07-29 12:30:13.456409 Failed to load current Xymon status, aborting page-update

2 2

Multiple xymon clients behind same IP
by vadud3＠gmail.com 30 Jul '20

30 Jul '20

I have multiple xymon clients (multiple devices at home) and the xymon server is at cloud. The outgoing IP is same for all the xymon clients. How do I monitor them? -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

2 2

Xymon and XSS vulnerability
by gatis.anee＠gmail.com 30 Jul '20

30 Jul '20

Hi, Looking for help. Does anyone know something about this rather old XSS vulnerability? https://infosec.rm-it.de/2012/04/08/xss-in-xymon/ It is kind of still there in the 4.3.30 version. Any ideas how can it be solved? Regards Gatis

2 1

mq.sh script
by neilsimmonds1808＠gmail.com 29 Jul '20

29 Jul '20

I've already tried running it with a set -x in the script and get the following in the log, + rm /xymon/client/tmp/mq-server_name.11677 + TMPFILE=/xymon/client/tmp/mq-server_name.11684 + touch /xymon/client/tmp/mq-server_name.11684 + echo 'client server_name.mqcollect mqcollect' + '[' 1 '!=' 0 ']' + QMGR=PPIM01 + shift + sudo -u mqm mqsc PPIM01 'display qstatus(*) curdepth msgage lgetdate lgettime' + '[' 0 '!=' 0 ']' + /xymon/client/bin/xymon ukclamon01 @ log:/opt/IBM/MDMPIM/logs/workflowengine_server_name/db_manager.log:10240 log:`find -L /opt/IBM/MDMPIM/logs/\w*server_name/\w*.log -maxdepth 1 -type f ! -size 0`:10240 log:/applogs/mdmpim/logs/appsvr_server_name/default.log:10240 log:/opt/IBM/MDMPIM/logs/workflowengine_server_name/customegl.log:20480 log:/opt/IBM/MDMPIM/logs/workflowengine_server_name/default.log:20480 ignore INFO trigger internal error file:`find /applogs/mdmpim/dump -name *.txt` file:`find /applogs/mdmpim/dump -name *.phd`` >From there I can see that it appears to be the line that sends the message to Xymon that's doing it but U can't see why. The line in question is underlined and BOLD below (copied the whole script for context) TMPFILE="$XYMONTMP/mq-$MACHINE.$$" touch $TMPFILE echo "client $MACHINE.mqcollect mqcollect" >$TMPFILE # while [ $# != 0 ] do QMGR=$1; shift sudo -u mqm mqsc $QMGR 'display qstatus(*) curdepth msgage lgetdate lgettime' >> $TMPFILE ##(echo 'dis ql(*) curdepth'; echo 'dis chs(*)'; echo 'end') | su - mqm -c "runmqsc $QMGR" >> $TMPFILE done # *$XYMON $XYMSRV "@" < $TMPFILE* rm $TMPFILE exit 0 On Mon, Jul 27, 2020 at 5:54 PM Damien Martins via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: Damien Martins <damien at makelofine.org> > To: xymon at xymon.com > Cc: > Bcc: > Date: Mon, 27 Jul 2020 18:44:19 +0200 > Subject: Re: [Xymon] mq.sh script > > Hello, > > I'm not using this script, but you may run it using: > > bash -x $XYMONCLIENTHOME/ext/mq.sh queuemanagername > > replace $XYMONCLIENTHOME with the correct path. > > Bash will generate output debug info, so you may see what it is doing. > > Also, you may check if the log destination is hardcoded in the script. > > > Le 27/07/2020 ? 16:39, Neil Simmonds a ?crit : > > Hi all, I'm seeing some really strange behaviour from the MQ.SH script/ > > We have the standard mq.sh script that comes with Xymon and we have it > setup in /xymon/client/etc/clientlaunch.cfg as follows > > [mq] > ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg > CMD $XYMONCLIENTHOME/ext/mq.sh queuemanagername > LOGFILE $XYMONCLIENTLOGS/xymonmq2client.log > INTERVAL 1m > > The behaviour that we are seeing is that,as expected MQ.SH writes nothing > in the way of errors into xymonmq2client.log but what it does write into > there is the contents of /xymon/client/tmp/logfetch.hostname.cfg > > (i.e. > log:/applogs/broker/broker.log:20480 > ignore MARK > ) > > I'm at a loss to explain this so am reaching out to the list to see if > anyone has come across this > > Regards, > Neil. > > _______________________________________________ > Xymon mailing listXymon at xymon.comhttp://lists.xymon.com/mailman/listinfo/xymon > > > > > ---------- Forwarded message ---------- > From: Damien Martins via Xymon <xymon at xymon.com> > To: xymon at xymon.com > Cc: > Bcc: > Date: Mon, 27 Jul 2020 18:44:19 +0200 > Subject: Re: [Xymon] mq.sh script > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

2 1