May 2021 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

4.3.30 RPMs released
by cleaver＠terabithia.org 23 May '22

23 May '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 1

Terabithia or compile
by ralphmitchell＠gmail.com 16 Mar '22

16 Mar '22

I downloaded the source rpm and installed it. The spec file for building the RPMs contains this: %if %{without trunk} %if 0%{?rhel} <= 7 Requires: ntp %endif %endif So, yeah, it builds into the binary RPMs the requirement to pull in ntp. I'm not sure if that can be made into a conditional, to require ntp OR chrony. Ralph Mitchell On Thu, Jun 18, 2020 at 5:05 PM Root, Paul T via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: "Root, Paul T" <Paul.Root at centurylink.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: RE: [Xymon] Terabithia or compile > That's odd. I did not do that to me. > > -----Original Message----- > From: Xymon <xymon-bounces at xymon.com> On Behalf Of John Thurston > Sent: Thursday, June 18, 2020 2:24 PM > To: xymon at xymon.com > Subject: Re: [Xymon] Terabithia or compile > > On 6/17/2020 4:38 PM, John Horne wrote: > > I used to build from source, but for our CentOS 7 and 8 servers I have > been > > using the Terabithia RPMs with no problems. The downside is that, like > Malcolm > > Hunter mentioned, I have subsequently applied some local patches - sent > to the > > list - so that meant rebuilding the RPMs. > > I'm now looking at the Terabithia package for the first time in years. > My first reaction is, wth? > > Installing the xymon package on my server has disabled chronyd and > installed (but not started) ntpd. why?! If I hadn't read through the > list of packages it was going to grab, and understand the significance > of this one, I'd be going blindly forward with an un-synchronized system > clock. > > Now I wonder how many other of my services it has disabled or replaced. > I feel like I've just walked into a mine field :( > > > -- > Do things because you should, not just because you can. > > John Thurston 907-465-8591 > John.Thurston at alaska.gov > Department of Administration > State of Alaska > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > > https://imss91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f… > This communication is the property of CenturyLink and may contain > confidential or privileged information. Unauthorized use of this > communication is strictly prohibited and may be unlawful. If you have > received this communication in error, please immediately notify the sender > by reply e-mail and destroy all copies of the communication and any > attachments. > > > > ---------- Forwarded message ---------- > From: "Root, Paul T via Xymon" <xymon at xymon.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: Re: [Xymon] Terabithia or compile > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

5 5

random errors with imaps/pop3s servers offering TLS 1.3
by lolo＠troll.free.org 10 Aug '21

10 Aug '21

Hi, I have found an issue when trying to monitor imaps and pop3s server offering TLS1.3 . My xymon server configuration: # freebsd-version -u 12.0-RELEASE-p5 # openssl version OpenSSL 1.1.1a-freebsd 20 Nov 2018 standard freebsd xymon-server pkg : # pkg info xymon-server-4.3.28 xymon-server-4.3.28 Name : xymon-server Version : 4.3.28 Installed on : Fri Feb 22 14:19:11 2019 CET Origin : net-mgmt/xymon-server Architecture : FreeBSD:12:amd64 Prefix : /usr/local Categories : net-mgmt www Licenses : GPLv2 Maintainer : feld at FreeBSD.org WWW : http://xymon.sourceforge.net/ Comment : System for monitoring servers and networks Options : DEBUG : off LDAP : off NETSNMP : off Shared Libs required: libcares.so.2 libpng16.so.16 libpcre.so.1 librrd.so.8 Annotations : FreeBSD_version: 1200086 cpe : cpe:2.3:a:xymon:xymon:4.3.28:::::freebsd12:x64 repo_type : binary repository : FreeBSD Flat size : 26.5MiB ... # ldd /usr/local/www/xymon/server/bin/xymonnet /usr/local/www/xymon/server/bin/xymonnet: libcares.so.2 => /usr/local/lib/libcares.so.2 (0x80027c000) libssl.so.111 => /usr/lib/libssl.so.111 (0x800297000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x80032c000) libpcre.so.1 => /usr/local/lib/libpcre.so.1 (0x800619000) libc.so.7 => /lib/libc.so.7 (0x8006bd000) libthr.so.3 => /lib/libthr.so.3 (0x800ab0000) When trying to monitor a pop3s or imaps server offering tls 1.3, I got random errors: WARNING: Flapping status Service imaps on xxx is not OK : Unexpected service response If I monitor the same services on an other server not offering tls 1.3, all is fine. Both servers where running dovecot. I wrote an extension in perl using IO::Socket::SSL (and the same local openssl) to monitor the server offerng tls 1.3 and all is fine with it. For some strange reason, I can monitor a https web servers (running nginx or apache) offering TLS 1.3 without this issue but not imaps/pop3s. It looks like the https test is different from other ssl/tls tests and does not have the TLS 1.3 issue Regards, -- Laurent Frigault | Free.org - BookMyName.com - ONLINE SAS - Registar ID 74

2 1

proper upgrade method from 4.3.27 to 4.3.30
by kspringer＠innovateteam.com 31 May '21

31 May '21

One of my Xymon servers is running Ubuntu v16.04 and Xymon Server v4.3.27 I was gonna build a brand new VM and install the Xymon v4.3.30 and go through all the config edits, but if I can just upgrade Xymon instead that will save me a lot of time, but I'm unclear on the correct method.? I'd like all the configs and paths to stay intact. Is it as simple as unzipping the 4.3.30.tar.gz and running ./configure ? I will of course be snapshoting my server prior to trying this, but I thought I'd ask if anyone has attempted this and run into any issues they could share. -- Kris Springer

3 3

xymonnet additional capability -- pause
by hobbit＠elyograg.org 30 May '21

30 May '21

Over ten years ago, I asked on this list how I could get rid of the "incorrect pipelining message" logged by postfix every time the smtp, smtps, or submission tests are done. https://lists.xymon.com/oldarchive/2010/11/msg00207.html The fix that I claim worked (multiple send commands) was for a job I had at the time. Now I am have a personal mail server (that also runs xymon) and that fix I mentioned so long ago is not working. I tweaked the code to allow a "pause" action in protocols.cfg and after modifying protocols.cfg to utilize it, I have eliminated the "incorrect command pipelining" message in mail.log. Here's the patch: --- xymon-4.3.28/lib/netservices.c 2017-01-05 19:00:06.000000000 -0700 +++ pause-xymon-4.3.28/lib/netservices.c 2021-05-29 12:51:09.717461323 -0600 @@ -259,6 +259,10 @@ } } } + else if (strncmp(l, "pause ", 6) == 0) { + int pausetime = atoi(skipwhitespace(l+5)); + sleep(pausetime); + } } if (fd) stackfclose(fd); Here's the new definitions I created in protocols.cfg: [smtp] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options banner port 25 [smtps] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options ssl,banner # No default port-number assignment for smtps - nonstandard according to IANA [submission|msa] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options banner port 587 And this is now what I see in mail.log (testing smtps and submission: May 29 13:51:15 bilbo postfix/submission/smtpd[16324]: connect from bilbo.elyograg.org[172.31.8.104] May 29 13:51:15 bilbo postfix/submission/smtpd[16324]: disconnect from bilbo.elyograg.org[172.31.8.104] quit=1 commands=1 May 29 13:51:15 bilbo postfix/smtps/smtpd[16325]: connect from bilbo.elyograg.org[172.31.8.104] May 29 13:51:15 bilbo postfix/smtps/smtpd[16325]: disconnect from bilbo.elyograg.org[172.31.8.104] quit=1 commands=1 Interesting thing here is that it's not actually pausing. Which probably means that I implemented it incorrectly. My training on C is ancient and I'm very rusty. But even though it doesn't pause, the error is gone, simply because each smtp command is now sent in a separate packet, which appears to *sometimes* be enough "delay" for postfix to not complain about pipelining. What happens now is occasionally I will get a yellow status on smtps with the xymon UI saying "Service smtps on bilbo.elyograg.org is not OK : Unexpected service response". So I think what I will do before submitting a patch is implement a "null" action (which will do nothing) as well as a "pause" action, and get some help from the real C developers here for making "pause" behave as advertised. Can somebody point me to some instructions on properly creating and submitting a patch? Also, if I could get some info on any other files I need to modify (man pages, readme files, etc), I would really appreciate it. Thanks, Shawn

4 4

PS Client in "local mode"?
by john.thurston＠alaska.gov 28 May '21

28 May '21

The old BB and Xymon clients for Windows rely on ancient libraries, so I'm looking at my options. It appears to me that the Powershell client is the only thing out there. Is there the correct location for the most recent? https://github.com/svn2github/xymon And can anyone tell me if it can be run in "local config mode", wherein the evaluation of red/green occurs on the host (rather than the server)? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Department of Administration State of Alaska

2 3

assistance with xymonPSclient
by kspringer＠innovateteam.com 28 May '21

28 May '21

client-local.cfg is for other things.? Check services using the analysis.cfg file.? Like this. HOST=your.hosts.name ??????? SVC???? SEEMISproxy Kris Springer On 5/28/21 10:34 AM, David Smith via Xymon wrote: > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon

1 0

assistance with xymonPSclient
by davieb＠gmail.com 28 May '21

28 May '21

David, I believe you would put the "SVC SEEMISproxy" in the analysis.cfg on the xymon server. Dave D On Fri, May 28, 2021 at 12:35 PM David Smith via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: David Smith <david.smith at renfrewshire.gov.uk> > To: "xymon at xymon.com" <xymon at xymon.com> > Cc: > Bcc: > Date: Fri, 28 May 2021 16:34:57 +0000 > Subject: assistance with xymonPSclient > > Hi > > > > I am looking for some help with Xymon client on windows > > > > I have a linux server running xymon server 4.3.30, and I have been using > bbwin clients for quite a while, but I was looking to migrate to the > powershell one as at least there seems to be some moderate development on > it. > > > > I put the newest xymonclient.ps1 + xymonclient_config + nssm.exe in > c:\program files\xymon > > > > Contents of xymonclient_config is > > XymonSettings> > > > > <servers>MYXYMONSERVER</servers> > > > > <clientlogfile>c:\program files\xymon\xymonclient.log</clientlogfile> > > <clientconfigfile>c:\program > files\xymon\clientconfig.cfg</clientconfigfile> > > > > <clientfqdn>0</clientfqdn> > > <clientlower>1</clientlower> > > > > <clientremotecfgexec>1</clientremotecfgexec> > > > > <wanteddisks>3</wanteddisks> > > > > <externalscriptlocation>c:\program > files\xymon\ext</externalscriptlocation> > > <externaldatalocation>c:\program > files\xymon\tmp</externaldatalocation> > > </XymonSettings> > > > > I have a line in client-local.cfg on my xymon server > > [win32] > > servicecheck:SEEMISproxy > > > > Now what I though that I would get is on the xymon page for services, for > my single client would be the single service I declared in the > client-local.cfg. However what I do get is a clear button, which shows all > services and it says No Services checks defined. > > Similarly there are clear button for files, ports and procs. > > All other checks like disks, cpu etc are working. > > Is my assumption for how this should work wrong? > > > > Do I have to do something locally on the client? > > > > Any help/suggestions would be appreciated > > > > Cheers > > > > David > > > > > > > > Renfrewshire Council Website -http://www.renfrewshire.gov.uk > > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. > If you have received this email in error please notify the system manager. > Renfrewshire Council may, in accordance with the Telecommunications(Lawful > Business Practice) (Interception of Communications) Regulations 2000, > intercept e-mail messages for the purpose of monitoring or keeping a record > of communications on the Council's system. If a message contains > inappropriate dialogue it will automatically be intercepted by the > Council's Internal Audit section who will decide whether or not the e-mail > should be onwardly transmitted to the intended recipient(s). > > > > ---------- Forwarded message ---------- > From: David Smith via Xymon <xymon at xymon.com> > To: "xymon at xymon.com" <xymon at xymon.com> > Cc: > Bcc: > Date: Fri, 28 May 2021 16:34:57 +0000 > Subject: [Xymon] assistance with xymonPSclient > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

1 0

assistance with xymonPSclient
by david.smith＠renfrewshire.gov.uk 28 May '21

28 May '21

Hi I am looking for some help with Xymon client on windows I have a linux server running xymon server 4.3.30, and I have been using bbwin clients for quite a while, but I was looking to migrate to the powershell one as at least there seems to be some moderate development on it. I put the newest xymonclient.ps1 + xymonclient_config + nssm.exe in c:\program files\xymon Contents of xymonclient_config is XymonSettings> <servers>MYXYMONSERVER</servers> <clientlogfile>c:\program files\xymon\xymonclient.log</clientlogfile> <clientconfigfile>c:\program files\xymon\clientconfig.cfg</clientconfigfile> <clientfqdn>0</clientfqdn> <clientlower>1</clientlower> <clientremotecfgexec>1</clientremotecfgexec> <wanteddisks>3</wanteddisks> <externalscriptlocation>c:\program files\xymon\ext</externalscriptlocation> <externaldatalocation>c:\program files\xymon\tmp</externaldatalocation> </XymonSettings> I have a line in client-local.cfg on my xymon server [win32] servicecheck:SEEMISproxy Now what I though that I would get is on the xymon page for services, for my single client would be the single service I declared in the client-local.cfg. However what I do get is a clear button, which shows all services and it says No Services checks defined. Similarly there are clear button for files, ports and procs. All other checks like disks, cpu etc are working. Is my assumption for how this should work wrong? Do I have to do something locally on the client? Any help/suggestions would be appreciated Cheers David Renfrewshire Council Website -http://www.renfrewshire.gov.uk This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. Renfrewshire Council may, in accordance with the Telecommunications(Lawful Business Practice) (Interception of Communications) Regulations 2000, intercept e-mail messages for the purpose of monitoring or keeping a record of communications on the Council's system. If a message contains inappropriate dialogue it will automatically be intercepted by the Council's Internal Audit section who will decide whether or not the e-mail should be onwardly transmitted to the intended recipient(s).

1 0