October 2021 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

4.3.30 RPMs released
by cleaver＠terabithia.org 23 May '22

23 May '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 1

Terabithia or compile
by ralphmitchell＠gmail.com 16 Mar '22

16 Mar '22

I downloaded the source rpm and installed it. The spec file for building the RPMs contains this: %if %{without trunk} %if 0%{?rhel} <= 7 Requires: ntp %endif %endif So, yeah, it builds into the binary RPMs the requirement to pull in ntp. I'm not sure if that can be made into a conditional, to require ntp OR chrony. Ralph Mitchell On Thu, Jun 18, 2020 at 5:05 PM Root, Paul T via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: "Root, Paul T" <Paul.Root at centurylink.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: RE: [Xymon] Terabithia or compile > That's odd. I did not do that to me. > > -----Original Message----- > From: Xymon <xymon-bounces at xymon.com> On Behalf Of John Thurston > Sent: Thursday, June 18, 2020 2:24 PM > To: xymon at xymon.com > Subject: Re: [Xymon] Terabithia or compile > > On 6/17/2020 4:38 PM, John Horne wrote: > > I used to build from source, but for our CentOS 7 and 8 servers I have > been > > using the Terabithia RPMs with no problems. The downside is that, like > Malcolm > > Hunter mentioned, I have subsequently applied some local patches - sent > to the > > list - so that meant rebuilding the RPMs. > > I'm now looking at the Terabithia package for the first time in years. > My first reaction is, wth? > > Installing the xymon package on my server has disabled chronyd and > installed (but not started) ntpd. why?! If I hadn't read through the > list of packages it was going to grab, and understand the significance > of this one, I'd be going blindly forward with an un-synchronized system > clock. > > Now I wonder how many other of my services it has disabled or replaced. > I feel like I've just walked into a mine field :( > > > -- > Do things because you should, not just because you can. > > John Thurston 907-465-8591 > John.Thurston at alaska.gov > Department of Administration > State of Alaska > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > > https://imss91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f… > This communication is the property of CenturyLink and may contain > confidential or privileged information. Unauthorized use of this > communication is strictly prohibited and may be unlawful. If you have > received this communication in error, please immediately notify the sender > by reply e-mail and destroy all copies of the communication and any > attachments. > > > > ---------- Forwarded message ---------- > From: "Root, Paul T via Xymon" <xymon at xymon.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: Re: [Xymon] Terabithia or compile > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

5 5

smtp test - how/what does it actually do?
by jeremy＠laidman.org 28 Oct '21

28 Oct '21

Ian On Thu, 28 Oct 2021 at 23:08, Ian Diddams via Xymon <xymon at xymon.com> wrote: > > we ahve a client configured thus > > 10.108.249.162 colt # ssh ntp MAINTNC:linux smtp > > This smtp c > heck has been green for ever - literally. It has suddenly alerted > overnight and is remaining so > > > Service smtp on colt is not OK : Service listening but unavailable > (connect timeout) > Seconds: 0.001814000 > > "listening" means it's accepting TCP connections. > I can telnet from the xymon server on port 25 fine. > "connect timeout" means xymonnet was waiting for something that didn't come in time. I wonder if you have an "expect" string in the [smtp] section of protocols.cfg, causing xymonnet to wait for a string containing "220" but that never arrives. postfix services on colt are working seemingly correctly (eg i can send a > mail successfully from a client that uses colt as a smtp server) > > So Im now trying to find out what it is that this smtp check atually does > in order to find out why it now thinks it cant do it. > > The error suggests indeed its a timeout issue - but the telnet test > connects immeditaely (no obvious lag anyway) > > thoughts/ > > ian > Maybe check your Postfix logs to see if something is happening there. Have you upgraded/reconfigured Postfix recently? In some configurations, Postfix has been known to reject "pipelining" which is sending multiple commands (eg "mail" and then "quit") without waiting for a response after each command. When this happens, Postfix rejects the command, but doesn't send a "220" nor does it close the connection. Or it might be as simple as your Xymon server being removed from Postfix's allowed senders lists. Here's the [smtp] section from the standard protocols.cfg: [smtp] send "mail\r\nquit\r\n" expect "220" options banner port 25 Compare with yours. You could do a packet capture to see what the dialog looks like, and compare it to the send/expect in protocols.cfg. You could try simulating the "send" string from protocols.cfg (after telnetting to port 25) and see if you get a 220 response. You could comment out the expect string in protocols.cfg and see if it starts working. You could take a look at (and perhaps even share here) the banner displayed in the smtp check status page, as that might give a clue as to what's not right. I seem to recall a discussion about SMTP pipelining on this list, only a few weeks ago. Perhaps re-read that discussion to see if there are any nuggets of wisdom that could help. Cheers Jeremy

1 0

smtp test - how/what does it actually do?
by didds3＠yahoo.co.uk 28 Oct '21

28 Oct '21

we ahve a client configured thus 10.108.249.162? colt # ssh ntp MAINTNC:linux smtp This smtp check has been green for ever - literally. It has suddenly alerted overnight and is remaining so Service smtp on colt is not OK : Service listening but unavailable (connect timeout) Seconds: 0.001814000 I can telnet from the xymon server on port 25 fine. postfix services on colt are working seemingly correctly? (eg i can send a mail successfully from a client that uses colt as a smtp server) So Im now trying to find out what it is that this smtp check atually does in order to find out why it now thinks it cant do it. The error suggests indeed its a timeout issue - but the telnet test connects immeditaely (no obvious lag anyway) thoughts/ ian

1 0

no reports because of full root disk - workaround
by zechner＠vrvis.at 22 Oct '21

22 Oct '21

Hi, a while ago, we encountered a rather annoying problem: if the root disk of our Linux servers got full between two xymon checks, we were not notified, because xymon was not able to report this, due to to remaining space. It did not matter whether the machine was in xymon local mode or not (we wanted to be sure). We then found a workaround for this problem: * creating a ramdisk (10 MB should suffice) * mount it as /var/lib/xymon/temp (unlike "tmp" originally) * change the tmp location in xymonclient.cfg * restart xymon-client This allows the system to send reports even if the root disk is completely full and there's no need for repartitioning or adding other disks. Any thoughts about that? Are there any repercussions I did not think about? I'd love to hear your opinions. Thanks in advance! Cheers Christoph

2 2

ports check for a VIP...
by didds3＠yahoo.co.uk 18 Oct '21

18 Oct '21

sorted.,... combostatus --debug --no-update does the job

1 0

ports check for a VIP...
by jeremy＠laidman.org 18 Oct '21

18 Oct '21

What was the output on the combo check page, during the outage? On Mon, 11 Oct 2021 at 19:16, Ian Diddams via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: Ian Diddams <didds3 at yahoo.co.uk> > To: xymon <xymon at xymon.com> > Cc: > Bcc: > Date: Mon, 11 Oct 2021 08:16:02 +0000 (UTC) > Subject: Re: [Xymon] ports check for a VIP... > Purely a sort of FYI I guess... not sure what was occurring but last > Friday early hours the clusetrr had a bit of a tizz and both nodes had no > mysqld running and so procs was not green on both nodes... but the combo > check remained green throughout.. > > > > On Thursday, 7 October 2021, 09:41:02 BST, Ian Diddams via Xymon < > xymon at xymon.com> wrote: > > > That's great Jeremy - many thanks > > Now > > (wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1 > [image: clear] wp-vldyn-estrela.ports > [image: green] wp-vldyn-rafeiro.ports > > ian > > > On Wednesday, 6 October 2021, 05:15:11 BST, Jeremy Laidman < > jeremy at laidman.org> wrote: > > > On Mon, 4 Oct 2021 at 23:15, Ian Diddams via Xymon <xymon at xymon.com> > wrote: > > > > > intrigiingly the actual output on that combo check shows > > > > (wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1 > > <snip> > > > i.e. its all showing clear for the individual lines which are also > separated where the hostname includes a hyphen... is that all correct ? > > I don't think so. From the manpage of combo.cfg: > > "Note: If the expression involves hostnames with a character that is also > an operator - e.g. if you have a host "t1-router-newyork.foo.com" with a > dash in the hostname - then the operator-character must be escaped with a > backslash '\' in the expression, or it will be interpreted as an operator. > E.g. like this: > nyc.conn = (t1\-router\-nyc.conn || backup\-router\-nyc.conn)" > > That's why it thinks you have hostnames like "wp" and "vldyn". It's almost > certainly not testing what you want it to be testing. > > Cheers > Jeremy > > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon > > > > ---------- Forwarded message ---------- > From: Ian Diddams via Xymon <xymon at xymon.com> > To: xymon <xymon at xymon.com> > Cc: > Bcc: > Date: Mon, 11 Oct 2021 08:16:02 +0000 (UTC) > Subject: Re: [Xymon] ports check for a VIP... > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

2 4

clients in local mode (Debian 10/11)
by zechner＠vrvis.at 18 Oct '21

18 Oct '21

Hi, this is just a little guide for anybody wishing to put xymon clients (all Debian 10 respectively 11) into local mode. We tried changing the appropriate config part [1] (but did not succeed, because in the Debian package the binary "xymond_client" was missing when installing the "xymon-client" package. This has previously been reported to the Debian bug tracker [2], but with little progress unfortunately. So we tried to copy the binary from our server and distributed it to the clients and it just worked. No need to manually compile the client package, just take the binary from the server package and use it on the client. Tested successfully with Debian 10 (xymon-client 4.3.28) and Debian 11 (xymon-client 4.3.30). Chers Christoph [1] LOCALMODE="yes" in /etc/xymon/xymonclient.cfg [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=903614

1 0

[bug report] logfetch: disable executing arbitrary commands
by zechner＠vrvis.at 15 Oct '21

15 Oct '21

Hi, I was surprised to learn that executing arbitrary commands via logfetch is the default in xymon and I think it shouldn't be. Overall it would make xymon a little bit more secure and if needed, it could be easily activated. Attached is the necessary patch for this, please let me know if this suits the general idea of xymon or if there are arguments against this measure. Thanks in advance! Cheers, Christoph patch: --- xymonclient.cfg.DIST +++ xymonclient.cfg.NEW @@ -15,7 +15,7 @@ XYMONCLIENTLOGS="$XYMONHOME/logs" # Where we store the client logfiles # Options to logfetch, the xymon binary which examines log files for recent changes. -LOGFETCHOPTS="" +LOGFETCHOPTS="--noexec" # Local Mode (Only) Options

1 0