September 2021 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

4.3.30 RPMs released
by cleaver＠terabithia.org 23 May '22

23 May '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 1

Terabithia or compile
by ralphmitchell＠gmail.com 16 Mar '22

16 Mar '22

I downloaded the source rpm and installed it. The spec file for building the RPMs contains this: %if %{without trunk} %if 0%{?rhel} <= 7 Requires: ntp %endif %endif So, yeah, it builds into the binary RPMs the requirement to pull in ntp. I'm not sure if that can be made into a conditional, to require ntp OR chrony. Ralph Mitchell On Thu, Jun 18, 2020 at 5:05 PM Root, Paul T via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: "Root, Paul T" <Paul.Root at centurylink.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: RE: [Xymon] Terabithia or compile > That's odd. I did not do that to me. > > -----Original Message----- > From: Xymon <xymon-bounces at xymon.com> On Behalf Of John Thurston > Sent: Thursday, June 18, 2020 2:24 PM > To: xymon at xymon.com > Subject: Re: [Xymon] Terabithia or compile > > On 6/17/2020 4:38 PM, John Horne wrote: > > I used to build from source, but for our CentOS 7 and 8 servers I have > been > > using the Terabithia RPMs with no problems. The downside is that, like > Malcolm > > Hunter mentioned, I have subsequently applied some local patches - sent > to the > > list - so that meant rebuilding the RPMs. > > I'm now looking at the Terabithia package for the first time in years. > My first reaction is, wth? > > Installing the xymon package on my server has disabled chronyd and > installed (but not started) ntpd. why?! If I hadn't read through the > list of packages it was going to grab, and understand the significance > of this one, I'd be going blindly forward with an un-synchronized system > clock. > > Now I wonder how many other of my services it has disabled or replaced. > I feel like I've just walked into a mine field :( > > > -- > Do things because you should, not just because you can. > > John Thurston 907-465-8591 > John.Thurston at alaska.gov > Department of Administration > State of Alaska > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > > https://imss91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f… > This communication is the property of CenturyLink and may contain > confidential or privileged information. Unauthorized use of this > communication is strictly prohibited and may be unlawful. If you have > received this communication in error, please immediately notify the sender > by reply e-mail and destroy all copies of the communication and any > attachments. > > > > ---------- Forwarded message ---------- > From: "Root, Paul T via Xymon" <xymon at xymon.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: Re: [Xymon] Terabithia or compile > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

5 5

Baffled by absence of "ssh" in certain "hosts.cfg" entries, yet SSH tests are run
by john.thurston＠alaska.gov 30 Sep '21

30 Sep '21

I suggest it is because you once had 'ssh' on each of them. The test ran, then you took it off of all but one. 30-mins later, the test result on the unwanted hosts turned purple. If you click on the purple bubble and look at the history, how long was it green? Exactly one test cycle? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Department of Administration State of Alaska On 9/29/2021 9:19 PM, Greg Earle wrote: > After putting in these entries, the vCenter shows up fine in the Web > interface but the ESXi hosts all show up with an "ssh" column - and red > entries for all! > > About 30 minutes later, they've turned to purple.

1 0

Xymon-client for Proxmox ve
by jgsuareza1984＠gmail.com 30 Sep '21

30 Sep '21

HI Where do I get the xymon client installer for proxmox ve? -- Saludos, GS

1 0

Baffled by absence of "ssh" in certain "hosts.cfg" entries, yet SSH tests are run
by earle＠isolar.DynDNS.ORG 30 Sep '21

30 Sep '21

I'm setting up a new instance of Xymon 4.3.30 on a RHEL 7.9 system and something simple has me absolutely baffled. -- [root at adm2 hosts.d]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.9 (Maipo) [root at adm2 hosts.d]# rpm -q xymon xymon-4.3.30-1.el7.x86_64 -- In the setup there are a couple of VMware vCenters and several VMware ESXi hosts. The vCenter(s) have SSH and Web server access, but the ESXi hosts have had their SSH servers disabled and they only run the Web services. A greatly oversimplified version of my "/etc/xymon/hosts.d/localhosts.cfg" file is as follows: -- [root at adm2 hosts.d]# grep https localhosts.cfg 192.168.1.1 vcenter01 # ssh https://vcenter01=192.168.1.1/ 192.168.1.100 esxi-1 # https://esxi-1=192.168.1.100/ 192.168.1.101 esxi-2 # https://esxi-2=192.168.1.101/ -- After putting in these entries, the vCenter shows up fine in the Web interface but the ESXi hosts all show up with an "ssh" column - and red entries for all! About 30 minutes later, they've turned to purple. As you can see, I didn't put "ssh" into the (local)hosts.cfg file entries for the ESXi hosts; so why are they showing up as red alerts? It's like as if the "https" test is implicitly causing an "ssh" test to be run, even if I didn't ask for it. Color me baffled. If I don't put "ssh" into the hosts.cfg entry, why is it running an "ssh" test? Any clues? I'm sure I could probably run "xymon 127.0.0.1 'drop <host> ssh'" for all of them, but I'd rather know why this is happening. Thanks in advance for any help. - Greg

1 0

SMTP checks
by jeremy＠laidman.org 29 Sep '21

29 Sep '21

Ian [disclaimer: I don't use the smtp check] I'm wondering if the "connect timeout" relates not to the TCP connection but the SMTP conversation. I would do a packet capture during the interaction between xymonnet and the mail server, and see if it closely matches your telnet interaction, or deviates. The smtp check is defined in protocols.cfg [smtp] send "mail\r\nquit\r\n" expect "220" options banner port 25 This expects a 220 in the response after the mail\r\nquit\r\n is sent: '*It will then expect a response beginning with "220".*' (my emphasis on the word "then") Your interaction shows 221 instead of 220, after the mail/quit strings. But if this is the problem, then protocols.cfg has been wrong for many, many years. I suspect (contrary to what I infer from the man page) that "expect" looks in the strings for the whole interaction, not just the ones after the "send" string is sent. In which case, the protocols.cfg entry for smtp should work fine. However, you might like to test removing the "expect" or altering to look for 221 instead of 220. Cheers Jeremy On Wed, 29 Sept 2021 at 00:22, Ian Diddams via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: Ian Diddams <didds3 at yahoo.co.uk> > To: xymon <xymon at xymon.com> > Cc: > Bcc: > Date: Tue, 28 Sep 2021 13:52:12 +0000 (UTC) > Subject: SMTP checks > xymon server 4.3.28, centos 7.9 > mail server centos 6 (about to be replaced anyway) > > I'm in a bit of a quandary... > > we have an smtp check of a server ("oscar"). > > Literally for "years" (3+) the xymon in house smtp check has been more > than happy. > > Then since Sept 11th [1] its been flipping constantly beyween green and > happy to red > eg > > Service listening but unavailable (connect timeout) > > > Seconds: 0.001842000 > > > But if I run a telnet test from the xymon server itself it always connects > immediately, and responds instantaneously. > > eg > > $ telnet oscar 25 > Trying <redacted>... > Connected to oscar. > Escape character is '^]'. > 220 a.b.co.uk ESMTP Postfix > mail > 501 5.5.4 Syntax: MAIL FROM:<address> > quit > 221 2.0.0 Bye > Connection closed by foreign host. > $ > > > > All xymon logs show are emails being sent because of the alerts, ad > falling statuses that are a result of the constant flipping from red to > green. maillog has nothing untoward, ditto messages. > > Anyone any ideas what on earth is suddenly occurring that may explain this > recent timeout issue? any expereince of similar? > > didds > > > > [1] no changes made around that time etc. Networks confirm nothing shoing > in their logs etc. > > > > ---------- Forwarded message ---------- > From: Ian Diddams via Xymon <xymon at xymon.com> > To: xymon <xymon at xymon.com> > Cc: > Bcc: > Date: Tue, 28 Sep 2021 13:52:12 +0000 (UTC) > Subject: [Xymon] SMTP checks > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

2 1

SMTP checks
by jeremy＠laidman.org 29 Sep '21

29 Sep '21

Ian I believe the postfix warning is because there's a "mail" command before a "helo" or "ehlo" command. Pipelining is when Postfix checks to see that the mail client is waiting for the response to one command, before sending the next command, rather than just sending a "pipeline" of commands without checking the results of each. Spamming robots do this kind of thing. You can either modify the postfix configuration (see reject_unauth_pipelining), or adjust the [smtp] protocol to not start sending a mail message. I'd suggest sending "ehlo xymonnet\r\nquit\r\n". What triggers the smtp check is: a) you have "smtp" on the host's line in hosts.cfg, and b) you have xymonnet being launched every 5 minutes, which reads hosts.cfg, sees "smtp" and looks up the [smtp] section in protocols.cfg When there's a test failure, xymonnet-again runs every minute and does a rapid re-check of any failed tests. So a test that goes red can go green again a minute later. In fact, xymonnet and xymonnet-again don't seem to be synchronised at all, so I guess it's possible for xymonnet to run immediately after xymonnet has finished. J On Wed, 29 Sept 2021 at 20:18, Ian Diddams via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: Ian Diddams <didds3 at yahoo.co.uk> > To: xymon <xymon at xymon.com> > Cc: > Bcc: > Date: Wed, 29 Sep 2021 10:18:19 +0000 (UTC) > Subject: Re: [Xymon] SMTP checks > I also can see from the maillog that when the smtp check occurs > > > Sep 29 11:05:01 oscar postfix/smtpd[27269]: connect from unknown[x.x.x.x] > Sep 29 11:05:01 oscar postfix/smtpd[27269]: improper command pipelining > after MAIL from unknown [x.x.x.x] > Sep 29 11:05:01 oscar postfix/smtpd[27269]: disconnect from unknow > [x.x.x.x] > > so obviously the smtp check is sending something postfix doesnt > like/handle. That isnt a bhiggy in itself - bgut it nojw makes me wonder > out of curiosity exactly what the logic of the smtp check is.. > > I also tried Jeremy's example of "ehlo xymonnet\r\nquit\r\n" in > protocols.cfg - that also has the same warning/error in maillog > > improper command pipelining after EHLO from unknown[x.x.x.x] > > all jolly wierd,.... > > ian > > > > On Wednesday, 29 September 2021, 10:59:09 BST, Ian Diddams via Xymon < > xymon at xymon.com> wrote: > > > > secondary question... > > what triggers the smtp check? > > Ive had instances where the test goes red, and then just a few seconds > later back to green. So this isnt a standard "every 5 minutes" clearly ? > > ... > Wed Sep 29 08:44:39 2021 green 0:02:13 > Wed Sep 29 08:44:15 2021 red 0:00:24 > Wed Sep 29 07:44:36 2021 green 0:59:39 > ... > > anybody know? > > cheers > > ian > > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon > > > > > ---------- Forwarded message ---------- > From: Ian Diddams via Xymon <xymon at xymon.com> > To: xymon <xymon at xymon.com> > Cc: > Bcc: > Date: Wed, 29 Sep 2021 10:18:19 +0000 (UTC) > Subject: Re: [Xymon] SMTP checks > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

1 0

SMTP checks
by lk＠man-da.de 29 Sep '21

29 Sep '21

On Mittwoch, 29. September 2021 12:18:19 CEST Ian Diddams via Xymon wrote: > > improper command pipelining after EHLO from unknown[x.x.x.x] Hello Ian, the following works for me without this log message: [smtp] send "ehlo xymonnet\r\n" expect "220" send "quit\r\n" options banner port 25 But I have the sends and expects commented out for smtps (with authentication) due to TLS 1.3 issues at the moment. Kind regards, Lars -- Lars Kollstedt Telefon: +49 6151 16-71027 E-Mail: lk at man-da.de man-da.de GmbH Dolivostra?e 11 64293 Darmstadt Sitz der Gesellschaft: Darmstadt Registergericht: Amtsgericht Darmstadt Handelsregisternummer: HRB 9484 Gesch?ftsf?hrer: Andreas Ebert

2 1

SMTP checks
by didds3＠yahoo.co.uk 29 Sep '21

29 Sep '21

I also can see from the maillog that when the smtp check occurs Sep 29 11:05:01 oscar postfix/smtpd[27269]: connect from unknown[x.x.x.x] Sep 29 11:05:01 oscar postfix/smtpd[27269]: improper command pipelining after MAIL from unknown [x.x.x.x]Sep 29 11:05:01 oscar postfix/smtpd[27269]: disconnect from unknow [x.x.x.x] so obviously the smtp check is sending something postfix doesnt like/handle.? That isnt a bhiggy in itself - bgut it nojw makes me wonder out of curiosity exactly what the logic of? the smtp check is.. I also tried Jeremy's example of "ehlo xymonnet\r\nquit\r\n" in protocols.cfg - that also has the same warning/error in maillog ?improper command pipelining after EHLO from unknown[x.x.x.x] all jolly wierd,.... ian On Wednesday, 29 September 2021, 10:59:09 BST, Ian Diddams via Xymon <xymon at xymon.com> wrote: secondary question... what triggers the smtp check? Ive had instances where the test goes red, and then just a few seconds later back to green.? So this isnt a standard "every 5 minutes" clearly ? ... Wed Sep 29 08:44:39 2021 green 0:02:13 Wed Sep 29 08:44:15 2021 red 0:00:24 Wed Sep 29 07:44:36 2021 green 0:59:39 ... anybody know? cheers ian _______________________________________________ Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

1 0