April 2022 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

4.3.30 RPMs released
by cleaver＠terabithia.org 23 May '22

23 May '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 1

Can Xymon alert for a missing disk on windows?
by me＠tdiehl.org 28 Apr '22

28 Apr '22

Hi, Is there a way to have Xymon alert if a disk is missing on a Windows server? I have the win_ps_client installed on them and I see the disks listed in the client data. Is it possible to configure Xymon to alert is one of those disks is missing? We had a server that one of the disks went off line after updates were installed and the machine was rebooted (Thanks Microsoft). Xymon was happy but the users were not. My Google foo has not yielded anything useful but I feel like this should be possible without resorting to Powershell weirdness. Regards, -- Tom me at tdiehl.org

4 7

Test page and alert message might specify different / wrong colour
by wim.nelis＠ziggo.nl 23 Apr '22

23 Apr '22

An HTML attachment was scrubbed... URL: <http://lists.xymon.com/pipermail/xymon/attachments/20220423/cb3492d9/attach…>

1 0

monitoring haproxy stats?
by mgoebel＠emich.edu 21 Apr '22

21 Apr '22

Hey, This is probably a longshot since I don't see any mentions of this in the mail archives but is anyone using xymon to monitor haproxy stats? Thanks, Matt -- Matthew Goebel : m <goebel at emunix.emich.edu>goebel at emich.edu : Unix Jockey @ EMU : Hail Eris Neo-Student, Net Lurker, Donut consumer, and procrastinating medher... "Always with the negative waves, Moriarty" - Oddball "Comfort the troubled, and trouble the comfortable." - Dietrich Bonhoeffer

1 0

clients in localmode cannot check files
by zechner＠vrvis.at 19 Apr '22

19 Apr '22

Hi, I've been running my clients in localmode for a while now and one thing that's not working at all, is checking for files. In a default configuration, there must be an entry in client-local.cfg and in analysis.cfg, but this does not apply to localmode. My file checks are configured in localclient.cfg on the clients (like the rest of the checks, which are working fine and as expected). I found an old question regarding this [1], but I was not able to solve my problems with this approach unfortunately. My file checks are just being ignored when clients are using localmode. Is there some trick to it respectively what am I doing wrong here? Thanks in advance! Cheers Christoph [1] https://lists.xymon.com/archive/2014-March/039417.html

2 4

Server migration
by stella＠rider.edu 11 Apr '22

11 Apr '22

Running an old version on old hardware and looking to migrate this to a new VM on RHEL8. I'd also most likely use Terbithia RPMs for this I saw a couple of posts on how to move all historical data over. Question for those who've done this, how did you handle IP addresses? I thought about setting up the server with a new temp IP address and have it do all outbound tests to make sure everything works. Then change the IP when decommissioning the old one. This would allow remote tests to report back. Or would you set up as new IP and have clients report to both, then remove the old one? TIA. -- ?(((=((===???(((================================================

3 3

Terebithia RPM's vs Build.
by john.horne＠plymouth.ac.uk 08 Apr '22

08 Apr '22

On Wed, 2022-03-30 at 09:51 -0400, Matt VanderWerf wrote: > > One thing to note if you're using NTP checks is that Xymon uses ntpdate by > default for these checks but ntpdate doesn't exist at all in any RHEL 8 repos > at all (neither does ntp), nor does it exist in any trusted third-party > repos, as it was completely replaced by chrony in RHEL 8 (chronyc is the > client tool). chrony also requires you make certain changes in the > chrony.conf file on any clients you use the NTP check on to allow the chronyc > remote commands to work. > > You can change the binary/path for what is used for the NTP checks and the > options used (NTPDATE and NTPDATEOPTS in xymonserver.cfg) but there are also > some hard coded parts in the source code which go by what it assumes the > ntpdate command output format looks like, which doesn't match chronyc output > at all. So if you use NTP checks, you'll need to figure out a way to extract > the chronyc output to match the ntpdate output or do without the NTP checks. > Note this is the same issue regardless if you're using the Xymon source or > the Terabithia RPMs. (See past discussion on this here and here.) > Hello, Yes, I noticed myself the lack of ntpdate in RHEL 8 based distributions. We are slowly moving our servers from CentOS 7 to Rocky Linux 8. As you say using chrony on a client server is not a problem. The problem is using chrony on the Xymon server itself. It is possible to get output from chrony containing some information by using: ===== chronyd -Q 'server <name or addr> iburst' 2022-04-08T17:33:22Z chronyd version 4.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 +DEBUG) 2022-04-08T17:33:22Z Disabled control of system clock 2022-04-08T17:33:27Z System clock wrong by 0.000660 seconds (ignored) 2022-04-08T17:33:27Z chronyd exiting ===== This works as a non-privileged user. So the above 'system clock wrong' bit is equivalent to the ntpdate offset value. Looking at the Xymon source, the ntp test already supports output formats from sntp and ntpdate. It should therefore be able to add in another section for the 'chronyd' output. I have also found that under Fedora 35, but not RHEL 8, Debian 11 or CentOS Stream 9, there is an 'ntpcheck' package which provides various NTP utility commands. One of those replicates 'ntpdate' (again it works as a non-privileged user): ======== ntpcheck utils ntpdate -s 10.10.10.10 Server: 10.10.10.10:123, Stratum: 2, Requests 3 Last Request: Offset: -0.000438s (-438us) | Delay: 0.028902s (28902us) Correct Time is 2022-04-08 18:07:55.767783923 +0100 BST Average (3 requests): Offset: -0.000519s (-519us) | Delay: 0.028867s (28867us) ======== I would say that if Xymon is to be modified, then it might be useful to cater for the 'ntpcheck' output as well. (The code itself seems to be relatively new; started in 2020 it seems on github.) In our case we already make a small modification to Xymon such that if the client stratum goes above a per-host configured value, then the test turns red. The problem is that the chrony output does not show the stratum value! I will have to think about this, but options look like build from source ntp/ntpdate (it is only the ntpdate command itself that we need); rebuild the ntpcheck package for Rocky 8; or modify the chronyd output so as to include the stratum value. (Really don't fancy doing that, and will need to check the very latest chrony code and site to see if this is already something in the pipeline. John. -- John Horne | Senior Operations Analyst | Technology and Information Services University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

2 1