October 2017 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

Need help with upgrade from 4.3.17 to 4.3.20
by vadud3＠gmail.com 17 Jul '18

17 Jul '18

I am not sure what I need to after the upgrade for the columns to show data instead of something like this ELF �� >� �� @��@�� Þ ��@�8� �@�%�"� �� @��@�@��@�@��ø ��ø �� 8 ��8 @��8 @�� @��@��ää��ää�� I removed FollowSymLinks from xymon-apache.conf file under /xymon-cgi/ and /xymon-seccgi/ and restarted. I am using ubuntu -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

9 29

Recovery notifications not generated when test goes red->yellow->green?
by hobbit＠elyograg.org 13 Dec '17

13 Dec '17

It appears that if a test goes from the red state, to the yellow state, and then to the green state, no recovery notification is sent. If it goes directly from red to green, then we get a notification. Initial notifications are only sent on RED and PURPLE. We have recovery notifications configured for most tests. Xymon server is 4.3.21, built using makerpm.sh and installed with yum on CentOS 6. I do not know whether our previous version (4.3.14) behaved similarly. It was almost an accident that I discovered the problem at all. Is this expected behavior? If it is, can I change it? Thanks, Shawn

4 3

CLASS not working as expected
by steve＠opendium.com 11 Dec '17

11 Dec '17

Under Xymon 4.3.25, I'm trying to use the class attribute to group hosts together by what monitoring and alerting they need. So for each host I have a hosts.cfg line like: 192.0.2.1 somehost.example.com # CLASS:Foo trace ssh Then in analysis.cfg I have: CLASS=Foo PROC this PROC that And alerts.cfg has: CLASS=Foo SCRIPT=blah However, the tests in analysis.cfg are not being applied. If I replace CLASS=Foo with HOST=somehost.example.com they work as expected. Also, CLASS=linux seems to work, which implies that the CLASS: directive in the hosts.cfg is being ignored. As far as I can tell, the messages on xymon's "client" channel don't contain the class name that was specified in hosts.cfg, so xymond_client is using the class that the client sent instead. Other channels, such as "status" do include the class name that was specified in hosts.cfg. Am I misunderstanding how the class is supposed to be used, or is this a bug? -- - Steve Hill Technical Director Opendium Limited http://www.opendium.com Direct contacts: Instant messager: xmpp:steve at opendium.com Email: steve at opendium.com Phone: sip:steve at opendium.com Sales / enquiries contacts: Email: sales at opendium.com Phone: +44-1792-824568 / sip:sales at opendium.com Support contacts: Email: support at opendium.com Phone: +44-1792-825748 / sip:support at opendium.com

7 9

How to calculate CPU Load Average
by velpulavenkat6＠gmail.com 02 Dec '17

02 Dec '17

Hi Team, 1)Please help me how to calculate cpu load average i need to configure alerts in xymon. 2)if i have 4 cores in server what is average load. 3)in xymon default values LOOAD 5.0 10.0 how to configured these values. NOTE : which values based on configured defaults values (LOOAD 5.0 10.0 ) in xymon. Regards Venkat Yadav

4 3

Alert transition yellow -> red with repeat problem
by john.r.rothlisberger＠accenture.com 27 Nov '17

27 Nov '17

This problem contributed to an outage last night - something is wrong. Last night we had a disk that was in the warning state - warning email sent. That disk then went into an alert state and an alert email was triggered (right away as the DURATION value took into account the time it was yellow) and then again 15 minutes later as designed. THEN - another disk on that server went yellow. It did NOT trigger any emails (as expected we should continue to focus on the alerts) but it somehow interfered with the REPEAT time of the alerts and those STOPPED. There is a bug somewhere. Thanks, John Upcoming PTO: _____________________________________________________________________ John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture 312.693.3136 office _____________________________________________________________________ From: Rothlisberger, John R. Sent: Tuesday, April 11, 2017 7:49 AM To: 'xymon >> xymon at xymon.com' <xymon at xymon.com> Subject: Alert transition yellow -> red with repeat problem This is a problem I have seen for a long long time and have actually brought it up on the list before. Xymon 4.3.21 Ubuntu 14.04LTS The problem I have (in this instance) a warning which is to be repeated daily suddenly goes red and triggers a single alert but doesn't repeat again until the repeat time of the warning has passed. >From the notification.log: Tue Mar 28 05:31:28 2017 ServerA.disk (IP) disk_warn 1490697082 100 <- warning sets a repeat time of 1 day Tue Mar 28 05:53:32 2017 ServerA.disk (IP) disk_alert 1490698405 100 <- minutes later it goes red (red repeat time is 15 minutes but no further alerts are generated) Next alert comes out 1 day after the above warning: Wed Mar 29 05:31:31 2017 ServerA.disk (IP) disk_alert 1490783486 100 <- 1 day after previous warning. This should have been repeated every 15 minutes. Wed Mar 29 05:46:40 2017 ServerA.disk (IP) disk_alert 1490784394 100 <- now, the repeat time is 15 minutes ... <- and continues every 15 minutes. This alert went a full 24 hours with only a single notification. :( I have seen this before (not always) where the repeat time in a warning overrides a follow up alert until the warning repeat time has expired. Alert rules: SCRIPT /home/xymon/server/ext/pg/exwarn_SCRIPT disk_warn DURATION>30 REPEAT=1d COLOR=yellow SERVICE=disk FORMAT=TEXT UNMATCHED SCRIPT /home/xymon/server/ext/pg/exalert_SCRIPT disk_alert DURATION>20 REPEAT=15 COLOR=red SERVICE=disk FORMAT=TEXT UNMATCHED Ideas/thoughts? Thanks, John Upcoming PTO: 4/3 _____________________________________________________________________ John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture 312.693.3136 office _____________________________________________________________________ ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com

3 5

Xymon to monitor HP OpenVMS
by Jose.LeusJr＠analog.com 19 Nov '17

19 Nov '17

Hi Xymon Team, I am a big fan of your monitoring system. I started to use hobbit since 2007 and now using xymon to watch my systems being manage. OpenVMS is one of the system that I need to monitor closely. At this time I was able to monitor hdd, connection , procs and cpu only. But as you can see below, the graph for cpu utilization is not displayed. I still need to add other system resources but im having difficulties to integrate it. I would like to seek for your expert assistance to fully monitor our OpenVMS system. Please advise if we can set a conference call to discuss. Our team will allocate budget for development work needed. I am hoping for your positive response on this matter. Have a nice day to all! [cid:image001.jpg at 01D32E15.9E17FF00][cid:image002.jpg at 01D32E15.9E17FF00] Sincerely, Leus

2 2

Big RRD files causing problems
by David.Mills＠hhsc.state.tx.us 09 Nov '17

09 Nov '17

Hi! Since upgrading to a new Xymon ( 4.3.28-1.el6.terabithia<http://xymon.sourceforge.net/> / Linux from 4.3.3 on Solaris ) , my RRD file storage consumption is *much* greater than on the old server. It's gotten so I have to manually groom the RRD files every 48 hours or they overflow the file system. Here are some stats: NOTE: Same set of hosts with same tests on both the old server (4.3.3 / Solaris) and the Terabithia distro for RHEL 6: Picking out the current biggest offender (the list varies but tracks pretty much along the lines of the largest hosts with the most data to send): NEW Server OLD Server Total space consumed for .../rrd 90GB 14GB # of files beneath .../rrd/pwow706/ 1,916 25,815 Avg file size (eyeballing it) 423K 38K Any ideas? Where can I read up on what RRD files Xymon puts into the RRD directory? Thx! ;-) David Mills ~~~~~~~~~~~~~~~~~~~~~~~~ System Administrator Northrop Grumman Texas Health and Human Services Commission

2 2

[XYMON] agents with pulldata - xymonfetch and ssh tunelling
by pierre.lapha＠gmail.com 02 Nov '17

02 Nov '17

Hello, I am trying to set up xymonfetch/msgcache with ssh tunnelling (because security does not want to open flows on port 1984). I have enabled xymonfetch (in tasks.cfg) and added pulldata (in hosts.cfg) on server side and also msgcache (in clientlaunch.cfg) and XYMSRV=127.0.0.1 (in xymonclient.cfg). I have done a ssh tunnel (ssh -L 1234:127.0.0.1:1984 IP_of_the_client) from the xymon server. My tunnel is ok but xymon logs still show that nothing happens, no pulldata. Is there something else I should do to enable this function? Any help would be appreciated. Thank you! Pierre

3 5

Bogus hosts filling up alert.log
by David.Mills＠hhsc.state.tx.us 01 Nov '17

01 Nov '17

Hi, all! I have recently set up a new Xymon (Xymon 4.3.28-1.el6.terabithia<http://xymon.sourceforge.net/>) server on RHEL 6.7 that, for the most part, is doing just fine. However, I've discovered that my /var/log/xymon/alert.log file is growing at a crazy rate to the point it periodically needs to zero'd-out or it will swamp the file system. The problem is every 15 seconds the /var/log/xymon/alert.log file receives a flurry of log entries like this: ... 2017-10-31 14:53:55 Checking criteria for host '0FS_94_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_94_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_94_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_94_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_94_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_94_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_94_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_94_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_95_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_95_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_95_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_95_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_95_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_95_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_95_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_95_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_96_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_96_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_96_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_96_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_96_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_96_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_96_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire 2017-10-31 14:53:55 Checking criteria for host '0FS_96_192_168_22_1__export_', which is not yet defined; some alerts may not immediately fire ... The '0FS_96_192_168_22_1__export_' is actually the name of a host I've defined in the past, but is no longer in xymon's memory (AFAIK!!) I have reduced the alerts.cfg down to a minimal stub, commenting out the "directory /etc/xymon/alerts.d/..." directive, "rm -r"'d any reference to this host (and others similar to it) under the data files directories (e.g. rrd/, hist/, histlogs/, hostdata/, etc.). I have gone as far as running "find /etc/xymon -type f | xargs egrep 0FS_" looking for "surprises". I've also stopped / restarted the server and scanned what's active in memory via "xymon localhost xymondboard | egrep 0FS_". This "host" is not a real client host but an artifact I've created on the server side to represent a file system I'm monitoring in a server-side ext script, so I know it is not announcing it's presence over port 1984. For the life of me I can't figure out where the alerts daemon is running across this hostname. Help? ~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~ David Mills Systems Administrator Northrop Grumman (512) 595-1238 (mobile)

3 7