September 2017 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

Need help with upgrade from 4.3.17 to 4.3.20
by vadud3＠gmail.com 17 Jul '18

17 Jul '18

I am not sure what I need to after the upgrade for the columns to show data instead of something like this ELF �� >� �� @��@�� Þ ��@�8� �@�%�"� �� @��@�@��@�@��ø ��ø �� 8 ��8 @��8 @�� @��@��ää��ää�� I removed FollowSymLinks from xymon-apache.conf file under /xymon-cgi/ and /xymon-seccgi/ and restarted. I am using ubuntu -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

9 29

Recovery notifications not generated when test goes red->yellow->green?
by hobbit＠elyograg.org 13 Dec '17

13 Dec '17

It appears that if a test goes from the red state, to the yellow state, and then to the green state, no recovery notification is sent. If it goes directly from red to green, then we get a notification. Initial notifications are only sent on RED and PURPLE. We have recovery notifications configured for most tests. Xymon server is 4.3.21, built using makerpm.sh and installed with yum on CentOS 6. I do not know whether our previous version (4.3.14) behaved similarly. It was almost an accident that I discovered the problem at all. Is this expected behavior? If it is, can I change it? Thanks, Shawn

4 3

CLASS not working as expected
by steve＠opendium.com 11 Dec '17

11 Dec '17

Under Xymon 4.3.25, I'm trying to use the class attribute to group hosts together by what monitoring and alerting they need. So for each host I have a hosts.cfg line like: 192.0.2.1 somehost.example.com # CLASS:Foo trace ssh Then in analysis.cfg I have: CLASS=Foo PROC this PROC that And alerts.cfg has: CLASS=Foo SCRIPT=blah However, the tests in analysis.cfg are not being applied. If I replace CLASS=Foo with HOST=somehost.example.com they work as expected. Also, CLASS=linux seems to work, which implies that the CLASS: directive in the hosts.cfg is being ignored. As far as I can tell, the messages on xymon's "client" channel don't contain the class name that was specified in hosts.cfg, so xymond_client is using the class that the client sent instead. Other channels, such as "status" do include the class name that was specified in hosts.cfg. Am I misunderstanding how the class is supposed to be used, or is this a bug? -- - Steve Hill Technical Director Opendium Limited http://www.opendium.com Direct contacts: Instant messager: xmpp:steve at opendium.com Email: steve at opendium.com Phone: sip:steve at opendium.com Sales / enquiries contacts: Email: sales at opendium.com Phone: +44-1792-824568 / sip:sales at opendium.com Support contacts: Email: support at opendium.com Phone: +44-1792-825748 / sip:support at opendium.com

7 9

Alert transition yellow -> red with repeat problem
by john.r.rothlisberger＠accenture.com 27 Nov '17

27 Nov '17

This problem contributed to an outage last night - something is wrong. Last night we had a disk that was in the warning state - warning email sent. That disk then went into an alert state and an alert email was triggered (right away as the DURATION value took into account the time it was yellow) and then again 15 minutes later as designed. THEN - another disk on that server went yellow. It did NOT trigger any emails (as expected we should continue to focus on the alerts) but it somehow interfered with the REPEAT time of the alerts and those STOPPED. There is a bug somewhere. Thanks, John Upcoming PTO: _____________________________________________________________________ John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture 312.693.3136 office _____________________________________________________________________ From: Rothlisberger, John R. Sent: Tuesday, April 11, 2017 7:49 AM To: 'xymon >> xymon at xymon.com' <xymon at xymon.com> Subject: Alert transition yellow -> red with repeat problem This is a problem I have seen for a long long time and have actually brought it up on the list before. Xymon 4.3.21 Ubuntu 14.04LTS The problem I have (in this instance) a warning which is to be repeated daily suddenly goes red and triggers a single alert but doesn't repeat again until the repeat time of the warning has passed. >From the notification.log: Tue Mar 28 05:31:28 2017 ServerA.disk (IP) disk_warn 1490697082 100 <- warning sets a repeat time of 1 day Tue Mar 28 05:53:32 2017 ServerA.disk (IP) disk_alert 1490698405 100 <- minutes later it goes red (red repeat time is 15 minutes but no further alerts are generated) Next alert comes out 1 day after the above warning: Wed Mar 29 05:31:31 2017 ServerA.disk (IP) disk_alert 1490783486 100 <- 1 day after previous warning. This should have been repeated every 15 minutes. Wed Mar 29 05:46:40 2017 ServerA.disk (IP) disk_alert 1490784394 100 <- now, the repeat time is 15 minutes ... <- and continues every 15 minutes. This alert went a full 24 hours with only a single notification. :( I have seen this before (not always) where the repeat time in a warning overrides a follow up alert until the warning repeat time has expired. Alert rules: SCRIPT /home/xymon/server/ext/pg/exwarn_SCRIPT disk_warn DURATION>30 REPEAT=1d COLOR=yellow SERVICE=disk FORMAT=TEXT UNMATCHED SCRIPT /home/xymon/server/ext/pg/exalert_SCRIPT disk_alert DURATION>20 REPEAT=15 COLOR=red SERVICE=disk FORMAT=TEXT UNMATCHED Ideas/thoughts? Thanks, John Upcoming PTO: 4/3 _____________________________________________________________________ John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture 312.693.3136 office _____________________________________________________________________ ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com

3 5

Xymon to monitor HP OpenVMS
by Jose.LeusJr＠analog.com 19 Nov '17

19 Nov '17

Hi Xymon Team, I am a big fan of your monitoring system. I started to use hobbit since 2007 and now using xymon to watch my systems being manage. OpenVMS is one of the system that I need to monitor closely. At this time I was able to monitor hdd, connection , procs and cpu only. But as you can see below, the graph for cpu utilization is not displayed. I still need to add other system resources but im having difficulties to integrate it. I would like to seek for your expert assistance to fully monitor our OpenVMS system. Please advise if we can set a conference call to discuss. Our team will allocate budget for development work needed. I am hoping for your positive response on this matter. Have a nice day to all! [cid:image001.jpg at 01D32E15.9E17FF00][cid:image002.jpg at 01D32E15.9E17FF00] Sincerely, Leus

2 2

Big RRD files causing problems
by David.Mills＠hhsc.state.tx.us 09 Nov '17

09 Nov '17

Hi! Since upgrading to a new Xymon ( 4.3.28-1.el6.terabithia<http://xymon.sourceforge.net/> / Linux from 4.3.3 on Solaris ) , my RRD file storage consumption is *much* greater than on the old server. It's gotten so I have to manually groom the RRD files every 48 hours or they overflow the file system. Here are some stats: NOTE: Same set of hosts with same tests on both the old server (4.3.3 / Solaris) and the Terabithia distro for RHEL 6: Picking out the current biggest offender (the list varies but tracks pretty much along the lines of the largest hosts with the most data to send): NEW Server OLD Server Total space consumed for .../rrd 90GB 14GB # of files beneath .../rrd/pwow706/ 1,916 25,815 Avg file size (eyeballing it) 423K 38K Any ideas? Where can I read up on what RRD files Xymon puts into the RRD directory? Thx! ;-) David Mills ~~~~~~~~~~~~~~~~~~~~~~~~ System Administrator Northrop Grumman Texas Health and Human Services Commission

2 2

Windows Powershell client v2.21
by zak.beck＠accenture.com 20 Oct '17

20 Oct '17

Hi v2.21 of XymonPSClient has been committed. This has a couple of new minor features and a minor fix, as follows: * Added reporting data to the server using UTF8 encoding. This is optional and the default remains to use ASCII. Use at your own risk - there's a possibility that not all server compiles will support receiving UTF8. (thanks Luboš Pavlíček) * Added the ability to include 'local' data in the core client message. This replicates functionality found in the Linux client - basically you can have external scripts write data to a file under the local folder (location specified by setting localdatalocation) and it will get written to a section in the client message like [local:<filename>]. You can then use a command like this to retrieve it: xymon 0 "clientlog <client host> section=local:<filename>" (thanks David Boyer for the inspiration!) * Fixed the Powershell provider appearing in file, log, dirsize and other tests - you would have only seen this if you were using UNC paths, it would have appeared like this: [file:Microsoft.PowerShell.Core\FileSystem::\\server\share\filename.txt], it should now appear like this: [file:\\server\share\filename.txt] I have also updated the documentation to include the new features. I have also updated nssm.exe in the repository. If you are running the client on Windows 10 Creator's Edition, you will need to use this new updated nssm.exe. Thanks Zak ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com

2 5

Forwarding custom status messages between Xymon servers
by James.E.Ness＠usdoj.gov 02 Oct '17

02 Oct '17

To start out, my environment. Two Xymon servers, XYMON1 and XYMON2. XYMON1 is the master, and XYMON2 is the servant. At this time, there are a handful of machines using the Xymon client to report on themselves to XYMON2, which is dutifully forwarding that info on to XYMON1 (as expected). The clients also provide some custom status messages, which are NOT being forwarded from XYMON2 to XYMON1. I see they are picked up and properly processed by XYMON2, but that's it. Previously, I had set XYMON2 as just a proxy, and both the client data and the custom status messages were sent to XYMON1. I can't go back to that, because XYMON1 will be decommissioned in the near future, and I need XYMON2 to take over. (I also have some other instances that I will eventually need to forward client data and status messages to XYMON2, but that is for later.) I've looked through so many man pages and can't find anything else that would point to what I might need to set to get this working, so I'm hoping someone on here can help me out. Or is this functionality that is not meant to work, no matter what I do? Thank you in advance for your help.

3 3

First time installed, and set up xymon, failed, NEED helps please.
by EDSchminke＠Hormel.com 28 Sep '17

28 Sep '17

*****PLEASE***** do NOT leave SELinux in permissive mode. ( http://stopdisablingselinux.com/) I have beat SELinux into submission to make Xymon work the way I need it to. You can do the same by following my procedure below, or from watching Thomas Cameron's lecture from RedHat Summit a couple years ago "SELinux For Mere Mortals" (https://www.youtube.com/watch?v=cNoVgDqqJmM) I built mine from the tips given in this video. As root: #> setsebool -P httpd_enable_homedirs on #> setsebool -P httpd_read_user_content on A few things can't be done for Xymon by simply changing SELinux booleans. I've curated a number of SELinux policy exceptions over the past couple years in order to make Xymon and SELinux play nice together. You can create yours by doing this: ## vvvvvv ---- copy everything below this line ---- vvvvvv ## module xymon 1.0; require { type unconfined_t; type var_log_t; type initrc_t; type admin_home_t; type httpd_t; type user_home_t; type fonts_cache_t; type port_t; class tcp_socket name_connect; class file { rename execute setattr read create execute_no_trans write getattr unlink open }; class sock_file write; class lnk_file { create unlink }; class unix_dgram_socket sendto; class dir { write rmdir setattr remove_name create add_name }; } #============= httpd_t ============== allow httpd_t admin_home_t:file { read getattr open }; allow httpd_t fonts_cache_t:dir setattr; allow httpd_t initrc_t:unix_dgram_socket sendto; allow httpd_t port_t:tcp_socket name_connect; allow httpd_t unconfined_t:unix_dgram_socket sendto; allow httpd_t user_home_t:dir rmdir; allow httpd_t user_home_t:dir { write remove_name create add_name }; allow httpd_t user_home_t:file setattr; allow httpd_t user_home_t:file { rename write execute create unlink execute_no_trans }; allow httpd_t user_home_t:lnk_file { create unlink }; allow httpd_t user_home_t:sock_file write; allow httpd_t var_log_t:file read; EOF ## ^^^^^ ---- to everything above this line ---- ^^^^^^ ## Paste what you've copied into a file-- doesn't matter where; I've used the name "xymon.te" #> vi xymon.te Run the following commands to build the SELinux policy module: #> checkmodule -M -m -o xymon.mod xymon.te #> semodule_package -m xymon.mod -o xymon.pp Run this command to install the policy module. #> semodule -i xymon.pp Change your /etc/sysconfig/selinux back to "enforcing". Reboot. If you see any funkiness, watch /var/log/audit/audit.log for AVC denials. #> grep type=AVC /var/log/audit/audit.log | grep denied If you see anything in there, it means it's time to "build a policy exception" not "disable SELinux". -------------------------------------------------------------------------------- Everyone was right on followings: added this line at the bottom of file /etc/httpd/conf/httpd.conf: include /home/xymon/server/etc/xymon-apache.conf and Paul Root was right about Selinux, so I did: modified file /etc/sysconfig/selinux #SELINUX=enforcing KERN testing .... SELINUX=permissive rebooted. it works now!!! Thank you!!!I can go home and feel good, will do more learning tomorrow :) Let's go GreenThis email contains 100% recycled electrons. Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com

2 1