February 2018 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

[Patch] xymond_rrd crash when parsing Linux ifconfig data
by steve＠opendium.com 22 Feb '19

22 Feb '19

This patch addresses two problems: 1. For Linux clients, do_ifstat_rrd parses the ifconfig output using the regex: ([a-z0-9]+(_[0-9]+)?:*|lo:?)\s pickdata() is called like this: pickdata(bol, ifstat_linux_pcres[0], 1, &ifname) With a network interface name such as "foo1", the regex produces 2 results: 0: "foo1 " 1: "foo1" pickdata() discards result 0 and assigns result 1 ("foo1") to the ifname parameter as expected. With a network interface name such as "foo_1", the regex produces 3 results: 0: "foo_1 " 1: "foo_1" 2: "_1" pickdata() discards result 0, assigns result 1 ("foo_1") to the ifname parameter and then tries to assign result 2 ("_1") to the next parameter in the variable argument list. Since there it no such parameter, this results in a segmentation fault and xymond_rrd crashes. The attached patch adds a "num_args" parameter to pickdata() to tell it how many arguments are in the variable argument list and ensure it never rolls off the end of that list. 2. I have some clients with network interface names in the form "foo.5", "foo_bar" and "foo-bar" ("eth0.5" is especially common when eth0 is carrying an 802.1q trunk, for example). Xymon does not collect statistics on those network interfaces because they don't match the regex. This patch relaxes the regex a bit so that it matches these network interfaces. -- - Steve Hill Technical Director Opendium Online Safety / Web Filtering http://www.opendium.com Enquiries: sales at opendium.com +44-1792-824568 Support: support at opendium.com +44-1792-825748 Opendium Limited is a company registered in England and Wales. Company No. 5465437 Highfield House, 1 Brue Close, Bruton, Somerset, BA10 0HY, England.

2 1

Problems with Content Security Policy in Safari, Chrome, and IE
by jtrott＠dancrai.com 17 Jul '18

17 Jul '18

Xymon 4.3.28-1.el7.terabithia with Safari 11 on High Sierra and Safari on iOS 11. Problem occurs on the trends page. https://xymon.domain.com.au/xymon-cgi/svcstatus.sh?HOST=host.com.au&SERVICE… If you click on any of the time based buttons, 48hrs for example, the requested page doesn't load. Safari on macOS look like it's loading a page but doesn't get anywhere. Safari on iOS does nothing at all when you tap the button. The console in Safari reveals the following error: Refused to load https://xymon.domain.com.au/xymon-cgi/svcstatus.sh?HOST=host.com.au&SERVICE… because it does not appear in the form-action directive of the Content Security Policy. Checking the headers shows this content security policy: Content-Security-Policy: script-src 'self' 'unsafe-inline'; connect-src 'self'; form-action 'self'; sandbox allow-forms allow-scripts; I'm not that well versed in the CSP stuff, but I note that it also fails with the same error in the latest Chrome 62.0.3202.89, and in Internet Explorer 11.0.9600.18817 (no error logged), but works in the latest Firefox 56.0.2. Has anyone else run into this issue, or has any more information on how I can modify the CSP headers to test? I tried using Header set Content-Security-Policy in apache but that seems to add an improperly formatted addition to the rules rather than overwriting them. Thanks, JT

7 11

Need help with upgrade from 4.3.17 to 4.3.20
by vadud3＠gmail.com 17 Jul '18

17 Jul '18

I am not sure what I need to after the upgrade for the columns to show data instead of something like this ELF �� >� �� @��@�� Þ ��@�8� �@�%�"� �� @��@�@��@�@��ø ��ø �� 8 ��8 @��8 @�� @��@��ää��ää�� I removed FollowSymLinks from xymon-apache.conf file under /xymon-cgi/ and /xymon-seccgi/ and restarted. I am using ubuntu -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

9 29

Force logfetch to only process complete lines?
by larry＠fni-stl.com 14 May '18

14 May '18

RHEL 6.9 and RHEL 7.4 Xymon v4.3.28 This may be documented somewhere and I'm just not able to find it. But is there a way to force logfetch to only scan complete lines and discard any partials it might retrieve based on the MAXCHECK setting? I've been getting quite a few alerts on highly active systems where the offending line would normally be excluded due to the first part of a search that is missing. A simple example, I want to ignore the alert triggers for /var/log/messages where the system name is test-system and :\sheader\ssubject: is also in the line. Since test-system comes right after the date/time stamp, that causes the ignore check to not work if test-system is not retrieved by logfetch. analysis.cfg # Red alert on CRITICAL or ERROR or SERIOUS (with exceptions) LOG %.* %(?-i)CRITICAL|ERROR|SERIOUS COLOR=red IGNORE=%(?-i)test-system.*:\sheader\ssubject: I've tried adjusting the MAXCHECK setting but it didn't make a difference one way or the other. client-local.cfg log:/var/log/messages:10240 # 10KB default log:/var/log/messages:1024000 # 1MB Thanks. ========================================================= Larry D. Bonham Financial Network Inc. 10401-F Baur Olivette, MO 63132 (314) 400-9412 voice (314) 997-5647 fax ========================================================= ________________________________ CONFIDENTIALITY NOTICE: This electronic mail message is intended exclusively for recipient to which it is addressed. The contents of this message and any attachments may contain confidential and privileged information. Any unauthorized review, use, print, storage, copy, disclosure or distribution is strictly prohibited. If you have received this message in error, please advise the sender immediately by replying to the message's sender and delete all copies of this message and its attachments without disclosing the contents to anyone, or using the contents for any purpose.

5 11

Help with removing top line of Combo Status page
by APatel＠kgsalpha.com 29 Mar '18

29 Mar '18

Hello all, I need assistance with customizing the status page generated of a combo test. On the top (under the date/time), there is a line that lists the following: test1&&test2&&test3... = 1&&1&&1&&1.. = 1. How do I get rid of this line? (See the yellow highlighted in the attached image) [cid:image002.png at 01D3ABD4.944F5580] This is informational only and is not a solicitation, commitment or confirmation. Transaction details are subject to official confirmation and terms of business. We may make markets, buy/sell as principal or have a position in financial instruments herein. We do not warrant that information is accurate, complete or that any returns will be achieved. Price/availability is subject to change. Changes to assumptions may have a material impact on price/returns. Other than transactions in financial instruments, any agreement KGS enters into must be executed by a KGS authorized signatory. If you received in error, please delete and advise sender. Communications are subject to review and archiving. Changes to assumptions may have material impact on price or returns. For more information, please contact your representative. If you received this message in error, please delete the message and advise the sender. Communications to and from our firm are subject to review and archiving.

2 4

Xymonproxy and an unreachable server
by wim.nelis＠ziggo.nl 27 Mar '18

27 Mar '18

On a Raspberry Pi zero W xymon-client is running to monitor some sensors and the PRi0 itself. As the xymon server itself is not reachable by times, a minimal xymon-server is running too on the RPi0. Xymonproxy is used to distribute the status and data messages to both the local xymon server and the primary xymon server. The intention of this setup is to have all the RRD's locally complete at the RPi0. If needed, the RRDs can be copied from the RPi0 to the primary xymon server. The local xymon server is listening to port 1985, the xymonproxy to port 1984. The latter distributes the messages to two servers, using parameter "--server=127.0.0.1:1985,192.168.178.72:1984". This setup is working, but the graphs created from the local RRD's contain gaps in the periods that the primary xymon server is not reachable. The logfiles of the clients running on the RPi0 contain messages like the following, about twice an hour: 2018-02-21 06:10:01.664965 Whoops ! Failed to send message (Connection failed) 2018-02-21 06:10:01.665673 -> Could not connect to Xymon daemon at 127.0.0.1:1984 (Connection refused) 2018-02-21 06:10:01.665767 -> Recipient '127.0.0.1', timeout 15 2018-02-21 06:10:01.665851 -> 1st line: 'status rpi00.mve green Wed 2018.02.21 06:10:01' This does explain the gaps. The logfile of xymonproxy shows that the proxy is restarted a dozen times per hour: 2018-02-21 05:55:38.272757 xymonproxy version 4.3.28 starting 2018-02-21 05:55:38.273605 Listening on 0.0.0.0:1984 2018-02-21 05:55:38.273751 Sending to Xymon server(s) 127.0.0.1:1985 192.168.178.72:1984 2018-02-21 05:56:05.304985 Server not responding, message lost 2018-02-21 06:00:30.195973 Server not responding, message lost 2018-02-21 06:00:36.221908 Server not responding, message lost 2018-02-21 06:00:41.231668 Server not responding, message lost 2018-02-21 06:00:41.236076 Server not responding, message lost *** buffer overflow detected ***: /usr/lib/xymon/server/bin/xymonproxy terminated 2018-02-21 06:00:42.269357 xymonproxy version 4.3.28 starting 2018-02-21 06:00:42.270200 Listening on 0.0.0.0:1984 2018-02-21 06:00:42.270346 Sending to Xymon server(s) 127.0.0.1:1985 192.168.178.72:1984 2018-02-21 06:01:09.301618 Server not responding, message lost 2018-02-21 06:05:29.188224 Server not responding, message lost 2018-02-21 06:05:40.201194 Server not responding, message lost 2018-02-21 06:05:45.208531 Server not responding, message lost 2018-02-21 06:05:45.208936 Server not responding, message lost 2018-02-21 06:05:45.209058 Server not responding, message lost *** buffer overflow detected ***: /usr/lib/xymon/server/bin/xymonproxy terminated 2018-02-21 06:10:45.237707 xymonproxy version 4.3.28 starting 2018-02-21 06:10:45.239061 Listening on 0.0.0.0:1984 2018-02-21 06:10:45.239219 Sending to Xymon server(s) 127.0.0.1:1985 192.168.178.72:1984 2018-02-21 06:11:11.272425 Server not responding, message lost I have been playing with the queue length, but to no avail. Is it possible to have xymonproxy not to terminate every 5 minutes, but just report the inability to send a message to a particular server? Regards, Wim Nelis.

2 1

Include an image on a page
by colin.coe＠gmail.com 04 Mar '18

04 Mar '18

Hi all Apologies if this has been asked and answered previously but is it possible to include a .png file at the end of a group (in the hosts.cfg file)? Thanks CC

2 1

critical.cfg format
by john.r.rothlisberger＠accenture.com 26 Feb '18

26 Feb '18

After many years of using Xymon I am taking a look at using critical.cfg to make things easier to read (as opposed to the all-nongreen screen). But, I am having trouble getting things to work when I want to use multiple templates for servers. Here is what works (multiple tests): Servername1|disk||||1|Resolver Group|Instructions|<modified by> Servername1|svcs||||2|Resolver Group|Instructions|<modified by> Servername1|msgs||||3|Resolver Group|Instructions|<modified by> This also works: Servername1|=TEMPLATE1 TEMPLATE1|disk||||1|Resolver Group|Instructions|<modified by> Does not work: Servername1|=TEMPLATE1 servername1|=TEMPLATE2 servername1|=TEMPLATE3 TEMPLATE1|disk||||1|Resolver Group|Instructions|<modified by> TEMPLATE2|svcs||||2|Resolver Group|Instructions|<modified by> TEMPLATE3|msgs||||3|Resolver Group|Instructions|<modified by> Also does not work: Servername1|=TEMPLATE1,TEMPLATE2,TEMPLATE3 TEMPLATE1|disk||||1|Resolver Group|Instructions|<modified by> TEMPLATE2|svcs||||2|Resolver Group|Instructions|<modified by> TEMPLATE3|msgs||||3|Resolver Group|Instructions|<modified by> Also does not work: Servername1|=TEMPLATE1 TEMPLATE1|disk,svcs,msgs||||1|Resolver Group|Instructions|<modified by> I have tried multiple combinations using spaces, commas, colons - nothing seems to work. I have tried adding doing cloning via the edit critical systems but that doesn't help either. Can I use regular expressions in critical.cfg to match multiple server names such as Servername*? Thanks, John Upcoming PTO: _____________________________________________________________________ John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture 312.693.3136 office _____________________________________________________________________ ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com

1 0

Acknowledgement via email not working
by APatel＠kgsalpha.com 22 Feb '18

22 Feb '18

Hello, I recently installed the Terabithia Xymon on CentOS 7 and having no luck to get the acknowledgements via email working. The procmail.log shows the following: >From me at myhost.com Thu Feb 22 09:00:49 2018 Subject: RE: myserver.cpu CRITICAL (RED) XYMON [628942188] Folder: /usr/share/xymon/server/bin/xymon-mailack --env=/usr/share/x 6083 2018-02-22 09:00:49.381891 Whoops ! Failed to send message (Connection failed) 2018-02-22 09:00:49.381979 -> connect to Xymon daemon at mycentosIP:1984 failed (Permission denied) 2018-02-22 09:00:49.381987 -> Recipient 'mycentosIP', timeout 50 2018-02-22 09:00:49.381994 -> 1st line: 'xymondack 628942188 3 --_000_B0101C9B9936244D9191B24572BF76C3012125BA7DKGSMBX1KGSLOC_' This is informational only and is not a solicitation, commitment or confirmation. Transaction details are subject to official confirmation and terms of business. We may make markets, buy/sell as principal or have a position in financial instruments herein. We do not warrant that information is accurate, complete or that any returns will be achieved. Price/availability is subject to change. Changes to assumptions may have a material impact on price/returns. Other than transactions in financial instruments, any agreement KGS enters into must be executed by a KGS authorized signatory. If you received in error, please delete and advise sender. Communications are subject to review and archiving. Changes to assumptions may have material impact on price or returns. For more information, please contact your representative. If you received this message in error, please delete the message and advise the sender. Communications to and from our firm are subject to review and archiving.

1 0