March 2018 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

[Patch] xymond_rrd crash when parsing Linux ifconfig data
by steve＠opendium.com 22 Feb '19

22 Feb '19

This patch addresses two problems: 1. For Linux clients, do_ifstat_rrd parses the ifconfig output using the regex: ([a-z0-9]+(_[0-9]+)?:*|lo:?)\s pickdata() is called like this: pickdata(bol, ifstat_linux_pcres[0], 1, &ifname) With a network interface name such as "foo1", the regex produces 2 results: 0: "foo1 " 1: "foo1" pickdata() discards result 0 and assigns result 1 ("foo1") to the ifname parameter as expected. With a network interface name such as "foo_1", the regex produces 3 results: 0: "foo_1 " 1: "foo_1" 2: "_1" pickdata() discards result 0, assigns result 1 ("foo_1") to the ifname parameter and then tries to assign result 2 ("_1") to the next parameter in the variable argument list. Since there it no such parameter, this results in a segmentation fault and xymond_rrd crashes. The attached patch adds a "num_args" parameter to pickdata() to tell it how many arguments are in the variable argument list and ensure it never rolls off the end of that list. 2. I have some clients with network interface names in the form "foo.5", "foo_bar" and "foo-bar" ("eth0.5" is especially common when eth0 is carrying an 802.1q trunk, for example). Xymon does not collect statistics on those network interfaces because they don't match the regex. This patch relaxes the regex a bit so that it matches these network interfaces. -- - Steve Hill Technical Director Opendium Online Safety / Web Filtering http://www.opendium.com Enquiries: sales at opendium.com +44-1792-824568 Support: support at opendium.com +44-1792-825748 Opendium Limited is a company registered in England and Wales. Company No. 5465437 Highfield House, 1 Brue Close, Bruton, Somerset, BA10 0HY, England.

2 1

Problems with Content Security Policy in Safari, Chrome, and IE
by jtrott＠dancrai.com 17 Jul '18

17 Jul '18

Xymon 4.3.28-1.el7.terabithia with Safari 11 on High Sierra and Safari on iOS 11. Problem occurs on the trends page. https://xymon.domain.com.au/xymon-cgi/svcstatus.sh?HOST=host.com.au&SERVICE… If you click on any of the time based buttons, 48hrs for example, the requested page doesn't load. Safari on macOS look like it's loading a page but doesn't get anywhere. Safari on iOS does nothing at all when you tap the button. The console in Safari reveals the following error: Refused to load https://xymon.domain.com.au/xymon-cgi/svcstatus.sh?HOST=host.com.au&SERVICE… because it does not appear in the form-action directive of the Content Security Policy. Checking the headers shows this content security policy: Content-Security-Policy: script-src 'self' 'unsafe-inline'; connect-src 'self'; form-action 'self'; sandbox allow-forms allow-scripts; I'm not that well versed in the CSP stuff, but I note that it also fails with the same error in the latest Chrome 62.0.3202.89, and in Internet Explorer 11.0.9600.18817 (no error logged), but works in the latest Firefox 56.0.2. Has anyone else run into this issue, or has any more information on how I can modify the CSP headers to test? I tried using Header set Content-Security-Policy in apache but that seems to add an improperly formatted addition to the rules rather than overwriting them. Thanks, JT

7 11

Need help with upgrade from 4.3.17 to 4.3.20
by vadud3＠gmail.com 17 Jul '18

17 Jul '18

I am not sure what I need to after the upgrade for the columns to show data instead of something like this ELF �� >� �� @��@�� Þ ��@�8� �@�%�"� �� @��@�@��@�@��ø ��ø �� 8 ��8 @��8 @�� @��@��ää��ää�� I removed FollowSymLinks from xymon-apache.conf file under /xymon-cgi/ and /xymon-seccgi/ and restarted. I am using ubuntu -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

9 29

Force logfetch to only process complete lines?
by larry＠fni-stl.com 14 May '18

14 May '18

RHEL 6.9 and RHEL 7.4 Xymon v4.3.28 This may be documented somewhere and I'm just not able to find it. But is there a way to force logfetch to only scan complete lines and discard any partials it might retrieve based on the MAXCHECK setting? I've been getting quite a few alerts on highly active systems where the offending line would normally be excluded due to the first part of a search that is missing. A simple example, I want to ignore the alert triggers for /var/log/messages where the system name is test-system and :\sheader\ssubject: is also in the line. Since test-system comes right after the date/time stamp, that causes the ignore check to not work if test-system is not retrieved by logfetch. analysis.cfg # Red alert on CRITICAL or ERROR or SERIOUS (with exceptions) LOG %.* %(?-i)CRITICAL|ERROR|SERIOUS COLOR=red IGNORE=%(?-i)test-system.*:\sheader\ssubject: I've tried adjusting the MAXCHECK setting but it didn't make a difference one way or the other. client-local.cfg log:/var/log/messages:10240 # 10KB default log:/var/log/messages:1024000 # 1MB Thanks. ========================================================= Larry D. Bonham Financial Network Inc. 10401-F Baur Olivette, MO 63132 (314) 400-9412 voice (314) 997-5647 fax ========================================================= ________________________________ CONFIDENTIALITY NOTICE: This electronic mail message is intended exclusively for recipient to which it is addressed. The contents of this message and any attachments may contain confidential and privileged information. Any unauthorized review, use, print, storage, copy, disclosure or distribution is strictly prohibited. If you have received this message in error, please advise the sender immediately by replying to the message's sender and delete all copies of this message and its attachments without disclosing the contents to anyone, or using the contents for any purpose.

5 11

Memory leak in Windows Virtual Disk service caused by Xymon Windows Powershell client
by spah＠syntec.co.uk 26 Apr '18

26 Apr '18

I have noticed a memory and thread leak in the Windows Virtual Disk service - or vds.exe from the process list. This service is set to Manual by default and gets started when starting the XymonPSClient service on Windows 2008 R2. It starts off using 32 threads, but grows forever, the virtual memory and physical memory (which starts under 6 MB) used slowly grow too. Has anyone noticed this problem and do they have a solution to it? On Windows Server 2012, the Virtual Disk service does not get started by Xymon Windows Powershell client, so it's obviously not started directly by the client, but by a request for information. I looked into known memory leaks for vds.exe and the one I found was not applicable to 2008 R2, only 2008 and Vista. The version of vds.exe we have is later than the version with the fix: 6.1.7601.17514 from 2010. The quick workaround is simply to stop the Virtual Disk Service - but as soon as the PSclient has used it, it does not stop properly and terminates unexpectedly, so one needs to set it not to restart on crash, or stop it immediately after starting, or to Disabled. But it would be good to know why it is starting, not stopping, and leaking threads / memory! Kind regards, SebA

3 6

HELP INSTALLATION-CLIENT
by pabdoom2011＠hotmail.com 24 Apr '18

24 Apr '18

Can I help with the list of software necessary to comply with all the dependencies and the order of installation-client for Solaris and Centos? Saludos cordiales, Pablo Padilla

3 2

findhost.sh URL
by john.r.rothlisberger＠accenture.com 03 Apr '18

03 Apr '18

Has anyone else noticed that in 4.3.28 the URL is written differently than in previous versions (4.3.21 below)? It is very useful to know what page a server resides on when using the findhost.sh and being able to send a link that contains the actual path: https://xymon.server_4.3.21/Chicago/ORD/ORD_TMP/#server1 In 4.3.28 the URL remains with the findhost.sh paramerters: https://xymon.server_4.3.28/Denver-cgi/findhost.sh?host=server2&jump=on&SUB… Is there a way to revert this back in 4.3.28? Thanks, John Upcoming PTO: _____________________________________________________________________ John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture 312.693.3136 office _____________________________________________________________________ ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy. ______________________________________________________________________________________ www.accenture.com

2 2

Xymon - Availability Report
by atalla＠zetaglobal.com 03 Apr '18

03 Apr '18

Hi, I need to generate Availability report. When I click on "Generate Spreadsheet" in Availability report page, I'm not able to download the report as .csv format. Instead the report is showing only in webview. I have the latest version of Xymon installed(v4.3.28) Could you please help me download the report? Thanks, Ajay

2 4

xymon communications program (server/bin/xymon) reading from stdin - first line only?
by glauber.ribeiro＠experian.com 30 Mar '18

30 Mar '18

Recently I tried to modify one of my xymon extension scripts, which passes the message data to the xymon command on the command line, to pass it via STDIN instead. It worked (sort of), but I was not able to get xymon to process more than the first line of the message (the one with color, status, timestamp). The rest of the message seemed to be ignored. This is xymon version 4.3.27. Is that a known problem? Thanks, glauber

2 2