August 2018 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

[Patch] xymond_rrd crash when parsing Linux ifconfig data
by steve＠opendium.com 22 Feb '19

22 Feb '19

This patch addresses two problems: 1. For Linux clients, do_ifstat_rrd parses the ifconfig output using the regex: ([a-z0-9]+(_[0-9]+)?:*|lo:?)\s pickdata() is called like this: pickdata(bol, ifstat_linux_pcres[0], 1, &ifname) With a network interface name such as "foo1", the regex produces 2 results: 0: "foo1 " 1: "foo1" pickdata() discards result 0 and assigns result 1 ("foo1") to the ifname parameter as expected. With a network interface name such as "foo_1", the regex produces 3 results: 0: "foo_1 " 1: "foo_1" 2: "_1" pickdata() discards result 0, assigns result 1 ("foo_1") to the ifname parameter and then tries to assign result 2 ("_1") to the next parameter in the variable argument list. Since there it no such parameter, this results in a segmentation fault and xymond_rrd crashes. The attached patch adds a "num_args" parameter to pickdata() to tell it how many arguments are in the variable argument list and ensure it never rolls off the end of that list. 2. I have some clients with network interface names in the form "foo.5", "foo_bar" and "foo-bar" ("eth0.5" is especially common when eth0 is carrying an 802.1q trunk, for example). Xymon does not collect statistics on those network interfaces because they don't match the regex. This patch relaxes the regex a bit so that it matches these network interfaces. -- - Steve Hill Technical Director Opendium Online Safety / Web Filtering http://www.opendium.com Enquiries: sales at opendium.com +44-1792-824568 Support: support at opendium.com +44-1792-825748 Opendium Limited is a company registered in England and Wales. Company No. 5465437 Highfield House, 1 Brue Close, Bruton, Somerset, BA10 0HY, England.

2 1

DISPLAYGROUP fails in analysis.cfg
by doughnut＠doughnut.net 01 Feb '19

01 Feb '19

I have a xymon server running "Debian GNU/Linux 9 (stretch)" with xymon 4.3.28-2. I'm having bad luck getting DISPLAYGROUP to work in analysis.cfg. It seems to only match on the first 7 characters. I also see this problem on "Debian GNU/Linux 8 (jessie)" with xymon 4.3.17-6+deb8u1. Below, I've shown that the display group is set, and that the analysis.cfg file is being parsed correctly by xymon, yet the the svcstatus page only shows the shortest of the DISPLAYGROUP enabled lines. root at misc-ten105-test:/etc/xymon# egrep -H Testnet.Database.Servers hosts.cfg hosts.cfg:group Testnet Database Servers root at misc-ten105-test:/etc/xymon# cat analysis.d/davetest.cfg HOST=* EXCLASS=linux PROC davetest-nonlinux 1 150 yellow PROC davetest-Testnet.Database.Servers 1 150 yellow DISPLAYGROUP=%(Testnet.Database.Servers) PROC davetest-Database 1 150 yellow DISPLAYGROUP=%(Database) PROC davetest-Databas 1 150 yellow DISPLAYGROUP=%(Databas) PROC davetest-Testnet.Database 1 150 yellow DISPLAYGROUP=%(Testnet.Database) PROC davetest-Testnet.Data 1 150 yellow DISPLAYGROUP=%(Testnet.Data) PROC davetest-Testnet.D 1 150 yellow DISPLAYGROUP=%(Testnet.D) PROC davetest-Testnet. 1 150 yellow DISPLAYGROUP=%(Testnet.) PROC davetest-Testnet 1 150 yellow DISPLAYGROUP=%(Testnet) PROC davetest-Testne 1 150 yellow DISPLAYGROUP=%(Testne) root at misc-ten105-test:/etc/xymon# sudo -u xymon xymoncmd xymond_client --dump-config | grep davetest [...] PROC davetest-nonlinux 1 150 yellow HOST=* EXCLASS=linux (line: 375) PROC davetest-Testnet.Database.Servers 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.Database.Servers) EXCLASS=linux (line: 377) PROC davetest-Database 1 150 yellow HOST=* DISPLAYGROUP=%(Database) EXCLASS=linux (line: 378) PROC davetest-Databas 1 150 yellow HOST=* DISPLAYGROUP=%(Databas) EXCLASS=linux (line: 379) PROC davetest-Testnet.Database 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.Database) EXCLASS=linux (line: 380) PROC davetest-Testnet.Data 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.Data) EXCLASS=linux (line: 381) PROC davetest-Testnet.D 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.D) EXCLASS=linux (line: 382) PROC davetest-Testnet. 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.) EXCLASS=linux (line: 383) PROC davetest-Testnet 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet) EXCLASS=linux (line: 384) PROC davetest-Testne 1 150 yellow HOST=* DISPLAYGROUP=%(Testne) EXCLASS=linux (line: 385) root at misc-ten105-test:/etc/xymon# lynx -dump -nolist 'http://localhost/xymon-cgi/svcstatus.sh?HOST=balin2012.collegenet.testnet&S…' [...] Xymon balin2012.mydomain.testnet - procs Wed Jul 11 13:29:40 2018 _______________________________________________________________________ HISTORY Wed Jul 11 13:29:40 2018 - Processes NOT ok yellow davetest-nonlinux (found 0, req. between 1 and 150) yellow davetest-Testnet (found 0, req. between 1 and 150) yellow davetest-Testne (found 0, req. between 1 and 150) Status unchanged in 8 days, 21 hours, 22 minutes Status message received from 10.101.1.35 Client data available

2 3

Project: Xymon dashboard
by dominique.frise＠unil.ch 08 Nov '18

08 Nov '18

Hi, My colleagues found this addon awesome but have a small feature request. Would it be possible to integrate the current date/time (like on all Xymon pages) on the same line as the title ? Thanks, Dominique ________________________________ De : Xymon <xymon-bounces at xymon.com> de la part de Christian Herzog <herzog at phys.ethz.ch> Envoyé : jeudi, 28 juin 2018 15:36 À : xymon at xymon.com Objet : Re: [Xymon] Project: Xymon dashboard Hi all, I just fixed a major issue that prevented Xymondash from properly working for some of you. If it didn't work yesterday, you might want to give the current commit another go. best, -Christian On Tue, Jun 26, 2018 at 06:15:11PM +0200, Dirk Kastens wrote: > Hi, > > I just got it to work on my system. > > - I had to install python 3.6. > - I had to copy the cgi scripts to /usr/share/xymon/cgi-bin, changing > main.js: > > const XYMONURL = '/xymon-cgi/svcstatus.sh'; > const XYMONACKURL = '/xymon-cgi/xymon-ack'; > const XYMONJSONURL = '/xymon-cgi/xymon2json'; > > - I had to change the path to xymon in xymon2json on line 77: > > return ['/usr/bin/xymon 0 ' + cmd] > > I didn't make any further testing for the moment. > > -- > Viele Gruesse, > > Dirk Kastens > Universitaet Osnabrueck, Rechenzentrum (Computer Center) > Albrechtstr. 28, 49069 Osnabrueck, Germany > Tel.: +49-541-969-2347, FAX: -2470 > > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon -- Dr. Christian Herzog <herzog at phys.ethz.ch> support: +41 44 633 26 68 IT Services Group, HPT H 8 voice: +41 44 633 39 50 Department of Physics, ETH Zurich 8093 Zurich, Switzerland http://nic.phys.ethz.ch/ _______________________________________________ Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

2 1

create an alert for all linux servers
by didds3＠yahoo.co.uk 31 Aug '18

31 Aug '18

OK - sorted :-) Somebody mentioned using "CLASS=" - success :-) HOST=* SERVICE=msgs CLASS=linux TIME=w:0800:1830 COLOR=red many thanks all for all your help. didds

1 0

create an alert for all linux servers
by didds3＠yahoo.co.uk 31 Aug '18

31 Aug '18

Ive an alert for msgs (var/log/messages entries) that I desire to have paged in office hours only Thing is its only linux clients that need this check., not windows clients. So if I set a stadard style alerts.cfg entry HOST=* SERVICE=msgs TIME=w:0800:1830 COLOR=red MAIL monitor at us.com of course if any windows msgs go red (constantly... zzzz) then they will page also, which is not the required outcome. There is the option I see of something like alerts.cfg: GROUP=SSSD_MSGS TIME=w:0800:1830 COLOR=red MAIL monitor at us.com analysis.cfgdefault section LOG %/var/log/messages "%TEST" COLOR=red GROUP=SSSD_MSGS [ search string here of course being TEST ] but that does exactly the same thing. So - aside from adding that LOG line to umpteen individual analysis.cfg entries for each linux server, or a single section listing all the linux servers ... is there a way to curtail this check to only linux servers (or more likely a bunch of hostnames of course). Somewhere it seems we'd have to manulayy maintain the correct "list" of hostnames to be caught - Im just trying to minimise the overhead. I had considered using a yellow alert to alert the LOG check... but other "things" also issue yellow alerts and we don;t want to trigger them either. I can;t see a way out of this? didds

3 3

How to install in windows?
by jgsuareza1984＠gmail.com 30 Aug '18

30 Aug '18

Hi greetings Please help with information on how to install XYMON in a Windows operating system, using BWIN *Gabriel *

2 1

powershell client data file excessive Port info enhancement request
by tlwilliams4＠vcu.edu 30 Aug '18

30 Aug '18

We are in the process of migrating 300+ servers from BBWin client to powershell client. All works well, except our anti-virus servers create a huge data file that gets truncated, causing FILES, SVCS, WHO to go Purple. We have set the MAXMSG_STATUS AND MAXMSG_CLIENT Xymon server parameters to 1024, but the AV servers still blow past that. The PORTS function reports all of the TCP ESTABLISHED and TIME_WAIT connections for every AV client on servers and desktops. We only need to know that the AV server is proper LISTENING. Using Slimmode and excluding the Ports section doesn't work as we do need to know the listening ports. I have modified the Xymonclient.ps1 script as follows to limit the data in the function XymonPorts "[ports]" netstat -an to: "[ports]" netstat -an | findstr LISTENING Rather than manually editing each new version of script released, I would rather have a client-local.cfg entry that would turn this on per group similar to what you have for IFSTAT. Less desirable would be a statement in Xymonclient_config.XML, or as a mode in Slimmode. Does this sound like a doable enhancement? *Timothy L. Williams* *Operating Systems Analyst* Virginia Commonwealth University Computer Center

3 3

Ignoring Portable Disks
by cdm46＠ast.cam.ac.uk 24 Aug '18

24 Aug '18

Hi all, I've got my clients setup for local config and am having problems with some commands in the localclient.cfg files. I am trying to ignore certain filesystems and have the line "DISK %^/media.*/ IGNORE" setup to ignore any portable hardware. This is identical to the example given in Xymon's man pages, except "oracle" has been changed to "media". However, this doesn't seem to do anything at all, it's as if Xymon is ignoring this configuration option. Other configuration options set in my client's localclient.cfg do work as expected, so it can't be a problem with xymon reading the localclient.cfg file. Anything to help me figure out what on earth is going on would be greatly appreciated. Cheers, Calum

3 3

CGI Security
by john＠executech.com 23 Aug '18

23 Aug '18

I was able to get CGI security working with the PAGE parameter but was unable to get it working with any SUBPAGE or SUBGROUP parameter. Is this an option or will I have to explicitly add each individual HOST? I have all of the servers and access in a SQL table and use PHP security to limit access to all other pages. Has anyone successfully configured XYMON to look to a database for CGI security based on SUBPAGE? This is in reference to: https://www.xymon.com/help/manpages/man5/xymonwebaccess.5.html Thanks, John

1 1