June 2019 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

Devmon template for Cisco ISR4000
by colin.coe＠gmail.com 19 Dec '19

19 Dec '19

Hi Just wondering if anyone already has a Devmon template for the ISR4000 series that they'd be willing to share? Thanks

4 5

Xymon and apache 2.4.35 security problem
by dl1025＠att.com 09 Oct '19

09 Oct '19

I recently upgraded to apache 2.4.35 and was having some issues with password file to secure xymon-seccgi. I got not get apache to read the password file. To get it to work I had to change from Require all granted to Require all denied. Now, it works. I get prompted to enter username and password. Here is the section from my httpd.conf file for your reference if you will run into similar problems. ScriptAlias /xymon-seccgi/ "/opt/app/workload/bbapp/bb/cgi-secure/" <Directory "/opt/app/workload/bbapp/bb/cgi-secure"> AllowOverride None Options ExecCGI Includes <IfModule mod_authz_core.c> # Apache 2.4+ Require all denied </IfModule> <IfModule !mod_authz_core.c> Order deny,allow Allow from all </IfModule> # Password file where users with access to these scripts are kept. # Although expected in $XYMONHOME/etc/ by the useradm and chpasswd # scripts, files here can be read with the "config" message type, # which allows status-privileged clients to read arbitrary regular files # from the directory. # # This file should be owned and readable only by the apache server user, # and ideally merely a symlink to a location outside of $XYMONHOME/etc/ # # Create it with: # htpasswd -c /opt/app/workload/bbapp/bb/server/etc/xymonpasswd USERNAME # chown apache:apache /opt/app/workload/bbapp/bb/server/etc/xymonpasswd # chmod 640 /opt/app/workload/bbapp/bb/server/etc/xymonpasswd # Add more users / change passwords with: "htpasswd /opt/app/workload/bbapp/bb/server/etc/xymonpasswd USERNAME" # # You can also use a group file to restrict admin access to members of a # group, instead of anyone who is logged in. In that case you must setup # the "xymongroups" file, and change the "Require" settings to require # a specific group membership. See the Apache docs for more details. AuthUserFile /opt/app/workload/bbapp/bb/server/etc/xymonpasswd AuthGroupFile /opt/app/workload/bbapp/bb/server/etc/xymongroups AuthType Basic AuthName "Xymon Administration" # "valid-user" restricts access to anyone who is logged in. Require valid-user # "group admins" restricts access to users who have logged in, AND # are members of the "admins" group in xymongroups. # Require group admins </Directory> I also enabled the following modules. LoadModule authn_file_module modules/mod_authn_file.so LoadModule authn_dbm_module modules/mod_authn_dbm.so LoadModule authn_anon_module modules/mod_authn_anon.so LoadModule authn_core_module modules/mod_authn_core.so LoadModule authz_host_module modules/mod_authz_host.so LoadModule authz_groupfile_module modules/mod_authz_groupfile.so LoadModule authz_user_module modules/mod_authz_user.so LoadModule authz_core_module modules/mod_authz_core.so LoadModule access_compat_module modules/mod_access_compat.so LoadModule auth_basic_module modules/mod_auth_basic.so LoadModule reqtimeout_module modules/mod_reqtimeout.so LoadModule filter_module modules/mod_filter.so LoadModule mime_module modules/mod_mime.so LoadModule log_config_module modules/mod_log_config.so LoadModule env_module modules/mod_env.so LoadModule headers_module modules/mod_headers.so LoadModule setenvif_module modules/mod_setenvif.so LoadModule version_module modules/mod_version.so LoadModule unixd_module modules/mod_unixd.so LoadModule status_module modules/mod_status.so LoadModule autoindex_module modules/mod_autoindex.so LoadModule cgid_module modules/mod_cgid.so LoadModule dir_module modules/mod_dir.so LoadModule alias_module modules/mod_alias.so LoadModule rewrite_module modules/mod_rewrite.so

2 3

E-Mail alert on conn ok xymon 4.3.28
by S.Pelzetter＠wvs-basa.de 01 Jul '19

01 Jul '19

Hello, I have 2 devices on xymon (version 4.3.28) alerting is driving me crazy. I get a lot of e-mail alerts on service conn even though the service conn is ok: Xymon [567211] gate......:conn CRITICAL (RED) green Wed Jun 26 07:16:54 2019 conn ok Service conn on gate..... is OK (up) &green 87.139.xx.xx is alive (0.95 ms) I can't see any differences on the configuration to other devices alerting works normal. Could this be related to a bug? But if there is a bug, why only on this 2 devices. Please help. -- Diese E-Mail wurde vom Sophos Security Gateway auf gefährliche Inhalte geprüft! http://www.sophos.com

2 1

duplicate sslcert entries when server-status is on SSL
by nskyrca＠syr.edu 28 Jun '19

28 Jun '19

Hello, We have some servers that we want to monitor the website (on https) as well as collect the apache performance statistics in the apache graphs in the trends column. The server-status is using SSL. Below is an example of the hosts.cfg entry: 0.0.0.0 examplehost.edu # https://example.edu ssldays=30:15 apache=https://example.edu/server-status?auto TRENDS:*,apache:apache|apache1|apache2|apache3,netstat:netstat0|netstat1|netstat2|netstat3,vmstat:vmstat1|vmstat3 The sslcert column is showing two entries, one for the website, and one for the server-status site. Is this expected because server-status is using SSL? Is there a way to remove the duplicate sslcert entry for https://example.syr.edu/server-status?auto" and only show the once entry? Below is an abbreviated example of the sslcert page: SSL certificate for https://example.edu/server-status?auto expires in 525 days Server certificate:{stuff removed} Available ciphers: {stuff removed} SSL certificate for https://example.edu/ expires in 525 days Server certificate: {stuff removed} Available ciphers: {stuff removed} Thank you! Nicole

1 0

tmp file system not being monitored.
by ljswan＠mweb.co.za 27 Jun '19

27 Jun '19

Good day, I have a problem whereby the /tmp partition is not being monitored. Is this because the filesystem is set to none? If so is there a way around this? It is important that we monitor that partition. Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 18446076 4627568 12858460 27% / /dev/sdb1 10321208 496652 9300268 6% /var/log /dev/sdb2 41283904 10817020 28369784 28% /opt/platform /dev/sdc1 51474044 2678776 46157496 6% /backups /dev/sdd1 262014980 2800716 259214264 2% /opt/platform/apps/mongodb/dbroot df -h Filesystem Size Used Avail Use% Mounted on udev 2.0G 0 2.0G 0% /dev tmpfs 791M 79M 712M 10% /run /dev/sda1 18G 4.5G 13G 27% / tmpfs 2.0G 0 2.0G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup none 2.0G 0 2.0G 0% /run/shm none 2.0G 40K 2.0G 1% /tmp /dev/sdb1 9.9G 448M 9.0G 5% /var/log /dev/sdb2 40G 11G 28G 28% /opt/platform cgmfs 100K 0 100K 0% /run/cgmanager/fs /dev/sdc1 50G 2.6G 45G 6% /backups DISK / 85% 90% DISK /tmp 85% 90% DISK /var/log 85% 90% DISK /opt/platform 85% 90% DISK /opt/platform/apps/mongodb/dbroot 85% 90% DISK /backups 85% 90% Thank you in advance. Regards, Leon

2 1

Lost procs
by chris.seip＠hpe.com 24 Jun '19

24 Jun '19

For one of my Xymon clients running xymon-client, the procs column has disappeared. My narrative of how this started & what we did lacks some clarity. I think procs was visible last week. If it was there I'd bet a paycheck the status was clear. One day last week it went purple. So we began troubleshooting & fiddling with it, puzzled by the purple status. I was able to drop the procs test for that host, and it vanished from the web UI. Then I defined a PROC test for this host in analysis.cfg. We do all our client configuration on the Xymon server and avoid the local configs of Xymon clients. This PROC test, in fact the PROC column for this group (of which only this one host is locally monitored), isn't showing up. Hm. We've been poking at this puzzle long enough to ask for some help, please. Any thoughts? --Chris- CLIENT: RHEL 6.0, Xymon-client 4.3.28-1 SERVER: Ubuntu 16.04.6, Xymon server 4.3.25-1 hosts.cfg excerpt: group-compress <I>blah blah </I> group-except files|msgs|ports 16.93.15.15 hostname.domain.com # ssh http://hostname.domain.com/accelerator/images/logo.gif include more-hosts.cfg analysis.cfg excerpt: HOST=hostname.domain.com PROC /sge 3 5 red

2 2

tmp filesystem not being monitored
by ljswan＠mweb.co.za 24 Jun '19

24 Jun '19

Good day, I have a problem whereby the /tmp partition is not being monitored. Is this because the filesystem is set to none? If so is there a way around this? It is important that we monitor that partition. Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 18446076 4627568 12858460 27% / /dev/sdb1 10321208 496652 9300268 6% /var/log /dev/sdb2 41283904 10817020 28369784 28% /opt/platform /dev/sdc1 51474044 2678776 46157496 6% /backups /dev/sdd1 262014980 2800716 259214264 2% /opt/platform/apps/mongodb/dbroot df -h Filesystem Size Used Avail Use% Mounted on udev 2.0G 0 2.0G 0% /dev tmpfs 791M 79M 712M 10% /run /dev/sda1 18G 4.5G 13G 27% / tmpfs 2.0G 0 2.0G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup none 2.0G 0 2.0G 0% /run/shm none 2.0G 40K 2.0G 1% /tmp /dev/sdb1 9.9G 448M 9.0G 5% /var/log /dev/sdb2 40G 11G 28G 28% /opt/platform cgmfs 100K 0 100K 0% /run/cgmanager/fs /dev/sdc1 50G 2.6G 45G 6% /backups DISK / 85% 90% DISK /tmp 85% 90% DISK /var/log 85% 90% DISK /opt/platform 85% 90% DISK /opt/platform/apps/mongodb/dbroot 85% 90% DISK /backups 85% 90% Thank you in advance. Regards, Leon

1 0

Can we discuss data flow? XYMON theory of operation
by bferrell＠baywinds.org 22 Jun '19

22 Jun '19

I think this might be useful for getting some of these "less that usual" features going I think we all know xymond listens to port 1984. It also starts up sub processes that some how communicate with xymond. Some handle delegated tasks from data provided like generating RRDs, updating web pages etc. Others independently collect data by performing network tests of various kinds (MY current favorite is snmp polling). Here is where I start to get hazy. Using lsof, it looks like the communications for most part is via FIFO pipes... Named pipes? Clients collect? Channels Process? xymond routes the data or clients and channels listen for stuff intended for themselves in a kind of free-for-all (not necessarily a bad thing)? Inquiring minds want to know

1 0

Plan for XYMON and SNMP
by jeremy＠laidman.org 22 Jun '19

22 Jun '19

On Fri, 17 May 2019 at 13:36, Bruce Ferrell <bferrell at baywinds.org> wrote: > Yes, I did Jeremy. My apologies. > No problem. > I did figure out I have to export SNMP=1 then configure and rebuild > Ah, I didn't know that. Or I forgot that. Just looking at the code now - are you sure this isn't "DOSNMP=1"? > What is the tag in the hosts file (if any)? > Nothing. The hostnames are found by xymon-snmpcollect in the snmphosts.cfg. You still need the hosts to be in hosts.cfg, but not tags are required. Jeremy > On 5/16/19 6:41 PM, Jeremy Laidman wrote: > > If you meant to say, "Hey Jeremy" then the answer is, that it's all > stuff for Xymon. > > > > On Fri, 17 May 2019 at 11:15, Bruce Ferrell <bferrell at baywinds.org > <mailto:bferrell at baywinds.org>> wrote: > > > > Hey Josh, I saw that before. Is that stuff that devmon looks at or > stuff for xymon. > > > > I'm not at all clear > > > > Thanks in advance > > > > On 5/16/19 6:00 PM, Jeremy Laidman wrote: > >> Xymon already has code for SNMP that works, but isn't documented. I > have it working for me, but it required a fair bit or digging to find the > right configuration. I've > >> posted on this previously; might help: > https://lists.xymon.com/archive/2019-March/046191.html > >> > >> J > >> > >> On Thu, 16 May 2019 at 19:26, Bruno Manzoni < > bruno.manzoni at ubi-network.ch <mailto:bruno.manzoni at ubi-network.ch>> wrote: > >> > >> Thanks you Josh, > >> > >> Yes I know about devmon (I put my own devmon version which is a > try to be a consolidate version) > >> > >> In fact, I just would like to know if there was another SNMP > development (but it seems not as you say it will be merged) inside Xymon . > (I am thinking a way to integrate > >> SNMP v3, but it do not uses the current SNMP_session perl lib > as it is not supported within this lib)? > >> > >> Regards > >> > >> Bruno Manzoni > >> (Bonomani) > >> > >> > >> Le 14.05.2019 à 15:20, Josh Luthman a écrit : > >>> You can just use Devmon these days. > >>> > >>> I've seen nothing to suggest the projects will be merged. > >>> > >>> Josh Luthman > >>> Office: 937-552-2340 > >>> Direct: 937-552-2343 > >>> 1100 Wayne St > >>> Suite 1337 > >>> Troy, OH 45373 > >>> > >>> > >>> On Fri, May 10, 2019 at 12:33 PM Bruno Manzoni < > bruno.manzoni at ubi-network.ch <mailto:bruno.manzoni at ubi-network.ch>> wrote: > >>> > >>> Hello, > >>> > >>> Is there any plan to have SNMP include with a new version > of xymon? If > >>> yes, what is the current state? > >>> > >>> > >>> _______________________________________________ > >>> Xymon mailing list > >>> Xymon at xymon.com <mailto:Xymon at xymon.com> > >>> http://lists.xymon.com/mailman/listinfo/xymon > >>> > >> _______________________________________________ > >> Xymon mailing list > >> Xymon at xymon.com <mailto:Xymon at xymon.com> > >> http://lists.xymon.com/mailman/listinfo/xymon > >> > >> > >> _______________________________________________ > >> Xymon mailing list > >> Xymon at xymon.com <mailto:Xymon at xymon.com> > >> http://lists.xymon.com/mailman/listinfo/xymon > > > > _______________________________________________ > > Xymon mailing list > > Xymon at xymon.com <mailto:Xymon at xymon.com> > > http://lists.xymon.com/mailman/listinfo/xymon > > > >

3 15