April 2019 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

Xymon and apache 2.4.35 security problem
by dl1025＠att.com 09 Oct '19

09 Oct '19

I recently upgraded to apache 2.4.35 and was having some issues with password file to secure xymon-seccgi. I got not get apache to read the password file. To get it to work I had to change from Require all granted to Require all denied. Now, it works. I get prompted to enter username and password. Here is the section from my httpd.conf file for your reference if you will run into similar problems. ScriptAlias /xymon-seccgi/ "/opt/app/workload/bbapp/bb/cgi-secure/" <Directory "/opt/app/workload/bbapp/bb/cgi-secure"> AllowOverride None Options ExecCGI Includes <IfModule mod_authz_core.c> # Apache 2.4+ Require all denied </IfModule> <IfModule !mod_authz_core.c> Order deny,allow Allow from all </IfModule> # Password file where users with access to these scripts are kept. # Although expected in $XYMONHOME/etc/ by the useradm and chpasswd # scripts, files here can be read with the "config" message type, # which allows status-privileged clients to read arbitrary regular files # from the directory. # # This file should be owned and readable only by the apache server user, # and ideally merely a symlink to a location outside of $XYMONHOME/etc/ # # Create it with: # htpasswd -c /opt/app/workload/bbapp/bb/server/etc/xymonpasswd USERNAME # chown apache:apache /opt/app/workload/bbapp/bb/server/etc/xymonpasswd # chmod 640 /opt/app/workload/bbapp/bb/server/etc/xymonpasswd # Add more users / change passwords with: "htpasswd /opt/app/workload/bbapp/bb/server/etc/xymonpasswd USERNAME" # # You can also use a group file to restrict admin access to members of a # group, instead of anyone who is logged in. In that case you must setup # the "xymongroups" file, and change the "Require" settings to require # a specific group membership. See the Apache docs for more details. AuthUserFile /opt/app/workload/bbapp/bb/server/etc/xymonpasswd AuthGroupFile /opt/app/workload/bbapp/bb/server/etc/xymongroups AuthType Basic AuthName "Xymon Administration" # "valid-user" restricts access to anyone who is logged in. Require valid-user # "group admins" restricts access to users who have logged in, AND # are members of the "admins" group in xymongroups. # Require group admins </Directory> I also enabled the following modules. LoadModule authn_file_module modules/mod_authn_file.so LoadModule authn_dbm_module modules/mod_authn_dbm.so LoadModule authn_anon_module modules/mod_authn_anon.so LoadModule authn_core_module modules/mod_authn_core.so LoadModule authz_host_module modules/mod_authz_host.so LoadModule authz_groupfile_module modules/mod_authz_groupfile.so LoadModule authz_user_module modules/mod_authz_user.so LoadModule authz_core_module modules/mod_authz_core.so LoadModule access_compat_module modules/mod_access_compat.so LoadModule auth_basic_module modules/mod_auth_basic.so LoadModule reqtimeout_module modules/mod_reqtimeout.so LoadModule filter_module modules/mod_filter.so LoadModule mime_module modules/mod_mime.so LoadModule log_config_module modules/mod_log_config.so LoadModule env_module modules/mod_env.so LoadModule headers_module modules/mod_headers.so LoadModule setenvif_module modules/mod_setenvif.so LoadModule version_module modules/mod_version.so LoadModule unixd_module modules/mod_unixd.so LoadModule status_module modules/mod_status.so LoadModule autoindex_module modules/mod_autoindex.so LoadModule cgid_module modules/mod_cgid.so LoadModule dir_module modules/mod_dir.so LoadModule alias_module modules/mod_alias.so LoadModule rewrite_module modules/mod_rewrite.so

2 3

Goodbye and thanks for all the fish
by EDSchminke＠Hormel.com 03 May '19

03 May '19

+1 We had been using "Big Brother" for years. We had also switched to a new fangled, whiz-bang monitoring solution and it just-- wasn't-- working out. We finally decided to pull the plug and switch back to Xymon because it just friggin' works. Good luck, I guess... we'll be here when you need us again. :) Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com

5 4

Xymon 4.3.29 scheduling and F30
by cleaver＠terabithia.org 03 May '19

03 May '19

Hi all, Just to give everyone an update, I have a few more patches I'm still looking at, but I mostly wanted to wait until I get a chance to test compile fixes on Fedora 30 final, which should drop today or tomorrow. There are still a number of compile warnings generally (mostly char mismatch warnings), but if you know of compile issues on other platforms specifically, it would be helpful if you could run a quick test on the latest 4.3.29 snapshot to see if it's resolved. The latest snap is available at https://sourceforge.net/p/xymon/code/HEAD/tree/branches/4.3.29/ in the upper right. Thanks, -jc

4 6

Patch - no-cpu-listing - cpu column
by john.horne＠plymouth.ac.uk 29 Apr '19

29 Apr '19

Hi, Using the Terabithia 4.3.28 RPMS, the 'cpu' column typically shows the output from the 'top' command received from the client. While we have no problem with users knowing if a server has a problem, we do not necessarily want them seeing the processes running on the server. (A bit of security by obscurity I suppose.) The attached patches implement a 'xymond_client' option named '--no-cpu- listing'. This is basically similar to the current '--no-ps-listing' and '--no- port-listing' options (both of which we already use). The default for the new option is to be disabled - that is, as before, the 'top' listing is shown. If the option is used (in the tasks.cfg file, under the 'clientdata' section), then no 'top' output is included in the 'cpu' column, and just the uptime output and CPU load graph are shown. Both patches are only a few lines long. We have used this patch for a long time without any problems. John. -- John Horne | Senior Operations Analyst | Technology and Information Services University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

2 1

Graph titles and NAME: in hosts.cfg - bug?
by john.horne＠plymouth.ac.uk 29 Apr '19

29 Apr '19

Hello, (A bit long, sorry...) Using Xymon 4.3.28 from the Terabithia RPMs. I've been adding a few new client servers to our Xymon monitor. However, I have noticed today that for some graphs no graph title appears. The graph appears, the values appear (on the graph) and the graph lines appear. Just no title. It seems that this affects those client servers which are configured in the hosts.cfg file with a NAME: entry. For example: 10.11.12.13 server1 # NAME:"server1 (pre-prod web)" testip !ftp ssh... (I don't use FQDNs for the server names.) It seems that entries we have added to the 'graphs.cfg' file, and where the TITLE uses the 'exec:' function have the problem. (See the 'graphs.cfg' man page for info about using 'exec:'.) So, for example we have in graphs.cfg: ======= [memory] TITLE exec:/bin/sh -c 'echo Memory Utilisation - $2' YAXIS % Used ... ======= Debugging the graph output (by setting some variables) using 'xymoncmd showgraph.cgi --debug >/tmp/graph.out' shows: ======= # xymoncmd showgraph.cgi --debug >/tmp/graph.out sh: -c: line 0: syntax error near unexpected token `(' sh: -c: line 0: `/bin/sh -c 'echo Memory Utilisation - $2' server1 (pre-prod web) memory "14/Apr/2019 - 16/Apr/2019" "memory.rrd"' ======= As can be seen the parameters passed to 'showgraph.cgi' should include the hostname, but it includes the hostname and the NAME: entry. The shell barfs on the parentheses. If I run the above shell command directly but quoting the whole server name, then it works fine: ====== $ /bin/sh -c 'echo Memory Utilisation - $2' "server1 (pre- prod web)" memory "14/Apr/2019 - 16/Apr/2019" "memory.rrd" Memory Utilisation - 14/Apr/2019 - 16/Apr/2019 ====== This seems to be a bug. If the hostname parameter being sent to showgraph includes spaces (because the NAME: entry is included), then the whole parameter should be quoted. I suspect that if the NAME: entry itself has double-quotes in it, then the graph title could be a bit messed up. If in the above example I changed the parentheses to '=' signs, then no quoting is required but the title comes out as 'Memory Utilisation - web='. Perhaps in those instances, as the man page for graphs.cfg suggests, a local script is run instead to do the processing and dig out the correct graph title. It's late now, and I'm going to be busy leading up to the Easter break, but if I can I'll take a look and see if I can get a patch for the code. I'm suspecting it should be a one-line - if the total hostname parameter has a space then quote the whole thing. John. -- John Horne | Senior Operations Analyst | Technology and Information Services University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

2 2

Goodbye and thanks for all the fish
by Phil.Crooker＠orix.com.au 29 Apr '19

29 Apr '19

Our company has decided to switch to a managed monitoring system, so goodbye, XYmon. I'm grateful to Henrik, Jeremy and everyone for all the work you've put in to XYmon over the years. I can't say I'll miss the regular expressions in analysis.cfg, but it certainly has saved my bacon many times. All the best, Phil

2 1

semaphore errors
by stef.coene＠docum.org 29 Apr '19

29 Apr '19

Hi, We are getting these errors on 2 xymon servers running 4.3.21 and 4.3.28. Any steps I can do to debug and/or prevent this error? The only solution is a restart of the xymon daemons. 2018-10-24 12:06:21.110407 semop failed, Invalid argument 2018-10-24 12:06:21.110456 Dropping message due to semaphore error ipcs ------ Message Queues -------- key msqid owner perms used-bytes messages ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x00000000 0 root 644 80 2 0x00000000 32769 root 644 16384 2 0x00000000 65538 root 644 280 2 0x01013343 2818051 xymon 600 10485760 2 0x02013343 2850820 xymon 600 10485760 2 0x03013343 2883589 xymon 600 10485760 2 0x04013343 2916358 xymon 600 10485760 2 0x05013343 2949127 xymon 600 262144 1 0x06013343 2981896 xymon 600 32768 1 0x07013343 3014665 xymon 600 10485760 2 0x08013343 3047434 xymon 600 10485760 2 0x09013343 3080203 xymon 600 131072 3 0xfa0101b9 1277964 root 666 30435368 2 0xfa0101bd 1310733 root 666 30435368 1 ------ Semaphore Arrays -------- key semid owner perms nsems 0x000000a7 98304 root 600 1 0x00000000 7995393 www-data 600 1 0x00000000 8028162 www-data 600 1 0x00000000 1769475 www-data 600 1 Stef

3 5

WordPress Monitoring
by matthew＠adultliteracy.ac.nz 28 Apr '19

28 Apr '19

I know this question may not be answered, but I am wondering if Xymon can monitor a WordPress website or 2 on DreamHost. I have a shared service account. Matthew Gregory IT Manager Rural Youth & Adult Literacy Trust www.adultliteracy.ac.nz m: 022 644 9544 t: 0800 891 339 e: matthew at adultliteracy.ac.nz

3 2

how to tune memory threshold for Windows PS client?
by rlhamil2＠gmail.com 28 Apr '19

28 Apr '19

For a wimpy laptop (latest Windows 10 Pro, 6 GB RAM), it goes orange pretty quick (three days or so), even though its mostly sitting idle. Is this MEMSWAP in analysis.cfg on the server that I want to tweak, or something else? What's reasonable for a Windows system like that? And does Windows grow swap automatically if needed (in which case the threshold could probably safely be higher)? Sun 28 Apr 08:13:01 2019 - Memory low Memory Used Total Percentage Real/Physical 3350M 5717M 58% Actual/Virtual 163M 960M 16% Swap/Page 4459M 6677M 66%

1 0