November 2021 - Xymon - Xymon mailman web

Capture and send data To Elasticsearch and Kibana
by francois.outteryck＠soprasteria.com 27 Sep '22

27 Sep '22

Hi Peter, Examples of task.cfg configuration with status and stachg events : [capture_status] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture.log LOGFILE /var/log/xymon/lunch_capture.log [capture_stachg] ENVFILE /home/xymon/server/etc/xymonserver.cfg NEED xymond CMD xymond_channel --channel=stachg /home/xymon/server/ext/capture.sh --log=$XYMONSERVERLOGS/xymond_task_capture_stachg.log LOGFILE /var/log/xymon/lunch_capture_stachg.log Capture.sh script simple made to call a php script : #!/bin/sh /usr/bin/php /var/www/kibana/capture.php $@ </dev/stdin >>/var/log/xymon/xymond_capture_php.log 2>&1 And capture.php make all job : <?php date_default_timezone_set('Europe/Paris'); $TODO=FALSE; $ARRAY_COLLECTOR=array(); $COLLECTOR=null; while (FALSE !== ($line = fgets(STDIN))) if ($line!='') { $tab=reduce_array($line); if (sizeof($tab)==0) continue; if ($line=="@@\n") { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } else if (substr($tab[0],0,8)=='@@stachg') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@stachg#seq|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime|disabletime|disablemsg|downtimeactive|clienttstamp|modifiers */ $COLLECTOR='stachg'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'PrevStatus'=>$metadata[8], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[9], 'DisableTime'=>$metadata[10], 'DisableMSg'=>$metadata[11], 'Message'=>'' ); $id=$metadata['ServerName'].'_'.$metadata['Type'].'_'.$meta['LastChange']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else if (substr($tab[0],0,8)=='@@status') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } /* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 /* @@status|timestamp|sender|origin|hostname|testname|expiretime|color|testflags|prevcolor|changetime|ackexpiretime|ackmessage|disableexpiretime|disablemessage|clientmsgtstamp|flapping */ $COLLECTOR='status'; $metadata=explode('|',$tab[0]); $sender=$metadata[2]; $hostname=$metadata[3]; $metadata=array( 'Collector'=>$COLLECTOR, 'Sender'=>$metadata[2], 'ServerName'=>$metadata[4], 'Type'=>$metadata[5], 'Status'=>$metadata[7], 'LogTime'=>$metadata[1], 'ValidTime'=>$metadata[6], 'LastChange'=>$metadata[10], 'DisableTime'=>$metadata[13], 'AckTime'=>$metadata[11], 'DisableMSg'=>$metadata[14], 'Message'=>'' ); $id=$metadata['ServerName'].'.'.$metadata['Type']; $ARRAY_COLLECTOR=null; $ARRAY_COLLECTOR=array(); $TODO=TRUE; } else { if (substr($tab[0],0,2)=='@@') { if ($TODO) { $metadata['Message']=implode('',$ARRAY_COLLECTOR); SendData($id,$metadata,$COLLECTOR); $TODO=FALSE; $COLLECTOR=null; } } if ($TODO) $ARRAY_COLLECTOR[]=$line; } } function CallAPI($method, $url, $data = false) { $curl = curl_init(); switch ($method) { case "POST": curl_setopt($curl, CURLOPT_POST, 1); if ($data) curl_setopt($curl, CURLOPT_POSTFIELDS, $data); break; case "PUT": curl_setopt($curl, CURLOPT_PUT, 1); break; default: if ($data) $url = sprintf("%s?%s", $url, http_build_query($data)); } curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($curl); curl_close($curl); return $result; } function reduce_array($line) { $tab=explode(' ',$line); $nr=array(); for ($i=0;$i<sizeof($tab);$i++) if (trim($tab[$i])!='') $nr[]=trim($tab[$i]); return $nr; } function SendData($id,$cc,$COLLECTOR=null) { if ($COLLECTOR==null) return; $d=new DateTime(); if ($cc['LastChange']!='') {$d->setTimestamp($cc['LastChange']);$cc['LastChange']=$d->format('c');} else {$cc['LastChange']='NaN';} if ($cc['LogTime']!='') {$d->setTimestamp($cc['LogTime']);$cc['LogTime']=$d->format('c');} else {$cc['LogTime']='NaN';} if ($cc['ValidTime']!='') {$d->setTimestamp($cc['ValidTime']);$cc['ValidTime']=$d->format('c');} else {$cc['ValidTime']='NaN';} if (isset($cc['AckTime'])) { if ($cc['AckTime']!='') {$d->setTimestamp($cc['AckTime']);$cc['AckTime']=$d->format('c');} else {$cc['AckTime']='NaN';} } if ($cc['DisableTime']!='') {$d->setTimestamp($cc['DisableTime']);$cc['DisableTime']=$d->format('c'); } else {$cc['DisableTime']='NaN';} $data = json_encode($cc); $data=str_replace('"NaN"','null',$data); echo $id.'=>'.$data."\n"; $url="http://127.0.0.1:9200/xymon/".$COLLECTOR."/".$id; $res=CallAPI("POST",$url,$data); } ?> De : Peter Welter [mailto:peter.welter at gmail.com] Envoyé : vendredi 23 décembre 2016 14:45 À : OUTTERYCK Francois Cc : xymon at xymon.com Objet : Re: [Xymon] Capture and send data To Elasticsearch and Kibana Hi Francois, Looks good, your sample. I am very interested in your integration efforts. How do you select what is going to be inserted in the ELK stack? All status messages that Xymon receives? And could you provide an example of the xymond_channel block in you task.cfg and the capture.sh? Tasks.cfg probably is something like: [es] ... CMD xymond_channel --channel=status /home/xymon/server/ext/capture.sh ... And the capture.sh contains some curl script to the ELK stack? Regards, Peter 2016-12-14 18:15 GMT+01:00 OUTTERYCK Francois <francois.outteryck at soprasteria.com<mailto:francois.outteryck at soprasteria.com>>: Hello, I just do a small work with Xymon in order to send status data to Elasticsearch. And with Kibana reporting is easy. How it work ? Simple : With capture of status message (xymond_channel --channel=status /home/xymon/server/ext/capture.sh ) script encode status message in json and send as document in elasticSearch. Example of Mapping document status in elasticsearch : "status"=> array( "properties"=>array( "ServerName"=> array("type"=> "string"), "Type"=> array("type"=> "string"), "Status"=> array("type"=> "string"), "LastChange"=> array("type"=> "date","format"=> "strict_date_optional_time"), "LogTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "ValidTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "AckTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "DisableTime"=> array("type"=> "date","format"=> "strict_date_optional_time"), "Sender"=> array("type"=> "string"), "DisableMSg"=> array("type"=> "string"), "Message"=> array("type"=> "string") ) ) All is done with a 130 lines script. So simple to integrate in existing environment and so powerfull usage of Xymon/elasticsearch/Kibana For those who don’t know kibana : https://www.elastic.co/products/kibana [cid:image001.png at 01D25EA3.F74489F0] _______________________________________________ Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

2 1

4.3.30 RPMs released
by cleaver＠terabithia.org 23 May '22

23 May '22

Hello all, After a weekend in /testing/, the xymon 4.3.30-1 Terabithia RPMs have been moved to the production repositories at https://repo.terabithia.org/rpms/xymon/ Regards, -jc

2 1

Terabithia or compile
by ralphmitchell＠gmail.com 16 Mar '22

16 Mar '22

I downloaded the source rpm and installed it. The spec file for building the RPMs contains this: %if %{without trunk} %if 0%{?rhel} <= 7 Requires: ntp %endif %endif So, yeah, it builds into the binary RPMs the requirement to pull in ntp. I'm not sure if that can be made into a conditional, to require ntp OR chrony. Ralph Mitchell On Thu, Jun 18, 2020 at 5:05 PM Root, Paul T via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: "Root, Paul T" <Paul.Root at centurylink.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: RE: [Xymon] Terabithia or compile > That's odd. I did not do that to me. > > -----Original Message----- > From: Xymon <xymon-bounces at xymon.com> On Behalf Of John Thurston > Sent: Thursday, June 18, 2020 2:24 PM > To: xymon at xymon.com > Subject: Re: [Xymon] Terabithia or compile > > On 6/17/2020 4:38 PM, John Horne wrote: > > I used to build from source, but for our CentOS 7 and 8 servers I have > been > > using the Terabithia RPMs with no problems. The downside is that, like > Malcolm > > Hunter mentioned, I have subsequently applied some local patches - sent > to the > > list - so that meant rebuilding the RPMs. > > I'm now looking at the Terabithia package for the first time in years. > My first reaction is, wth? > > Installing the xymon package on my server has disabled chronyd and > installed (but not started) ntpd. why?! If I hadn't read through the > list of packages it was going to grab, and understand the significance > of this one, I'd be going blindly forward with an un-synchronized system > clock. > > Now I wonder how many other of my services it has disabled or replaced. > I feel like I've just walked into a mine field :( > > > -- > Do things because you should, not just because you can. > > John Thurston 907-465-8591 > John.Thurston at alaska.gov > Department of Administration > State of Alaska > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > > https://imss91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f… > This communication is the property of CenturyLink and may contain > confidential or privileged information. Unauthorized use of this > communication is strictly prohibited and may be unlawful. If you have > received this communication in error, please immediately notify the sender > by reply e-mail and destroy all copies of the communication and any > attachments. > > > > ---------- Forwarded message ---------- > From: "Root, Paul T via Xymon" <xymon at xymon.com> > To: "'John Thurston'" <john.thurston at alaska.gov>, "xymon at xymon.com" < > xymon at xymon.com> > Cc: > Bcc: > Date: Thu, 18 Jun 2020 21:04:50 +0000 > Subject: Re: [Xymon] Terabithia or compile > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

5 5

Xymond Oversize data/client on Xymon 4.3.30
by tp6891＠att.com 25 Nov '21

25 Nov '21

Hi, I am running Xymon 4.3.30 on Redhat 7.9 and I still get "Oversize data/client msg" under xymond and xymond.log. I have followed the advicean MAN page to modify the xymonserver.cfg file with no luck. I have increased MAXMSG_CLIENT, MAX, MSG_STATUS MAXMSG_USER & MAXMSG_DATA. Any ideas or am I missing something? Error: Oversize data/client msg from x.x.x.x truncated (n=894639, limit 524288) First line: somehost.mynetwork.com|sunos|sunos| xymonserver.cfg MAXMSG_CLIENT=1024 # clientdata messages (default=512k) MAXMSG_CLICHG=$MAXMSG_CLIENT # client change messages (default=MAXMSG_CLIENT) MAXMSG_STATUS=1024 # general "status" messages (default=256k) MAXMSG_STACHG=$MAXMSG_STATUS # status change messages (default=MAXMSG_STATUS) MAXMSG_PAGE=$MAXMSG_STATUS # page messages (default=MAXMSG_STATUS) #MAXMSG_ENADIS=1024 # enable and disable control messages (default=32k) MAXMSG_USER=1024 # "user" messages (default=128k) MAXMSG_DATA=1024 # "data" messages, if enabled (default=256k) #MAXMSG_NOTES=256 # "notes" messages, if enabled (default=256k Tim

4 4

4.3.30 core dumps with directory directive in hosts.cfg
by zak.beck＠accenture.com 03 Nov '21

03 Nov '21

Hi I?m using 4.3.30 and I?m experiencing core dumps on some servers using the directory directive in hosts.cfg. The contents of my directory sometimes change and this sometimes leads to a core dump. In hosts.cfg I have: directory /home/xymon/server/etc/include_xxxxx/xxxxx/ this directory contains one file per host which contain something like this: 0.0.0.0 a_unique_name # noconn NOCOLUMNS:conn,info,trends nonongreen A cron job adds/removes files from this directory periodically, however, the directory should contain at least one file at all times. It?s been a long time since I did any C but I think I?ve been able to track this down a bit with gdb and --debug on xymond. Here?s some xymond debug output: 5315 2021-11-03 12:08:45.037113 File /home/xymon/server/etc/include_xxxxx/xxxxx//xxxxxxxxx new timestamp 16223 2021-11-03 12:08:45.037136 -> save_checkpoint 5315 2021-11-03 12:08:45.037218 Opening file /home/xymon/server/etc/hosts.cfg 5315 2021-11-03 12:08:45.040573 Opening file /home/xymon/server/etc/include_xxxxx/xxxxx//xxxxxxxxxx .. (opens all the files) 5315 2021-11-03 12:08:45.047109 -> handle_dropnrename 5315 2021-11-03 12:08:45.047128 -> posttochannel 5315 2021-11-03 12:08:45.047144 Posting message 13651 to 1 readers 5315 2021-11-03 12:08:45.047155 <- posttochannel 5315 2021-11-03 12:08:45.047159 -> posttochannel 5315 2021-11-03 12:08:45.047168 Posting message 4683 to 1 readers 5315 2021-11-03 12:08:45.047176 <- posttochannel 5315 2021-11-03 12:08:45.047179 -> posttochannel 5315 2021-11-03 12:08:45.047188 Posting message 1367 to 1 readers 5315 2021-11-03 12:08:45.047195 <- posttochannel 5315 2021-11-03 12:08:45.047199 -> posttochannel 5315 2021-11-03 12:08:45.047207 Posting message 1905 to 1 readers 5315 2021-11-03 12:08:45.047214 <- posttochannel 5315 2021-11-03 12:08:45.047218 -> posttochannel 5315 2021-11-03 12:08:45.047221 Dropping message - no readers 5315 2021-11-03 12:08:45.047225 -> posttochannel 5315 2021-11-03 12:08:45.047228 Dropping message - no readers 5315 2021-11-03 12:08:45.047232 -> posttochannel 5315 2021-11-03 12:08:45.047240 Posting message 785 to 1 readers 5315 2021-11-03 12:08:45.047247 <- posttochannel 5315 2021-11-03 12:08:45.047251 -> posttochannel 5315 2021-11-03 12:08:45.047259 Posting message 6 to 1 readers 5315 2021-11-03 12:08:45.047266 <- posttochannel 5315 2021-11-03 12:08:45.047269 -> posttochannel 5315 2021-11-03 12:08:45.047273 Dropping message - no readers 5315 2021-11-03 12:08:45.047287 -> free_log_t 5315 2021-11-03 12:08:45.047300 <- free_log_t 5315 2021-11-03 12:08:45.047308 <- handle_dropnrename 5315 2021-11-03 12:08:45.047312 -> handle_dropnrename 16223 2021-11-03 12:08:45.240284 <- save_checkpoint 2021-11-03 12:09:00.061085 Whoops ! Failed to send message (timeout) 2021-11-03 12:09:00.061303 -> On the last handle_dropnrename, the core dump occurs: Program terminated with signal 6, Aborted. #0 0x00007f39f751e387 in __GI_raise (sig=sig at entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55 55 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); (gdb) bt #0 0x00007f39f751e387 in __GI_raise (sig=sig at entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55 #1 0x00007f39f751fa78 in __GI_abort () at abort.c:90 #2 0x0000000000415f63 in sigsegv_handler (signum=<optimized out>) at sig.c:57 #3 <signal handler called> #4 __strlen_sse2_pminub () at ../sysdeps/x86_64/multiarch/strlen-sse2-pminub.S:38 #5 0x000000000040a5ac in handle_dropnrename (cmd=cmd at entry=CMD_DROPSTATE, sender=sender at entry=0x425e3c "xymond", hostname=0x0, n1=n1 at entry=0x0, n2=n2 at entry=0x0) at xymond.c:2511 #6 0x0000000000403eb1 in main (argc=<optimized out>, argv=<optimized out>) at xymond.c:5702 What is interesting is that handle_dropnrename appears to be passed an empty hostname or null pointer if I understand gdb correctly. In handle_dropnrename strlen is called on hostname and I think this causes the core dump? (xymond.c line 2511): char *msgbuf = (char *)malloc(20 + strlen(hostname) + (n1 ? strlen(n1) : 0) + (n2 ? strlen(n2) : 0)) I _think_ handle_dropnrename is being called around line 5700 of xymond.c: else if (hostinfo(hwalk->hostname) == NULL) { /* Remove all state info about this host. This will NOT remove files. */ handle_dropnrename(CMD_DROPSTATE, "xymond", hwalk->hostname, NULL, NULL); What am I doing wrong ? Thanks Zak ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy. ______________________________________________________________________________________ www.accenture.com

1 0