February 2019 - Xymon - Xymon mailman web

file check flapping and HTTP500 for xymoncgimsg.cgi
by kii.noda＠gmail.com 08 Feb '19

08 Feb '19

Hi, I have a problem where a file check for a machine hosting an ELK stack is flapping between green and white, triggering alerts because of this. FYI, the host is sending its status information over authenticated HTTPS POST calls to xymoncgimsg.cgi. I got to the point where I found out that the list of files that the client is reporting on is received from the server on every call and then, on subsequent runs, it is used to check the files and then report back. Upon further investigation I observed that on some runs, this HTTP POST returns an HTTP500 error, thus the client then points $LOGFETCHCFG to a file that contains the HTTP500 page instead of the list of files to check. This means that on the next run it will no longer report back to the server the status of the files. It will recover when the POST returns HTTP200, hence looping between green and white in between successful/unsuccessful POST requests. Now, I would very much like to understand what happens BUT xymoncgimsg.cgi which is receiving the HTTP POST is a binary and I'm not aware of how to enable debugging to see what happens on those runs that fail with HTTP500. FWIW, the calls only fail w/ HTTP500 for this particular host and I did consider the size of the status message - it's just around the 1MB mark. The MSG size set in the configs sits at 2MB currently so I thought I should be covered here but maybe you guys know better than me. For now, my quick/dirty hack was to `chattr +i` on a working $LOGFETCHCFG just so that I no longer get the flapping behavior but I would like to see this get a proper fix. Would anyone be able to advise on a next step? Looking through C source files is not exactly what I want to do since I don't have the expertise to do so but would be willing to give it some love. So... help?!? :) Thanks!

1 0

History broken after cleaning up old data with trimhistory
by Alessandro.Tinivelli＠faacgroup.com 07 Feb '19

07 Feb '19

good day all, my xymon 4.3.28-1.el7.terabithia running on Centos7 has encountered a problem since i made a cleanup using trimhistory command. If a test (i.e. conn or anyelse) has been failing for a few hours, clicking the History Button shows all green bars, no red bars for the KO period. The bars show red lines for previous failures only until the time i used the trimhistory. What is the possible cause and solution? I issued this command: /usr/libexec/xymon/trimhistory --drop --droplogs --dropsvcs --cutoff=`date +%s --date="1 Oct 2018"` --env=/etc/xymon/xymonserver.cfg Thanks in advance!

3 3

[External] XymonPS upgrade source
by tlwilliams4＠vcu.edu 06 Feb '19

06 Feb '19

Thanks! So for us it is just the bb:// which points to the default /download folder. It works! We may make a XymonPS subfolder down the road. Tim On Wed, Feb 6, 2019 at 12:43 PM Andy Smith <abs at shadymint.com> wrote: > All, > > Spot on Zak, we use the bb:// urls for our upgrades, we have > ~xymon/server/download/XymonPS/ populated with for example > xymonclient_2.28.ps1 > > and the client configured with lines such as :- > > > clientversion:2.28:bb://XymonPS:SHA1:0737ef113b23f1994c21495e678eaa04bfd1e1c7 > > -- > Andy > > On Wed, Feb 06, 2019 at 05:15:40PM +0000, Beck, Zak wrote: > > > > I???d forgotten we added bb:// style urls, I must admit I don???t use > them regularly. > > > > From the server side, the manpage< > https://www.xymon.com/help/manpages/man1/xymon.1.html> says: > > > > download FILENAME > > Download a file from the Xymon server???s download directory. > > > > Looking at the xymond code (I???m hoping this is the right place): > > > > sprintf(fullfn, "%s/download/%s", xgetenv("XYMONHOME"), fn) > > > > so I think the download directory should be under your XYMONHOME and the > path in the bb: url is relative to that. > > > > You will not be able to download from an absolute path per your config > currently. I guess you might want to create an updates directory e.g. > $XYMONHOME/download/updates/, put your new client files there and your bb > url will be bb://updates. > > > > Zak > > > > > > From: Xymon <xymon-bounces at xymon.com> On Behalf Of Timothy Williams > > Sent: Wednesday, 6 February 2019 16:54 > > To: xymon at xymon.com > > Subject: [External] [Xymon] XymonPS upgrade source > > > > In our main 'regular' server environment we have a web server that holds > the newest XymonPS client for successful self-upgrading on slow scan. > However, in our restrictive PCI environment, the Windows servers are not > permitted to access the outside web folder or even other servers in the > environment excepting WSUS, antivirus, etc on specific ports. They do send > and receive files on port 1984 to Xymon server. > > > > 2019-02-05 14:11:23 Main and optional tests finished. > > 2019-02-05 14:11:23 Sending to server > > 2019-02-05 14:11:23 Using UTF8 encoding > > 2019-02-05 14:11:23 Connecting to host 192.168.47.233 > > 2019-02-05 14:11:23 Sent 69413 bytes to server > > 2019-02-05 14:11:23 Received 107 bytes from server > > > > I have tried to use the bb: pseudo-URL to access the download folder on > the Xymon (3.28) Linux server. World Read permissions have been verified. > From the client logs, it appears to find the file and initiates a download, > but the new file has zero bytes. The upgrade process continues to restart > the service, but obviously hangs as there is now no script file. I have > tried both ASCII and UTF8 encoding. > > > > 2019-02-05 14:11:23 Found a command: > clientversion:2.35:bb://usr/local/xymon/server/download > > .. > > 2019-02-05 14:11:23 Executing XymonCheckUpdate > > 2019-02-05 14:11:24 Running version 2.28; config version 2.35; > attempting upgrade > > 2019-02-05 14:11:24 XymonDownloadFromServer - Downloading > usr/local/xymon/server/download/xymonclient_2.35.ps1 to > C:\Utils\xymonclient_2.35.ps1 > > 2019-02-05 14:11:24 Using UTF8 encoding > > 2019-02-05 14:11:24 Connecting to host 192.168.47.233 > > 2019-02-05 14:11:24 Sent 62 bytes to server > > 2019-02-05 14:11:24 Wrote 0 bytes from server to > C:\Utils\xymonclient_2.35.ps1 > > 2019-02-05 14:11:24 Launching update > > 2019-02-05 14:11:24 Upgrading C:\Utils\xymonclient.ps1 to > C:\Utils\xymonclient_2.35.ps1 > > 2019-02-05 14:11:24 Restarting service... > > > > The error message on the XymonD process indicates that the file was not > found. > > > > Latest error messages: > > Download file usr/local/xymon/server/xymonclient_2.35.ps1 not found > > > > Anybody have ideas about where the fault lies? Using file share or https > is beyond where InfoSec wants to go. We currently manually copy in new > clients to each PCI server, I'd like to have self-upgrading working. > > > > Tim Williams > > Virginia Commonwealth University Computer Center >

1 0

XymonPS upgrade source
by tlwilliams4＠vcu.edu 06 Feb '19

06 Feb '19

In our main 'regular' server environment we have a web server that holds the newest XymonPS client for successful self-upgrading on slow scan. However, in our restrictive PCI environment, the Windows servers are not permitted to access the outside web folder or even other servers in the environment excepting WSUS, antivirus, etc on specific ports. They do send and receive files on port 1984 to Xymon server. 2019-02-05 14:11:23 Main and optional tests finished. 2019-02-05 14:11:23 Sending to server 2019-02-05 14:11:23 Using UTF8 encoding 2019-02-05 14:11:23 Connecting to host 192.168.47.233 2019-02-05 14:11:23 Sent 69413 bytes to server 2019-02-05 14:11:23 Received 107 bytes from server I have tried to use the bb: pseudo-URL to access the download folder on the Xymon (3.28) Linux server. World Read permissions have been verified. From the client logs, it appears to find the file and initiates a download, but the new file has zero bytes. The upgrade process continues to restart the service, but obviously hangs as there is now no script file. I have tried both ASCII and UTF8 encoding. 2019-02-05 14:11:23 Found a command: clientversion:2.35:bb://usr/local/xymon/server/download .. 2019-02-05 14:11:23 Executing XymonCheckUpdate 2019-02-05 14:11:24 Running version 2.28; config version 2.35; attempting upgrade 2019-02-05 14:11:24 XymonDownloadFromServer - Downloading usr/local/xymon/server/download/xymonclient_2.35.ps1 to C:\Utils\xymonclient_2.35.ps1 2019-02-05 14:11:24 Using UTF8 encoding 2019-02-05 14:11:24 Connecting to host 192.168.47.233 2019-02-05 14:11:24 Sent 62 bytes to server 2019-02-05 14:11:24 Wrote 0 bytes from server to C:\Utils\xymonclient_2.35.ps1 2019-02-05 14:11:24 Launching update 2019-02-05 14:11:24 Upgrading C:\Utils\xymonclient.ps1 to C:\Utils\xymonclient_2.35.ps1 2019-02-05 14:11:24 Restarting service... The error message on the XymonD process indicates that the file was not found. Latest error messages: Download file usr/local/xymon/server/xymonclient_2.35.ps1 not found Anybody have ideas about where the fault lies? Using file share or https is beyond where InfoSec wants to go. We currently manually copy in new clients to each PCI server, I'd like to have self-upgrading working. Tim Williams Virginia Commonwealth University Computer Center

2 1

SMS options
by doughnut＠doughnut.net 05 Feb '19

05 Feb '19

For quite some time, we've been sending alerts via emails to <number>@<telco>.tld. It works, mostly, but I'd like to do better. I'm considering getting a wireless modem (without data), or using a service like Twilio. Does anyone have any warnings or advice beyond the obvious?

4 3

Xymon WinPSclient uptime bug
by spah＠syntec.co.uk 04 Feb '19

04 Feb '19

I've noticed a minor bug in the Xymon WinPSclient uptime report - after stopping and starting the service it says " Machine recently rebooted " in the uptime column on the Xymon server display. I don't mind it going yellow - arguably a warning might be in order here, but if so, it should say 'Xymon client recently started'. Once the client has been running for a bit, it reports the correct uptime. Kind regards, SebA

2 3

monitoring Domain Controllers
by Paul.Root＠CenturyLink.com 04 Feb '19

04 Feb '19

Hi, Recently, I inherited some Windows machines. Including a pair of Domain controllers. I'm wondering what are somethings I'm going to want to monitor to make sure these things keep running? I have the Powershell client installed no problem. I see DNS and DHCP services, and am adding those. Then I see ADWS, the active director. Each machine has it as starting automatically, but only one is running. I assume that's correct, and that they watch each other so only one is running, right? Is there a way to do a combo test to make sure it is running on only one? Thanks, Paul. Paul Root Lead Engineer 390 Commerce Dr Woodbury, Mn 55125 651-312-5207 paul.root at centurylink.com This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.

5 6

DISPLAYGROUP fails in analysis.cfg
by doughnut＠doughnut.net 01 Feb '19

01 Feb '19

I have a xymon server running "Debian GNU/Linux 9 (stretch)" with xymon 4.3.28-2. I'm having bad luck getting DISPLAYGROUP to work in analysis.cfg. It seems to only match on the first 7 characters. I also see this problem on "Debian GNU/Linux 8 (jessie)" with xymon 4.3.17-6+deb8u1. Below, I've shown that the display group is set, and that the analysis.cfg file is being parsed correctly by xymon, yet the the svcstatus page only shows the shortest of the DISPLAYGROUP enabled lines. root at misc-ten105-test:/etc/xymon# egrep -H Testnet.Database.Servers hosts.cfg hosts.cfg:group Testnet Database Servers root at misc-ten105-test:/etc/xymon# cat analysis.d/davetest.cfg HOST=* EXCLASS=linux PROC davetest-nonlinux 1 150 yellow PROC davetest-Testnet.Database.Servers 1 150 yellow DISPLAYGROUP=%(Testnet.Database.Servers) PROC davetest-Database 1 150 yellow DISPLAYGROUP=%(Database) PROC davetest-Databas 1 150 yellow DISPLAYGROUP=%(Databas) PROC davetest-Testnet.Database 1 150 yellow DISPLAYGROUP=%(Testnet.Database) PROC davetest-Testnet.Data 1 150 yellow DISPLAYGROUP=%(Testnet.Data) PROC davetest-Testnet.D 1 150 yellow DISPLAYGROUP=%(Testnet.D) PROC davetest-Testnet. 1 150 yellow DISPLAYGROUP=%(Testnet.) PROC davetest-Testnet 1 150 yellow DISPLAYGROUP=%(Testnet) PROC davetest-Testne 1 150 yellow DISPLAYGROUP=%(Testne) root at misc-ten105-test:/etc/xymon# sudo -u xymon xymoncmd xymond_client --dump-config | grep davetest [...] PROC davetest-nonlinux 1 150 yellow HOST=* EXCLASS=linux (line: 375) PROC davetest-Testnet.Database.Servers 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.Database.Servers) EXCLASS=linux (line: 377) PROC davetest-Database 1 150 yellow HOST=* DISPLAYGROUP=%(Database) EXCLASS=linux (line: 378) PROC davetest-Databas 1 150 yellow HOST=* DISPLAYGROUP=%(Databas) EXCLASS=linux (line: 379) PROC davetest-Testnet.Database 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.Database) EXCLASS=linux (line: 380) PROC davetest-Testnet.Data 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.Data) EXCLASS=linux (line: 381) PROC davetest-Testnet.D 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.D) EXCLASS=linux (line: 382) PROC davetest-Testnet. 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet.) EXCLASS=linux (line: 383) PROC davetest-Testnet 1 150 yellow HOST=* DISPLAYGROUP=%(Testnet) EXCLASS=linux (line: 384) PROC davetest-Testne 1 150 yellow HOST=* DISPLAYGROUP=%(Testne) EXCLASS=linux (line: 385) root at misc-ten105-test:/etc/xymon# lynx -dump -nolist 'http://localhost/xymon-cgi/svcstatus.sh?HOST=balin2012.collegenet.testnet&S…' [...] Xymon balin2012.mydomain.testnet - procs Wed Jul 11 13:29:40 2018 _______________________________________________________________________ HISTORY Wed Jul 11 13:29:40 2018 - Processes NOT ok yellow davetest-nonlinux (found 0, req. between 1 and 150) yellow davetest-Testnet (found 0, req. between 1 and 150) yellow davetest-Testne (found 0, req. between 1 and 150) Status unchanged in 8 days, 21 hours, 22 minutes Status message received from 10.101.1.35 Client data available

2 4

Advanced xymondboard query
by davidenzomedina＠gmail.com 01 Feb '19

01 Feb '19

Hello! I am new to Xymon and I am trying to extract xymon's data from command line. What I need: extract from Xymon list of services with the following fields: Host;ServiceName;ServiceStatus,ServicePerfValue(if applicable);ServiceOutputText For example: localhost;bbd;green;;OK - bbd is ok. localhost;cpu;green;0.23;OK - cpu is ok. localhost;memory;yellow;0.85;WARNING - memory is 85%. ... So far, I found xymon command and xymondboard subcommand: # /home/xymon/server/bin/xymon 127.0.0.1 "xymondboard fields=hostname,testname,color,line1" checkmk|trends|green| checkmk|info|green| checkmk|http|green|green Thu Jan 31 12:09:41 2019: OK linux04|trends|green| linux04|info|green| linux04|clientlog|green| linux04|memory|green|green Thu Jan 31 12:09:41 -03 2019 - Memory OK My problem: I cannot find a way to extract performance data from services. I tried reading RRD files, but I cannot find a general association rule between services and rrd files. It seems it is pretty much dependent of each service. How is the best way to do this? Kind regards Enzo

2 4

xymongen core dump
by jainyashaswi＠gmail.com 01 Feb '19

01 Feb '19

Hi all, I am running the xymonreports.sh to generate reports daily, weekly and monthly. But recently i am getting following error when we execute this script : *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_col* *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_dsc* *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_err* *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_load* *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_stat* *2019-02-01 07:17:39 build_reportspecs: Found too few items in memory* *2019-02-01 07:17:39 build_reportspecs: Found too few items in serial)* *2019-02-01 07:17:39 build_reportspecs: Found too few items in rtd* *2019-02-01 07:17:39 build_reportspecs: Found too few items in jitter* *2019-02-01 07:17:39 build_reportspecs: Found too few items in bfd* *2019-02-01 07:17:39 build_reportspecs: Found too few items in bgp* *2019-02-01 07:17:39 build_reportspecs: Found too few items in cpu* *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_col* *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_dsc* *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_err* *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_load* *2019-02-01 07:17:39 build_reportspecs: Found too few items in if_stat* *2019-02-01 07:17:39 build_reportspecs: Found too few items in memory* *2019-02-01 07:17:39 build_reportspecs: Found too few items in serial)* */opt/xymon/server/bin/xymonreports.sh: line 83: 7553 Aborted (core dumped) XYMONWEB=$REPORTTOPURL/$REPDIR $XYMONHOME/bin/xymongen --reportopts=$STIME:$ETIME:0:nongr $XYMONGENREPOPTS $REPORTTOPDIR/$REPDIR* *+ exit 0* i have also tried to analyse the core dump , but not able to get much help from there. *-bash-4.2$ gdb /opt/xymon/server/bin/xymongen -c /tmp/core.42915* *GNU gdb (GDB) SLES Expanded Support platform 7.6.1-110.el7* *Copyright (C) 2013 Free Software Foundation, Inc.* *License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html <http://gnu.org/licenses/gpl.html>>* *This is free software: you are free to change and redistribute it.* *There is NO WARRANTY, to the extent permitted by law. Type "show copying"* *and "show warranty" for details.* *This GDB was configured as "x86_64-redhat-linux".* *For bug reporting instructions, please see:* *<http://www.gnu.org/software/gdb/bugs/ <http://www.gnu.org/software/gdb/bugs/>>...* *Reading symbols from /xymon/xymon/server/bin/xymongen...done.* *[New LWP 42915]* *Core was generated by `/opt/xymon/server/bin/xymongen --reportopts=1548889200:1548975599:0:nongr --rec'.* *Program terminated with signal 6, Aborted.* *#0 0x000000374f8324f5 in ?? ()* *(gdb)* I have done all the restart but no help. Please help me the way forward. Thanks YJ

1 0