November 2023 - Xymon - Xymon mailman web

Possible bug in xymon svcstatus causing segfault (buffer overrun?)
by jeremy＠laidman.org 31 May '24

31 May '24

Hi all I'm not a C programmer by any stretch, so I'm seeking confirmation that my working is sound. At our site we run 4.3.30-1 from Terabithia RPM. For most hosts, the "trends" page displays just fine, but for some hosts, the webserver gives a CGI error, and when I manually run the CGI directly it crashes with a seg fault. This seems to be more likely for hosts with a lot of entries on the trends page, but some with more trends graphs are OK than others with fewer. I believe I've tracked the issue down to the last few lines of code in svcstatus-trends.c, where a URL is constructed for one graph entry, before being added to the list of trends in a buffer. My reading of the code suggests that it first pre-allocates (using malloc()) a 16k buffer for the trends data, then sequentially appends trend strings (which include a URL) into the buffer, until the length of a string would exceed the size of the buffer. In that case, an extra 4k of memory is added to the buffer (using realloc()), and then the string is added to the buffer. The problem with the code, as I see it, is that the string might be more than 4k long, in which case, adding string would cause a buffer overrun. I don't really know what's in each string, so I don't know if it's ever likely to be 4k, but we're seeing the process dumping core immediately after this bit of code, so that's the suspicion. Here's the bit of code, which is attempting to add the string "onelink" to the buffer, after seeing if it's going to exceed the buffer, and extending the buffer if required: if ((buflen + strlen(onelink)) >= allrrdlinks_buflen) { SBUF_REALLOC(allrrdlinks, allrrdlinks_buflen+4096); allrrdlinksend = allrrdlinks + buflen; } allrrdlinksend += snprintf(allrrdlinksend, (allrrdlinks_buflen - (allrrdlinksend - allrrdlinks)), "%s", onelink); Elsewhere in other code, the SBUF_REALLOC macro is used to add extra bufferspace by calculating a new buffer size of [existing buffer size plus 4k plus the length of the string being added]. So if the string to be added had 20 bytes, SBUF_REALLOC would be used to add 4k+20 bytes to the buffer size. Notably, the size of the string is included in the calculation. For example this bit of code elsewhere in the same file does a similar buffer extension: if ((strlen(rrdlink) + strlen(partlink) + 1) >= rrdlink_buflen) { SBUF_REALLOC(rrdlink, rrdlink_buflen + strlen(partlink) + 4096); } strncat(rrdlink, partlink, (rrdlink_buflen - strlen(rrdlink))); Note the extra "+ strlen(<stringvar>)" bit in the extra memory being allocated that isn't there in the first block of code above. So my thinking is that the SBUF_REALLOC() needs to take the string's size into account, like so: SBUF_REALLOC(allrrdlinks, allrrdlinks_buflen+strlen(onelink)+4096); Is anyone with C skills able to confirm that this does indeed look like a bug, and also that my proposed change is a suitable fix? After applying this change and recompiling, we haven't yet seen a seg fault from this bit of code, so I'm hoping it's all fixed, and without negative side effects. Cheers Jeremy

3 3

Update on 4.4 (Alpha1 release)
by tom＠4schmidts.com 15 May '24

15 May '24

J.C., Thank you again J.C. for reviving the updates for Xymon. I am willing to help update it as well, even though I am now retired and only have a small install of Xymon 4.3.30 at home for my home network. I have contributed to Xymon several times in the past when I had it installed in a large corporate environment before I retired. J.C. and 4.4-alpha1 testers, I have done a little bit of testing of 4.4-alpha1 on Rocky Linux 8.8 (server and client) and on a busybox Linux install on a NAS. I looked into the gcc 8.5.0 compiler warnings that I got on Rocky Linux, and have corrected them, or silenced the ones that should not be an issue. Attached are two context diff patch files, one to fix the compiler warnings, and one to make a couple updates that I had applied to 4.3.30 previously. My updates fix the busybox Linux client build, and enhance the temperature graph to optionally display both Celsius and Fahrenheit readings. I have not fully tested 4.4-alpha1 yet, but wanted to get these first patches released for 4.4-alpha2. Tom Schmidt On Wed, Sep 20, 2023 at 2:38?PM J.C. Cleaver <cleaver at terabithia.org> wrote: > Hello all! Just wanted to update everyone on a few things. > > Going through the 4.x codebase re-familiarizing myself again with the > state of everything, I recalled that I'd actually built a release artifact > before development was paused, including a testing RPM that has actually > been running unbeknownst on a VM for quite a while. Unfortunately, while > validating that compiles from source worked I ran into what looked like > showstopper bugs. These weren't showing up in the package, so I had to > pore through a lot more code* than I expected to try to figure out where > the package was accidentally fixing it. > > (*message-sending code, which changes a lot in 4.4 to support compression, > TLS, HTTP, and size-prefixed messages) > > Fortunately, it ended up being a relatively simple issue (isn't it > always?), with easy-enough validation and workaround (given below). > Although more fixes could be put in (and I'm sure will be needed!), I > believe there's some benefit to having a stable starting point using a > build artifact from 2019. > > As a result, I'm pleased to announce the first alpha for Xymon 4.4! As > this is an alpha release, this is absolutely NOT suitable for production > use and is intended mainly for smoketesting by source users. (And I expect > there will be smoke.) The release tarball is available now at: > > > https://sourceforge.net/projects/xymon/files/Xymon/4.4-alpha/xymon-4.4-alph… > > > Major improvements are too many to list here but are noted in the Changes > and RELEASENOTES files. README documents for compression, TLS, and IPv6 > are in progress. > > Some of the changes and improvements other than TLS support have been in > the Terabithia packages (to some extent) for a while, so there will be > less of a delta for those users to this alpha. > > For the moment, I'd like to focus on source tarball users and others > running xymon from scratch to help fix underlying issues and any smoketest > failures, which I expect may crop up particularly on untested non-Linux > platforms. Once the lower-hanging fruit is fixed and more patches are > upstreamed and cleaned up, I'll release an alpha2 release at which point > packages may be released to help with testing iteration and downstream > distributors. > > Please flag bug reports or patches as related to "alpha1" in the subject > line to help keep things clear. > > > Thank you to everyone in the community for remaining part of it -- and > xymon users! -- during this interregnum. Restarting development has been a > process, but I'm glad to take this important first step to our next > release. > > > Regards, > -jc > > > > *** KNOWN ISSUES -- IMPORTANT *** > > The configure script is likely setting XYMONHOME/XYMONCLIENTHOME and > XYMSRV/XYMSERVERS/XYMONSERVERS incorrectly, which will prevent xymonlaunch > from correctly launching anything. Double check the xymon environment > files (xymonserver/xymonclient.cfg) before starting either the server or > client. > > The tools/xymon-client.default and tools/xymonlaunch files may be placed > into /etc/{sysconfig,default}/{xymon-client,xymonlaunch} to more easily > set $XYMONSERVERS > > Revealed by the above issue is a bug causing a crash in the client (or > anywhere) when given an empty string as the destination argument (instead > of 0, 127.0.0.1 or no variable set). Don't do this --> [ xymon '' "ping" ] > > The client and logfetch binary may be being re-compiled during 'make > install' > > Compilation may fail if c-ares and at least one compression library is not > present (lzo/lz4/zlib are all options). xxHash libraries are also highly > recommended as the plan is for xxhash to replace md5 for checksumming > functionality. libtirpc may be necessary on some platforms, and it's > possible the build code is still expecting traditional (Sun) RPC > > > > > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

5 6

Is this thing on? SUMMARY UPDATE and POC
by bruno.manzoni＠ubi-network.ch 29 Jan '24

29 Jan '24

Hi every one *I did a POC on GitHub! *- https://github.com/xymon-monitoring To include/manage the summary of problem (I did previously on this mailing list). The mailing list is not really good to follow problems that take a long time to be resolved (my point of view) This summary also evolve with some more details (but mainly from me) and take a also new form for a better follow up (I hope so) : https://github.com/xymon-monitoring/problem-solving This is only a POC so not official at all, but just for you to see what I did/what could be done (currently limited by my poor skills) You are invited to help me to improve this POC, so to be a member of this GitHub organization: *let me know your GitHub name* Remark: - *Let me know what you think!* - *Focus on the general ideas*: *the support (GitHub)* and *the form* I choose and not to much in the content (I try to make also a good content, but for sure this will need many more work, I would like first to add latest feedback: WIP) - Constructive comment about anything is always welcome! - And as English is not my language: do not hesitate also to correct it. - Positive feedback also motivate... *We still have an urgent problem: the mailing list *(see detail on the POC or in this mailing list) Bruno

4 10

Xymon Server on RHEL9
by john.thurston＠alaska.gov 29 Nov '23

29 Nov '23

Colin's question was related directly to NTP with RHEL 9. Mine is more general, but I think related enough to go here. Our Xymon servers are running on CentOS 7. I need to look at my options to get those replaced (or turned into) Linux 8 or 9. Alma Linux is where I've taken other hosts in this situation; mostly to 9, but one to 8 due to package availability. When we moved our Xymon service from Solaris to Linux, it was (obviously) a completely new deployment. Our current Xymon servers have lots of things besides Xymon running on them. There are external hosts pushing files in via scp through service accounts, and local cron jobs consuming those files to generate test results (and special Xymon web pages). There are customization to the apache web server handling authentication and delivering non-Xymon (but related) content. And other stuff I'm sure I've forgotten about. It's going to be whole lot of work to re-build all of this on a new host, and move traffic to it. I've used 'elevate <https://almalinux.org/elevate/>' (from Red Hat's 'Leapp') with several other systems to lift them from CentOS 7 to CentOS Stream 8, or Alma 8 & 9. It wasn't as simple as falling off a log, and it wasn't successful in all cases, but in several cases it did what it was supposed to do and saved us a huge amount of work. Has anyone tried "elevating" their older Linux Xymon server to Linux 9 ? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Department of Administration State of Alaska On 10/9/2023 4:00 PM, Colin Coe wrote: > Hi all > > I'm getting ready to replace our venerable RHEL6 Xymon server with one > running RHEL 9. > > We use the terabithia.org > <http://terabithia.org/> > RPMs. > > NTP tests are all returning red.? I've updated xymonserver.cfg (and > restarted the xymon service) by changing NTPDATE="/usr/bin/chronyc" > and adding NTPDATEOPTS="tracking". > > When I run "/usr/bin/chronyc tracking <IP HERE>" from the shell, it > works fine however the Xymon page is red. > > Service ntp on <CENSORED> is not OK : Service unavailable > > Command: /usr/bin/chronyc tracking <CENSORED> 2>&1 > > Any ideas? > > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon

1 0

'Invalid timespec' in xymond.log
by john.thurston＠alaska.gov 27 Nov '23

27 Nov '23

I'm seeing messages like this > Invalid timespec - expected 4 digits, got: '*:*' > Invalid timespec - expected 4 digits, got: '*' > Invalid timespec - expected 4 digits, got: '*:*' > Invalid timespec - expected 4 digits, got: '*' pop up in my xymond.log. They've been there for ages, so this is nothing new. I've poked at them before, to no satisfaction. I think this came out of the 'minutes' function in lib/timefunc.c But what was the input which was passed through that function, to generate the error? I have 'DOWNTIME' specifications like > DOWNTIME=60:*:*;*:1630:0800 > But there are 21 of those in hosts.cfg, and I don't have that many complaints in the log. Does anyone else have a suggestion where to look? or how to interactively run some parsing commands to tease out the source? Maybe that function is used for more than parsing the configs? Could the source of the errors be from messages passed into the server from clients? -- -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Department of Administration State of Alaska

2 2

Acknowledgements and page color.
by cleaver＠terabithia.org 09 Nov '23

09 Nov '23

On Thu, November 9, 2023 11:35, Grant Taylor wrote: > On 11/8/23 11:46???AM, J.C. Cleaver wrote: >> This isn't something currently there, but I could see adding it in >> in the future as an option or toggle. > > That would be nice to have. :-) > >> For the moment, the distinction here between one Ack'd and one >> a page-viewer "doesn't need to care about" would be achieved by >> sending a "Disabled until OK" message, which sets the color to the >> non-alerting Blue. > > Yep, that's what we've been doing. > > But I figured I'd stop and ask instead of doing the same thing has > always been done out of ignorance. > > I should clarify that I was referring to the individual status pages for a host (pulled live via svcstatus.cgi, or a static HTML generation if enabled). If you want to prevent an entire *page* from reflecting the worst status if that status has been acked, you can pass the --nopropack=* option to xymongen in your tasks.cfg entry. See https://xymon.sourceforge.io/xymon/help/manpages/man1/xymongen.1.html -jc

1 0

Acknowledgements and page color.
by cleaver＠terabithia.org 09 Nov '23

09 Nov '23

On Wed, November 8, 2023 09:24, Grant Taylor wrote: > Hi, > > Is there any way to have the page color go back to green when a yellow / > red / purple is acknowledged? > > Or will the page color always and forever reflect the worst color > (purple -> yellow -> red) on the page? > > This isn't something currently there, but I could see adding it in in the future as an option or toggle. For the moment, the distinction here between one Ack'd and one a page-viewer "doesn't need to care about" would be achieved by sending a "Disabled until OK" message, which sets the color to the non-alerting Blue. -jc

2 1

Missing support for HTTP/2 ?
by jeremy＠laidman.org 08 Nov '23

08 Nov '23

On Wed, 8 Nov 2023 at 23:14, Ingeborg Hellemo via Xymon <xymon at xymon.com> wrote: > > ~/server/bin/xymonnet --debug --no-update <servername> > > 35422 2023-11-08 13:06:06.168385 1 status messages merged into 1 > transmissions > Address=[IP]:443, open=1, res=0, err=5, connecttime=0.000303, > totaltime=0.002631, > httpstatus = -5, open=1, errcode=5, parsestatus=0 > Response: > (no headers) > URL : https://<servername>/ > HTTP status : -5 > I'm surprised there isn't some kind of error message being displayed. err=5 is "CONTEST_ESSL" meaning something went wrong in the SSL comms, or setup. In most cases where the error code is set to CONTEST_ESSL, there's an "errprintf()" that describes the error. I'd have thought "--debug" would show these on STDERR. The missing error message is likely going to help narrow down the problem. These are all of the error messages in contest.c where err is set to CONTEST_ESSL: errprintf("SSL test, but xymonnet was built without SSL support\n"); errprintf("Failed to find enough entropy on your system"); errprintf("Cannot create SSL context - IP %s, service %s: %s\n", inet_ntoa(item->addr.sin_addr), item->svcinfo->svcname, sslerrmsg); errprintf("Cannot load SSL client certificate/key %s: %s\n", item->ssloptions->clientcert, sslerrmsg); errprintf("SSL_new failed - IP %s, service %s: %s\n", inet_ntoa(item->addr.sin_addr), item->svcinfo->svcname, sslerrmsg); errprintf("Private/public key mismatch for certificate %s\n", item->ssloptions->clientcert); errprintf("Could not initiate SSL on connection - IP %s, service %s: %s\n", inet_ntoa(item->addr.sin_addr), item->svcinfo->svcname, sslerrmsg); errprintf("IO error in SSL_connect to %s on host %s: %s\n", portinfo, inet_ntoa(item->addr.sin_addr), sslerrmsg); errprintf("Unspecified SSL error in SSL_connect to %s on host %s: %s\n", portinfo, inet_ntoa(item->addr.sin_addr), sslerrmsg); errprintf("Unknown error %d in SSL_connect to %s on host %s: %s\n", err, portinfo, inet_ntoa(item->addr.sin_addr), sslerrmsg); errprintf("Cannot get peer certificate for %s on host %s\n", portinfo, inet_ntoa(item->addr.sin_addr)); Perhaps these can give you some clues about what might be going wrong. I also suggest taking a look at the webserver logs for any errors, warnings or notice messages associated with the xymonnet connection. J

1 0

Acknowledgements and page color.
by gtaylor＠tnetconsulting.net 08 Nov '23

08 Nov '23

Hi, Is there any way to have the page color go back to green when a yellow / red / purple is acknowledged? Or will the page color always and forever reflect the worst color (purple -> yellow -> red) on the page? -- Grant. . . . unix || die

1 0

Missing support for HTTP/2 ?
by tom＠4schmidts.com 08 Nov '23

08 Nov '23

Ingeborg, HTTP/2 support is currently not in xymon, including in the 4.4alpha1 development tree. I did a little testing, and sites like google that support HTTP/2 still pass the xymonnet checks as seen below: $ curl --head https://www.google.com | head -1 HTTP/2 200 $ grep google ~/server/etc/hosts.cfg 0.0.0.0 www.google.com # conn https://www.google.com/ $ ~/server/bin/xymonnet --no-update www.google.com |grep -B2 HTTP &green https://www.google.com/ - OK HTTP/1.1 200 OK $ ~/server/bin/xymonnet --no-update www.google.com |grep -i ssl status+30 www,google,com.sslcert green Wed Nov 1 08:38:21 2023 &green SSL certificate for https://www.google.com/ expires in 2252 days Perhaps the website you are testing is configured to support HTTP/2 and not support HTTP/1? Try running the xymonnet command manually similar to what I did here. Tom Schmidt On Wed, Nov 1, 2023 at 7:28?AM Ingeborg Hellemo via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: Ingeborg Hellemo <ingeborg.hellemo at uit.no> > To: xymon at xymon.com > Cc: > Bcc: > Date: Wed, 01 Nov 2023 14:21:04 +0100 > Subject: Missing support for HTTP/2 ? > I have a webserver which works ok when you visit it in a browser or when > you > use curl, but Xymon http test shows "SSL error". Tests with "openssl > s_client" > from the command line works as expected. > > Could the culprit be that xymonnet does not support HTTP/2 ? > > > /#curl --head https://<servername>/ > HTTP/2 200 > cache-control: private > content-length: 4423 > content-type: text/html > strict-transport-security: max-age=31536000 > date: Wed, 01 Nov 2023 13:15:44 GMT > > > > > --Ingeborg > -- > Ingeborg ?strem Hellemo -- ingeborg.hellemo at uit.no > Dep. of Information Technology --- Univ. of Troms? > > > > > > ---------- Forwarded message ---------- > From: Ingeborg Hellemo via Xymon <xymon at xymon.com> > To: xymon at xymon.com > Cc: > Bcc: > Date: Wed, 01 Nov 2023 14:21:04 +0100 > Subject: [Xymon] Missing support for HTTP/2 ? > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

2 1