September 2023 - Xymon - Xymon mailman web

Possible bug in xymon svcstatus causing segfault (buffer overrun?)
by jeremy＠laidman.org 31 May '24

31 May '24

Hi all I'm not a C programmer by any stretch, so I'm seeking confirmation that my working is sound. At our site we run 4.3.30-1 from Terabithia RPM. For most hosts, the "trends" page displays just fine, but for some hosts, the webserver gives a CGI error, and when I manually run the CGI directly it crashes with a seg fault. This seems to be more likely for hosts with a lot of entries on the trends page, but some with more trends graphs are OK than others with fewer. I believe I've tracked the issue down to the last few lines of code in svcstatus-trends.c, where a URL is constructed for one graph entry, before being added to the list of trends in a buffer. My reading of the code suggests that it first pre-allocates (using malloc()) a 16k buffer for the trends data, then sequentially appends trend strings (which include a URL) into the buffer, until the length of a string would exceed the size of the buffer. In that case, an extra 4k of memory is added to the buffer (using realloc()), and then the string is added to the buffer. The problem with the code, as I see it, is that the string might be more than 4k long, in which case, adding string would cause a buffer overrun. I don't really know what's in each string, so I don't know if it's ever likely to be 4k, but we're seeing the process dumping core immediately after this bit of code, so that's the suspicion. Here's the bit of code, which is attempting to add the string "onelink" to the buffer, after seeing if it's going to exceed the buffer, and extending the buffer if required: if ((buflen + strlen(onelink)) >= allrrdlinks_buflen) { SBUF_REALLOC(allrrdlinks, allrrdlinks_buflen+4096); allrrdlinksend = allrrdlinks + buflen; } allrrdlinksend += snprintf(allrrdlinksend, (allrrdlinks_buflen - (allrrdlinksend - allrrdlinks)), "%s", onelink); Elsewhere in other code, the SBUF_REALLOC macro is used to add extra bufferspace by calculating a new buffer size of [existing buffer size plus 4k plus the length of the string being added]. So if the string to be added had 20 bytes, SBUF_REALLOC would be used to add 4k+20 bytes to the buffer size. Notably, the size of the string is included in the calculation. For example this bit of code elsewhere in the same file does a similar buffer extension: if ((strlen(rrdlink) + strlen(partlink) + 1) >= rrdlink_buflen) { SBUF_REALLOC(rrdlink, rrdlink_buflen + strlen(partlink) + 4096); } strncat(rrdlink, partlink, (rrdlink_buflen - strlen(rrdlink))); Note the extra "+ strlen(<stringvar>)" bit in the extra memory being allocated that isn't there in the first block of code above. So my thinking is that the SBUF_REALLOC() needs to take the string's size into account, like so: SBUF_REALLOC(allrrdlinks, allrrdlinks_buflen+strlen(onelink)+4096); Is anyone with C skills able to confirm that this does indeed look like a bug, and also that my proposed change is a suitable fix? After applying this change and recompiling, we haven't yet seen a seg fault from this bit of code, so I'm hoping it's all fixed, and without negative side effects. Cheers Jeremy

3 3

Fwd: Re. Is this thing on?
by bruno.manzoni＠ubi-network.ch 30 Sep '23

30 Sep '23

Hi. Seem that Ye-Fee is unable to email to the mailing list? *Can you check that if there is a problem ( I think Henrik) ?* By to way, I remember that we have to migrate to another service. *What about: https://mailman3.com/ which is free for opensource ?* (comment to do or not to do are welcome!) *Can someone take the job ? * Thing to do: - Ask malman3 if possible to have a freemailing list - The we can see how to do it! (That will need for sure to contact Henrik), but I can help also. Bruno -------- Forwarded Message -------- Subject: Re. [Xymon] Is this thing on? Date: Thu, 28 Sep 2023 16:22:00 +0000 From: Liang, Ye-Fee [PHSA] <yefee.liang at phsa.ca> To: bruno.manzoni at ubi-network.ch <bruno.manzoni at ubi-network.ch> Hi Bruno, ?? I was able to email xymon at xymon.com long time ago, and now find myself unable to email anymore.? I have used xymon at different organizations over the decades, and would love to help the tool/platform to continue for decades more.? Simply there is no other product that is more versatile, easy to manage, and light server+client requirement.? Many more people are eager to contribute, but can only just read the xymon list email, from the side line. ?? Thanks for your summary of Xymon non-technical and technical needs. Just to keep the enthusiasm going, I would suggest a section on "making the better use of Xymon as now" I am only a sys admin, not a coder, so my focus is on management and usability areas. 1. enhanced cpu utilization graph (adds wait IO; better colors); see 1a. 2. menu script (see 2a) to clean up of accumulating rrd, hostdata and histlogs, eg. due to short-term ???ad-hoc metrics like nfs file systems, container nic devices. 3. menu script (see 3a) to manage clients easier Keep up the great work! Ye-Fee 1a 2a ------------------------------------------------------ c) change Client:?????? yyy ct) change days to trim: 30 1) summary???? 1a) list pings?? 1b) list cpu clients 1bbb) list to scp file?????????? 1bb) list in one column 1c) list non-pings tests?????? ??1d) list non-green cpu tests 2)? duplicate report 3)? Info on client?????????????? 3a) formatted 4)? Search client in hosts.cfg+ 6)? find clients with too many rrd's???? 6z) list -5 RRD's 6a) list top 10?? 6af) find top 10?? 6at) trim hostdata's ?6b) list top 10?? 6bf) find top 10?? 6bt) trim histlogs's 6c) list top 10?? 6cf) find top 10?? 6ct) trim Select from top 10 to trim??? 6ta) RRD? 6tb) hostdata? 6tc) histlogs 7a) diff host_cfg files????????? 7u) update host_cfg files ------------------------------------------------------ 3a ----------------------------------------------------------- c) Change Client: vag7 c1) check ssh???????????????? c2) ps -ef | grep xym c3) copy-id?????????????????? c4) check client os version 2a) info xymonclient/logs???? 3a) log: logs/xymonclient.log 2b) info xymonclient/tmp????? 3i) log info? 3t) log tail 2c) info xymonclient dir????? 3u) log util 3s) scan xymonclient.log 5a) start xymonclient???????? 5c) status 5b) stop t1) curl nag1 (10.248.224.149) 1984 test???? t3) telnet t2) curl nag2 (142.71.43.51)?? 1984 test???? t4) telnet t8) rm known_hosts entry dc) compare config file?????? dx) cp? + extract client file d2) diff vmstat2.sh?????????? d3) cp vmstat2.sh to client d6) client file: xymonclient_v4_3_28_RH7.gz d7) config file: xymonclient.cfg d9) copy config file to client -----------------------------------------------------------

2 1

svcstatus.cgi
by Christoph.Mueller＠dwd.de 29 Sep '23

29 Sep '23

Dear all, I hope you can give a hint. Xymonserver (4.30-3.el9) is installed on an alma Linux Server an running fine. There is an output in /var/log/messages every 5 minutes which looks like this: "kernel: traps: svcstatus.cgi[3519701] general protection fault ip:411288 sp:7ffcbdd46878 error:0 in svcstatus.cgi[402000+23000]" Any ideas where to start my investigations? Thanks a lot. Best regards Christoph Christoph M?ller DWD Offenbach /Germany

2 1

Terabithia - Certificate expired
by john.horne＠plymouth.ac.uk 27 Sep '23

27 Sep '23

Hi, Seeing a few of these: Curl error $60$: SSL peer certificate or SSH remote key was not OK for https://repo.terabithia.org/rpms/xymon/el9/x86_64/repodata/repomd.xml [SSL certificate problem: certificate has expired] John. -- John Horne | Senior Operations Analyst | Technology and Information Services University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK ________________________________ [https://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

2 1

Update on 4.4 (Alpha1 release)
by cleaver＠terabithia.org 26 Sep '23

26 Sep '23

Hello all! Just wanted to update everyone on a few things. Going through the 4.x codebase re-familiarizing myself again with the state of everything, I recalled that I'd actually built a release artifact before development was paused, including a testing RPM that has actually been running unbeknownst on a VM for quite a while. Unfortunately, while validating that compiles from source worked I ran into what looked like showstopper bugs. These weren't showing up in the package, so I had to pore through a lot more code* than I expected to try to figure out where the package was accidentally fixing it. (*message-sending code, which changes a lot in 4.4 to support compression, TLS, HTTP, and size-prefixed messages) Fortunately, it ended up being a relatively simple issue (isn't it always?), with easy-enough validation and workaround (given below). Although more fixes could be put in (and I'm sure will be needed!), I believe there's some benefit to having a stable starting point using a build artifact from 2019. As a result, I'm pleased to announce the first alpha for Xymon 4.4! As this is an alpha release, this is absolutely NOT suitable for production use and is intended mainly for smoketesting by source users. (And I expect there will be smoke.) The release tarball is available now at: https://sourceforge.net/projects/xymon/files/Xymon/4.4-alpha/xymon-4.4-alph… Major improvements are too many to list here but are noted in the Changes and RELEASENOTES files. README documents for compression, TLS, and IPv6 are in progress. Some of the changes and improvements other than TLS support have been in the Terabithia packages (to some extent) for a while, so there will be less of a delta for those users to this alpha. For the moment, I'd like to focus on source tarball users and others running xymon from scratch to help fix underlying issues and any smoketest failures, which I expect may crop up particularly on untested non-Linux platforms. Once the lower-hanging fruit is fixed and more patches are upstreamed and cleaned up, I'll release an alpha2 release at which point packages may be released to help with testing iteration and downstream distributors. Please flag bug reports or patches as related to "alpha1" in the subject line to help keep things clear. Thank you to everyone in the community for remaining part of it -- and xymon users! -- during this interregnum. Restarting development has been a process, but I'm glad to take this important first step to our next release. Regards, -jc *** KNOWN ISSUES -- IMPORTANT *** The configure script is likely setting XYMONHOME/XYMONCLIENTHOME and XYMSRV/XYMSERVERS/XYMONSERVERS incorrectly, which will prevent xymonlaunch from correctly launching anything. Double check the xymon environment files (xymonserver/xymonclient.cfg) before starting either the server or client. The tools/xymon-client.default and tools/xymonlaunch files may be placed into /etc/{sysconfig,default}/{xymon-client,xymonlaunch} to more easily set $XYMONSERVERS Revealed by the above issue is a bug causing a crash in the client (or anywhere) when given an empty string as the destination argument (instead of 0, 127.0.0.1 or no variable set). Don't do this --> [ xymon '' "ping" ] The client and logfetch binary may be being re-compiled during 'make install' Compilation may fail if c-ares and at least one compression library is not present (lzo/lz4/zlib are all options). xxHash libraries are also highly recommended as the plan is for xxhash to replace md5 for checksumming functionality. libtirpc may be necessary on some platforms, and it's possible the build code is still expecting traditional (Sun) RPC

4 5

Case sensitivity in host+svc names
by cleaver＠terabithia.org 25 Sep '23

25 Sep '23

Curious if anyone out there has run into issues resulting from case sensitivity in hostnames (either self-report from the clientlog, or in hosts.cfg) or test names. Although I imagine this is fairly uncommon in the Linux/BSD world, I've seen AIX and Solaris systems with mixed-case hostnames, and I imagine users of BBWin can run into this a lot. Have you had issues with graphs not displaying, or any mismatch from filesystem assets (like static-generated status HTML pages or notes) that become unavailable after a name change? Or analysis.cfg or custom xymondboard queries failing? Looking at the PCRE->PCRE2 transition, there are a few places where case options are called out differently. If xymon were to canonically store/convert to lowercase internally that would help keep things cleaner, and also allow a lot of strcasecmp's in xtree lookups to become strcmp's. Regards, -jc

2 1

Monitoring SQL certs
by driley＠malone.edu 18 Sep '23

18 Sep '23

Hello, We are using Xymon to monitor our web server, and of course the https test works well watching those certs. We are moving toward using certs in MS SQL for encryption. Does anybody have a way to monitor those within xymon? Thanks. Dennis Riley IT Systems Administrator Malone University 2600 Cleveland Ave. NW Canton, Ohio 44709

2 3

/snap/core/* Filesystems NOT ok
by metron6＠gmail.com 11 Sep '23

11 Sep '23

hello all, i have the known problem with /snap/* Mon Sep 4 14:33:50 EEST 2023 - Filesystems NOT ok [image: red] /snap/core/15511 (100% used) has reached the PANIC level (95%)[image: red] /snap/core20/1974 (100% used) has reached the PANIC level (95%)[image: red] /snap/certbot/3024 (100% used) has reached the PANIC level (95%)[image: red] /snap/certbot/2913 (100% used) has reached the PANIC level (95%)[image: red] /snap/core20/2015 (100% used) has reached the PANIC level (95%)[image: red] /snap/core/15925 (100% used) has reached the PANIC level (95%) Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 98497780 25367880 68080352 28% / tmpfs 5120 4 5116 1% /run/lock /dev/loop1 121088 121088 0 100% /snap/core/15511 /dev/loop0 65024 65024 0 100% /snap/core20/1974 /dev/loop2 45440 45440 0 100% /snap/certbot/3024 /dev/loop3 45696 45696 0 100% /snap/certbot/2913 /dev/loop6 65024 65024 0 100% /snap/core20/2015 /dev/loop4 108416 108416 0 100% /snap/core/15925 I added this to my analysis.cfg no good... HOST=* DISK %^/snap IGNORE INODE %^/snap/.+ IGNORE this is my default analysis.cfg CLASS=linux # Ignore some usually uninteresting tmpfs mounts. DISK /dev IGNORE DISK /snap IGNORE DISK /lib/init/rw IGNORE DISK /sys/fs/cgroup IGNORE DISK %^/run/user/\d+$ IGNORE INODE /dev IGNORE INODE /snap IGNORE INODE /lib/init/rw IGNORE INODE /sys/fs/cgroup IGNORE INODE %^/run/user/\d+$ IGNORE DEFAULT # Ignore some usually uninteresting tmpfs mounts. DISK /dev IGNORE DISK /dev/shm IGNORE DISK /lib/init/rw IGNORE DISK /run IGNORE # These are the built-in defaults. You should only modify these # lines, not add new ones (no PROC, DISK, LOG ... lines). UP 1h LOAD 5.0 10.0 DISK * 90 95 INODE * 70 90 MEMPHYS 100 101 MEMSWAP 50 80 MEMACT 90 97 -- regards, George Michalopoulos born & raise @ Metron 6 (six) Metron6 at gmail.com

5 4

Could not determine size of directory
by kspringer＠innovateteam.com 09 Sep '23

09 Sep '23

Anyone have a clue why the following folder size can't be determined when I add it to analysis.cfg and client-local.cfg ? "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\LOGS" Could not determine size of directory "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\LOGS" These other folders are working fine. C:\inetpub\logs C:\Windows\Temp C:\Windows\Logs\CBS C:\ProgramData\Microsoft\Windows\WER The only difference I see is spaces in the path.? I've put "" around it, but that doesn't seem to fix it. Does someone have a working example that includes spaces in the path? -- Kris Springer

3 3

INFO: NTPQ update
by bruno.manzoni＠ubi-network.ch 08 Sep '23

08 Sep '23

Hello, Thanks to S. Maurim, NTPQ test now support NTPd, Chrony and Timedatectl https://github.com/bonomani/ntpq Bruno

2 1