October 2023 - Xymon - Xymon mailman web

Possible bug in xymon svcstatus causing segfault (buffer overrun?)
by jeremy＠laidman.org 03 Jun '24

03 Jun '24

Hi all I'm not a C programmer by any stretch, so I'm seeking confirmation that my working is sound. At our site we run 4.3.30-1 from Terabithia RPM. For most hosts, the "trends" page displays just fine, but for some hosts, the webserver gives a CGI error, and when I manually run the CGI directly it crashes with a seg fault. This seems to be more likely for hosts with a lot of entries on the trends page, but some with more trends graphs are OK than others with fewer. I believe I've tracked the issue down to the last few lines of code in svcstatus-trends.c, where a URL is constructed for one graph entry, before being added to the list of trends in a buffer. My reading of the code suggests that it first pre-allocates (using malloc()) a 16k buffer for the trends data, then sequentially appends trend strings (which include a URL) into the buffer, until the length of a string would exceed the size of the buffer. In that case, an extra 4k of memory is added to the buffer (using realloc()), and then the string is added to the buffer. The problem with the code, as I see it, is that the string might be more than 4k long, in which case, adding string would cause a buffer overrun. I don't really know what's in each string, so I don't know if it's ever likely to be 4k, but we're seeing the process dumping core immediately after this bit of code, so that's the suspicion. Here's the bit of code, which is attempting to add the string "onelink" to the buffer, after seeing if it's going to exceed the buffer, and extending the buffer if required: if ((buflen + strlen(onelink)) >= allrrdlinks_buflen) { SBUF_REALLOC(allrrdlinks, allrrdlinks_buflen+4096); allrrdlinksend = allrrdlinks + buflen; } allrrdlinksend += snprintf(allrrdlinksend, (allrrdlinks_buflen - (allrrdlinksend - allrrdlinks)), "%s", onelink); Elsewhere in other code, the SBUF_REALLOC macro is used to add extra bufferspace by calculating a new buffer size of [existing buffer size plus 4k plus the length of the string being added]. So if the string to be added had 20 bytes, SBUF_REALLOC would be used to add 4k+20 bytes to the buffer size. Notably, the size of the string is included in the calculation. For example this bit of code elsewhere in the same file does a similar buffer extension: if ((strlen(rrdlink) + strlen(partlink) + 1) >= rrdlink_buflen) { SBUF_REALLOC(rrdlink, rrdlink_buflen + strlen(partlink) + 4096); } strncat(rrdlink, partlink, (rrdlink_buflen - strlen(rrdlink))); Note the extra "+ strlen(<stringvar>)" bit in the extra memory being allocated that isn't there in the first block of code above. So my thinking is that the SBUF_REALLOC() needs to take the string's size into account, like so: SBUF_REALLOC(allrrdlinks, allrrdlinks_buflen+strlen(onelink)+4096); Is anyone with C skills able to confirm that this does indeed look like a bug, and also that my proposed change is a suitable fix? After applying this change and recompiling, we haven't yet seen a seg fault from this bit of code, so I'm hoping it's all fixed, and without negative side effects. Cheers Jeremy

3 6

Update on 4.4 (Alpha1 release)
by cleaver＠terabithia.org 15 May '24

15 May '24

Hello all! Just wanted to update everyone on a few things. Going through the 4.x codebase re-familiarizing myself again with the state of everything, I recalled that I'd actually built a release artifact before development was paused, including a testing RPM that has actually been running unbeknownst on a VM for quite a while. Unfortunately, while validating that compiles from source worked I ran into what looked like showstopper bugs. These weren't showing up in the package, so I had to pore through a lot more code* than I expected to try to figure out where the package was accidentally fixing it. (*message-sending code, which changes a lot in 4.4 to support compression, TLS, HTTP, and size-prefixed messages) Fortunately, it ended up being a relatively simple issue (isn't it always?), with easy-enough validation and workaround (given below). Although more fixes could be put in (and I'm sure will be needed!), I believe there's some benefit to having a stable starting point using a build artifact from 2019. As a result, I'm pleased to announce the first alpha for Xymon 4.4! As this is an alpha release, this is absolutely NOT suitable for production use and is intended mainly for smoketesting by source users. (And I expect there will be smoke.) The release tarball is available now at: https://sourceforge.net/projects/xymon/files/Xymon/4.4-alpha/xymon-4.4-alph… Major improvements are too many to list here but are noted in the Changes and RELEASENOTES files. README documents for compression, TLS, and IPv6 are in progress. Some of the changes and improvements other than TLS support have been in the Terabithia packages (to some extent) for a while, so there will be less of a delta for those users to this alpha. For the moment, I'd like to focus on source tarball users and others running xymon from scratch to help fix underlying issues and any smoketest failures, which I expect may crop up particularly on untested non-Linux platforms. Once the lower-hanging fruit is fixed and more patches are upstreamed and cleaned up, I'll release an alpha2 release at which point packages may be released to help with testing iteration and downstream distributors. Please flag bug reports or patches as related to "alpha1" in the subject line to help keep things clear. Thank you to everyone in the community for remaining part of it -- and xymon users! -- during this interregnum. Restarting development has been a process, but I'm glad to take this important first step to our next release. Regards, -jc *** KNOWN ISSUES -- IMPORTANT *** The configure script is likely setting XYMONHOME/XYMONCLIENTHOME and XYMSRV/XYMSERVERS/XYMONSERVERS incorrectly, which will prevent xymonlaunch from correctly launching anything. Double check the xymon environment files (xymonserver/xymonclient.cfg) before starting either the server or client. The tools/xymon-client.default and tools/xymonlaunch files may be placed into /etc/{sysconfig,default}/{xymon-client,xymonlaunch} to more easily set $XYMONSERVERS Revealed by the above issue is a bug causing a crash in the client (or anywhere) when given an empty string as the destination argument (instead of 0, 127.0.0.1 or no variable set). Don't do this --> [ xymon '' "ping" ] The client and logfetch binary may be being re-compiled during 'make install' Compilation may fail if c-ares and at least one compression library is not present (lzo/lz4/zlib are all options). xxHash libraries are also highly recommended as the plan is for xxhash to replace md5 for checksumming functionality. libtirpc may be necessary on some platforms, and it's possible the build code is still expecting traditional (Sun) RPC

9 13

Fwd: Re. Is this thing on?
by bruno.manzoni＠ubi-network.ch 29 Jan '24

29 Jan '24

Hi. Seem that Ye-Fee is unable to email to the mailing list? *Can you check that if there is a problem ( I think Henrik) ?* By to way, I remember that we have to migrate to another service. *What about: https://mailman3.com/ which is free for opensource ?* (comment to do or not to do are welcome!) *Can someone take the job ? * Thing to do: - Ask malman3 if possible to have a freemailing list - The we can see how to do it! (That will need for sure to contact Henrik), but I can help also. Bruno -------- Forwarded Message -------- Subject: Re. [Xymon] Is this thing on? Date: Thu, 28 Sep 2023 16:22:00 +0000 From: Liang, Ye-Fee [PHSA] <yefee.liang at phsa.ca> To: bruno.manzoni at ubi-network.ch <bruno.manzoni at ubi-network.ch> Hi Bruno, ?? I was able to email xymon at xymon.com long time ago, and now find myself unable to email anymore.? I have used xymon at different organizations over the decades, and would love to help the tool/platform to continue for decades more.? Simply there is no other product that is more versatile, easy to manage, and light server+client requirement.? Many more people are eager to contribute, but can only just read the xymon list email, from the side line. ?? Thanks for your summary of Xymon non-technical and technical needs. Just to keep the enthusiasm going, I would suggest a section on "making the better use of Xymon as now" I am only a sys admin, not a coder, so my focus is on management and usability areas. 1. enhanced cpu utilization graph (adds wait IO; better colors); see 1a. 2. menu script (see 2a) to clean up of accumulating rrd, hostdata and histlogs, eg. due to short-term ???ad-hoc metrics like nfs file systems, container nic devices. 3. menu script (see 3a) to manage clients easier Keep up the great work! Ye-Fee 1a 2a ------------------------------------------------------ c) change Client:?????? yyy ct) change days to trim: 30 1) summary???? 1a) list pings?? 1b) list cpu clients 1bbb) list to scp file?????????? 1bb) list in one column 1c) list non-pings tests?????? ??1d) list non-green cpu tests 2)? duplicate report 3)? Info on client?????????????? 3a) formatted 4)? Search client in hosts.cfg+ 6)? find clients with too many rrd's???? 6z) list -5 RRD's 6a) list top 10?? 6af) find top 10?? 6at) trim hostdata's ?6b) list top 10?? 6bf) find top 10?? 6bt) trim histlogs's 6c) list top 10?? 6cf) find top 10?? 6ct) trim Select from top 10 to trim??? 6ta) RRD? 6tb) hostdata? 6tc) histlogs 7a) diff host_cfg files????????? 7u) update host_cfg files ------------------------------------------------------ 3a ----------------------------------------------------------- c) Change Client: vag7 c1) check ssh???????????????? c2) ps -ef | grep xym c3) copy-id?????????????????? c4) check client os version 2a) info xymonclient/logs???? 3a) log: logs/xymonclient.log 2b) info xymonclient/tmp????? 3i) log info? 3t) log tail 2c) info xymonclient dir????? 3u) log util 3s) scan xymonclient.log 5a) start xymonclient???????? 5c) status 5b) stop t1) curl nag1 (10.248.224.149) 1984 test???? t3) telnet t2) curl nag2 (142.71.43.51)?? 1984 test???? t4) telnet t8) rm known_hosts entry dc) compare config file?????? dx) cp? + extract client file d2) diff vmstat2.sh?????????? d3) cp vmstat2.sh to client d6) client file: xymonclient_v4_3_28_RH7.gz d7) config file: xymonclient.cfg d9) copy config file to client -----------------------------------------------------------

4 12

Xymon Server on RHEL9
by colin.coe＠gmail.com 29 Nov '23

29 Nov '23

Hi all I'm getting ready to replace our venerable RHEL6 Xymon server with one running RHEL 9. We use the terabithia.org RPMs. NTP tests are all returning red. I've updated xymonserver.cfg (and restarted the xymon service) by changing NTPDATE="/usr/bin/chronyc" and adding NTPDATEOPTS="tracking". When I run "/usr/bin/chronyc tracking <IP HERE>" from the shell, it works fine however the Xymon page is red. Service ntp on <CENSORED> is not OK : Service unavailable Command: /usr/bin/chronyc tracking <CENSORED> 2>&1 Any ideas?

5 5

Looking for clarification on Xymon client / server hierarchy.
by jeremy＠laidman.org 17 Oct '23

17 Oct '23

On Tue, 17 Oct 2023 at 14:19, Grant Taylor via Xymon <xymon at xymon.com> wrote: > Now I get the impression that it's more that xymonnet needs a properly > configured environment and that the easiest way to do that is to use > Xymon / xymonlaunch to manage that for you. But it sounds like it's > quite possible to crate said environment and call xymonnet directly. > Correct. The man page for xymonnet should cover the various env vars that it supports. I think the only ones that might be required are some combination of (XYMSRV, XYMONSERVERS, XYMSERVERS) as per doco to specify the display servers, and access to the hosts.cfg file, and XYMONTMP. In fact if you run xymonnet with the --loadhostsfromxymond switch, it doesn't even need a local copy of hosts.xfg. Try something like this from a device in your enclave, and see if the output looks OK, then drop the --no-update to run it for real: XYMSERVERS=xymon-server-or-IP XYMONTMP=/tmp /usr/lib/xymon/server/bin/xymonnet --no-update --loadhostsfromxymond Optionally, try the --debug and/or --dump switches. You can specify the FQDN of a target server if you just want to probe one device (eg for testing). You can set XYMONNETWORK to something in the NET: tags in hosts.cfg, and only those devices will be polled. So in hosts.cfg: 10.1.2.3 mywebserver.example.com # NET:enclave XYMSERVERS=xymon-server-or-IP XYMONTMP=/tmp XYMONNETWORK=enclave /usr/lib/xymon/server/bin/xymonnet --loadhostsfromxymond But if Xymon is running as anything other than root, it's chances of > reading the /etc/shadow file are -- dare I say -- exceedingly small. > Yep, agreed. And if that's the only exposure, then we're golden. Might be a private key that the Xymon user accesses to authenticate for one of its probes to an HTTPS server with client auth. Might be /var/tmp/ps-dump that someone is running from their cron, to diagnose some fault, not realising that someone else's process has "--with-password=M0nkey", because that user didn't realise that args are visible to all. Might be a core dump that someone copied to /tmp/, without knowing that they can contain passwords and private keys. Of course most possible exploits are really unlikely, and require a bit of stupidity and a bit of mis-fortune. I'm not trying to suggest likely exposures, just possible exposures under some circumstances. We can't mitigate all of them, but we can address the likely ones even if it takes effort, and we also should address the unlikely ones where it takes very little effort. I suppose my point here is, if a client doesn't need to access an arbitrary file on a server, then it shouldn't be able to. If a client doesn't need to access any file on a server, then it shouldn't be able to. And a feature that permits this, and is on by default, and is seldom used or necessary, and is possibly not known about by the sysadmin, probably should be addressed, or at least assessed. > I don't know if there's a way to prevent "config" messages from being > > honoured by the xymond process. It's possible to have xymond ignore > > messages (including config) from IP addresses that aren't in hosts.cfg, > > so this will minimise any exposure. > > I should investigate that. > > I assume that a firewall also helps here. > Only for some threat models. Every xymon client has to be able to connect to the xymon server on TCP/1984[*]. A firewall can allow or deny that. But a firewall can't allow status and data messages while blocking config messages. I'm not aware of a DPI firewall that can parse Xymon packets. > There's a similar feature where you can send a "download" message and > > receive a copy of the named file. The download feature can be disabled > > by adding "--no-download" to the xymond command line arguments. > > Doesn't the download feature also have a more lax mode where it will > allow you to download things that aren't config files? Or am I > conflating config and download? > I actually don't know. I don't use either of these features. I also wonder where the command comes into play to have clients download > updated (tar files) from the central server and install them. > Yes. If my memory serves, the original BigBrother, on which Xymon was based, was often deployed so that it could upgrade itself. This process required that the BB server had tarballs of the various clients. The clients would periodically check their versions with the latest available, and if there was an updated, the client would fetch it and untar over itself. This mechanism was in place when compiling from source code was the only way to deploy open source. So it seemed like a good idea at the time. And it has merit. But these days, we're all used to using packages, and we make good use of the additional features a package manage can bring. So the way I see the download command is that it's a relic from a previous generation, that most people won't ever use. Might come in handy, but if there's another way to do it, probably use the nother way. J

2 1

Can custom client-side tests leverage the server-side value in analysis.cfg?
by jeremy＠laidman.org 17 Oct '23

17 Oct '23

On Tue, 17 Oct 2023 at 13:55, Grant Taylor via Xymon <xymon at xymon.com> wrote: > > Oh my! Isn't AIX5.3 like a decade beyond end-of-support? > > At least. > > Does "legacy" and "out to pasture" or even "business critical" mean > anything to you? My $DAY_JOB is care and feeding of old systems that > nobody else wants to touch. I enjoy it and it pays the bills. > Oh dear. I'm glad you enjoy that sort of thing. I suppose if the expectations aren't high, then you can always exceed expectations. Do you happen to program in COBOL? ;-) J

2 1

Looking for clarification on Xymon client / server hierarchy.
by jeremy＠laidman.org 17 Oct '23

17 Oct '23

On Tue, 17 Oct 2023 at 12:34, Grant Taylor via Xymon <xymon at xymon.com> wrote: > The scenario that I'm working with can be described as a primary Xymon > (display) server in one network with a small lab network behind a NATing > / SPI firewall. Clients on the inside side / opposite of the Xymon > server are free to send outgoing packets. It's just that xymonnet > running on the Xymon server can't send probes into the clients. > > > You can run xymonnet stand-alone, and set environment variables to tell > > it where to send its messages. If you already have a xymon client > > installed on the client host, you can execute xymonnet from > > clientlaunch.cfg and it should then know where to send packets due to > > the environment that is setup. > > Oh! This is promising. > > I misinterpreted comments in the tasks.cfg file to mean that xymonnet > depended on xymond. Now it sounds like xymonnet can be satisified by > the xymon client. > Indeed. Perhaps more precisely, it can be satisfied by xymonlaunch, which also runs the main Xymon client script (xymonclient.sh). It's also possible to run xymonnet from cron, or set it up as a systemd service, or even trigger its execution by running from a remote ssh session (something like "ssh client-host /usr/local/bin/xymonnet --env=/usr/local/etc/xymonnet.conf"). In most cases, running from within a xymoncmd environment (which is implied when run by xymonlaunch) takes away the need to explicitly set all of the required options. So you could have a cron job like so: */5 * * * * xymon /usr/lib/xymon/client/bin/xymoncmd /usr/local/bin/xymonnet >>/var/log/xymon/xymonnet.log 2>&1 But it's worth taking a look at the xymonnet entries in the tasks.cfg file on the Xymon server. There is a xymonnet-again (I think) task that runs more frequently but only performs probes for failed tests. In this way when a fault is detected, the polling rate increases so that you can more quickly see when a service is restored, and SLA times are more accurate. Running xymonproxy + xymonnet + xymonclient on a system inside of the > firewall might do what I'm wanting to do. > I don't think xymonproxy is required for this. If your clients can talk directly to your xymon server on TCP/1984, then that's all xymonclient.sh and xymonnet need to be able to do. I think xymonproxy is only required if your clients can't connect directly to your server. And it will add delays, and become an extra point of failure. If you haven't already, look into msgcache and xymonfetch. These can be used to bridge networks in a slightly different way to xymonproxy. I'm quite okay with '${XYMON} ${XYMSRV} "config hosts.cfg" > hosts.cfg' > on the internal system running xymonnet. > Makes sense, and should work fine. I actually like "xymongrep" for its ability to do searches. By default it looks for hosts.cfg, but if given the --loadhostsfromxymond switch, it'll connect to the Xymon server and send a "config hosts.cfg" command and filter for the tags you're interested in. So you could do something like: xymoncmd xymongrep "NET:behindfirewall" > hosts.cfg (multiple search strings can be used, and lines that match any of them will be returned) This brings up a possible issue for some installations and security postures. In a multi-network multi-security-level environment, some consider the "config" command to be a feature that's not worth the potential risk. For example, if there's a coding error in Xymon, there might be a way to request the config file /etc/shadow. Or if someone doesn't realise that hosts.cfg is potentially exposed outside the Xymon server in this way, they might put a password or private key in the hosts.cfg file (eg in a tag for a custom extension). An example of this is that the "devmon" add-on uses the DEVMON: tag to specify SNMP parameters, including community string, and the operator might not realise that the community strings are accessible by any Xymon client. If this is of concern to you, then you should consider ways to mitigate this. Choosing to not use this feature in your scripts doesn't mean an attacker can't do so. I don't know if there's a way to prevent "config" messages from being honoured by the xymond process. It's possible to have xymond ignore messages (including config) from IP addresses that aren't in hosts.cfg, so this will minimise any exposure. There's a similar feature where you can send a "download" message and receive a copy of the named file. The download feature can be disabled by adding "--no-download" to the xymond command line arguments. Cheers Jeremy

2 1

Can custom client-side tests leverage the server-side value in analysis.cfg?
by jeremy＠laidman.org 17 Oct '23

17 Oct '23

On Tue, 17 Oct 2023 at 12:22, Grant Taylor via Xymon <xymon at xymon.com> wrote: > Sounds like use what's best for the situation, even if there might be a > slight affinity one way or the other. > Yes. One of the things I love about Xymon is that it's extremely flexible so that you get to choose from a range of different ways to solve a problem, based on your skillset, environment, and preferences. It's modular and extensible. > > With central mode, client data messages are sent to the server, into > > the xymond daemon. Then code on the server asks the xymond daemon for > > the client message of each host, parses them for possible problems, > > and constructs a status message with the colour in the header and a > > text report in the body. > > My understanding is that's one way to do it. > Yes, one way (see my previous comment ;-). But it's also the way Xymon does it internally. Well, kind-of. Instead of "asks the xymond daemon" a xymon module connects to a channel and receives from the xymon daemon. But the data flow is pretty much the same, and I suppose that's what I was trying to convey, while avoiding the technical complexities. My understanding that another way to do it is to hook in as a channel > and receive a copy of all messages (of the proper type) and act > appropriately. > Yes indeed. You're really getting into the nitty-gritty of things. Most Xymon admins never have to know that channels are a thing, because they're mostly used for Xymon to talk to itself (eg to parse a client data message and create status messages). So consider yourself levelled-up! When writing server-side extensions that use client data, you can write your own parser that periodically fetches the latest client message, or you can tap into the channel to get it. Xymon taps into the channel. I'm trying to get my hands on a compilation of any version for AIX 5.3 > (?). For now I'm running the hobbitclient-aix.sh via telnet w/ echo > scheduled via cron. It's working and providing most of what I need and > much of what I want. > Oh my! Isn't AIX5.3 like a decade beyond end-of-support? I've sort of taken what previous colleagues poked once in a while and > ran with it to make things better for $DAY_JOB, myself, and a friend > that I help. I've also intrigued another friend enough to install and > play with it. > > I think I say that I'm liking Xymon. ;-) > Good to see. > > The patch is applied to the packages available from the Terabithia > > repository, but I don't believe it's included when Xymon is built from > > original source code. > > Please elaborate and / or point me where I can go do some more reading. > About the "sections" patch? I don't know much about the background to its conception and existence, but there's probably a CHANGELOG and that might provide some info. If you haven't yet, perhaps install an RPM from the Terabithia website, or unpack an SRPM to review any extra README files or the like. I've seen references to 4.4 being in alpha status. > Yes indeed. I've not started testing this yet myself. Yep. Naming things and knowing where / how to start counting are three > of the hardest things in computers. > LoL. Cheers Jeremy

2 1

Looking for clarification on Xymon client / server hierarchy.
by jeremy＠laidman.org 17 Oct '23

17 Oct '23

Hi Grant The xymonnet process needs to be able to send probe packets (eg ping, web requests, and whatever you're trying to monitor) to the clients. If the firewall is blocking the probe traffic, then it's not going to work. The xymon proxy only proxies xymon messages, such as the ones sent by the xymonnet process to the xymond process when reporting the status of the probes (success or failure, and round-trip times). It seems to me that you need a xymonnet process running on the client side of the firewall. For example, if you can run xymonnet on one of the clients, then the firewall only needs to allow xymon traffic from the client to the Xymon server, so that xymonnet can report the status of its probes. You can run xymonnet stand-alone, and set environment variables to tell it where to send its messages. If you already have a xymon client installed on the client host, you can execute xymonnet from clientlaunch.cfg and it should then know where to send packets due to the environment that is setup. The only thing I'm not certain of, is how xymonnet knows which hosts to probe and what probes to send to them. When xymonnet is running on the Xymon server, it has access to the hosts.cfg file that's there. When running elsewhere, I'm not sure. I know that there's a way to fetch the hosts.cfg contents using xymon messages, so my guess is that xymonnet can do that too, but might need to be told to do so. And if so, you would only want that xymonnet instance to probe devices inside the client network, so you might need to make use of the "NET:" tags in hosts.cfg. J On Tue, 17 Oct 2023 at 02:51, Grant Taylor via Xymon <xymon at xymon.com> wrote: > > > > ---------- Forwarded message ---------- > From: Grant Taylor <gtaylor at tnetconsulting.net> > To: xymon at xymon.com > Cc: > Bcc: > Date: Mon, 16 Oct 2023 10:49:42 -0500 > Subject: Looking for clarification on Xymon client / server hierarchy. > Hi, > > Would someone help me understand the Xymon client / server / proxy > hierarchy a little bit better? > > My scenario is I have two locations separated by a firewall wherein > clients inside can send things out to the larger network, but the > xymonnet can't reach in to probe clients in the private LAN. > > I had thought that an Xymon proxy might be the answer for this. -- I > did get internal clients to relay updates out through the xymonproxy to > the Xymon (display) server. However xymonnet seems to not utilize the > xymonproxy to initiate tests therefrom. > > What is the recommended way to have Xymon monitor internal clients that > can't be directly reached from the Xymon (display) server? > > Aside: It seems as if the xymonproxy might be for the other way around, > to have clients in the wild get messages into a protected Xymon server > which can reach out and touch the clients. > > Thank you and have a good day. > > > > -- > Grant. . . . > unix || die > > > > ---------- Forwarded message ---------- > From: Grant Taylor via Xymon <xymon at xymon.com> > To: xymon at xymon.com > Cc: > Bcc: > Date: Mon, 16 Oct 2023 10:49:42 -0500 > Subject: [Xymon] Looking for clarification on Xymon client / server > hierarchy. > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon >

2 1

Can custom client-side tests leverage the server-side value in analysis.cfg?
by jeremy＠laidman.org 17 Oct '23

17 Oct '23

Responding inline: On Tue, 17 Oct 2023 at 02:47, Grant Taylor via Xymon <xymon at xymon.com> wrote: > > > The "DS" means "data source". It's a term used by the RRD (round-robin > > database) library (and tools) that Xymon uses for storing historical > data. > > Thank you for clarifying. I sort of deduced that as I got further into > the custom graphs, which I've now got working. But it's one of those > things that when you're newer to something "DS" is unfortunately vague > and lost in a sea of new information. > Agreed. Xymon/Hobbit, and its spiritual predecessor BigBrother, come from a tradition of using a common collection of disparate tools for monitoring networked devices and hosts. One of these foundational tools is RRD, which is used by other network monitors (Cacti, MRTG, etc). The "culture" of Xymon can sometimes assume knowledge of how the other tools work. And while there's references to RRD in the Xymon doco, it couldn't hurt to add some extra definitions or explanations within the Xymon doco, to help those who don't have a history in this space that extends back to the last millenium (showing my age, aren't I). > As a general principle, Xymon clients operate in local mode or central > > mode. In (the older) local mode, the clients do all of their own > > analysis and create their own data and status messages. In (the more > > modern) central mode, the clients do a lot less of that, and instead > > simply gather data for the server to parse and alert on. > > Would it be fair to say that the older local mode uses status messages > with color and newer central mode use data messages without color? -- > As a rough generalization, not a hard and fast rule. > Yes and no. The central/local mode choice relates to the bulk of the core OS resource metric collection and processing. Choosing central mode doesn't prevent the use of client-side processing and status messages (with colour) being sent from the client. I know you didn't ask this, but I'm just making sure I'm not giving the wrong idea. With local mode, the client looks for faults, and sends a status message with the colour in the header and a text report in the body. With central mode, client data messages are sent to the server, into the xymond daemon. Then code on the server asks the xymond daemon for the client message of each host, parses them for possible problems, and constructs a status message with the colour in the header and a text report in the body. So the last part of the process is the same. The Xymon server receives status messages from itself for clients in central mode, or from the clients when they're in local mode. To bring it back to your phrasing, local mode uses status messages with colo(u)r and central mode also uses status messages with colo(ur) but those messages come from the Xymon server itself. In central mode, the colours are determined by the server (eg in analysis.cfg), whereas in local mode, the colours are determined by the client. > You could create a script in the "sections" subdirectory, and the > > Xymon client will automatically run it and include the output in > > the [mailq] section of the client message. > > This is very intriguing. I spent a moment looking for "section" (case > insensitive) in the client directory tree (`fgrep -ir "section"` type > thing) and found reference to the client/local directory. Upon testing, > I see that I end up with the output of my executable (currently a > script) with a heading named `[local:test_script.sh]`. > Yes, the "client/sections" directory is a relatively recent feature that was added as a "patch" to the original client-side script. My mistake for assuming that everyone had this. The patch is applied to the packages available from the Terabithia repository, but I don't believe it's included when Xymon is built from original source code. The "client/local" directory is "standard" Xymon (as long as you create the directory), but it has the side-effect of making section names with the "local" prefix. There are probably good reasons for this prefixing (I'm guessing to prevent collisions with standard section names), so if it works for you, that's great. My guess is that the next release of Xymon will include client/sections in the source, in addition to supporting client/local. A snippet from the README in client/sections/ from the Terabithia package: "This directory is intended for use with automatically deployed files. Local scripts for this host should be placed in the client/local/ directory instead." It seems like the gist of the idea is to receive the client status > message, extract the section, and generate updates therefor. > Exactly. The client is as simple as it can be, so that it doesn't need constant tweaking. The server does all the smarts, including parsing, graphing, thresholding, so that the client host can spend more cycles doing its main function. > And optionally turn the mail queue size into a graphable data point > > (either create a trends message with the number, or use a > > colon-separated value format for the Xymon parser to detect). The Xymon > > doco has a "how to setup custom graphs" page with a few different ways > > to do this type of thing. > > Yep. I used that very page to get NCV based custom graphs working > Friday evening. > > I still need to explore the DS analysis / color declaration therefor. > You're well on your way, so well done! Feel free to ask more questions on this list. The more questions the better our shared knowledge! (Although it's easier to respond to if each question is in a separate thread.) J

2 1