- Xymon - Xymon mailman web

Analysis proc match problem and a feature request
by jlaidman＠rebel-it.com.au 04 Sep '17

04 Sep '17

Hi all I'm having some difficulty with a process counting/alerting problem for a Linux client (Xymon client v4.3.27). I know the cause, but can't find a good solution, short of hacking the code, or requesting a feature enhancement. The "ps" command produces output that shows child processes in a kind-of tree structure ("forest" mode). So I sometimes see this (with some columns removed for brevity): 3636 1 root 21720 xinetd -stayalive 29956 3636 nobody 21720 \_ xinetd -stayalive For xinetd, usually there's only the "master" process. Occasionally there is one extra like the above, and sometimes more than this. These are simply child xinetd processes that haven't yet performed an exec of the real worker process, so nothing unusual. >From my perspective, the correct number of "master" xinetd processes is 1. We don't want to have a second master instance of xinetd running. However, if I add a PROC entry to analysis.cfg for xinetd, alerting when not 1, any child processes are counted, and this causes an alert that I don't want. This should be quite easy to solve. I just need to use a regular expression that only matches when "xinetd" is at the start of the command string. So I setup analysis.cfg as follows: HOST=* PROC "%^xinetd" 1 1 red Awesomely, this worked! But only when the Xymon server was v4.3.10. It seems that in v4.3.23, when the "forest" option for "ps" was introduced in the client scripts, so was some code to strip out the leading "\_" characters before matching. This means that by the time the counting routine is reached, all of these lines from "ps" look identical. My work-around is to simply increase the "maximumcount" from 1 to 10 or some arbitrary number (perhaps based on some historical data). But it's not ideal. I'd like to be able to tell when a process is a child or a parent (PPID=1) process and select for those when counting and thresholding. I can't think of any other way to solve this without reverting xymonclient to pre-v4.3.23. I thought about this for a while, pondering the idea of adding a PPID=1 option, or some other way of specifying other fields from the "ps" output. Then I realised that it might generally be very helpful to be able to optionally match against the full line from the "ps" output, and not just the "CMD" field or some other pre-defined field. For example, if I could append a FIELD=all parameter to the PROC line in analysis.cfg, which allows me to match anywhere along the "ps" output line, I could not only match/exclude the "forest" characters, but also the username or the PPID number or the time started or anything else that's interesting to me. The fact that the counting routine add_count() is the same for proc, disk and inode, means that I could do similar things there - such as using the "IGNORE" keyword for any "df" output lines that started with "/dev/myspecialfs". The same type of feature for PORT could allow me to match UDP ports only, whereas I don't believe this is currently possible. What are peoples' thoughts on this idea? Is there a more obvious solution I haven't thought of yet?

1 0

Human readable disk space
by mailinglist＠tikklik.nl 31 Aug '17

31 Aug '17

Hello List, Im using xymon for many years I just wondering is there a easy way to make the output more readable? Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/sysvg-root 10321208 4397592 5399328 45% / /dev/sda1 1032088 152244 827416 16% /boot /dev/mapper/sysvg-vz 1898719656 249544968 1552725360 14% /vz As you see there is no scaling in the output to GB etc Thanxs Steffan

5 8

Xymon Issue : some monitors not displayed on xymon page
by Mohit.Jain＠colt.net 31 Aug '17

31 Aug '17

Hi Team, We have observed that some devices are not being displayed on xymon page. There is no change in configuration and it was working fine few days ago. While we search the same through find host option, we found the device. After clicking on that search link the devices/monitor wasn't available on next page. Please find core dump output for xymonpage. [xymon at cmspg-mn36 server]$ gdb bin/xymonpage /opt/xymon/core.26958 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /opt/rxymon/server/bin/xymonpage...done. warning: core file may not match specified executable file. [New Thread 26958] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Core was generated by `xymond --pidfile=/var/log/xymon/xymond.pid --restart=/opt/xymon/server/tmp/xymo'. Program terminated with signal 6, Aborted. #0 0x0000003d1ce32625 in ?? () Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.149.el6_6.7.x86_64 (gdb) bt #0 0x0000003d1ce32625 in ?? () #1 0x0000003d1ce33e05 in ?? () #2 0x00007fff2fb69688 in ?? () #3 0x00007fff2fb69670 in ?? () #4 0x00007fff2fb6ee7c in ?? () #5 0x0000000000000006 in ?? () #6 0x0000003d1cf5758e in ?? () #7 0x0000000000000003 in ?? () #8 0x00007fff2fb6967a in ?? () #9 0x0000000000000006 in ?? () #10 0x0000003d1cf57592 in ?? () #11 0x0000000000000002 in ?? () #12 0x00007fff2fb6966e in ?? () #13 0x0000000000000002 in ?? () #14 0x0000003d1cf55b43 in ?? () #15 0x0000000000000001 in ?? () #16 0x0000003d1cf5758e in ?? () #17 0x0000000000000003 in ?? () #18 0x00007fff2fb69676 in ?? () #19 0x000000000000000a in ?? () #20 0x0000003d1cf57592 in ?? () #21 0x0000000000000002 in ?? () #22 0x0000000000000020 in ?? () #23 0x0000000000000000 in ?? () Is there any monitors display limit on a single page for xymon? Please let me know if any details required. Thanks & Regards, Mohit Jain [Colt Disclaimer] This email is from an entity of the Colt group of companies. Colt Group S.A., K2 Building, Forte 1, 2a rue Albert Borschette, L-1246 Luxembourg, R.C.S. B115679. Corporate and contact information for our entities can be found at http://colt.net/uk/en/Colt-Group-of-Companies/index.htm. Internet communications are not secure and Colt does not accept responsibility for the accurate transmission of this message. Content of this email or its attachments is not legally or contractually binding unless expressly previously agreed in writing by Colt

1 1

Suggestion: monitor CPU utilization by process
by glauber.ribeiro＠experian.com 30 Aug '17

30 Aug '17

Currently we can monitor for presence of certain processes in the "ps" listing, under the "procs" test and configured in analysis.cfg. It would be nice to be able to monitor also on CPU utilization. E.g.: "process db2sysc taking more than 90% of the CPU" glauber ribeiro

3 3

How to have a test not specifically associated with any given host?
by rlhamil2＠gmail.com 30 Aug '17

30 Aug '17

Let's say I want to add a test that probes for a DHCP server or relay...and that there should be one, and only one on a given LAN (or if more than one, only authorized ones). The test could be run on any client on the LAN, or on the xymon server for the LAN it's on. The test results have nothing to do with the system the test runs on, save that it's on the LAN in question. (The test isn't that hard; nmap --script broadcast-dhcp-discover -e net0 (change interface name as needed) can collect the data, and the output be examined to look for "Response 1 of 1:" and check that the IP following "Server Identifier:" is as expected; if not, either the DHCP server is missing, or there's one or more that shouldn't be there, or, if there's a single response but the IP is wrong, worst of all, there's one DHCP server, but it's bogus or misconfigured.) Do I make up a pseudo-hostname (e.g. give each LAN a name of its own), put that in hosts.cfg with a reserved-as-bogus address (can't use 0.0.0.0 if that will cause lookup failures), maybe mark it noconn (since the IP should not be pinged) and multihomed (since the IP is bogus, and the IP that the report comes from may be different), and with the test name? And then make the test script report as if the hostname was the made-up LAN name? And of course NOT use any tests handled by xymonnet for that entry? I suppose that would work, and if I put those pseudo-hosts in a separate page or section, would only have the DHCPprobe test (or similar tests) column for them. But is that (a) a reasonable thing to do, or just plain crazy, and (b) is that a reasonable abuse (if that's not an oxymoron) of the facilities xymon provides, or is there some better way to do that sort of thing (e.g. a test that's per-LAN or at any rate doesn't really relate to a single system as such)? Keep in mind that putting in entries specifically for the DHCP servers/relays themselves may not be helpful insofar the test is not truly just for them (why xymonnet can't do it), that one might want to have a variation to allow more than one legitimate DHCP server per LAN (redundant, perhaps with only static allocation tied to MAC addresses so they don't have to coordinate allocation from a dynamic address range), etc. Not to mention that a DHCP server may be colocated on a router or other infrastructure box, which one might either not want to report at all, or report separately, and which might not support a client extension script, thus requiring the script to run elsewhere anyway. My case, with a single home LAN, is pretty trivial; but if there were more LANs, the script might end up hard-coding LAN names to use individually (each version of script different), unless it parsed ifconfig output to see what LAN it was on, and then consulted an entire table of LAN addresses (either hardcoded or retrieved from the server) to see what name to use for the LAN. So writing the script to be scalable to multiple LANs without needing each instance tweaked would be harder. Could xymon handle (pseudo-) hostnames that looked like LAN-192.168.0, or would it choke on those? Names like that would require no bother for how to make them up (no table needed, could be generated from the interface IP and net mask), and would be informative...although it wouldn't work well for net masks that were other than /8, /16, or /24. And how the heck could one generate acceptable IPv6 LAN names that were similarly useful and digestible? :-) Even with auto-generated LAN names, there would still need to be a table of legitimate DHCP server(s) per LAN, so I suppose that would best be retrieved from the server. Have I finally come up with an unusual enough idea to make someone's head hurt? :-)

2 1

Graph lines for disk and memory showing gaps
by sbarrie＠brightsolid.com 30 Aug '17

30 Aug '17

Hi We have a few Xymon servers running on Ubuntu 14.04 and 16.04 that show gaps in the graphs for disk and memory metrics. This specifically relates to Linux and Unix clients reporting to these servers. The graph lines for Windows clients look ok. If the same Unix and Linux clients are pointed to another server running Red Hat we do not see the same gaps in the graphs. Upgrading the client version does not resolve this. So it seems there is a problem with graph data being created on the Xymon server on Ubuntu. We are using the package version 4.3.25. Is there a known issue with this? Regards Stephen Barrie ________________________________ This message is confidential and may contain privileged information. You should not disclose its contents to any other person. If you are not the intended recipient, please notify the sender named above immediately. It is expressly declared that this e-mail does not constitute nor form part of a contract or unilateral obligation. Opinions, conclusions and other information in this message that do not relate to the official business of brightsolid shall be understood as neither given nor endorsed by it. ________________________________ __________________________________________________________________________ This email has been checked for virus and other malicious content prior to leaving our network. __________________________________________________________________________

2 5

Removing Custom Column question.
by kelvin.mitchell＠4mresearch.com 29 Aug '17

29 Aug '17

Hello all, I know this is probably a horribly stupid question. I created at custom column called ASDSim coming from a client xymon agent. I have tried removing the using the drop command, removing the [ASDSim] tag from the cfg file, and removing the script itself, etc. But it still displays on the monitor. The only way I can get the column off is to put it in the NOCOLUMNS section of the cfg file. Is that the only way, or am I just blind and totally missing something in the documentation? Thanks! Kel ________________________________ From: Xymon <xymon-bounces at xymon.com> on behalf of xymon-request at xymon.com <xymon-request at xymon.com> Sent: Tuesday, August 29, 2017 5:00 AM To: xymon at xymon.com Subject: Xymon Digest, Vol 79, Issue 11 Send Xymon mailing list submissions to xymon at xymon.com To subscribe or unsubscribe via the World Wide Web, visit http://lists.xymon.com/mailman/listinfo/xymon Xymon -- Discussions about the Xymon monitor<http://lists.xymon.com/mailman/listinfo/xymon> lists.xymon.com The Xymon monitor - www.xymon.com - is an Open Source (GPL) licensed monitoring system. This mailing list is for discussions about installing and using Xymon. or, via email, send a message with subject or body 'help' to xymon-request at xymon.com You can reach the person managing the list at xymon-owner at xymon.com When replying, please edit your Subject line so it is more specific than "Re: Contents of Xymon digest..." Today's Topics: 1. how to enable/disable an alert automatically (or make an alert conditional upon something else)? (Ribeiro, Glauber) 2. Re: how to enable/disable an alert automatically (or make an alert conditional upon something else)? (Root, Paul T) 3. Loaded client stops reporting to the server (Ribeiro, Glauber) 4. Re: Critical System Page -- HTTP 500 Error (EDSchminke at Hormel.com) 5. Re: Loaded client stops reporting to the server (Ribeiro, Glauber) 6. Suggestion: monitor CPU utilization by process (Ribeiro, Glauber) 7. Re: Suggestion: monitor CPU utilization by process (Root, Paul T) 8. Re: Suggestion: monitor CPU utilization by process (Jeremy Laidman) ---------------------------------------------------------------------- Message: 1 Date: Mon, 28 Aug 2017 19:46:24 +0000 From: "Ribeiro, Glauber" <glauber.ribeiro at experian.com> To: "xymon at xymon.com" <xymon at xymon.com> Subject: [Xymon] how to enable/disable an alert automatically (or make an alert conditional upon something else)? Message-ID: <135DD93F19DFE347BC300E008E6744BCFB92C42B at MCKEXCH04.ena.us.experian.local> Content-Type: text/plain; charset="utf-8" Hello, We have a very common situation where there are 2 servers, one being the production server, and the other its “spare”. If anything happens to the production server, we need people to be alerted even if it’s the middle of the night. On the other hand, if the spare server has problems, we may not need to wake people up. The server roles swap periodically (the one that is production this month may be spare next month, and vice versa). I can tell which one is production by watching for the existence of certain “flag files”, but I can imagine other places would have different mechanisms. The question is: is there a way in xymon, to make an alert conditional on some other piece of information? Some way to handle this kind of situation, where the “decision” on the criticality of a server is outside xymon? I think I could handle this by sending the alert to a script, and having logic in the script to send the communication or not. But I’m wondering if there is a better way, and I imagine that this kind of situation is common enough that people here must have seen it. Thanks, glauber

2 1

Loaded client stops reporting to the server
by glauber.ribeiro＠experian.com 28 Aug '17

28 Aug '17

We seem to be having a problem with AIX clients (I’m not sure if the OS type is relevant, but these happen to be AIX servers) where the client stops sending data to the xymon server when its CPU usage is high. This is making it hard to monitor for high CPU utilization, because when it happens, the server doesn’t hear about it. Any ideas? Is this a known problem? Thanks, glauber

1 1

Critical System Page -- HTTP 500 Error
by EDSchminke＠Hormel.com 28 Aug '17

28 Aug '17

I have another update to this issue. I think I can pin point the problem a little more specifically. In my previous message, I blamed the "uptime" test for causing the Critical Systems Page to crash. After this weekend, I discovered that there's a little more to it than that. This weekend, the Critical Systems Page crashed due to a "disk" test being non-green. After picking random hosts/test with varying degrees of success, I finally noticed that isn't necessarily "disk" or "uptime" that causes it; rather whichever test is the LAST test defined for a host (or cloned host). It just so happens, that "uptime" usually ends up being the LAST test defined for most of my hosts since the Critical Systems Page Editor sorts them as it gets written. To test, I made "disk" the last (only) test defined for a host. I would then modify thresholds for memory, disk and procs to put the tests into a non-green state. I set the "monitoring time" window 11:58PM to 11:59PM. First, disk crashed the page. I then duplicated the "disk" entry to "memory", making that the last test for that host. Disk no longer crashed the page, but when I put memory into a non-green state, it would crash. I then made "procs" the last test for the host. Memory no longer crashed the page, but procs would after putting that test into a non-green state. So in short, the test conditions. - Current time OUTSIDE monitoring time window. - Target test in a non-green state - Target test is the last (or only) test defined for a given host. I imagine this must be caused by something running off the end of the loop. Erik D. Schminke | Associate Systems Programmer Hormel Foods Corporation | One Hormel Place | Austin, MN 55912 Phone: (507) 434-6817 edschminke at hormel.com | www.hormelfoods.com

1 0

how to enable/disable an alert automatically (or make an alert conditional upon something else)?
by glauber.ribeiro＠experian.com 28 Aug '17

28 Aug '17

Hello, We have a very common situation where there are 2 servers, one being the production server, and the other its “spare”. If anything happens to the production server, we need people to be alerted even if it’s the middle of the night. On the other hand, if the spare server has problems, we may not need to wake people up. The server roles swap periodically (the one that is production this month may be spare next month, and vice versa). I can tell which one is production by watching for the existence of certain “flag files”, but I can imagine other places would have different mechanisms. The question is: is there a way in xymon, to make an alert conditional on some other piece of information? Some way to handle this kind of situation, where the “decision” on the criticality of a server is outside xymon? I think I could handle this by sending the alert to a script, and having logic in the script to send the communication or not. But I’m wondering if there is a better way, and I imagine that this kind of situation is common enough that people here must have seen it. Thanks, glauber

2 1