On Sun, January 10, 2016 6:12 pm, David Boyer wrote:
JC, I see it takes stdout just fine. Might I add that it might be beneficial if the script would output the heading say [script] vice the name of the the script "script.sh"?
True, but then you could just rename the script to "script" and leave off the ".sh" :) As long as it's chmod +x (executable) and not named README or a .rpm/.dpkg debris file it will be executed and added in.
I java a bunch of specific info on each host that is 90% there for ref by our security folks, developers, etc... for instance which mode selinux is in, whether auditing is running, backup logs are current, etc. Using the "local" directory feature would gather the info, put a header on it and allow reports to more easily be run against the data.
I can definitely see where you're coming from there, but the current /local (and /sections, for Terabithia RPM users) functionality is geared towards xymonclient.sh handling the wrapping of text with the [script] headers itself.
Fortunately (or not), though, it's all just plain text once it gets to xymond :) You could easily have your script "myscript" add in the headers for sub-sections however you like...
=== #!/bin/sh
echo "this line get put into the xymonclient.sh-created [myscript] section" echo "[foobar]" /some/command/here --debug cat /proc/some/other/data
echo "[snafu]" /some/other/commands
===== etc... It'll look and be treated just like any other [section].
Alternatively, you can have your script execute completely independently of the default "xymonclient.sh" script. All you need to do is copy the output format that xymonclient is generating, but use a custom "collectorid" string:
client[/COLLECTORID] HOSTNAME.OSTYPE [HOSTCLASS]
and then run it on a regular basis on the host. (This is convenient if you have complex additional scripts or need to run something as root to collect the data.) All of the un-expired client data will be concatenated together when displayed/saved. If it can run as the xymon user, just make a new section in the clientlaunch.cfg file on the client, and xymonlaunch will handle it for you. It's up to you.
Also, on the xymond_rootlogin.pl script, where would this be set upto run? In the tasks.cfg?
Correct. This is server-side and should work pretty much as a drop-in task. Take a look at how "xymond_client" is being run from there via xymond_channel and duplicate that section (changing the name of the header).
(There are multiple ways to do this sort of thing, especially at scale, but this can get you started.)
-jc
On Sun, Jan 10, 2016 at 8:05 PM, J.C. Cleaver <cleaver at terabithia.org> wrote:
Glad it could help :)
There should be a README-local file in the client home directory (or documentation directory, depending on what install method you used). By default, the client will look in $XYMONHOME/local/ for anything executable. If anything is found, it's run (xymonclient.sh:65) and the output is added to the client report near the end, under the heading "[local:<filename>]". This allows a server admin to easily add new things on a one-off basis.
Xymon v4.4 will have a similar directory named /sections/ with does the same thing, except without the "local:" prefix on the output. (It's intended for packaged site or distro/vendor-specific scripts.)
Ryan: Yes, this is one area where it definitely needs more highlighting. A lot of users miss that this happens, but it's really an invaluable resource for forensics.
Remember, though, that this requires xymon to be running in server-config mode, not --local client mode. In --local client mode, this raw data never leaves the original box and so can't be saved.
Regards, -jc
On Sun, January 10, 2016 3:51 pm, David Boyer wrote:
JC, Yes, it does! Thanks for the background.. I've written/modified several tests for my environment while we were still in the "big brother" era. I'm still learning the ins/outs of xymon. I'm intrigued by the use of the "local" directory on the clientside. Where might I find more information about using this feature.
Dave
On Sun, Jan 10, 2016 at 3:35 PM, Novosielski, Ryan <novosirj at ca.rutgers.edu> wrote:
I just wanted to thank you, JC, for this information. I wanted to know whether it would be possible to check the kernel version back some time even though that is something I did not test. I figured the info might show something, but that isn't kept in history or anything. If I had realized that a client data snapshot is kept when the status changes, I would have had the answer (which I still need, so it's very helpful).
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences* || \\UTGERS |---------------------*O*--------------------- ||_// Biomedical | Ryan Novosielski - Senior Technologist || \\ and Health | novosirj at rutgers.edu- 973/972.0922 (2x0922) || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark `'
On Jan 10, 2016, at 06:48, J.C. Cleaver <cleaver at terabithia.org> wrote:
Hi David,
On Sat, January 9, 2016 3:15 pm, David Boyer wrote:
I see that there is data in the client data that is not turned into
columns? Can I turn this data into columns to report? I'm using xymon
4.3.20 on both the client/server.
On linux, I see that this data is not turned into any columns:
[who]
[route]
[netstat]
[ifstat]
Not that I'd use all of them, just wondering what the reasoning is behind
it.
This is correct. The client data is a full raw text of various bits of information that can be used to create status messages out of, but there isn't a mandated 1:1 correspondence between each section and each test. By keeping them independent (raw data and central processing), you have the flexibility to write up new tests based off of already-existing incoming data, and/or add new "raw" data without having a specific test in mind.
For the 'who' data in particular, there's a sample processor in the source tarball that can demonstrate how easy it is to add simple new tests at
https://sourceforge.net/p/xymon/code/HEAD/tree/branches/4.3.24/xymond/xymond...
You can easily add new sections to the client data by adding files to
a
"/local/" directory on the client machine, or editing the xymonclient-${OS}.sh shell script by hand, and running any command that the unprivileged 'xymon' user can execute.
What's the benefit to all this additional data if it's not used? Primarily forensics and triage. As the xymon(7) man page puts it:
The Xymon user-interface is simple, but engineers will also find lots of
relevant information. E.g. the data that clients report to Xymon contain the raw output from a number of system commands. That information is available directly in Xymon, so an administrator no longer needs to login to a server to get an overview of how it is behaving - the very commands they would normally run have already been performed, and the results are on-line in Xymon.
https://www.xymon.com/help/manpages/man7/xymon.7.html
This becomes even more relevant when you consider snapshoting. When a status goes "red", a snapshot of the client data at that time is kept. So if you went back later to try to figure out why (e.g.) CPU was rising, the output of the '[who]' section tells you who might have been doing something then, even if the data wasn't used for making a test out of at that time.
Also, I notice that getting the data via xymondboard is not 100% either...
[xymon at ztest bin]$ ./xymon 192.168.1.230 "clientlog yumlist section=who"
*snip*
As you can see below, requesting a status on all the tests, there is no
"who" being reported.
But just a moment ago, asking for just that data it works.
[xymon at ztest bin]$ ./xymon 192.168.1.230 "xymondboard host=yumlist"
*snip*
Correct. The "clientlog" command retrieves the most recent raw client data, while the "xymondboard" command retrieves just the status messages (the tests that you see on the webpages).
The "clientlog" column on the web page, much like the "info" and "trends" columns, isn't a real test... it's just present to provide easy access to the most recent data. You can also access it from the "Client data" link on the bottom of any of that host's status pages.
Hope that helps!
-jc
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon