All -
I know I did something bad, but don't know exactly what...
I ran across this posting from Henrik (http://lists.xymon.com/archive/2011-August/032267.html) indicating one could see the data coming across to the server from all the clients using a command similar to this:
xymoncmd xymond_channel --channel=data egrep "egrep '^\@\@'"(Henrik's example used "status" rather than "data")
So, I kicked it off and walked away for about an hour. Fine. When I returned some colleagues came by with worried expressions asking me to look at our Non-Green report. Rather than the usual handful of non-green hosts, there were rows and rows of hosts showing 'purple' in various columns. I killed my manual "xymond_channel..." command and watched as the Non-Green report shrank quickly to a normal display.
What happened?
It appeared the appearance of various "purple" hosts in the environment was random: random hosts with random columns (but I won't swear to that)At least one monitored host's "proc"s ended up as "red", rather than "purple", but this transition was long after the initial onset of my command. The user believes there was no real "crash" of his procs on his host.We have two servers monitoring the same hosts in our environment. On the host used to run my manual command, pretty much all graph data for the Xymon server itself shows a gap: no data available while my command ran.From the brother server, looking at graphs for the deranged server, we see an odd *decrease* in CPU load for the duration of my extra command, and an *increase* in TCP traffic
I'm guessing my manual command competed / randomly intercepted incoming data for this server, but does this fit all the oddities observed?
In any case, I'm back at square one for my original goal of *safely* seeing what raw data is coming into the server from my clients. (FWIW - all hosts are Solaris 10 / Xymon 4.3.3)
Help?
Thanks!
david
David Mills
Systems Administrator
Northrop Grumman
512-595-1238
david.mills at ngc.com