April 2014 - Xymon - Xymon mailman web

Possible BUG in how Xymon interprets data for monitoring "FILE" and "MSGS"
by waa-hobbitml＠revpol.com 14 May '14

14 May '14

Hi everyone! We have a random issue with a Samba server where it core dumps each new smbd process as users log in. Generally the server functions 24/7/365, but once something (we don't know what yet) triggers this, each new smbd process spawned core dumps and no new logins are possible. Simple way to monitor with Xymon is by looking for the non-existance (NOEXIST) of one of these core files: /var/log/samba/cores/smbd/core The Xymon issue I am reporting here is that by default, the directories above this file were created with 600 root:root permissions, and as such, Xymon can not see/access this file even when it exists and will report GREEN for the 'files' test. Interesting thing is that if I remove the NOEXIST parameter from the analysis.cfg line for this file - meaning that this file needs to exist - and the file does exist, Xymon also reports it GREEN. Clicking on the files test and then the /var/log/samba/cores/smbd/core link on the test's page brings you to a white web page with the following: [file:/var/log/samba/cores/smbd/core] ERROR: Permission denied At the bare minimum, I would think that Xymon would report this test as yellow, not green because "something is wrong" that an admin monitoring, or configuring Xymon needs to address. For now, I have modified the permissions in the directory tree to allow for Xymon to access the dir that this file gets created in, but if for some reason, perhaps on syslog-ng startup or some other reason these permissions are reverted, Xymon will always think this test is GREEN, and no one will ever be notified when the file does appear. Of course, I can add additional tests for this host to monitor each directory's ownership and permissions down to the file, but that becomes more cumbersome and opens things up to more places where an admin can make mistakes. I have noticed a similar thing with regards to monitoring MSGS. By default (on Gentoo systems with syslog-ng, at least) /var/log/messages is created root:root 640, and as such, the Xymon user can not read this file to parse it. The MSGS test however shows GREEN. But on its page shows: --[snip]-- No entries in /var/log/messages Full log /var/log/messages Cannot open logfile /var/log/messages : Permission denied --[snip]-- I generally fix this by greating a group 'log', adding the xymon user to it and configuring syslog-ng to create the /var/log/messages file with root:log 640 perms.. But again, if my monitoring server is (default) set to monitor a log file and is told "permission denied" I would consider that to be at least a yellow status that an admin needs to investigate and address. Thanks for listening -- Bill Arlofski Reverse Polarity, LLC http://www.revpol.com/ -- Not responsible for anything below this line --

1 1

Problem with HTTPS proxy and CPU load report with Multi CPU Windows 2008 BBWin machines
by linuxmasterjedi＠free.fr 06 May '14

06 May '14

Hello I've switched from 4.2.2 to 4.3.17. Unfortunately, I can't check anymore HTTPS via proxy. If I remember correctly, it worked before, not anymore. How do you manage to check HTTPS website through HTTP Proxy with authentication? Also, I have a suspicious behavior with Windows 2008 R2 64 bits with 4 CPU and BBWin. XYmon trigger a red alert even the CPU is under the yellow alert. I'm guessing BBWIn or XYmon mix up load information of each CPU and the total CPU load. For example : 2 CPUs got 20% of load and 2 CPUs got 30% of CPU. the server total average load is 25%. XYmon or BBwin add CPU loads 20 + 20 + 30 + 30 = 100% => red alert, CPU load is over 95% limit... Anyone experimented this issue before ? Thanks

2 2

Multi-host graphs
by colin.coe＠gmail.com 30 Apr '14

30 Apr '14

Hi all Could someone point me at a guide for $SUBJECT? What I want to end up with is CPU/Load graphs for 4 related servers on a separate page. Similarly, I want CPU/Load and networking graphs for a dozen different related servers on a separate page. Thanks CC -- RHCE#805007969328369

1 0

flushing rrd files
by Martin.Sperl＠amdocs.com 29 Apr '14

29 Apr '14

Hi! Is there a means to flush the rrd files filled in from xymond_rrd say via the "xymon" command? As an alternative is it possible to use the generic RRDCACHED as a replacement for the caching? Any experience with that? Thanks, Martin This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp

2 2

High CPU Load Rendering Graphs
by everett.vernon＠gmail.com 29 Apr '14

29 Apr '14

Hi all My Xymon server 4.3.10 is burning the CPU cycles when we view multiple graphs, like the trends page, and takes about 5 seconds to render a single graph in a single-graph page view. It's a Sun Fire X4150 with 4Gb of RAM, running Solaris 10 update 5.. Version Location Tag -------------------------------- -------------------------- Intel(R) Xeon(R) CPU X5460 @ 3.16GHz CPU 1 Not a very powerful box, and a bit dated, but I have seen significantly better performance on far lesser systems. So I am not really thinking the issue is with the hardware. It's been slow since it was installed. If I view the trends column, I can see the CPU load jump from below 1 to over 10 at times. Running prstat or top in another window while viewing the trends column, the process ranking by CPU gets dominated by showgraph.cgi, owned by the web server user. Top under normal conditions. CPU states: 99.9% idle, 0.0% user, 0.1% kernel, 0.0% iowait, 0.0% swap Top rendering the trends column. CPU states: 0.0% idle, 93.8% user, 6.2% kernel, 0.0% iowait, 0.0% swap Also getting this error (128)Network is unreachable: connect to listener on [::]:443 in my Apache error.log file, repeated every second while rendering the graphs. And from time to time, I get this one. File does not exist: /opt/csw/apache2/share/htdocs/server-status Anybody seen anything like this? Perhaps know of somewhere I can look for more info? I have looked at this http://lists.xymon.com/archive/2014-January/038780.html But it doesn't seem relevant. Only 2 errant files, and deleting them made absolutely no difference. Other info that may be important.... bash-3.00# ./httpd -v Server version: Apache/2.2.22 (Unix) Server built: Jun 1 2012 05:09:20 bash-3.00# ./httpd -V Server version: Apache/2.2.22 (Unix) Server built: Jun 1 2012 05:09:20 Server's Module Magic Number: 20051115:30 Server loaded: APR 1.4.5, APR-Util 1.3.12 Compiled using: APR 1.4.6, APR-Util 1.3.12 Architecture: 32-bit Server MPM: Prefork threaded: no forked: yes (variable process count) Server compiled with.... -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_FCNTL_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=128 -D HTTPD_ROOT="/opt/csw/apache2" -D SUEXEC_BIN="/opt/csw/apache2/sbin/suexec" -D DEFAULT_PIDLOG="/var/run/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="/var/run/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="etc/mime.types" -D SERVER_CONFIG_FILE="etc/httpd.conf" Thanks Vernon -- "Accept the challenges so that you can feel the exhilaration of victory" - General George Patton

4 14

custom xymon web pages
by warnold＠websense.com 29 Apr '14

29 Apr '14

We are using xymon to monitor about 2500+ nodes in our production system. We have broken up our system to pages, however, but even with that criteria the pages are getting much bigger and taking longer to load. We are coming to a point that we need to be able to filter certain pages based on flexible criteria. For instance, we have all of one type of system on a page set which has 300 nodes. It is helpful at times to see all 300 nodes, but at some point you need to filter down. Say you only want to see the systems that are getting a release, or the systems in a region, or the systems in one location in the region, or even down to the one system you are working on at the moment. We aren't going to be able to define the different filters up front as there are just so many options. I am thinking of some script akin to bb2board.pl which we adopted and have modified to great effect as our current non-green board. We have added multiple filters to pass on to xymondboard (alert color, host name, time slices, you name it) and then we build a web page with the resulting alerts found from xymondboard and use the xymon header and footers. What I am asking in this particular case is to see all the tests for a server and their current status much as I would in a regular xymon page view. I know I can use xymondboard to get all the test values for a system, but that doesn't seem to handle the combo tests, and it seems like rewriting a lot of existing code. So what I'm looking for is there a way to create a hobbit page on the fly for just a section of the hosts.cfg file? Anybody else come across this need before? Thank you! WHITNEY ARNOLD Cloud Operations Lead (Americas) WEBSENSE, INC.

1 0

RRD Duplicate DS Name
by dirk.winning＠junidas.de 29 Apr '14

29 Apr '14

Hi Phil, may I refer to your xymon-question from some years back:-) http://lists.xymon.com/archive/2012-July/035049.html I think that the answer from Hendrik was very weak, in that respect that I do not regard it as an answer at all; what did you think? What I found when running into the same issue was, that the output given to xymon has to start with a newline! After I added a newline at the top of the output the trigger-string was found and hence the graph appeared. Did you have similar solutions? Thanks greets Dipl. Ing. Dirk Winning, Systemberater dirk.winning at junidas.de junidas GmbH, Aixheimer Str. 12, 70619 Stuttgart Tel. +49 (711) 4599799-12 Geschäftsführer: Dr. Markus Stoll, Matthias Zepf Amtsgericht Stuttgart, HRB 21939

2 1

Is this logic right?
by Paul.Root＠CenturyLink.com 25 Apr '14

25 Apr '14

I have two machines that run redundantly. One is used more than the other, but both can perform the same function. Most of the time, I want to know when either of them are having problems. However, from 9:15 until about midnight, I have a backup job that sometimes causes I/O delays, and makes some of the tests, start flapping. So I get paged. But nothing is really wrong. So what I want is to have pages go normally at other times, but from 9:15pm until midnight, only page if both machines are red. HOST=hpna1,hpna2 COLOR=red IGNORE TIME=*:2115:2359 MAIL $NG-IM-ADMIN MAIL $NG-IM-ONCALL FORMAT=SMS REPEAT=20 RECOVERED # During Veeam's backup time, just watch the combo test HOST=HPNA COLOR=red TIME=*:2115:2359 MAIL $NG-IM-ADMIN RECOVERED REPEAT=30m DURATION>30 MAIL $NG-IM-ONCALL FORMAT=SMS REPEAT=20 RECOVERED DURATION>30 And combo is: HPNA.conn = (hpna1.conn || hpna2.conn) HPNA.hpnaproxy = (hpna1.hpnaproxy || hpna2.hpnaproxy) HPNA.http = (hpna1.http || hpna2.http) HPNA.hpnadetails = (hpna1.hpnadetails || hpna2.hpnadetails) Paul Root Lead Engineer CenturyLink Network Reliability Operations Center 600 Stinson Blvd, N.E. Flr 2N Minneapolis, MN 55413 Direct: (651)312-5207 Paul.Root at centurylink.com

1 1

rrd crashing and graph gaps on 'trends' charts with 4.3.17
by john.thurston＠alaska.gov 23 Apr '14

23 Apr '14

I'm seeing gaps in some charts and not in others. The charts for vmstat tcp ifstat show no gaps. All of the other charts show consistent gaps of several hours, followed by a brief period of data, followed by more multi-hour data gaps. After appending --debug to the rrd_data line in tasks.cfg, tailing rrd-data.log shows many messages of the form: "Peer not up, flushing message queue" and several messages of the form: "Child process 10502 died: Signal 6" This makes me think some portion of the rrd listening process is crashing, leaving gaps in my data. Can anyone suggest how to determine why the rrd is dying? Could the absence of gaps in the previously mentioned charts be because they are being populated by rrd_status rather than rrd_data? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Enterprise Technology Services Department of Administration State of Alaska

1 0

lrpc testing
by tawilcox＠fs.fed.us 22 Apr '14

22 Apr '14

In Big Brother we tested lrpc on port 1352. This test is not working in Xymon. How can we test lrpc in Xymon? [USDA USFS] Theodore (Ted) Wilcox, PMP , ITIL Incident Management Program Manager, EOC Monitor CIO/EOC USDA Forest Service p: 505-500-7648 tawilcox at fs.fed.us<mailto:tawilcox at fs.fed.us> 100 Sun Avenue NE Albuquerque, NM 87109 www.fs.fed.us<http://www.fs.fed.us/> [Twitter]<https://twitter.com/forestservice>[Facebook]<http://facebook.com/USDA> Caring for the land and serving people This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

2 1