Jeremy,
It seems LDAP surely is the issue. The check has no settable timeout in hosts.cfg. As for what happens to our LOVELY OpenDirectory, it just hangs both SSL and plain text. All ports remain open, ping even still works.
I caught it this time and it does seem that the lack of a timeout is the problem here (the check only terminated because we rebooted the box)
Error output: WARNING: Runtime 521 longer than time limit (300)
TIME SPENT Event Start time Duration xymonnet startup 1383315414.744906
Service definitions loaded 1383315414.746373 0.001467 Tests loaded 1383315414.815137 0.068764 DNS lookups completed 1383315414.815838 0.000701 Test engine setup completed 1383315414.839164 0.023326 TCP tests completed 1383315426.045943 11.206779 PING test completed (104 hosts) 1383315434.035841 7.989898 PING test results sent 1383315434.036641 0.000800 Test result collection completed 1383315434.037506 0.000865 LDAP test engine setup completed 1383315434.037568 0.000062 LDAP tests executed 1383315935.053479 501.015911 LDAP tests result collection completed 1383315935.053491 0.000012 Test results transmitted 1383315935.055777 0.002286 xymonnet completed 1383315935.062680 0.006903 TIME TOTAL 520.317774
In the host.cfg we have: $IP commander.example.com # ldap:// commander.example.com:389/dc=commander,dc=example,dc=com
It is our only ldap check (its over a IPsec S2S link with 50/50 connection)
Xymonnet is standard install "CMD xymonnet --report --ping --checkresponse", the proc only exits when we reboot the server or it passes normally.
I couldn't get the command to just do the one server, will try again in a bit. I think the ultimate problem is ldap check never times out, so how can one be set?
-- Thomas Vachon
On Thu, Oct 31, 2013 at 12:31 AM, Jeremy Laidman <jlaidman at rebel-it.com.au>wrote:
Thomas
In what way is the LDAP server in a "bad state"?
Are you using LDAP or LDAPS?
Can you connect to the server on the LDAP port using telnet?
What does your hosts.cfg file entry look like?
What parameters do you have for xymonnet in tasks.cfg?
Does the xymonnet process eventually exit?
Can you run xymonnet manually (as the xymon user, under a xymoncmd shell), such as:
$ xymonnet --debug --timeout=1 <name-of-server>
If this fails in the same way, perhaps you can tweak some parameters, such as adding "--dns=ip" or "--noping" or other options.
J
On 31 October 2013 00:01, Thomas Vachon <vachon at sessionm.com> wrote:
We are having issues with xymonnet handing on 4.3.12 (happened on 4.3.10 too). As soon as we added an ldap check which can hang (due to the ldap server being in a bad state), xymonnet hangs and goes purple on all remote checks.
Nothing shows in the logs. Here is the historical xymonnet info from the last purple:
Wed Oct 30 12:12:10 2013
xymonnet version 4.3.12 SSL library : OpenSSL 1.0.1e 11 Feb 2013 LDAP library: OpenLDAP 20431
Statistics: Hosts total : 70 Hosts with no tests : 1 Total test count : 183 Status messages : 183 Alert status msgs : 0 Transmissions : 3
DNS statistics:
hostnames resolved : 115
succesful : 69
failed : 0
calls to dnsresolve : 182
TCP test statistics:
TCP tests total : 113
HTTP tests : 45
Simple TCP tests : 68
Connection attempts : 113
bytes written : 7192
bytes read : 528667
TIME SPENT Event Start time Duration xymonnet startup 1383135130.081755
Service definitions loaded 1383135130.089141 0.007386 Tests loaded 1383135130.147247 0.058106 DNS lookups completed 1383135130.147712 0.000465 Test engine setup completed 1383135130.159468 0.011756 TCP tests completed 1383135142.406914 12.247446 PING test completed (69 hosts) 1383135149.073309 6.666395 PING test results sent 1383135149.074005 0.000696 Test result collection completed 1383135149.075192 0.001187 LDAP test engine setup completed 1383135149.075263 0.000071 LDAP tests executed 1383135151.403612 2.328349 LDAP tests result collection completed 1383135151.403621 0.000009 Test results transmitted 1383135151.405148 0.001527 xymonnet completed 1383135151.407244 0.002096 TIME TOTAL 21.325489
-- Thomas Vachon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon