On Mon, 2012-07-09 at 11:02 -0700, cleaver at terabithia.org wrote:
Does it go green when you restart the client, then eventually go purple again?
Yes. The initial 'vmstat' process started by xymonlaunch runs, then ends but isn't restarted (so it seems). Hence I get one run, then nothing and so the colours go green once then purple until xymon is restarted.
Can you try running xymonlaunch with --debug to see what's happening, or strace it as it's running and send the output?
I did both yesterday but as far as I could see it showed nothing useful. (I'll repeat this and add the output below.)
I also ran it with the '--dump' option and that showed that the entries in clientlaunch.cfg were okay. The intervals were reported as 300 seconds (5 mins), so the processes should have been restarted. (I have 2 tasks configured.)
Looking at the code (in ./common/xymonlaunch.c) I can see the ('running') loop that it runs through and it looks fine. But because nothing at all is logged with debug, and as far as I can see all eventualities should report something, then the 'for' loop of the task list is probably not being entered (the initial ('for') loop works, but subsequently is failing?) I'll add some more logging to the client xymonlaunch code and see what happens.
Output from using 'xymonlaunch --debug':
=================================================== 2012-07-10 12:01:21 xymonlaunch starting 2012-07-10 12:01:21 Loading tasklist configuration from /home/xymon/client/etc/clientlaunch.cfg 29487 2012-07-10 12:01:21 Opening file /home/xymon/client/etc/clientlaunch.cfg 29487 2012-07-10 12:01:21 29487 2012-07-10 12:01:21 Starting tasklist scan 29487 2012-07-10 12:01:21 About to start task client 29488 2012-07-10 12:01:21 client -> Loading environment from /home/xymon/client/etc/xymonclient.cfg area 29488 2012-07-10 12:01:21 Opening file /home/xymon/client/etc/xymonclient.cfg 29488 2012-07-10 12:01:21 client -> Assigning stdout/stderr to log '/home/xymon/client/logs/xymonclient.log' 29487 2012-07-10 12:01:21 About to start task dns 29490 2012-07-10 12:01:21 dns -> Loading environment from /home/xymon/client/etc/xymonclient.cfg area 29490 2012-07-10 12:01:21 Opening file /home/xymon/client/etc/xymonclient.cfg 29490 2012-07-10 12:01:21 dns -> Assigning stdout/stderr to log '/home/xymon/client/logs/xymonclient.log' 29487 2012-07-10 12:01:21 29487 2012-07-10 12:01:21 Starting tasklist scan 29487 2012-07-10 12:01:21 Task client active with PID 29488 29487 2012-07-10 12:01:26 29487 2012-07-10 12:01:26 Starting tasklist scan 29487 2012-07-10 12:01:26 Task client active with PID 29488 29487 2012-07-10 12:01:28 29487 2012-07-10 12:01:28 Starting tasklist scan 29487 2012-07-10 12:01:33 29487 2012-07-10 12:01:33 Starting tasklist scan 29487 2012-07-10 12:01:38 29487 2012-07-10 12:01:38 Starting tasklist scan 29487 2012-07-10 12:01:43 29487 2012-07-10 12:01:43 Starting tasklist scan 29487 2012-07-10 12:01:48 29487 2012-07-10 12:01:48 Starting tasklist scan 29487 2012-07-10 12:01:53 29487 2012-07-10 12:01:53 Starting tasklist scan 29487 2012-07-10 12:01:58 29487 2012-07-10 12:01:58 Starting tasklist scan 29487 2012-07-10 12:02:03 29487 2012-07-10 12:02:03 Starting tasklist scan 29487 2012-07-10 12:02:08 29487 2012-07-10 12:02:08 Starting tasklist scan 29487 2012-07-10 12:02:13 29487 2012-07-10 12:02:13 Starting tasklist scan 29487 2012-07-10 12:02:18 29487 2012-07-10 12:02:18 Starting tasklist scan 29487 2012-07-10 12:02:23 29487 2012-07-10 12:02:23 Starting tasklist scan 29487 2012-07-10 12:02:28 29487 2012-07-10 12:02:28 Starting tasklist scan 29487 2012-07-10 12:02:33 29487 2012-07-10 12:02:33 Starting tasklist scan 29487 2012-07-10 12:02:38 29487 2012-07-10 12:02:38 Starting tasklist scan 29487 2012-07-10 12:02:43 29487 2012-07-10 12:02:43 Starting tasklist scan 29487 2012-07-10 12:02:48 29487 2012-07-10 12:02:48 Starting tasklist scan 29487 2012-07-10 12:02:53 29487 2012-07-10 12:02:53 Starting tasklist scan 29487 2012-07-10 12:02:58 29487 2012-07-10 12:02:58 Starting tasklist scan 29487 2012-07-10 12:03:03 29487 2012-07-10 12:03:03 Starting tasklist scan 29487 2012-07-10 12:03:08 29487 2012-07-10 12:03:08 Starting tasklist scan 29487 2012-07-10 12:03:13 29487 2012-07-10 12:03:13 Starting tasklist scan 29487 2012-07-10 12:03:18 29487 2012-07-10 12:03:18 Starting tasklist scan 29487 2012-07-10 12:03:23 29487 2012-07-10 12:03:23 Starting tasklist scan 29487 2012-07-10 12:03:28 29487 2012-07-10 12:03:28 Starting tasklist scan 29487 2012-07-10 12:03:33 29487 2012-07-10 12:03:33 Starting tasklist scan 29487 2012-07-10 12:03:38 29487 2012-07-10 12:03:38 Starting tasklist scan 29487 2012-07-10 12:03:43 29487 2012-07-10 12:03:43 Starting tasklist scan 29487 2012-07-10 12:03:48 29487 2012-07-10 12:03:48 Starting tasklist scan 29487 2012-07-10 12:03:53 29487 2012-07-10 12:03:53 Starting tasklist scan 29487 2012-07-10 12:03:58 29487 2012-07-10 12:03:58 Starting tasklist scan 29487 2012-07-10 12:04:03 29487 2012-07-10 12:04:03 Starting tasklist scan 29487 2012-07-10 12:04:08 29487 2012-07-10 12:04:08 Starting tasklist scan 29487 2012-07-10 12:04:13 29487 2012-07-10 12:04:13 Starting tasklist scan 29487 2012-07-10 12:04:18 29487 2012-07-10 12:04:18 Starting tasklist scan 29487 2012-07-10 12:04:23 29487 2012-07-10 12:04:23 Starting tasklist scan 29487 2012-07-10 12:04:28 29487 2012-07-10 12:04:28 Starting tasklist scan 29487 2012-07-10 12:04:33 29487 2012-07-10 12:04:33 Starting tasklist scan 29487 2012-07-10 12:04:38 29487 2012-07-10 12:04:38 Starting tasklist scan 29487 2012-07-10 12:04:43 29487 2012-07-10 12:04:43 Starting tasklist scan 29487 2012-07-10 12:04:48 29487 2012-07-10 12:04:48 Starting tasklist scan 29487 2012-07-10 12:04:53 29487 2012-07-10 12:04:53 Starting tasklist scan 29487 2012-07-10 12:04:58 29487 2012-07-10 12:04:58 Starting tasklist scan 29487 2012-07-10 12:05:03 29487 2012-07-10 12:05:03 Starting tasklist scan 29487 2012-07-10 12:05:08 29487 2012-07-10 12:05:08 Starting tasklist scan 29487 2012-07-10 12:05:13 29487 2012-07-10 12:05:13 Starting tasklist scan 29487 2012-07-10 12:05:18 29487 2012-07-10 12:05:18 Starting tasklist scan 29487 2012-07-10 12:05:23 29487 2012-07-10 12:05:23 Starting tasklist scan 29487 2012-07-10 12:05:28 29487 2012-07-10 12:05:28 Starting tasklist scan 29487 2012-07-10 12:05:33 29487 2012-07-10 12:05:33 Starting tasklist scan 29487 2012-07-10 12:05:38 29487 2012-07-10 12:05:38 Starting tasklist scan 29487 2012-07-10 12:05:43 29487 2012-07-10 12:05:43 Starting tasklist scan 29487 2012-07-10 12:05:48 29487 2012-07-10 12:05:48 Starting tasklist scan 29487 2012-07-10 12:05:53 29487 2012-07-10 12:05:53 Starting tasklist scan 29487 2012-07-10 12:05:58 29487 2012-07-10 12:05:58 Starting tasklist scan 29487 2012-07-10 12:06:03 29487 2012-07-10 12:06:03 Starting tasklist scan 29487 2012-07-10 12:06:08 29487 2012-07-10 12:06:08 Starting tasklist scan 29487 2012-07-10 12:06:13 29487 2012-07-10 12:06:13 Starting tasklist scan 29487 2012-07-10 12:06:18 29487 2012-07-10 12:06:18 Starting tasklist scan 29487 2012-07-10 12:06:23 29487 2012-07-10 12:06:23 Starting tasklist scan 29487 2012-07-10 12:06:28 29487 2012-07-10 12:06:28 Starting tasklist scan 29487 2012-07-10 12:06:33 29487 2012-07-10 12:06:33 Starting tasklist scan 29487 2012-07-10 12:06:38 29487 2012-07-10 12:06:38 Starting tasklist scan 29487 2012-07-10 12:06:43 29487 2012-07-10 12:06:43 Starting tasklist scan 29487 2012-07-10 12:06:48 29487 2012-07-10 12:06:48 Starting tasklist scan 29487 2012-07-10 12:06:53 29487 2012-07-10 12:06:53 Starting tasklist scan 29487 2012-07-10 12:06:58 29487 2012-07-10 12:06:58 Starting tasklist scan 29487 2012-07-10 12:07:03 29487 2012-07-10 12:07:03 Starting tasklist scan 29487 2012-07-10 12:07:08 29487 2012-07-10 12:07:08 Starting tasklist scan 29487 2012-07-10 12:07:13 29487 2012-07-10 12:07:13 Starting tasklist scan 29487 2012-07-10 12:07:18 29487 2012-07-10 12:07:18 Starting tasklist scan 29487 2012-07-10 12:07:23 29487 2012-07-10 12:07:23 Starting tasklist scan 29487 2012-07-10 12:07:28 29487 2012-07-10 12:07:28 Starting tasklist scan
The 'vmstat' task has ended by 12:06:33. As can be seen the loop (scan) carries on but no tasks are restarted. (The 'dns' task is a task we run on the clients to report to the 'dnsr' column on the Xymon server. It runs once then should be started again by xymonlaunch after 5 mins.)
The output of 'strace -f -p <xymonlaunch pid>' is attached. It shows the loop (scan) occurring but nothing else - hence it doesn't seem that the 'for' loop of the task list is being executed. By the end of the strace output the 'vmstat' task of the client has ended but not been restarted.
John.
-- John Horne Tel: +44 (0)1752 587287 Plymouth University, UK Fax: +44 (0)1752 587001