Bump...

This issue just happened again, and as I feared, when the smbd process core dumped, it created a core file and set its permissions and ownership to:

0600 root:root making i impossible for xymon to access this file.

but worse, it set every directory in the path to 0700 root:root

This of course makes it impossible for the xymon user to even traverse to the directory where the file exists.

As I reported previously, the client data sent back shows:

[file:/var/log/samba/cores/smbd/core] ERROR: Permission denied

But the test shows green, even though this a "red" (file server inaccessible, no one can log in) condition.

I still say that this issue of a file/dir that an admin wants to monitor for EXIST/NOEXIST but has been made unreadable/inaccessible to the xymon client should trigger a non-green condition to alert the admin that something is not right and needs to be investigated.

P.S. same idea for msgs file(s) that are configured for monitoring, but are inaccessible to the xymon user.

Thanks for listening (again)

:)

On 04/09/14 10:11, Bill Arlofski wrote:

Hi everyone!

We have a random issue with a Samba server where it core dumps each new smbd process as users log in. Generally the server functions 24/7/365, but once something (we don't know what yet) triggers this, each new smbd process spawned core dumps and no new logins are possible.

Simple way to monitor with Xymon is by looking for the non-existance (NOEXIST) of one of these core files:

/var/log/samba/cores/smbd/core

The Xymon issue I am reporting here is that by default, the directories above this file were created with 600 root:root permissions, and as such, Xymon can not see/access this file even when it exists and will report GREEN for the 'files' test.

Interesting thing is that if I remove the NOEXIST parameter from the analysis.cfg line for this file - meaning that this file needs to exist - and the file does exist, Xymon also reports it GREEN.

Clicking on the files test and then the /var/log/samba/cores/smbd/core link on the test's page brings you to a white web page with the following:

[file:/var/log/samba/cores/smbd/core] ERROR: Permission denied

At the bare minimum, I would think that Xymon would report this test as yellow, not green because "something is wrong" that an admin monitoring, or configuring Xymon needs to address.

For now, I have modified the permissions in the directory tree to allow for Xymon to access the dir that this file gets created in, but if for some reason, perhaps on syslog-ng startup or some other reason these permissions are reverted, Xymon will always think this test is GREEN, and no one will ever be notified when the file does appear.

Of course, I can add additional tests for this host to monitor each directory's ownership and permissions down to the file, but that becomes more cumbersome and opens things up to more places where an admin can make mistakes.

I have noticed a similar thing with regards to monitoring MSGS. By default (on Gentoo systems with syslog-ng, at least) /var/log/messages is created root:root 640, and as such, the Xymon user can not read this file to parse it. The MSGS test however shows GREEN. But on its page shows:

--[snip]-- No entries in /var/log/messages

Full log /var/log/messages Cannot open logfile /var/log/messages : Permission denied --[snip]--

I generally fix this by greating a group 'log', adding the xymon user to it and configuring syslog-ng to create the /var/log/messages file with root:log 640 perms..

But again, if my monitoring server is (default) set to monitor a log file and is told "permission denied" I would consider that to be at least a yellow status that an admin needs to investigate and address.

Thanks for listening

-- Bill Arlofski Reverse Polarity, LLC http://www.revpol.com/ -- Not responsible for anything below this line --