Hi List,
I've just set up Xymon monitoring of a pfSense system and I thought I'd
share a few details for others.
pfSense is a firewall system based on FreeBSD using the 'pf' packet
filter system. The FreeBSD system is trimmed down to a smaller operating
system than the normal distribution and has a very small number of
available packages available in the default repository, all related to
it's firewall functionality. I've seen reference to pulling in packages
from the standard FreeBSD pkg system but have chosen not to go down that
path.
My starting point is Jeremy Laidman's xymon-rclient.sh script from
xymonton. This is a quite well thought out shell script (in contrast to
many of the add-ons, including mine), that connects to the target system
using ssh (or others that don't matter) and fires off a combination of
the standard xymon client scripts and a few extra shell commands. The
resulting output looks like the normal client log which is then handed
back to xymon. The idea is that no special software is required on the
target system.
After installing the script in the server/ext directory, you add a tasks
entry (I use a tasks.d directory file) to run the script every 5
minutes. All pretty standard stuff so far. The recommended task entry is:
[xymon-rclient]
??????? ENVFILE $XYMONHOME/etc/xymonserver.cfg
??????? CMD /$XYMONHOME/ext/xymon-rclient.sh
??????? LOGFILE $XYMONSERVERLOGS/xymon-rclient.log
??????? INTERVAL 5m
This looks for RCLIENT tags on entries in your hosts.cfg file. My tag
looks like this:
??? "RCLIENT:cmd(ssh -T -o BatchMode=yes admin at 10.0.1.54
/bin/sh),ostype(freebsd)"
Key points include the ostype(freebsd) part which identifies the target
system type. This identifies the client script to send to the target
system, in this case client/bin/xymonclient-freebsd.sh.
The /bin/sh at the end of the command solves a particular problem. It
tells the ssh daemon to run this program rather than the normal login
shell. This is important because if you connect to a pfSense system by
ssh (BTW this must be enabled first in the pfSense web interface), you
are greeted by a menu rather than a command prompt. It would be possible
to patch the rclient.sh script to send an '8' (the option for a shell)
as the first thing it does, but that would mess up using the script for
any other target systems. Including the /bin/sh command bypasses the
menu and sets up the system in a shell to begin work. Note that this is
a more or less standard bourne shell. Bash would only be available as an
unsupported package but is also completely unnecessary.
SSH Setup
======
To use ssh you will need to generate some key pairs and transport the
public key parts to the target system.
On the xymon server machine I did this (as root), assuming the xymon
server runs as user called 'xymon':
??? cd ~xymon
??? mkdir .ssh
??? chown xymon .ssh
??? cd .ssh
??? ssh-keygen -t rsa -f id_rsa -P ''
??? ssh-keygen -t dsa -f id_dsa -P ''
??? ssh-keygen -t ecdsa -f id_ecdsa -P ''
??? ssh-keygen -t ed25519 -f id_ed25519 -P ''
??? chown xymon *
Your ssh may have less or additional key types than mine, check your
manpage for ssh-keygen. Also, that's two consecutive single quotes at
the end of the line.
You then need to transfer your public keys over to the target system. To
get your public keys, type:
??? cat ~xymon/.ssh/*.pub
And copy the resulting 4 lines of output (some of which may be wrapped
over a few lines).
We need to exchange public keys with the target server. Log on to the
pfSense system as the local xymon user. This is important because we
need this user to capture the remote pfSense host's public key in
xymon's .ssh/known_hosts file.
??? su - xymon -c "ssh admin at 10.0.1.54"
If you are a 'sudo' fan, adjust as necessary. Accept the remote system's
key and then give the remote password. The pfSense menu should appear,
select 8 to get a shell.
Create a .ssh directory:
??? mkdir .ssh
And add our public keys to the system.
??? cat >>.ssh/authorized_keys
Paste in our public keys from earlier and press control+D on a new line
to end input. Type control+D again to exit the shell, then select option
'0' to logout from the menu.
You should now be able to ssh to the pfSense system (as the xymon user)
without accepting a new key or supplying a password. Test and verify.
??? su - xymon -c "ssh admin at 10.0.1.54"
Fixes to xymon-rclient.sh
================
I've sent a diff file to Jeremy Laidman a couple of days ago. He may or
may not change the code on xymonton. That's how these things work.
Essentially there are two essential changes to reflect the target system
not being Linux. One is a problem with the sent command stream
over-running a fresh shell invocation intended to set up proper
environment variables. The other change is to some sed GNU extensions
used. There shouldn't be any impact on other systems.
There are a couple of other minor fixes that I'd recommend but are not
essential.
I'm including the patch in this email (not sure if the mail list server
approves) incase you have an unpatched copy.
Fixes for Xymon
==========
The client script for a FreeBSD system includes a small C program to
extract memory data. This is a problem since we don't want to install a
program on the target machine. We can avoid including that section in
the client data if the program isn't available but then we don't get
memory tracking by xymon.
The reason for the program is that the installed memory on the system is
available as a number of bytes. With recent systems this often excedes a
32 bit number. The requirement is a memory size in MB and shell
arithmetic is limited to 32 bit integers. I've implemented two
workarounds for this:
1) A massive hack - drop the last three digits of the size and divide by
1048 to get the size in MB. This should be surprisingly good up to
memory sizes in TB. I've only tested to 32GB though.
2) Send the size in bytes back to xymon where it can properly process
the numbers as unsigned longs.
Xymon already uses "memory free" from a later section (vmtotal) so we
don't need to fake that up, although it also quite doable but commented
out in my fix.
The other diff file includes patches for both the FreeBSD client script,
and the xymond reader for freebsd client messages.
You have three choices depending on circumstances and how much you care:
1) Do nothing. You won't have a memory column. Remember to include a
NOCOLUMNS:memory tag on your pfSense machine in this case.
2) Just patch the client script. This should be good to just short of
2TB pfSense memory. pfSense doesn't need anything like that much memory.
3) Patch the script and the xymond code. Should be good up to a full 64
bit used address space. No nasty hacks involved.
[Side note: netbsd and openbsd also have client programs for the same
purpose. I don't have ready access to either of these systems to
develop/test equivalent patches.]
Other systems
=========
I've also tested this on a nanobsd system (the ssh stuff is a little
harder because of read-only file systems) and a dell server running my
hypervisor on a recent version of FreeBSD. Also found that msdosfs file
system on FreeBSD doesn't have inodes (I've normally handled this with
exclusions) so added it to the inodes exclusion list.
I hope this helps someone in the future,
Brian
P.S. If this email doesn't have the diff files attached, check for
another email following close behind with them included as the message body.
