Hi,
We solved the encryption by using a wget and/or curl script alternative for the xymon client. It's a drop-in replacement. We have a setup script that checks if wget or cups exists and it creates a symlink for the xymon command to the script that works. The script uses a username and password to connect to xymoncgimsg.cgi over https to send the data. We use 1 username / paswoord for all clients but with some scripting you can give each client it's own username / password.
If wget or cups is not available (some old AIX servers can not connect to a https server...), the good old binary is used without encryption :(
I can document it somewhere if anyone is interested. It's not that complicated.
For the Windows clients we also use https as much as possible.
I have some patches for the Powershell client that allows for testing a new XML file. We want to manage the XML files centrally and want to avoid mistakes that can disable the client. So this allows us to test a new config file before overwriting the old one. I also added the ping test command for this and did some changes so data can be send to multiple Xymon servers. I will create some patches and send them to the mailinglist.
Stef
Stef
Nice idea on the Powershell patch. We don't monitor Windows hosts where I use Xymon, but others in this list might benefit.
Is the PS client able to use https natively, or do you need to install curl.exe on those hosts?
I'm interested in the wget/curl option. Have you had an opportunity to compare performance impact on the server running xymond and the CGI script vs the native TCP/1984 method? This is perhaps my biggest concern, knowing that xymond is written with high-performance socket handling, and although Apache also has code optimised thusly, the CGI script execution might become a bottleneck. I fear stunnel and similar solutions would suffer from the same problem, due to (I assume) being written for low socket turn-over environments.
Does the CGI provide the client local data (from client-local.cfg) to the client?
I'd be interested to see a HOWTO on this, even though it seems simple enough. If the CGI option works out for us, we'll be able to cut over a large proportion of hosts.
However we have a sizeable proportion of hosts that cannot make connections to the Xymon server, and so we have to do funky things like ssh from xymons server to client, setup a reverse tunnel and adjust XYMSRV to 127.0.0.1. It's encrypted, but it's not particularly clean, and we have to all sorts of extra hacks to support dual Xymon servers (so that the client message tempfile for one Xymon server isn't stomped on by the second instance running for the other Xymon server, and so the reverse tunnels don't interfere with each others' listening ports). I'm investigating using msgcache/xymonfetch option designed for server->client connections, but it's non-trivial to have it work through an ssh tunnel, and so far I've not succeeded.
J
On Wed, 23 Aug 2023 at 22:02, Stef Coene <stef.coene at docum.org> wrote:
Hi,
We solved the encryption by using a wget and/or curl script alternative for the xymon client. It's a drop-in replacement. We have a setup script that checks if wget or cups exists and it creates a symlink for the xymon command to the script that works. The script uses a username and password to connect to xymoncgimsg.cgi over https to send the data. We use 1 username / paswoord for all clients but with some scripting you can give each client it's own username / password.
If wget or cups is not available (some old AIX servers can not connect to a https server...), the good old binary is used without encryption :(
I can document it somewhere if anyone is interested. It's not that complicated.
For the Windows clients we also use https as much as possible.
I have some patches for the Powershell client that allows for testing a new XML file. We want to manage the XML files centrally and want to avoid mistakes that can disable the client. So this allows us to test a new config file before overwriting the old one. I also added the ping test command for this and did some changes so data can be send to multiple Xymon servers. I will create some patches and send them to the mailinglist.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
On Wed, August 23, 2023 16:41, Jeremy Laidman wrote:
Stef
Nice idea on the Powershell patch. We don't monitor Windows hosts where I use Xymon, but others in this list might benefit.
Is the PS client able to use https natively, or do you need to install curl.exe on those hosts?
I'm interested in the wget/curl option. Have you had an opportunity to compare performance impact on the server running xymond and the CGI script vs the native TCP/1984 method? This is perhaps my biggest concern, knowing that xymond is written with high-performance socket handling, and although Apache also has code optimised thusly, the CGI script execution might become a bottleneck. I fear stunnel and similar solutions would suffer from the same problem, due to (I assume) being written for low socket turn-over environments.
Does the CGI provide the client local data (from client-local.cfg) to the client?
I'd be interested to see a HOWTO on this, even though it seems simple enough. If the CGI option works out for us, we'll be able to cut over a large proportion of hosts.
However we have a sizeable proportion of hosts that cannot make connections to the Xymon server, and so we have to do funky things like ssh from xymons server to client, setup a reverse tunnel and adjust XYMSRV to 127.0.0.1. It's encrypted, but it's not particularly clean, and we have to all sorts of extra hacks to support dual Xymon servers (so that the client message tempfile for one Xymon server isn't stomped on by the second instance running for the other Xymon server, and so the reverse tunnels don't interfere with each others' listening ports). I'm investigating using msgcache/xymonfetch option designed for server->client connections, but it's non-trivial to have it work through an ssh tunnel, and so far I've not succeeded.
The two-way communication for the client message is indeed an obstacle, since a full conversation with xymond is necessary, but only for really high-concurrency situations.
The Terabithia RPM has a STATUSMODE option that leveraged a bb-compatibility hack in xymond which allowed client messages to be one-way. This was updated to a SUBMITMODE option ~4.3.19 with some server patches to accept new types of client(submit/config) commands. This was added into the 4.x branch at https://sourceforge.net/p/xymon/code/7852/ and should be working in 4.4
Basically, both of these (SUBMITMODE preferred now) make client messages much more efficient for processing, either by the .cgi or by xymonproxy (which does much the same thing). This also means the xymon binary can be forked by xymonclient.sh when submitting the message (we don't need a response), which helps if you're sending to multiple servers at once.
As to how to actually download the client-local.cfg snippet? In the RPM, this is handled via a separate client task that once every 30m downloads client-local.cfg instead (with a null client body, so the connection is just like any other single-line command). I need to double-check that this script made it into SF, however.
A better recommendation in a truly large farm would probably be to have this managed out-of-band by a config mgmt system, but lowering the frequency of two-way comms to 30m seemed a reasonable middle ground.
Unfortunately, yeah; the encryption, SSL, and massive-scale performance questions are all a bit intertwined still.
Regards, -jc
J
On Wed, 23 Aug 2023 at 22:02, Stef Coene <stef.coene at docum.org> wrote:
Hi,
We solved the encryption by using a wget and/or curl script alternative for the xymon client. It's a drop-in replacement. We have a setup script that checks if wget or cups exists and it creates a symlink for the xymon command to the script that works. The script uses a username and password to connect to xymoncgimsg.cgi over https to send the data. We use 1 username / paswoord for all clients but with some scripting you can give each client it's own username / password.
If wget or cups is not available (some old AIX servers can not connect to a https server...), the good old binary is used without encryption :(
I can document it somewhere if anyone is interested. It's not that complicated.
For the Windows clients we also use https as much as possible.
I have some patches for the Powershell client that allows for testing a new XML file. We want to manage the XML files centrally and want to avoid mistakes that can disable the client. So this allows us to test a new config file before overwriting the old one. I also added the ping test command for this and did some changes so data can be send to multiple Xymon servers. I will create some patches and send them to the mailinglist.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
I've used Powershell to talk to an Apache server. It was a while back, but I'm pretty sure it used https builtins. I certainly didn't have access to install curl or anything else.
As for curl->cgi, I'm using that on far too many clients. I'm not sure of the exact number, but it's probably 2500+. I could probably do a better job configuring Apache, if I had any clue what I was doing. What I see is that occasionally one Apache socket doesn't close properly and the next incoming connection picks it up, so the second report gets awarded to the first reporter.
I have some servers where I can't install the RPMs or configure anything to run the client and report to the Xymon server. What I did there was copy over the various client bits and then from another machine did something like this:
ssh -q <server> /opt/xymon/bin/xymonclient.sh | /home/xymon/bin/xymon<xymonserver> "@"
I'm off work this week, so I can't go look up exactly what I have working, but it's similar to the above. SSH keys on the remote machine, then ssh over to run the command and pass any output through the xymon command to the xymon server. The message from the remote machine has its own name, so it gets reported properly. I don't think the client-local.cg file gets shipped back, but I'd need to go look to be sure.
Ralph Mitchell
On Thu, Aug 24, 2023 at 12:12?AM J.C. Cleaver <cleaver at terabithia.org> wrote:
On Wed, August 23, 2023 16:41, Jeremy Laidman wrote:
Stef
Nice idea on the Powershell patch. We don't monitor Windows hosts where I use Xymon, but others in this list might benefit.
Is the PS client able to use https natively, or do you need to install curl.exe on those hosts?
I'm interested in the wget/curl option. Have you had an opportunity to compare performance impact on the server running xymond and the CGI script vs the native TCP/1984 method? This is perhaps my biggest concern, knowing that xymond is written with high-performance socket handling, and although Apache also has code optimised thusly, the CGI script execution might become a bottleneck. I fear stunnel and similar solutions would suffer from the same problem, due to (I assume) being written for low socket turn-over environments.
Does the CGI provide the client local data (from client-local.cfg) to the client?
I'd be interested to see a HOWTO on this, even though it seems simple enough. If the CGI option works out for us, we'll be able to cut over a large proportion of hosts.
However we have a sizeable proportion of hosts that cannot make connections to the Xymon server, and so we have to do funky things like ssh from xymons server to client, setup a reverse tunnel and adjust XYMSRV to 127.0.0.1. It's encrypted, but it's not particularly clean, and we have to all sorts of extra hacks to support dual Xymon servers (so that the client message tempfile for one Xymon server isn't stomped on by the second instance running for the other Xymon server, and so the reverse tunnels don't interfere with each others' listening ports). I'm investigating using msgcache/xymonfetch option designed for server->client connections, but it's non-trivial to have it work through an ssh tunnel, and so far I've not succeeded.
The two-way communication for the client message is indeed an obstacle, since a full conversation with xymond is necessary, but only for really high-concurrency situations.
The Terabithia RPM has a STATUSMODE option that leveraged a bb-compatibility hack in xymond which allowed client messages to be one-way. This was updated to a SUBMITMODE option ~4.3.19 with some server patches to accept new types of client(submit/config) commands. This was added into the 4.x branch at https://sourceforge.net/p/xymon/code/7852/ and should be working in 4.4
Basically, both of these (SUBMITMODE preferred now) make client messages much more efficient for processing, either by the .cgi or by xymonproxy (which does much the same thing). This also means the xymon binary can be forked by xymonclient.sh when submitting the message (we don't need a response), which helps if you're sending to multiple servers at once.
As to how to actually download the client-local.cfg snippet? In the RPM, this is handled via a separate client task that once every 30m downloads client-local.cfg instead (with a null client body, so the connection is just like any other single-line command). I need to double-check that this script made it into SF, however.
A better recommendation in a truly large farm would probably be to have this managed out-of-band by a config mgmt system, but lowering the frequency of two-way comms to 30m seemed a reasonable middle ground.
Unfortunately, yeah; the encryption, SSL, and massive-scale performance questions are all a bit intertwined still.
Regards, -jc
J
On Wed, 23 Aug 2023 at 22:02, Stef Coene <stef.coene at docum.org> wrote:
Hi,
We solved the encryption by using a wget and/or curl script alternative for the xymon client. It's a drop-in replacement. We have a setup script that checks if wget or cups exists and it creates a symlink for the xymon command to the script that works. The script uses a username and password to connect to xymoncgimsg.cgi over https to send the data. We use 1 username / paswoord for all clients but with some scripting you can give each client it's own username / password.
If wget or cups is not available (some old AIX servers can not connect to a https server...), the good old binary is used without encryption :(
I can document it somewhere if anyone is interested. It's not that complicated.
For the Windows clients we also use https as much as possible.
I have some patches for the Powershell client that allows for testing a new XML file. We want to manage the XML files centrally and want to avoid mistakes that can disable the client. So this allows us to test a new config file before overwriting the old one. I also added the ping test command for this and did some changes so data can be send to multiple Xymon servers. I will create some patches and send them to the mailinglist.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Hi,
If anyone is interested, I documented the steps I did to use wget and curl as alternative: https://www.docum.org/drupal/content/client-server-encryption
@Jeremy Laidman: yes, data from client-local.cfg is also provided. In fact, when a client connects to the server, the data returned is the relevant lines from client-local.cfg
Stef
On 2023-08-23 13:57, Stef Coene wrote:
Hi,
We solved the encryption by using a wget and/or curl script alternative for the xymon client. It's a drop-in replacement. We have a setup script that checks if wget or cups exists and it creates a symlink for the xymon command to the script that works. The script uses a username and password to connect to xymoncgimsg.cgi over https to send the data. We use 1 username / paswoord for all clients but with some scripting you can give each client it's own username / password.
If wget or cups is not available (some old AIX servers can not connect to a https server...), the good old binary is used without encryption :(
I can document it somewhere if anyone is interested. It's not that complicated.
For the Windows clients we also use https as much as possible.
I have some patches for the Powershell client that allows for testing a new XML file. We want to manage the XML files centrally and want to avoid mistakes that can disable the client. So this allows us to test a new config file before overwriting the old one. I also added the ping test command for this and did some changes so data can be send to multiple Xymon servers. I will create some patches and send them to the mailinglist.
Stef
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
participants (4)
-
cleaver＠terabithia.org
-
jeremy＠laidman.org
-
ralphmitchell＠gmail.com
-
stef.coene＠docum.org