Yes, this wasn't terribly hard at all, now was it? I feel dumb.

Server script (still need to beef it up with error checking):

export SOCKS5_USER=xxx export SOCKS5_PASSWD=xxx export DMZ_MACH=my.machine.dmz export TMPFILE=something_or_other runsocks scp ${DMZ_MACH}:tmp/msg.${DMZ_MACH}.txt ${TMPFILE} ${BB} $(BBDISP} "cat ${TMPFILE}" rm -f ${TMPFILE}

The one thing I appears to be missing doing things like this is that "[Client data available]" link on the generated webpages. I don't know why that's missing yet.

And I also need to investigate the possible problems of having the client script and the server script run asynchronously. It may be possible to upload the same msg.XXX.txt file twice in a row if timings/delays fall just right. I need to see how that would affect the RRD/graphing part of Hobbit. It might confuse the heck out of it.

I'm getting there in my quest for an ssh replacement for hobbitfetch. But comments and suggestions still greatly appreciated!

Thanks

From: Haertig, David F (Dave) [mailto:haertig at avaya.com] Sent: Friday, June 22, 2007 8:11 PM To: hobbit at hswn.dk Subject: RE: [hobbit] hobbitfetch replacement with ssh

Oops. Looks like I missed the obvious (again!).

bb accepts a message of type "client". What do you bet I've answered my own question here? (again!) I think I'll try a "client" message, and I bet I'll be pleasantly surprized.

And if anyone wants to know the answer to the question running around in your minds ... YES, it does hurt to be this stupid! Ouch! ;-)

Thanks

From: Haertig, David F (Dave) [mailto:haertig at avaya.com] Sent: Friday, June 22, 2007 7:27 PM To: hobbit at hswn.dk Subject: [hobbit] hobbitfetch replacement with ssh

I'm still thinking about my hobbitfetch/msgcache replacement I asked about a few weeks ago. Using only ssh from the Hobbit server. Client machine is out in DMZ, thus unable to talk to Hobbit server. I cannot run a listener (msgcache) on the client (policy reasons, not technical ones).

Here are my thoughts. Maybe someone can expand on them or suggest a different approach.

(1) Run normal Hobbit client on DMZ machine. (1a) Client appears to collect data just fine into ~hobbit/tmp/msg.XXX.txt (1b) Client would try to connect to Hobbit server to transmit this file, and fail. Fine. Let it fail. It appears to do so gracefully. Question: Any simple way to make it stop trying to connect and still behave gracefully? chmod 000 ~hobbit/bin/bb? Change BBDISP="" in ~hobbit/etc/hobbitclient.cfg?

(2) Create a Hobbit server script to use socks/ssh to retrieve that ~hobbit/tmp/msg.XXX.txt file from the client machine (it's trivial to write such a thing)

(3) Have this server script feed the raw data retrieved in step (2) into the Hobbit listener (which would be localhost:1984), such that it appears to the Hobbit listener the data came in from the client computer in the normal manner.

Step (3) is what I don't quite know how to implement yet. I'm familiar with using the bb command to send messages, but those are formatted messages, not raw data as would be present in the retrieved msg.XXX.tmp file. I want to assure that the guts of the Hobbit server processes this proxied client raw data just as if the client had sent it instead of my custom server script.

Is there some trivial way to accomplish (3) that I'm just missing searching through the man pages? I could have my custom server script parse the raw data itself and then create individual Hobbit messages (to send via the bb command), but that sounds like I'd be reinventing the wheel. Hobbit server processes already know how to parse this data ... if I could just tap into that part of it.

Thanks!

Good idea! I just need to see how hobbitlaunch normally passes ENVFILE and LOGFILE to CMD (hobbit-client.sh) when it normally envokes CMD. So I can do that when I invoke hobbit-cient.sh manually via ssh. Probably just as simple env variables. I'll check the source code for hobbitlaunch, or just "guess-and-test".

BTW, the clients appear to NOT attempt to connect to Hobbit server, and still behave gracefully, if you set BBDISP=0.0.0.0 and BBDISPLAYS="". At least the logfile quits complaining about that "Whoops!" message. And the msx.XXX.txt file is still generated normally. Great!

One last thing is to figure out why there aren't the normal "Client data available" links on the Hobbit webpages (not a show-stopper if they aren't there).

Thanks

-----Original Message----- From: Stef Coene [mailto:stef.coene at docum.org] Sent: Saturday, June 23, 2007 6:42 AM To: hobbit at hswn.dk Subject: Re: [hobbit] hobbitfetch replacement with ssh

On Saturday 23 June 2007, Haertig, David F (Dave) wrote:

And I also need to investigate the possible problems of having the client script and the server script run asynchronously. It may be possible to upload the same msg.XXX.txt file twice in a row if timings/delays fall just right. I need to see how that would affect the RRD/graphing part of Hobbit. It might confuse the heck out of it. Why not ssh to the client, execute hobbit-client.sh and pick up the resulting file ?

Stef

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

Cool! Thanks for the tip.

I was a bit naive in thinking hobbitlaunch might just pass ENVFILE and LOGFILE to CMD as env variables. I had just reviewed hobbitlaunch.c and environ.c and found that hobbitlaunch redirects stdout/stderr to LOGFILE before calling CMD. And also the ENVFILE processing is a little more complex due to the AREA stuff (which implies hobbitclient.cfg is not a sourceable env file, as it almost appears to be in it's default state).

I'll look at bbcmd. I was going to hack my own way through this, but if there's already an existing path, I'm all for the easy way!

Thanks

-----Original Message----- From: Stef Coene [mailto:stef.coene at docum.org] Sent: Saturday, June 23, 2007 11:55 AM To: hobbit at hswn.dk Subject: Re: [hobbit] hobbitfetch replacement with ssh

On Saturday 23 June 2007, Haertig, David F (Dave) wrote:

Good idea! I just need to see how hobbitlaunch normally passes ENVFILE and LOGFILE to CMD (hobbit-client.sh) when it normally envokes

CMD. So I can do that when I invoke hobbit-cient.sh manually via ssh.

Probably just as simple env variables. I'll check the source code for

hobbitlaunch, or just "guess-and-test". You can use bbcmd. So "bbcmd hobbitclient.sh". bbcmd will set the needed environment variabels. You probably have to specify the full path to hobbitclient.sh.

Stef

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

On Saturday 23 June 2007, Haertig, David F (Dave) wrote:

Using bbcmd to run hobbitclient.sh was a good idea, but presented some problems of it's own. No problem getting it to set the environment though.

For one, invoking the client in this manner leaves lots of iostatcpu, iostatdisk, and vmstat files laying around in the tmp directory. I guess it must be hobbitlaunch that cleans these up normally.

Second, I could not trick logfetch into grabbing custom logfiles. I tried putting the appropriate entries in ..../tmp/logfetch.XXX.cfg on the client end, but logfetch ignored these. This file is normally overwritten each time the client contacts the host, so normally you wouldn't edit it. But in my case the client is not connecting to the host so I thought it might be "safe" to manually create this file, but alas logfetch appears to ignore it.

I may well run into this logfetch gotcha no matter how I invoke hobbitclient.sh If the client cannot talk to the Hobbit server, as in my DMZ case, is it even possible to configure logfetch to grab custom logfiles? The logfetch config is normally downloaded from the server. I don't see any way (yet) to configure it locally on the client. If "I" should have this problem, I will try this: change hobbit client.sh so it:

• is not trying to contact the hobbit server (so the bb command is not used)
• the message file is saved in a fixed file (so the pid is not appended to it)
• the logfetch can still work if you update the input file manually

make a daemon on the hobbit server:

• that runs every minute
• contacts the hobbit client with ssh and checks for a message file, if a message file is found, the content is send to the hobbit server and the file is deleted

Or something like this ;)

Stef

Below is what I ended up doing. It's working like a champ. It probably doesn't fit your environment exactly, but it may be of some help anyway.

My Hobbit server is Linux 2.6, the clients I'm pulling data from are Solaris 9. All running OpenSSH. The client domains are picked up on the Hobbit server via /etc/resolv.conf mechanism. Hobbit server has socks client installed and configured for our environment. ssh public keys were distributed from host to clients so ssh/scp will work without passwords being required.

The file "hobbitclient-custom.sh" you see below is nothing more than the standard hobbitclient.sh file with some stuff I didn't want stripped out. I modified the LOGFETCHCFG line to point to the "scripts" directory rather than the default "tmp" directory because I wanted to keep all my customized stuff in that scripts directory.

Custom HOST script, "pulldata.sh", found on server in $BBHOME/ext directory, run via clientlaunch.cfg

============================= export SOCKS5_USER='my_socks_userid' export SOCKS5_PASSWD='my_socks_password' USER=stars for DMZ_MACH in 'machine1' 'machine2' 'machine3' do TMPFILE=${BBTMP}/pulldata.${DMZ_MACH}.tmp.$$ runsocks ssh ${USER}@${DMZ_MACH} hobbit/scripts/exec_client.sh runsocks scp ${USER}@${DMZ_MACH}:~${USER}/hobbit/tmp/msg.${DMZ_MACH}.txt ${TMPFILE} ${BB} ${BBDISP} "cat ${TMPFILE}" rm -f ${TMPFILE} done

Custom CLIENT script, "exec_client.sh", found on client in /home/stars/hobbit/scripts directory

======================= #!/bin/sh BBHOME=/home/stars/hobbit BBCMD=${BBHOME}/bin/bbcmd CMD=${BBHOME}/scripts/hobbitclient-custom.sh ENVFILE=${BBHOME}/etc/hobbitclient.cfg LOGFILE=${BBHOME}/logs/hobbitclient-custom.log ${BBCMD} --env=${ENVFILE} ${CMD} >${LOGFILE} 2>&1 exit 0

Custom CLIENT script, "hobbitclient-custom.sh", found on client in /home/stars/hobbit/scripts directory

=============================== #!/bin/sh LANG=C LC_ALL=C LC_MESSAGES=C export LANG LC_ALL LC_MESSAGES if test "$BBOSSCRIPT" = ""; then BBOSSCRIPT="hobbitclient-uname -s | tr '[A-Z]' '[a-z]'.sh" fi MSGFILE="$BBTMP/msg.$MACHINEDOTS.txt" MSGTMPFILE="$MSGFILE.$$" LOGFETCHCFG=$BBHOME/scripts/logfetch.$MACHINEDOTS.cfg LOGFETCHSTATUS=$BBTMP/logfetch.$MACHINEDOTS.status export LOGFETCHCFG LOGFETCHSTATUS rm -f $MSGTMPFILE touch $MSGTMPFILE echo "client $MACHINE.$BBOSTYPE $CONFIGCLASS" >>$MSGTMPFILE $BBHOME/bin/$BBOSSCRIPT >> $MSGTMPFILE if test -f $LOGFETCHCFG then $BBHOME/bin/logfetch $LOGFETCHCFG $LOGFETCHSTATUS >>$MSGTMPFILE fi echo "[clock]" >>$MSGTMPFILE $BBHOME/bin/logfetch --clock >>$MSGTMPFILE rm -f $MSGFILE mv $MSGTMPFILE $MSGFILE exit 0

Custom CLIENT file, "logfetch.machine1.txt", found on client in /home/stars/hobbit/scripts directory

============================ log:/my_app/logs/file1:4096 log:/my_app/logs/file2:4096 log:/my_app/logs/file3:4096

-----Original Message----- From: John G [mailto:jg2727 at gmail.com] Sent: Monday, June 25, 2007 8:30 PM To: hobbit at hswn.dk Subject: Re: [hobbit] hobbitfetch replacement with ssh

Here are some old messages on the topic that are similar to what you are doing.

http://www.hswn.dk/hobbiton/2006/01/msg00045.html http://www.hswn.dk/hobbiton/2006/01/msg00047.html

On 6/23/07, Stef Coene <stef.coene at docum.org> wrote:

On Saturday 23 June 2007, Haertig, David F (Dave) wrote:

...

Using bbcmd to run hobbitclient.sh was a good idea, but presented some problems of it's own. No problem getting it to set the environment though.

For one, invoking the client in this manner leaves lots of iostatcpu, iostatdisk, and vmstat files laying around in the tmp directory. I guess it must be hobbitlaunch that cleans these up normally.

Second, I could not trick logfetch into grabbing custom logfiles. I

...

tried putting the appropriate entries in ..../tmp/logfetch.XXX.cfg on the client end, but logfetch ignored these. This file is normally overwritten each time the client contacts the host, so normally you wouldn't edit it. But in my case the client is not connecting to the host so I thought it might be "safe" to manually create this file, but alas logfetch appears to ignore it.

I may well run into this logfetch gotcha no matter how I invoke hobbitclient.sh If the client cannot talk to the Hobbit server, as in my DMZ case, is it even possible to configure logfetch to grab custom logfiles? The logfetch config is normally downloaded from the server. I don't see any way (yet) to configure it locally on the client. If "I" should have this problem, I will try this: change hobbit client.sh so it:

• is not trying to contact the hobbit server (so the bb command is not

used)

• the message file is saved in a fixed file (so the pid is not appended to it)
• the logfetch can still work if you update the input file manually

make a daemon on the hobbit server:

• that runs every minute
• contacts the hobbit client with ssh and checks for a message file, if a message file is found, the content is send to the hobbit server and the file is deleted

Or something like this ;)

Stef

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

p.s. - I posted these scripts a monent ago. I see in the email replay that my cut-n-paste did not work 100%. It appears that everything is there, but some newlines were lost in a few places. e.g., you will see:

${BBCMD} --env=${ENVFILE} ${CMD} >${LOGFILE} 2>&1 exit 0

... but if you know shell scripts, you'll recognize that the "exit 0" part should be on a line by itself. I do not know why cut-n-paste didn't work right. I guess some Linux/Solaris/Windows/Thunderbird glitch. There are a few other places where commands were run-on like the above example. Watch for them, and manually fix them.

Sorry about that!

-----Original Message----- From: Haertig, David F (Dave) Sent: Monday, June 25, 2007 9:20 PM To: 'hobbit at hswn.dk' Subject: RE: [hobbit] hobbitfetch replacement with ssh

Below is what I ended up doing. It's working like a champ. It probably doesn't fit your environment exactly, but it may be of some help anyway.

My Hobbit server is Linux 2.6, the clients I'm pulling data from are Solaris 9. All running OpenSSH. The client domains are picked up on the Hobbit server via /etc/resolv.conf mechanism. Hobbit server has socks client installed and configured for our environment. ssh public keys were distributed from host to clients so ssh/scp will work without passwords being required.

The file "hobbitclient-custom.sh" you see below is nothing more than the standard hobbitclient.sh file with some stuff I didn't want stripped out. I modified the LOGFETCHCFG line to point to the "scripts" directory rather than the default "tmp" directory because I wanted to keep all my customized stuff in that scripts directory.

Custom HOST script, "pulldata.sh", found on server in $BBHOME/ext directory, run via clientlaunch.cfg

============================= export SOCKS5_USER='my_socks_userid' export SOCKS5_PASSWD='my_socks_password' USER=stars for DMZ_MACH in 'machine1' 'machine2' 'machine3' do TMPFILE=${BBTMP}/pulldata.${DMZ_MACH}.tmp.$$ runsocks ssh ${USER}@${DMZ_MACH} hobbit/scripts/exec_client.sh runsocks scp ${USER}@${DMZ_MACH}:~${USER}/hobbit/tmp/msg.${DMZ_MACH}.txt ${TMPFILE} ${BB} ${BBDISP} "cat ${TMPFILE}" rm -f ${TMPFILE} done

Custom CLIENT script, "exec_client.sh", found on client in /home/stars/hobbit/scripts directory

======================= #!/bin/sh BBHOME=/home/stars/hobbit BBCMD=${BBHOME}/bin/bbcmd CMD=${BBHOME}/scripts/hobbitclient-custom.sh ENVFILE=${BBHOME}/etc/hobbitclient.cfg LOGFILE=${BBHOME}/logs/hobbitclient-custom.log ${BBCMD} --env=${ENVFILE} ${CMD} >${LOGFILE} 2>&1 exit 0

Custom CLIENT script, "hobbitclient-custom.sh", found on client in /home/stars/hobbit/scripts directory

=============================== #!/bin/sh LANG=C LC_ALL=C LC_MESSAGES=C export LANG LC_ALL LC_MESSAGES if test "$BBOSSCRIPT" = ""; then BBOSSCRIPT="hobbitclient-uname -s | tr '[A-Z]' '[a-z]'.sh" fi MSGFILE="$BBTMP/msg.$MACHINEDOTS.txt" MSGTMPFILE="$MSGFILE.$$" LOGFETCHCFG=$BBHOME/scripts/logfetch.$MACHINEDOTS.cfg LOGFETCHSTATUS=$BBTMP/logfetch.$MACHINEDOTS.status export LOGFETCHCFG LOGFETCHSTATUS rm -f $MSGTMPFILE touch $MSGTMPFILE echo "client $MACHINE.$BBOSTYPE $CONFIGCLASS" >>$MSGTMPFILE $BBHOME/bin/$BBOSSCRIPT >> $MSGTMPFILE if test -f $LOGFETCHCFG then $BBHOME/bin/logfetch $LOGFETCHCFG $LOGFETCHSTATUS >>$MSGTMPFILE fi echo "[clock]" >>$MSGTMPFILE $BBHOME/bin/logfetch --clock >>$MSGTMPFILE rm -f $MSGFILE mv $MSGTMPFILE $MSGFILE exit 0

Custom CLIENT file, "logfetch.machine1.txt", found on client in /home/stars/hobbit/scripts directory

============================ log:/my_app/logs/file1:4096 log:/my_app/logs/file2:4096 log:/my_app/logs/file3:4096

-----Original Message----- From: John G [mailto:jg2727 at gmail.com] Sent: Monday, June 25, 2007 8:30 PM To: hobbit at hswn.dk Subject: Re: [hobbit] hobbitfetch replacement with ssh

Here are some old messages on the topic that are similar to what you are doing.

http://www.hswn.dk/hobbiton/2006/01/msg00045.html http://www.hswn.dk/hobbiton/2006/01/msg00047.html

On 6/23/07, Stef Coene <stef.coene at docum.org> wrote:

On Saturday 23 June 2007, Haertig, David F (Dave) wrote:

...

Using bbcmd to run hobbitclient.sh was a good idea, but presented some problems of it's own. No problem getting it to set the environment though.

For one, invoking the client in this manner leaves lots of iostatcpu, iostatdisk, and vmstat files laying around in the tmp directory. I guess it must be hobbitlaunch that cleans these up normally.

Second, I could not trick logfetch into grabbing custom logfiles. I

...

tried putting the appropriate entries in ..../tmp/logfetch.XXX.cfg on the client end, but logfetch ignored these. This file is normally overwritten each time the client contacts the host, so normally you wouldn't edit it. But in my case the client is not connecting to the host so I thought it might be "safe" to manually create this file, but alas logfetch appears to ignore it.

I may well run into this logfetch gotcha no matter how I invoke hobbitclient.sh If the client cannot talk to the Hobbit server, as in my DMZ case, is it even possible to configure logfetch to grab custom logfiles? The logfetch config is normally downloaded from the server. I don't see any way (yet) to configure it locally on the client. If "I" should have this problem, I will try this: change hobbit client.sh so it:

• is not trying to contact the hobbit server (so the bb command is not used)
• the message file is saved in a fixed file (so the pid is not appended to it)
• the logfetch can still work if you update the input file manually

make a daemon on the hobbit server:

• that runs every minute
• contacts the hobbit client with ssh and checks for a message file, if a message file is found, the content is send to the hobbit server and the file is deleted

Or something like this ;)

Stef

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk