Any ideas on how to solve the following problem.
hamilton is shown as ssh ok, status unchanged for a week, but you can't ssh in:
% ssh -v hamilton OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008 debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug1: Connecting to hamilton [128.210.3.42] port 22. debug1: Connection established. debug1: identity file /homes/jflack/.ssh/identity type -1 debug1: identity file /homes/jflack/.ssh/id_rsa type -1 debug1: identity file /homes/jflack/.ssh/id_dsa type -1 debug1: loaded 3 keys debug1: Remote protocol version 1.99, remote software version OpenSSH_3.9p1 debug1: match: OpenSSH_3.9p1 pat OpenSSH_3.* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_4.3 debug1: SSH2_MSG_KEXINIT sent Read from socket failed: Connection reset by peer
Apparently something goes wrong in the server just at the start of key exchange. The xymon ssh test reports the remote protocol and software versions, so it must converse at least that far, but I guess it doesn't go on through the key exchange.
The ssh server going wrong that way seems to be a familiar failure mode for our linux boxes, so it would be nice to have a test for it in xymon. An ssh identity that's allowed to run some single very restricted command would work. Actually ssh-keyscan does the trick too, and doesn't require any logging in or permission on the host. The only trick is ssh-keyscan exits with status 0 whether it succeeded or not, so it would have to be used in a script that actually parses its output to see if it worked.
Dustin's script that does ssh-keyscan for updating the keys list could be a useful starting point.
Robert P. McGraw, Jr.
Manager, Computer System EMAIL: rmcgraw at purdue.edu
Purdue University ROOM: MATH-807
Department of Mathematics PHONE: (765) 494-6055
150 N. University Street
West Lafayette, IN 47907-2067
Why not use simply ssh %host% cat /etc/SuSE-release
mfg
Andreas
--
Andreas Kunberger Körschtalstraße 26, 73770 Denkendorf
What is something that can be executed to make that universal? Is /etc/*release on all *nix systems? I know it is on rhel but that's it.
On 6/11/10, Andreas Kunberger <andreas.kunberger at itv-denkendorf.de> wrote:
Why not use simply ssh %host% cat /etc/SuSE-release
mfg
Andreas
--
Andreas Kunberger Körschtalstraße 26, 73770 Denkendorf
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
“Success is not final, failure is not fatal: it is the courage to continue that counts.” --- Winston Churchill
You need something that's on all *nix systems. /etc/passwd /etc/hosts or even /etc/motd
But why use cat? ssh $HOST ls -la /etc/ will do just as well. ssh $HOST ls -la /tmp is probably even less of a risk.
If you are testing ssh, any command will do. ssh $HOST ls -d /tmp is probably best. You know what the result will be, on all unix systems, and if you get "/tmp" then ssh works.
Cheers Vernon
On Fri, Jun 11, 2010 at 2:32 PM, Josh Luthman <josh at imaginenetworksllc.com>wrote:
What is something that can be executed to make that universal? Is /etc/*release on all *nix systems? I know it is on rhel but that's it.
On 6/11/10, Andreas Kunberger <andreas.kunberger at itv-denkendorf.de> wrote:
Why not use simply ssh %host% cat /etc/SuSE-release
mfg
Andreas
--
Andreas Kunberger Körschtalstraße 26, 73770 Denkendorf
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
“Success is not final, failure is not fatal: it is the courage to continue that counts.” --- Winston Churchill
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Am 11.06.2010 08:32, schrieb Josh Luthman:
What is something that can be executed to make that universal? Is /etc/*release on all *nix systems? I know it is on rhel but that's it.
'All' ist difficult. You could of course create a link in /etc, do a 'grep CLIENTHOSTNAME /etc/sysconfig/hobbit' or 'ls /etc/passwd'.
mfg
Andreas
Andreas Kunberger Körschtalstraße 26, 73770 Denkendorf
I like:
ls -d /tmp
Same result, always and everywhere.
On 6/11/10, Andreas Kunberger <andreas.kunberger at itv-denkendorf.de> wrote:
Am 11.06.2010 08:32, schrieb Josh Luthman:
What is something that can be executed to make that universal? Is /etc/*release on all *nix systems? I know it is on rhel but that's it.
'All' ist difficult. You could of course create a link in /etc, do a 'grep CLIENTHOSTNAME /etc/sysconfig/hobbit' or 'ls /etc/passwd'.
mfg
Andreas
Andreas Kunberger Körschtalstraße 26, 73770 Denkendorf
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
“Success is not final, failure is not fatal: it is the courage to continue that counts.” --- Winston Churchill
Simplicity is the ultimate sophistication. -- Leonardo da Vinci
:-)
On Fri, Jun 11, 2010 at 2:52 PM, Josh Luthman <josh at imaginenetworksllc.com>wrote:
I like:
ls -d /tmp
Same result, always and everywhere.
On 6/11/10, Andreas Kunberger <andreas.kunberger at itv-denkendorf.de> wrote:
Am 11.06.2010 08:32, schrieb Josh Luthman:
What is something that can be executed to make that universal? Is /etc/*release on all *nix systems? I know it is on rhel but that's it.
'All' ist difficult. You could of course create a link in /etc, do a 'grep CLIENTHOSTNAME /etc/sysconfig/hobbit' or 'ls /etc/passwd'.
mfg
Andreas
Andreas Kunberger Körschtalstraße 26, 73770 Denkendorf
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
“Success is not final, failure is not fatal: it is the courage to continue that counts.” --- Winston Churchill
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Friday, 11 June 2010 07:15:06 Andreas Kunberger wrote:
Why not use simply ssh %host% cat /etc/SuSE-release
Maybe just run a command that is likely to be present on almost any host running ssh.
For example, almost any unix should have 'uptime' or 'true'.
Then again, cisco devices don't have that, but they do have 'who'.
On Thursday, 10 June 2010 18:35:33 McGraw, Robert P wrote:
Any ideas on how to solve the following problem.
hamilton is shown as ssh ok, status unchanged for a week, but you can't ssh in:
% ssh -v hamilton OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008 debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug1: Connecting to hamilton [128.210.3.42] port 22. debug1: Connection established. debug1: identity file /homes/jflack/.ssh/identity type -1 debug1: identity file /homes/jflack/.ssh/id_rsa type -1 debug1: identity file /homes/jflack/.ssh/id_dsa type -1 debug1: loaded 3 keys debug1: Remote protocol version 1.99, remote software version OpenSSH_3.9p1
This is quite an old version. Time to consider an upgrade?
debug1: match: OpenSSH_3.9p1 pat OpenSSH_3.* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_4.3 debug1: SSH2_MSG_KEXINIT sent Read from socket failed: Connection reset by peer
Apparently something goes wrong in the server just at the start of key exchange. The xymon ssh test reports the remote protocol and software versions, so it must converse at least that far, but I guess it doesn't go on through the key exchange.
The ssh server going wrong that way seems to be a familiar failure mode for our linux boxes,
In quite a few years in production environments with hundreds of linux servers, I haven't seen that myself ...
Have you managed to find a way to reproduce it? Have you filed a bug? IOW, maybe prevention of the problem is better than identification.
Regards, Buchan
Hi, I'm the guy who documented the original issue Robert forwarded.
On Friday, 11 June 2010 Buchan Milne wrote:
This is quite an old version. Time to consider an upgrade?
Red Hat Enterprise is very conservative about switching package versions during the lifetime of a single RHEL major release, though they do frequently issue backported patches. We tend to avoid replacing the shipped packages unless we can't help it.
debug1: match: OpenSSH_3.9p1 pat OpenSSH_3.* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_4.3 debug1: SSH2_MSG_KEXINIT sent Read from socket failed: Connection reset by peer
Apparently something goes wrong in the server just at the start of key exchange. The xymon ssh test reports the remote protocol and software versions, so it must converse at least that far, but I guess it doesn't go on through the key exchange.
In quite a few years in production environments with hundreds of linux servers, I haven't seen that myself ...
Have you managed to find a way to reproduce it? Have you filed a bug? IOW, maybe prevention of the problem is better than identification.
It won't be that easy to prevent until the kinks are worked out of RHEL's NFSv4 state recovery code. I.e., it's not some bug in sshd we are talking about. At the times in question, the box is partially moribund: some kernel services are deadlocked, but the sshd is still able to run just enough to accept connections and get as far as key exchange but no further. The details don't matter that much; the only point of reporting the issue on this list was to point out that xymon's current ssh test doesn't confirm the complete handshake, and maybe it would be a stronger test if it did.
...
Most of the suggestions I've seen so far on the list seem to involve running some arbitrary command on the host. That was my first idea too, but it needs some cautions.
that requires giving xymon an identity with which to log in to the host. To avoid saving a password, that should be done with a public/ private key pair, the public key being installed in authorized_keys on each machine to be tested.
the identity should not be allowed to run arbitrary commands. an entry in authorized_keys can be limited to running a single fixed command.
the discussions of whether certain commands or files are present everywhere still seem Unix-centric; you can get ssh daemons for other OSes too, which may or may not even have something called /tmp etc.
because a fixed command should be assigned for the identity key anyway, OS heterogeneity isn't such a big problem; the fixed command could just be "echo OK" or whatever the equivalent is on another OS.
However, because that approach does require extra setup on each client machine (create a uid or pick some existing one like nobody, set up authorized_keys to accept the xymon identity and run the fixed command), an approach based on ssh-keyscan might be a reasonable compromise. It doesn't require extra setup on the host or permission to log in, and it does confirm at least that enough of the OS is working for the ssh daemon to retrieve its keys.
Chapman Flack
On Fri, June 11, 2010 09:30, chap at anastigmatix.net wrote:
- the identity should not be allowed to run arbitrary commands. an entry in authorized_keys can be limited to running a single fixed command.
Just give the identity a login shell of /bin/true in /etc/passwd and you won't have to be concerned about commands from a shell at all.
On Fri, June 11, 2010 09:30, chap at anastigmatix.net wrote: Just give the identity a login shell of /bin/true in /etc/passwd and you won't have to be concerned about commands from a shell at all.
Yes, that works too, if you will create a new dedicated identity (or reuse one that already has true for a shell). command="/bin/true" in authorized_keys will work in any event (though something like /bin/echo OK might give a more positive confirmation).
The line in authorized_keys should also disallow all the extra goodies like port forwarding, X tunneling, and so on.
-Chap
On Fri, Jun 11, 2010 at 11:21 AM, Xymon User in Richmond < hobbit at epperson.homelinux.net> wrote:
On Fri, June 11, 2010 09:30, chap at anastigmatix.net wrote:
- the identity should not be allowed to run arbitrary commands. an entry in authorized_keys can be limited to running a single fixed command.
Just give the identity a login shell of /bin/true in /etc/passwd and you won't have to be concerned about commands from a shell at all.
You can also use a command such as /bin/hostname - that would give you a way to verify you reached the target system.
Ralph Mitchell
On Fri, June 11, 2010 12:41, Ralph Mitchell wrote:
On Fri, Jun 11, 2010 at 11:21 AM, Xymon User in Richmond < hobbit at epperson.homelinux.net> wrote:
On Fri, June 11, 2010 09:30, chap at anastigmatix.net wrote:
- the identity should not be allowed to run arbitrary commands. an entry in authorized_keys can be limited to running a single fixed command.
Just give the identity a login shell of /bin/true in /etc/passwd and you won't have to be concerned about commands from a shell at all.
You can also use a command such as /bin/hostname - that would give you a way to verify you reached the target system.
/bin/true will return exit 0. If you don't get that far, ssh will return nonzero.
participants (8)
-
andreas.kunberger＠itv-denkendorf.de
-
bgmilne＠staff.telkomsa.net
-
chap＠anastigmatix.net
-
everett.vernon＠gmail.com
-
hobbit＠epperson.homelinux.net
-
josh＠imaginenetworksllc.com
-
ralphmitchell＠gmail.com
-
rmcgraw＠purdue.edu