This is an important one to patch your systems on, if you haven't already.
The xymon CGI interface runs via shell wrappers around the actual C cgi code (to set the environment properly), which means this would be an avenue for attack.
Alternatively, using /bin/dash or some other shell besides bash (often /bin/sh on Linux distros) is another work around. (This is the default on the Terabithia RPMS for EL6.)
More info: http://seclists.org/oss-sec/2014/q3/650
https://securityblog.redhat.com/2014/09/24/bash-specially-crafted-environmen... https://access.redhat.com/articles/1200223
Regards, -jc
Oh, yes, very terrible.
And if you want to test to see that you are vulnerable through Xymon, you can try this harmless exploit:
your_workstation$ curl -k -H 'User-Agent: () { :;}; echo vulnerable>/tmp/test-xymon-shellshock' http://your_xymon_server/xymon-cgi/svcstatus.sh <html><head><title>Invalid request</title></head> <body>Invalid request</body></html> your_workstation$ ssh your_xymon_server 'cat /tmp/test-xymon-shellshock' vulnerable your_workstation$ ...which creates a file (if you are vulnerable) in your Xymon server '/tmp/':
your_workstation$ ssh your_xymon_server 'cat /tmp/test-xymon-shellshock' vulnerable your_workstation$ ...so then, you can verify before and after patching.
cheers,
Troy
----- Original Message ----- From: "J.C. Cleaver" <cleaver at terabithia.org> To: xymon at xymon.com Sent: Wednesday, September 24, 2014 11:54:35 AM GMT -07:00 US/Canada Mountain Subject: [Xymon] FYI: CVE-2014-6271 - bash vulnerability
This is an important one to patch your systems on, if you haven't already.
The xymon CGI interface runs via shell wrappers around the actual C cgi code (to set the environment properly), which means this would be an avenue for attack.
Alternatively, using /bin/dash or some other shell besides bash (often /bin/sh on Linux distros) is another work around. (This is the default on the Terabithia RPMS for EL6.)
More info: http://seclists.org/oss-sec/2014/q3/650
https://securityblog.redhat.com/2014/09/24/bash-specially-crafted-environmen... https://access.redhat.com/articles/1200223
Regards, -jc
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
-- This communication is intended for the use of the recipient to whom it is addressed, and may contain confidential, personal, and or privileged information. Please contact us immediately if you are not the intended recipient of this communication, and do not copy, distribute, or take action relying on it. Any communications received in error, or subsequent reply, should be deleted or destroyed.
The xymon CGI interface runs via shell wrappers around the actual C cgi code (to set the environment properly), which means this would be an avenue for attack. Indeed, this one is nasty. Fortunately, most Linux systems I know of have /bin/sh linked to /bin/dash and hence are not vulnerable.
In light of this, I think it is about time we retire the shell-script wrappers from Xymon. I have written a replacement which is now checked into the 4.3.18 branch.
There is a preliminary release of 4.3.18 available on https://www.xymon.com/patches/ - feel free to try it out. I will release 4.3.18 over the weekend unless I find some problems with it.
NOTE: Replacing the shell script wrappers means that the cgioptions.cfg file is no longer processed as a shell script. The new wrapper works fine with the default version of cgioptions.cfg, but it you have modified it in a way that it relies on being processed by a shell, then it will break.
Regards, Henrik
? Just checked fully patched RHEL5, RHEL6, and Fedora 20 systems and found sh linked to bash on all.
On 2014-09-26 10:54, Henrik Størner wrote:
Fortunately, most Linux systems I know of have /bin/sh linked to /bin/dash and hence are not vulnerable.
Ubuntu (and maybe debian) links sh to dash...I've not seen dash used on other linux systems I've used.
=G=
From: Xymon <xymon-bounces at xymon.com> on behalf of Another Xymon User <xymon at epperson.homelinux.net> Sent: Friday, September 26, 2014 12:36 PM To: xymon at xymon.com Subject: Re: [Xymon] 'Shell shock' mitigation
? Just checked fully patched RHEL5, RHEL6, and Fedora 20 systems and found sh linked to bash on all.
On 2014-09-26 10:54, Henrik St?rner wrote:
Fortunately, most Linux systems I know of have /bin/sh linked to /bin/dash and hence are not vulnerable.
Yes, Debian as well. At least on 6.0.
-- ____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences* || \\UTGERS |---------------------*O*--------------------- ||_// Biomedical | Ryan Novosielski - Senior Technologist || \\ and Health | novosirj at rutgers.edu<mailto:novosirj at rutgers.edu>- 973/972.0922 (2x0922) || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark `'
On Sep 26, 2014, at 17:03, Galen Johnson <Galen.Johnson at sas.com<mailto:Galen.Johnson at sas.com>> wrote:
Ubuntu (and maybe debian) links sh to dash...I've not seen dash used on other linux systems I've used.
=G=
From: Xymon <xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>> on behalf of Another Xymon User <xymon at epperson.homelinux.net<mailto:xymon at epperson.homelinux.net>> Sent: Friday, September 26, 2014 12:36 PM To: xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] 'Shell shock' mitigation
? Just checked fully patched RHEL5, RHEL6, and Fedora 20 systems and found sh linked to bash on all.
On 2014-09-26 10:54, Henrik Størner wrote:
Fortunately, most Linux systems I know of have /bin/sh linked to /bin/dash and hence are not vulnerable.
Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon
/bin/sh to /bin/bash is standard on Red Hat-derived systems.
dash is present as a package in RHEL6 and Fedora, but not EL7 or EL5. Prior to that (<=EL4) 'ash' was available.
-jc
On Fri, September 26, 2014 2:03 pm, Galen Johnson wrote:
Ubuntu (and maybe debian) links sh to dash...I've not seen dash used on other linux systems I've used.
From: Xymon <xymon-bounces at xymon.com> on behalf of Another Xymon User <xymon at epperson.homelinux.net> Sent: Friday, September 26, 2014 12:36 PM To: xymon at xymon.com Subject: Re: [Xymon] 'Shell shock' mitigation
? Just checked fully patched RHEL5, RHEL6, and Fedora 20 systems and found sh linked to bash on all.
On 2014-09-26 10:54, Henrik St?rner wrote:
Fortunately, most Linux systems I know of have /bin/sh linked to /bin/dash and hence are not vulnerable.
On Fri, 26 Sep 2014, J.C. Cleaver wrote:
/bin/sh to /bin/bash is standard on Red Hat-derived systems.
dash is present as a package in RHEL6 and Fedora, but not EL7 or EL5. Prior to that (<=EL4) 'ash' was available.
So, is changing the shell in /etc/passwd for the xymon user to /bin/dash sufficient to get xymon to use dash or are other changes required.
I really do not want to change the symlink for /bin/sh to point to dash as I am not sure what other things might break.
This is on a Centos 6.5 box.
Regards,
-- Tom me at tdiehl.org Spamtrap address me123 at tdiehl.org
On Fri, September 26, 2014 4:57 pm, me at tdiehl.org wrote:
On Fri, 26 Sep 2014, J.C. Cleaver wrote:
/bin/sh to /bin/bash is standard on Red Hat-derived systems.
dash is present as a package in RHEL6 and Fedora, but not EL7 or EL5. Prior to that (<=EL4) 'ash' was available.
So, is changing the shell in /etc/passwd for the xymon user to /bin/dash sufficient to get xymon to use dash or are other changes required.
The key exposure is the shebang line for the wrappers in ~/server/xymon-cgi/ and/or ~/server/xymon-seccgi/. After that, ensure that SHELL= in xymonserver.cfg is set to /bin/dash (if you're using at least 4.3.12).
The /etc/passwd shell is what the system account will use, but there's no real need for the xymon user to have a valid shell at all there (unless you're doing a remote login or something. I typically use /sbin/nologin there.
I really do not want to change the symlink for /bin/sh to point to dash as I am not sure what other things might break.
This is on a Centos 6.5 box.
I wouldn't advise switching /bin/sh to /bin/dash on a RH/CentOS box -- probably lots of small breakages here and there due to bashisms.
HTH,
-jc
Red hat has an updated bash rpm out for RHEL6. And the GNU sites make available the bash source code for 4.3 plus the patch to apply. I used both on a large number of systems yesterday, without reboots and without problems. Both verified using Nessus security scans. Could you use those instead of relying on dash?
-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of me at tdiehl.org Sent: Friday, September 26, 2014 4:58 PM To: J.C. Cleaver Cc: Xymon Mailing List Subject: Re: [Xymon] 'Shell shock' mitigation
On Fri, 26 Sep 2014, J.C. Cleaver wrote:
/bin/sh to /bin/bash is standard on Red Hat-derived systems.
dash is present as a package in RHEL6 and Fedora, but not EL7 or EL5. Prior to that (<=EL4) 'ash' was available.
So, is changing the shell in /etc/passwd for the xymon user to /bin/dash sufficient to get xymon to use dash or are other changes required.
I really do not want to change the symlink for /bin/sh to point to dash as I am not sure what other things might break.
This is on a Centos 6.5 box.
Regards,
-- Tom me at tdiehl.org Spamtrap address me123 at tdiehl.org
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This message may contain confidential or proprietary information intended only for the use of the
addressee(s) named above or may contain information that is legally privileged. If you are
not the intended addressee, or the person responsible for delivering it to the intended addressee,
you are hereby notified that reading, disseminating, distributing or copying this message is strictly
prohibited. If you have received this message by mistake, please immediately notify us by
replying to the message and delete the original message and any copies immediately thereafter.
Thank you.
CLLD
On any box, you need to check the patched bash as to whether the vendor has addressed both CVE-2014-6271 and CVE-2014-7179.
Redhat appears to have addressed both with bash updates for RHEL and Fedora Core releases. Centos though appears to only address 6271 - patched this morning and the Centos uppdate still fails 7179; may take a couple more days for the maintainers to kick out an update covering 6271.
HPUX Porting Archive (UK) is advertising a binary suite for HPUX 11.11/11.23/11.31 PA-RISC/Itanium that addresses CVE-2014-6271 and CVE-2014-7179 (have not had time to check on these - caveat emptor).
Doubt anyone has addressed CVE-2014-7187 yet; too new.
-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Michael Short Sent: Friday, September 26, 2014 8:54 PM To: me at tdiehl.org; J.C. Cleaver Cc: Xymon Mailing List Subject: Re: [Xymon] 'Shell shock' mitigation
Red hat has an updated bash rpm out for RHEL6. And the GNU sites make available the bash source code for 4.3 plus the patch to apply. I used both on a large number of systems yesterday, without reboots and without problems. Both verified using Nessus security scans. Could you use those instead of relying on dash?
-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of me at tdiehl.org Sent: Friday, September 26, 2014 4:58 PM To: J.C. Cleaver Cc: Xymon Mailing List Subject: Re: [Xymon] 'Shell shock' mitigation
On Fri, 26 Sep 2014, J.C. Cleaver wrote:
/bin/sh to /bin/bash is standard on Red Hat-derived systems.
dash is present as a package in RHEL6 and Fedora, but not EL7 or EL5. Prior to that (<=EL4) 'ash' was available.
So, is changing the shell in /etc/passwd for the xymon user to /bin/dash sufficient to get xymon to use dash or are other changes required.
I really do not want to change the symlink for /bin/sh to point to dash as I am not sure what other things might break.
This is on a Centos 6.5 box.
Regards,
-- Tom me at tdiehl.org Spamtrap address me123 at tdiehl.org
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter.
Thank you.
CLLD
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Hi Henrik,
On Fri, 26 Sep 2014, Henrik Størner wrote:
The xymon CGI interface runs via shell wrappers around the actual C cgi code (to set the environment properly), which means this would be an avenue for attack. Indeed, this one is nasty. Fortunately, most Linux systems I know of have /bin/sh linked to /bin/dash and hence are not vulnerable.
In light of this, I think it is about time we retire the shell-script wrappers from Xymon. I have written a replacement which is now checked into the 4.3.18 branch.
There is a preliminary release of 4.3.18 available on https://www.xymon.com/patches/ - feel free to try it out. I will release 4.3.18 over the weekend unless I find some problems with it.
NOTE: Replacing the shell script wrappers means that the cgioptions.cfg file is no longer processed as a shell script. The new wrapper works fine with the default version of cgioptions.cfg, but it you have modified it in a way that it relies on being processed by a shell, then it will break.
I just upgraded bash to the latest from RH/Centos and I can report that it breaks the data from machines using bbwin. They all went purple. To be sure my hunch was correct, I downgraded bash to bash-4.1.2-15.el6_5.1.x86_64 and the purple went away.
Is it expected that the fix you reference above will work with bbwin? I have not modified cgioptions.cfg.
I need to wait until the terabithia rpms are updated to upgrade xymon.
Regards,
-- Tom me at tdiehl.org Spamtrap address me123 at tdiehl.org
On Fri, September 26, 2014 1:14 pm, me at tdiehl.org wrote:
Hi Henrik,
On Fri, 26 Sep 2014, Henrik Størner wrote:
The xymon CGI interface runs via shell wrappers around the actual C cgi code (to set the environment properly), which means this would be an avenue for attack. Indeed, this one is nasty. Fortunately, most Linux systems I know of have /bin/sh linked to /bin/dash and hence are not vulnerable.
In light of this, I think it is about time we retire the shell-script wrappers from Xymon. I have written a replacement which is now checked into the 4.3.18 branch.
There is a preliminary release of 4.3.18 available on https://www.xymon.com/patches/ - feel free to try it out. I will release 4.3.18 over the weekend unless I find some problems with it.
NOTE: Replacing the shell script wrappers means that the cgioptions.cfg file is no longer processed as a shell script. The new wrapper works fine with the default version of cgioptions.cfg, but it you have modified it in a way that it relies on being processed by a shell, then it will break.
I just upgraded bash to the latest from RH/Centos and I can report that it breaks the data from machines using bbwin. They all went purple. To be sure my hunch was correct, I downgraded bash to bash-4.1.2-15.el6_5.1.x86_64 and the purple went away.
Is it expected that the fix you reference above will work with bbwin? I have not modified cgioptions.cfg.
That's very strange. Was there anything at all in the logs anywhere when that was happening? Does BBWin use anything special to communicate in to Xymon or is it simply submitting on port 1984 like normal?
I need to wait until the terabithia rpms are updated to upgrade xymon.
Regards,
I've posted a set of 4.3.18-0.0.7471.1 RPMs at http://terabithia.org/rpms/xymon/testing/ if you're curious to take a look, but I'm still testing myself and would use caution.
One note: The apache config file needs to be updated to allow FollowSymLinks in the /xymon-(sec)cgi/ directory, or all dynamic pages will return with a 403 error. The following line on upgrade should fix it:
perl -pe 's/Options ExecCGI Includes/Options ExecCGI FollowSymLinks Includes/' -i /etc/httpd/conf.d/xymon.conf && /sbin/service httpd graceful
Regards, -jc
On Fri, 26 Sep 2014, J.C. Cleaver wrote:
On Fri, September 26, 2014 1:14 pm, me at tdiehl.org wrote:
Hi Henrik,
On Fri, 26 Sep 2014, Henrik Størner wrote:
The xymon CGI interface runs via shell wrappers around the actual C cgi code (to set the environment properly), which means this would be an avenue for attack. Indeed, this one is nasty. Fortunately, most Linux systems I know of have /bin/sh linked to /bin/dash and hence are not vulnerable.
In light of this, I think it is about time we retire the shell-script wrappers from Xymon. I have written a replacement which is now checked into the 4.3.18 branch.
There is a preliminary release of 4.3.18 available on https://www.xymon.com/patches/ - feel free to try it out. I will release 4.3.18 over the weekend unless I find some problems with it.
NOTE: Replacing the shell script wrappers means that the cgioptions.cfg file is no longer processed as a shell script. The new wrapper works fine with the default version of cgioptions.cfg, but it you have modified it in a way that it relies on being processed by a shell, then it will break.
I just upgraded bash to the latest from RH/Centos and I can report that it breaks the data from machines using bbwin. They all went purple. To be sure my hunch was correct, I downgraded bash to bash-4.1.2-15.el6_5.1.x86_64 and the purple went away.
Is it expected that the fix you reference above will work with bbwin? I have not modified cgioptions.cfg.
That's very strange. Was there anything at all in the logs anywhere when that was happening? Does BBWin use anything special to communicate in to Xymon or is it simply submitting on port 1984 like normal?
I agree it is strange and it makes no sense to me. bbwin sends its data over 1984 like any other client. FWIW, bbwin is running in central mode and I am using xymon-4.3.17-1.el6.x86_64.
I saw a bunch of the following in the hostdata.log and distribute.log: 2014-09-26 15:42:58 Could not get shm of size 5242880: No such file or directory 2014-09-26 15:42:58 xymond_channel: Channel not available 2014-09-26 15:52:29 Could not get shm of size 5242880: No such file or directory 2014-09-26 15:52:29 xymond_channel: Channel not available
In the alert.log I also saw the following: Could not get shm of size 4194304: No such file or directory 2014-09-26 15:52:29 xymond_channel: Channel not available 2014-09-26 15:52:29 Whoops ! Failed to send message (Connection failed) 2014-09-26 15:52:29 -> Could not connect to Xymon daemon at 192.168.0.2:1984 (Connection refused) 2014-09-26 15:52:29 -> Recipient '192.168.0.2', timeout 15 2014-09-26 15:52:29 -> 1st line: 'xymondboard color=red,yellow,purple fields=hostname,testname,color' 2014-09-26 15:52:29 xymond status-board not available, code 5
I have similar stuff in the xymongen.log but in looking at all of these errors I suspect they occured when I was restarting xymon.
The other weird thing I saw was the bbwin service would not reconnect after I downgraded bash. I had to go to each of the machines and restart the service by hand before they would communicate. Fortunately there are not very many of them.
In looking at the logs on the windoze hosts, I see the following: BBWin failed to send the client data successfuly to the Xymon server. The error was : Can't send message : An established connection was aborted by the software in your host machine..
I am not sure if this is revelant or not as everything is reporting normally but I still see that error in the eventvwr.
The really weird thing is that the linux clients kept working through all of this.
I need to wait until the terabithia rpms are updated to upgrade xymon.
Regards,
I've posted a set of 4.3.18-0.0.7471.1 RPMs at http://terabithia.org/rpms/xymon/testing/ if you're curious to take a look, but I'm still testing myself and would use caution.
Thanks, I will take a look.
One note: The apache config file needs to be updated to allow FollowSymLinks in the /xymon-(sec)cgi/ directory, or all dynamic pages will return with a 403 error. The following line on upgrade should fix it:
perl -pe 's/Options ExecCGI Includes/Options ExecCGI FollowSymLinks Includes/' -i /etc/httpd/conf.d/xymon.conf && /sbin/service httpd graceful
Regards,
-- Tom me at tdiehl.org Spamtrap address me123 at tdiehl.org
On Fri, 26 Sep 2014, J.C. Cleaver wrote:
On Fri, September 26, 2014 1:14 pm, me at tdiehl.org wrote:
Hi Henrik,
On Fri, 26 Sep 2014, Henrik Størner wrote:
The xymon CGI interface runs via shell wrappers around the actual C cgi code (to set the environment properly), which means this would be an avenue for attack. Indeed, this one is nasty. Fortunately, most Linux systems I know of have /bin/sh linked to /bin/dash and hence are not vulnerable.
In light of this, I think it is about time we retire the shell-script wrappers from Xymon. I have written a replacement which is now checked into the 4.3.18 branch.
There is a preliminary release of 4.3.18 available on https://www.xymon.com/patches/ - feel free to try it out. I will release 4.3.18 over the weekend unless I find some problems with it.
NOTE: Replacing the shell script wrappers means that the cgioptions.cfg file is no longer processed as a shell script. The new wrapper works fine with the default version of cgioptions.cfg, but it you have modified it in a way that it relies on being processed by a shell, then it will break.
I just upgraded bash to the latest from RH/Centos and I can report that it breaks the data from machines using bbwin. They all went purple. To be sure my hunch was correct, I downgraded bash to bash-4.1.2-15.el6_5.1.x86_64 and the purple went away.
Is it expected that the fix you reference above will work with bbwin? I have not modified cgioptions.cfg.
That's very strange. Was there anything at all in the logs anywhere when that was happening? Does BBWin use anything special to communicate in to Xymon or is it simply submitting on port 1984 like normal?
I need to wait until the terabithia rpms are updated to upgrade xymon.
Regards,
I've posted a set of 4.3.18-0.0.7471.1 RPMs at http://terabithia.org/rpms/xymon/testing/ if you're curious to take a look, but I'm still testing myself and would use caution.
One note: The apache config file needs to be updated to allow FollowSymLinks in the /xymon-(sec)cgi/ directory, or all dynamic pages will return with a 403 error. The following line on upgrade should fix it:
perl -pe 's/Options ExecCGI Includes/Options ExecCGI FollowSymLinks Includes/' -i /etc/httpd/conf.d/xymon.conf && /sbin/service httpd graceful
I did some poking over the weekend and discovered that when I upgraded xymon a long time ago, I never looked at the associated .rpmnew files. After updating the various .rpmnew file including xymonserver.cfg and then applying the bash update all seems to be working normal.
In addition, I found that the default shell used in the xymon scripts is /bin/dash. So it looks like the bash exploit was never an issue for my systems.
Regards,
-- Tom me at tdiehl.org Spamtrap address me123 at tdiehl.org
participants (9)
-
cleaver＠terabithia.org
-
Galen.Johnson＠sas.com
-
henrik＠hswn.dk
-
Mark.Deiss＠xerox.com
-
me＠tdiehl.org
-
mshort＠corelogic.com
-
novosirj＠ca.rutgers.edu
-
troy＠athabascau.ca
-
xymon＠epperson.homelinux.net