Hi all
Over the last day or so,been getting a lot of xymonnet core dumps.
Here's the debugging info as per http://xymon.sourceforge.net/xymon/help/known-issues.html
[root at xymon_srv 09:06 /var/lib/xymon/tmp]# file core.15685 core.15685: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from '/usr/libexec/xymon/xymonnet --ping --checkresponse portal01 portal02 TAB002' [root at xymon_srv 09:06 /var/lib/xymon/tmp]# gdb /usr/libexec/xymon/xymonnet core.15685 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/libexec/xymon/xymonnet...Reading symbols from /usr/lib/debug/usr/libexec/xymon/xymonnet.debug...done. done. [New Thread 15685] Missing separate debuginfo for Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/f1/5837b82aa5f929c8237a97e4a8b924b457ad7e Reading symbols from /usr/lib64/libcares.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcares.so.2 Reading symbols from /lib64/libldap-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libldap-2.4.so.2 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libz.so.1 Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libssl.so.10 Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcrypto.so.10 Reading symbols from /lib64/libpcre.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libpcre.so.0 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/liblber-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/liblber-2.4.so.2 Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /usr/lib64/libsasl2.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libsasl2.so.2 Reading symbols from /usr/lib64/libssl3.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libssl3.so Reading symbols from /usr/lib64/libsmime3.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libsmime3.so Reading symbols from /usr/lib64/libnss3.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libnss3.so Reading symbols from /usr/lib64/libnssutil3.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libnssutil3.so Reading symbols from /lib64/libplds4.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libplds4.so Reading symbols from /lib64/libplc4.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libplc4.so Reading symbols from /lib64/libnspr4.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libnspr4.so Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libgssapi_krb5.so.2 Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5.so.3 Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libcom_err.so.2 Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libk5crypto.so.3 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libcrypt.so.1 Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5support.so.0 Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libkeyutils.so.1 Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libfreebl3.so Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libselinux.so.1 Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libnss_files.so.2 Core was generated by `/usr/libexec/xymon/xymonnet --ping --checkresponse portal01 portal02 TAB002'. Program terminated with signal 6, Aborted. #0 0x0000003fcda32625 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install c-ares-1.10.0-3.el6.x86_64 cyrus-sasl-lib-2.1.23-15.el6_6.1.x86_64 glibc-2.12-1.149.el6_6.5.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 nspr-4.10.6-1.el6_5.x86_64 nss-3.16.2.3-3.el6_6.x86_64 nss-softokn-freebl-3.14.3-22.el6_6.x86_64 nss-util-3.16.2.3-2.el6_6.x86_64 openldap-2.4.39-8.el6.x86_64 openssl-1.0.1e-30.el6_6.5.x86_64 pcre-7.8-6.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x0000003fcda32625 in raise () from /lib64/libc.so.6 #1 0x0000003fcda33e05 in abort () from /lib64/libc.so.6 #2 0x0000000000420ab3 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x0000003fcda81171 in __strlen_sse2 () from /lib64/libc.so.6 #5 0x0000003fcda80e86 in strdup () from /lib64/libc.so.6 #6 0x000000000040d2a4 in setup_ssl (item=0xa136c0) at contest.c:653 #7 0x000000000040edef in do_tcp_tests (timeout=10, concurrency=256) at contest.c:1150 #8 0x000000000040aaf3 in main (argc=6, argv=0x7fffe1c7a008) at xymonnet.c:2461 (gdb) [root at xymon_srv 09:08 /var/lib/xymon/tmp]# cat ping-stderr.15685.00 ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27 ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27 ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27 ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27 [root at xymon_srv 09:08 /var/lib/xymon/tmp]#
Notes:
- TAB002 is 172.31.64.27
- TAB002 hosts.cfg entry is "172.31.64.27 TAB002 # conn nonongreen"
- server is RHEL6 x86_64 with RPM 4.3.18-1.el6.x86_64 from Terabithia
Anyone else seeing this?
Thanks
CC
Found the problem. There was a missing closing round bracket ')' on the hosts.cfg line for portal02.
On Thu, Mar 5, 2015 at 9:18 AM, Colin Coe <colin.coe at gmail.com> wrote:
Hi all
Over the last day or so,been getting a lot of xymonnet core dumps.
Here's the debugging info as per http://xymon.sourceforge.net/xymon/help/known-issues.html
[root at xymon_srv 09:06 /var/lib/xymon/tmp]# file core.15685 core.15685: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from '/usr/libexec/xymon/xymonnet --ping --checkresponse portal01 portal02 TAB002' [root at xymon_srv 09:06 /var/lib/xymon/tmp]# gdb /usr/libexec/xymon/xymonnet core.15685 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/libexec/xymon/xymonnet...Reading symbols from /usr/lib/debug/usr/libexec/xymon/xymonnet.debug...done. done. [New Thread 15685] Missing separate debuginfo for Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/f1/5837b82aa5f929c8237a97e4a8b924b457ad7e Reading symbols from /usr/lib64/libcares.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcares.so.2 Reading symbols from /lib64/libldap-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libldap-2.4.so.2 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libz.so.1 Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libssl.so.10 Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcrypto.so.10 Reading symbols from /lib64/libpcre.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libpcre.so.0 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/liblber-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/liblber-2.4.so.2 Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /usr/lib64/libsasl2.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libsasl2.so.2 Reading symbols from /usr/lib64/libssl3.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libssl3.so Reading symbols from /usr/lib64/libsmime3.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libsmime3.so Reading symbols from /usr/lib64/libnss3.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libnss3.so Reading symbols from /usr/lib64/libnssutil3.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libnssutil3.so Reading symbols from /lib64/libplds4.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libplds4.so Reading symbols from /lib64/libplc4.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libplc4.so Reading symbols from /lib64/libnspr4.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libnspr4.so Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libgssapi_krb5.so.2 Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5.so.3 Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libcom_err.so.2 Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libk5crypto.so.3 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libcrypt.so.1 Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5support.so.0 Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libkeyutils.so.1 Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libfreebl3.so Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libselinux.so.1 Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libnss_files.so.2 Core was generated by `/usr/libexec/xymon/xymonnet --ping --checkresponse portal01 portal02 TAB002'. Program terminated with signal 6, Aborted. #0 0x0000003fcda32625 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install c-ares-1.10.0-3.el6.x86_64 cyrus-sasl-lib-2.1.23-15.el6_6.1.x86_64 glibc-2.12-1.149.el6_6.5.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 nspr-4.10.6-1.el6_5.x86_64 nss-3.16.2.3-3.el6_6.x86_64 nss-softokn-freebl-3.14.3-22.el6_6.x86_64 nss-util-3.16.2.3-2.el6_6.x86_64 openldap-2.4.39-8.el6.x86_64 openssl-1.0.1e-30.el6_6.5.x86_64 pcre-7.8-6.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x0000003fcda32625 in raise () from /lib64/libc.so.6 #1 0x0000003fcda33e05 in abort () from /lib64/libc.so.6 #2 0x0000000000420ab3 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x0000003fcda81171 in __strlen_sse2 () from /lib64/libc.so.6 #5 0x0000003fcda80e86 in strdup () from /lib64/libc.so.6 #6 0x000000000040d2a4 in setup_ssl (item=0xa136c0) at contest.c:653 #7 0x000000000040edef in do_tcp_tests (timeout=10, concurrency=256) at contest.c:1150 #8 0x000000000040aaf3 in main (argc=6, argv=0x7fffe1c7a008) at xymonnet.c:2461 (gdb) [root at xymon_srv 09:08 /var/lib/xymon/tmp]# cat ping-stderr.15685.00 ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27 ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27 ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27 ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27 [root at xymon_srv 09:08 /var/lib/xymon/tmp]#
Notes:
- TAB002 is 172.31.64.27
- TAB002 hosts.cfg entry is "172.31.64.27 TAB002 # conn nonongreen"
- server is RHEL6 x86_64 with RPM 4.3.18-1.el6.x86_64 from Terabithia
Anyone else seeing this?
Thanks
CC
On 3/4/2015 4:31 PM, Colin Coe wrote:
Found the problem. There was a missing closing round bracket ')' on the hosts.cfg line for portal02.
yowser. That's a pretty brutal failure mode for a syntax error in hosts.cfg If you feed that file to xymongrep or xymoncfg, do either of them complain of the syntax error?
(I'm just wondering if I should ad a post-edit syntax reality check to my workflow.)
Do things because you should, not just because you can.
John Thurston 907-465-8591 John.Thurston at alaska.gov Enterprise Technology Services Department of Administration State of Alaska
On Wed, March 4, 2015 5:39 pm, John Thurston wrote:
On 3/4/2015 4:31 PM, Colin Coe wrote:
Found the problem. There was a missing closing round bracket ')' on the hosts.cfg line for portal02.
yowser. That's a pretty brutal failure mode for a syntax error in hosts.cfg If you feed that file to xymongrep or xymoncfg, do either of them complain of the syntax error?
(I'm just wondering if I should ad a post-edit syntax reality check to my workflow.)
Colin, can you forward me a copy of the relevant snippet?
If we died here, it was probably a result of an un-sanity-checked URL trying to get analyzed for SSL. We can probably add a little more protection around this.
Regards,
-jc
Replied direct with hosts.cfg entries.
Thanks
CC
On Fri, Mar 6, 2015 at 2:43 AM, J.C. Cleaver <cleaver at terabithia.org> wrote:
On Wed, March 4, 2015 5:39 pm, John Thurston wrote:
On 3/4/2015 4:31 PM, Colin Coe wrote:
Found the problem. There was a missing closing round bracket ')' on the hosts.cfg line for portal02.
yowser. That's a pretty brutal failure mode for a syntax error in hosts.cfg If you feed that file to xymongrep or xymoncfg, do either of them complain of the syntax error?
(I'm just wondering if I should ad a post-edit syntax reality check to my workflow.)
Colin, can you forward me a copy of the relevant snippet?
If we died here, it was probably a result of an un-sanity-checked URL trying to get analyzed for SSL. We can probably add a little more protection around this.
Regards,
-jc
participants (3)
-
cleaver@terabithia.org
-
colin.coe@gmail.com
-
john.thurston@alaska.gov