xymonnet crashing after upgrade to 4.3.15
Hi.
Updated our server from 4.3.13 to 4.3.15 today, and after that xymonnet has crashed a few times. Is it possible to enable some debug log or something to find out what happens?
The error in the xymonnet test is:
"red - Program crashed
Fatal signal caught!"
/Johan
Den 04-02-2014 22:17, Johan Sjöberg skrev:
Hi.
Updated our server from 4.3.13 to 4.3.15 today, and after that xymonnet has crashed a few times. Is it possible to enable some debug log or something to find out what happens?
The error in the xymonnet test is:
“red - Program crashed
I have a couple of fixes, but none that really relate to xymonnet. You should get a 'core' file in ~xymon/server/tmp/ - if you do, could you try getting a stack trace using this procedure: http://www.xymon.com/xymon/help/known-issues.html#bugreport
Regards, Henrik
-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Henrik Størner Sent: den 4 februari 2014 22:23 To: xymon at xymon.com Subject: Re: [Xymon] xymonnet crashing after upgrade to 4.3.15
Den 04-02-2014 22:17, Johan Sjöberg skrev:
Hi.
Updated our server from 4.3.13 to 4.3.15 today, and after that xymonnet has crashed a few times. Is it possible to enable some debug log or something to find out what happens?
The error in the xymonnet test is:
"red - Program crashed
I have a couple of fixes, but none that really relate to xymonnet. You should get a 'core' file in ~xymon/server/tmp/ - if you do, could you try getting a stack trace using this procedure: http://www.xymon.com/xymon/help/known-issues.html#bugreport
Regards, Henrik
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Hi.
I ran gdb on the core file and got the output below. We also see the same crashes on our staging xymon server after the upgrade.
mon01:/usr/local/xymon/server/tmp# gdb /usr/local/xymon/server/bin/xymonnet core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/local/xymon/server/bin/xymonnet...done.
warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/libcares.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcares.so.2 Reading symbols from /usr/lib/libldap_r-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libldap_r-2.4.so.2 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/libpcre.so.3...(no debugging symbols found)...done. Loaded symbols for /lib/libpcre.so.3 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/liblber-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib/liblber-2.4.so.2 Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /usr/lib/libsasl2.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libsasl2.so.2 Reading symbols from /usr/lib/libgnutls.so.26...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgnutls.so.26 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /usr/lib/libtasn1.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libtasn1.so.3 Reading symbols from /usr/lib/libgcrypt.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgcrypt.so.11 Reading symbols from /usr/lib/libgpg-error.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgpg-error.so.0 Core was generated by `/usr/local/xymon/server/bin/xymonnet --ping --checkresponse asx-b5.xxxx.d'. Program terminated with signal 6, Aborted. #0 0x00007f66ca0171b5 in raise () from /lib/libc.so.6 (gdb) bt #0 0x00007f66ca0171b5 in raise () from /lib/libc.so.6 #1 0x00007f66ca019fc0 in abort () from /lib/libc.so.6 #2 0x000000000041ed43 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x00000000004239dc in xmh_find_item (host=0xbbad20, item=<value optimized out>) at loadhosts.c:226 #5 0x000000000040fefe in send_content_results (httptest=<value optimized out>, host=<value optimized out>, nonetpage=<value optimized out>, contenttestname=<value optimized out>, failgoesclear=<value optimized out>) at httpresult.c:456 #6 0x000000000040a8ed in main (argc=6, argv=<value optimized out>) at xymonnet.c:2484
/Johan
Hi Johan,
Den 2014-02-05 9:07, Johan Sjöberg skrev:
#2 0x000000000041ed43 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x00000000004239dc in xmh_find_item (host=0xbbad20, item=<value optimized out>) at loadhosts.c:226 #5 0x000000000040fefe in send_content_results (httptest=<value optimized out>, host=<value optimized out>, nonetpage=<value optimized out>, contenttestname=<value optimized out>, failgoesclear=<value optimized out>) at httpresult.c:456
could you try this patch, please?
Regards, Henrik
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] Sent: den 5 februari 2014 09:14 To: Johan Sjöberg Cc: xymon at xymon.com Subject: RE: [Xymon] xymonnet crashing after upgrade to 4.3.15
Hi Johan,
Den 2014-02-05 9:07, Johan Sjöberg skrev:
#2 0x000000000041ed43 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x00000000004239dc in xmh_find_item (host=0xbbad20, item=<value optimized out>) at loadhosts.c:226 #5 0x000000000040fefe in send_content_results (httptest=<value optimized out>, host=<value optimized out>, nonetpage=<value optimized out>, contenttestname=<value optimized out>, failgoesclear=<value optimized out>) at httpresult.c:456
could you try this patch, please?
Regards, Henrik
Thanks. Can I just copy the new xymonnet binary after compiling, and skip the make install?
/Johan
Den 2014-02-05 9:24, Johan Sjöberg skrev:
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] could you try this patch, please?
Thanks. Can I just copy the new xymonnet binary after compiling, and skip the make install?
Yes, that will work fine.
Regards, Henrik
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] Sent: den 5 februari 2014 09:50 To: Johan Sjöberg Cc: xymon at xymon.com Subject: RE: [Xymon] xymonnet crashing after upgrade to 4.3.15
Den 2014-02-05 9:24, Johan Sjöberg skrev:
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] could you try this patch, please?
Thanks. Can I just copy the new xymonnet binary after compiling, and skip the make install?
Yes, that will work fine.
Regards, Henrik
Ok, I have applied it on one of our machines. I'll wait and see if it makes any difference. The crashes are not too frequent, it has happened 3-4 times on either server after the upgrade.
/Johan
Just reporting that I too have seen 1 x core dump from xymonnet on our test xymon setup since upgrading to 4.3.15 on Monday. Relevant gdb output follows:
[xymon at aztec server]$ gdb bin/xymonnet tmp/core.30031 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1) .... .... Core was generated by `xymonnet --report --ping --ping-tasks=5 --test-untagged --concurrency=64 --cont'. Program terminated with signal 6, Aborted. .... (gdb) bt #0 0x0000003cfea32925 in raise () from /lib64/libc.so.6 #1 0x0000003cfea34105 in abort () from /lib64/libc.so.6 #2 0x0000000000426433 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x000000000042b07c in xmh_find_item (host=<value optimized out>, item=XMH_FLAG_HTTP_HEADER_MATCH) at loadhosts.c:226 #5 0x000000000040f84e in send_content_results (httptest=<value optimized out>, host=<value optimized out>, nonetpage=<value optimized out>, contenttestname=0x967030 "cont", failgoesclear=1) at httpresult.c:456 #6 0x000000000040a26d in main (argc=14, argv=0x7fffaf67cce8) at xymonnet.c:2484
I have also applied the supplied patch and will report any issues.
Cheers, Gavin Stone-Tolcher, IT Support Officer, Network Operations and Incident Response Information Technology Services The University of Queensland Level 4, Prentice Building, St Lucia 4072 T: +61 7 334 66645, M: +61 401 140 838 E: g.stone-tolcher at its.uq.edu.au W: www.its.uq.edu.au
ITS: Service. Team. Accountability. Results.
IMPORTANT: This email and any attachments are intended solely for the addressee(s), contain copyright material and are confidential. We do not waive any legal privilege or rights in respect of copyright or confidentiality. Except as intended addressees are otherwise permitted, you do not have permission to use, disclose, reproduce or communicate any part of this email or its attachments. Statements, opinions and information not related to the official business of The University of Queensland are neither given nor endorsed by us. By using this email (including accessing any attachments or links) you agree we are not liable for any loss or damage of any kind arising in connection with any electronic defect, virus or other malicious code we did not intentionally include.
Please consider the environment before printing this email.
CRICOS Code 00025B
-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Johan Sjöberg Sent: Wednesday, 5 February 2014 8:00 PM To: henrik at hswn.dk Cc: xymon at xymon.com Subject: Re: [Xymon] xymonnet crashing after upgrade to 4.3.15
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] Sent: den 5 februari 2014 09:50 To: Johan Sjöberg Cc: xymon at xymon.com Subject: RE: [Xymon] xymonnet crashing after upgrade to 4.3.15
Den 2014-02-05 9:24, Johan Sjöberg skrev:
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] could you try this patch, please?
Thanks. Can I just copy the new xymonnet binary after compiling, and skip the make install?
Yes, that will work fine.
Regards, Henrik
Ok, I have applied it on one of our machines. I'll wait and see if it makes any difference. The crashes are not too frequent, it has happened 3-4 times on either server after the upgrade.
/Johan
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Hi.
Just wanted to report that I installed the patch yesterday, and have not seen any crash since then.
/Johan
-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Gavin Stone-Tolcher Sent: den 6 februari 2014 05:23 To: xymon at xymon.com Subject: Re: [Xymon] xymonnet crashing after upgrade to 4.3.15
Just reporting that I too have seen 1 x core dump from xymonnet on our test xymon setup since upgrading to 4.3.15 on Monday. Relevant gdb output follows:
[xymon at aztec server]$ gdb bin/xymonnet tmp/core.30031 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1) .... .... Core was generated by `xymonnet --report --ping --ping-tasks=5 --test- untagged --concurrency=64 --cont'. Program terminated with signal 6, Aborted. .... (gdb) bt #0 0x0000003cfea32925 in raise () from /lib64/libc.so.6 #1 0x0000003cfea34105 in abort () from /lib64/libc.so.6 #2 0x0000000000426433 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x000000000042b07c in xmh_find_item (host=<value optimized out>, item=XMH_FLAG_HTTP_HEADER_MATCH) at loadhosts.c:226 #5 0x000000000040f84e in send_content_results (httptest=<value optimized out>, host=<value optimized out>, nonetpage=<value optimized out>, contenttestname=0x967030 "cont", failgoesclear=1) at httpresult.c:456 #6 0x000000000040a26d in main (argc=14, argv=0x7fffaf67cce8) at xymonnet.c:2484
I have also applied the supplied patch and will report any issues.
Cheers, Gavin Stone-Tolcher, IT Support Officer, Network Operations and Incident Response Information Technology Services The University of Queensland Level 4, Prentice Building, St Lucia 4072 T: +61 7 334 66645, M: +61 401 140 838 E: g.stone-tolcher at its.uq.edu.au W: www.its.uq.edu.au
ITS: Service. Team. Accountability. Results.
IMPORTANT: This email and any attachments are intended solely for the addressee(s), contain copyright material and are confidential. We do not waive any legal privilege or rights in respect of copyright or confidentiality. Except as intended addressees are otherwise permitted, you do not have permission to use, disclose, reproduce or communicate any part of this email or its attachments. Statements, opinions and information not related to the official business of The University of Queensland are neither given nor endorsed by us. By using this email (including accessing any attachments or links) you agree we are not liable for any loss or damage of any kind arising in connection with any electronic defect, virus or other malicious code we did not intentionally include.
Please consider the environment before printing this email.
CRICOS Code 00025B
-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Johan Sjöberg Sent: Wednesday, 5 February 2014 8:00 PM To: henrik at hswn.dk Cc: xymon at xymon.com Subject: Re: [Xymon] xymonnet crashing after upgrade to 4.3.15
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] Sent: den 5 februari 2014 09:50 To: Johan Sjöberg Cc: xymon at xymon.com Subject: RE: [Xymon] xymonnet crashing after upgrade to 4.3.15
Den 2014-02-05 9:24, Johan Sjöberg skrev:
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] could you try this patch, please?
Thanks. Can I just copy the new xymonnet binary after compiling, and skip the make install?
Yes, that will work fine.
Regards, Henrik
Ok, I have applied it on one of our machines. I'll wait and see if it makes any difference. The crashes are not too frequent, it has happened 3-4 times on either server after the upgrade.
/Johan
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
We've seen this also.
[bb at prdexp server]$ gdb bin/xymonnet tmp/core.64739 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1) ... ... Core was generated by `xymonnet --report --ping --checkresponse'. Program terminated with signal 6, Aborted. ... (gdb) bt #0 0x00000035bdc32925 in raise () from /lib64/libc.so.6 #1 0x00000035bdc34105 in abort () from /lib64/libc.so.6 #2 0x000000000041e5c3 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x000000000042320c in xmh_find_item (host=<value optimized out>, item=XMH_FLAG_HTTP_HEADER_MATCH) at loadhosts.c:226 #5 0x000000000040f7de in send_content_results ( httptest=<value optimized out>, host=<value optimized out>, nonetpage=<value optimized out>, contenttestname=0x42abde "content", failgoesclear=1) at httpresult.c:456 #6 0x000000000040a1fd in main (argc=4, argv=0x7fff3c97eaf8) at xymonnet.c:2484
Dominique
On 02/06/2014 05:23 AM, Gavin Stone-Tolcher wrote:
Just reporting that I too have seen 1 x core dump from xymonnet on our test xymon setup since upgrading to 4.3.15 on Monday. Relevant gdb output follows:
[xymon at aztec server]$ gdb bin/xymonnet tmp/core.30031 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1) .... .... Core was generated by `xymonnet --report --ping --ping-tasks=5 --test-untagged --concurrency=64 --cont'. Program terminated with signal 6, Aborted. .... (gdb) bt #0 0x0000003cfea32925 in raise () from /lib64/libc.so.6 #1 0x0000003cfea34105 in abort () from /lib64/libc.so.6 #2 0x0000000000426433 in sigsegv_handler (signum=<value optimized out>) at sig.c:57 #3 <signal handler called> #4 0x000000000042b07c in xmh_find_item (host=<value optimized out>, item=XMH_FLAG_HTTP_HEADER_MATCH) at loadhosts.c:226 #5 0x000000000040f84e in send_content_results (httptest=<value optimized out>, host=<value optimized out>, nonetpage=<value optimized out>, contenttestname=0x967030 "cont", failgoesclear=1) at httpresult.c:456 #6 0x000000000040a26d in main (argc=14, argv=0x7fffaf67cce8) at xymonnet.c:2484
I have also applied the supplied patch and will report any issues.
Cheers, Gavin Stone-Tolcher, IT Support Officer, Network Operations and Incident Response Information Technology Services The University of Queensland Level 4, Prentice Building, St Lucia 4072 T: +61 7 334 66645, M: +61 401 140 838 E: g.stone-tolcher at its.uq.edu.au W: www.its.uq.edu.au
ITS: Service. Team. Accountability. Results.
IMPORTANT: This email and any attachments are intended solely for the addressee(s), contain copyright material and are confidential. We do not waive any legal privilege or rights in respect of copyright or confidentiality. Except as intended addressees are otherwise permitted, you do not have permission to use, disclose, reproduce or communicate any part of this email or its attachments. Statements, opinions and information not related to the official business of The University of Queensland are neither given nor endorsed by us. By using this email (including accessing any attachments or links) you agree we are not liable for any loss or damage of any kind arising in connection with any electronic defect, virus or other malicious code we did not intentionally include.
Please consider the environment before printing this email.
CRICOS Code 00025B
-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Johan Sjöberg Sent: Wednesday, 5 February 2014 8:00 PM To: henrik at hswn.dk Cc: xymon at xymon.com Subject: Re: [Xymon] xymonnet crashing after upgrade to 4.3.15
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] Sent: den 5 februari 2014 09:50 To: Johan Sjöberg Cc: xymon at xymon.com Subject: RE: [Xymon] xymonnet crashing after upgrade to 4.3.15
Den 2014-02-05 9:24, Johan Sjöberg skrev:
-----Original Message----- From: henrik at hswn.dk [mailto:henrik at hswn.dk] could you try this patch, please?
Thanks. Can I just copy the new xymonnet binary after compiling, and skip the make install?
Yes, that will work fine.
Regards, Henrik
Ok, I have applied it on one of our machines. I'll wait and see if it makes any difference. The crashes are not too frequent, it has happened 3-4 times on either server after the upgrade.
/Johan
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
participants (4)
-
dominique.frise＠unil.ch
-
g.stone-tolcher＠its.uq.edu.au
-
henrik＠hswn.dk
-
Johan.Sjoberg＠deltamanagement.se