Whoops sent this before I finished typing
This is telling me some xymond_channel isn't exiting properly and it can't load? It's not telling me much about invalid data for hosts (which is where henrik pointed me back in the day)
On 3/15/13 3:31 PM, "Clark, Sean" <sean.clark at twcable.com> wrote:
Just as a note of perl vs straight C code
Using mysql libs & C to insert stachg channel -- handles about 1200 msgs/5 minutes before it starts flushing on a dual core machine with 8 GB RAM Same hardware using Perl, DBD:Mysql -- tops out @ about 300
/sw/xymon/server/bin/xymond --listen=127.0.0.1:1985 --debug --checkpoint-file=./xymond.chk.crashed
As to the debug loading of chk file:
31911 2013-03-15 15:23:17 Opening file /sw/xymon/server/etc/hosts.cfg 31911 2013-03-15 15:23:19 Opening file /sw/xymon/server/etc/client-local.cfg 2013-03-15 15:23:19 Setting up network listener on 127.0.0.1:1985 2013-03-15 15:23:19 Setting up signal handlers 2013-03-15 15:23:19 Setting up xymond channels 31911 2013-03-15 15:23:19 Setting up status channel (id=1) 31911 2013-03-15 15:23:19 calling ftok('/sw/xymon/server',1) 31911 2013-03-15 15:23:19 ftok() returns: 0x1000047 31911 2013-03-15 15:23:19 shmget() returns: 0xD6800C 2013-03-15 15:23:19 FATAL: xymond sees clientcount 1, should be 0 Check for hanging xymond_channel processes or stale semaphores 2013-03-15 15:23:19 Cannot setup status channel
That is telling me
On 3/15/13 2:41 PM, "cleaver at terabithia.org" <cleaver at terabithia.org> wrote:
That's odd. If you're on a box with a lot of memory, writing out to a tmpfs might help. For your worker, I'd suggest just adding a debug line or two in front of that section.
WRT the checkpoint file, the only real corruption I've seen myself has occurred when malformed utf-8 packets came in -- I'd accidentally included gzip output in a script I'd put in my /local directory :/.
You could try modifying the init startup/shutdown script to copy over the checkpoint file every once in a while, and then point a copy of xymond over to it in --debug mode and see if it chokes... and if so, how far in.
Thinking about it, a --validate flag to xymond might not be too hard to whip up.
Regards,
-jc
--- Original Message ---
Heh , I'd have to look at the whole stachg channel to find needle in haystack for that
Got a couple (once every 2-3 day) core dumps here:
Program terminated with signal 11, Segmentation fault. #0 main (argc=2, argv=0xbfd1a444) at xymond_mysql.c:371
xymond_mysql.c line 371: mysql_escape_string(timestamp,metadata[1],timestampbytes); Timestampbytes is strln of timestamp
I am not strong in C , however, so to find that needle, I wrote a perl version that pipes hist to mysql (that way, it logs exceptions etc etc), However, the perl version can't handle the rate of messages (between 300-500/sec)
Bleh
What I STRONGLY need help with is my xymond.chk getting corrupted - henrik looked at one a while back, and gave me something to look at/fix Which I did, but it's still getting corrupted (and then any time it crashes, lose all states)
Do you know of a good way to parse/manage the chk file to see what it doesn't like?
This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.