I am running the latest version of xymon, with J.C. Cleaver's patches.
I have a xymonproxy that sits dual homed, forwarding messages from one network segment to the main xymon display. It's roughly 4,000 hosts, about half using bbwin as their client, the other half are linux xymon clients.
The proxy is mostly working, but is dropping/retransmitting messages as well – the log indicates a problem with getting SIGPIPE like this:
2014-12-11 10:29:28.204598 conn 8182371, socket 238: error writing to socket: Broken pipe 2014-12-11 10:29:28.204608 conn 8182371, socket 238: attempting recovery from write error to server 0 (192.168.0.100:1984): Broken pipe 2014-12-11 10:29:28 Got SIGPIPE... 2014-12-11 10:29:28.208113 conn 8179105, socket 131: error writing to socket: Broken pipe 2014-12-11 10:29:28.208118 conn 8179105, socket 131: attempting recovery from write error to server 0 (192.168.0.100:1984): Broken pipe
I've played around with lqueue settings, and timeout settings, to no avail – it seems to constantly be posting those messages in log no matter what.
Ulimit for open files for xymon is 65535, and I don't see it going above 200-300 open files
The report lists this under resources: (also looks like there is a spelling error for resource in xymonproxy.c, which is duplicated in the man pages)
Proxy ressources
- Connection table size : 134
- Buffer space : 6504124 kByte
Timeout/failure details
- reading from client : 24
- connecting to server : 0
- sending to bfq : 0
- bfq recovered : 0
- sending to server : 593820
- recovered : 53130
- reading from server : 24
- sending to client : 0
On the receiving server, it tells me in xymond.log --> Discarding timed-out partial msg from X.X.X.X
Wondering what the next steps I can take for troubleshooting/fixing — any thoughts? Tweaks on receiving server? Changes to xymonproxy? Network pathing issues?
-Sean
This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.