[hobbit] fping failing error code 98
Thanks for the response Henrik,
The weird thing is that the files were created, and contained a mix of:
IP is alive (0.## ms) IP is unreachable
So it was able to ping some things, but not others.
Looking at the space utilization for that server, which only runs hobbit on the root file system, it hasn't gone over 8% utilization (8.7GB out of 115GB), it's currently up from 7% in the last month :), so unlikely that it was a space issue.
The morning it paged us at 4:15, that was the first thing I looked at, was did it create the file indicated in the hobbit log. I also cheked the df, and to see if there had been a router problem with the router that the server itself is plugged into. Our network engineers were unable to find any issues with that router. After the second "non-event", we decided to reboot to clear out what appeared to be defunct hobbit processes after 92 days of uptime. Not sure if these were related or not. Only time will tell. Another weird thing is that immediately after getting the pages and e-mails about the connection failures, we were getting pages and e-mails that everything had recovered, almost as if there was only a few seconds delay. In the time we've been running hobbit (and bb before), this is the first time something like this has happened.
Al
-----Original Message----- From: Henrik Stoerner [mailto:henrik at hswn.dk] Sent: Wednesday, June 14, 2006 5:28 PM To: hobbit at hswn.dk Subject: Re: [hobbit] fping failing error code 98
On Wed, Jun 14, 2006 at 03:27:53PM -0400, Cortes, Manny wrote:
On two occasions within a week hobbit alerted that several servers lost connectivity and then apparently recovered. Here is a sample of the error in the bb-network.log:
2006-06-11 04:16:07 Execution of '/usr/sbin/fping -Ae' failed with error-code 98
2006-06-11 04:16:07 Cannot open fping output file /hobbit/server/tmp/fping-stdout.2246
The error code basically means what it says: Hobbit couldn't create the output file it needs for fping output. In the newer versions it also prints out the system error-message which would have been helpful here, but you should look for a cause that prevents files from being created in this directory.
Most likely cause is that the /hobbit/server/tmp/ filesystem is full.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
This e-mail message and any attached files are confidential and are intended solely for the use of the addressee(s) named above. If you are not the intended recipient, any review, use, or distribution of this e-mail message and any attached files is strictly prohibited. This communication may contain material protected by Federal privacy regulations, attorney-client work product, or other privileges. If you have received this confidential communication in error, please notify the sender immediately by reply e-mail message and permanently delete the original message. To reply to our email administrator directly, send an email to: postmaster at orlandoregional.org . If this e-mail message concerns a contract matter, be advised that no employee or agent is authorized to conclude any binding agreement on behalf of Orlando Regional Healthcare by e-mail without express written confirmation by an officer of the corporation. Any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Orlando Regional Healthcare.
On Wed, Jun 14, 2006 at 05:49:00PM -0400, Jeffcoat, Al wrote:
Looking at the space utilization for that server, which only runs hobbit on the root file system, it hasn't gone over 8% utilization (8.7GB out of 115GB), it's currently up from 7% in the last month :), so unlikely that it was a space issue.
How about inodes ? Hobbit can use an extraordinary amount of inodes because it stores lots of small files (for historical logs). "df -i" is often useful.
Henrik
participants (2)
-
Al.Jeffcoat@orhs.org
-
henrik@hswn.dk