[hobbit] Large number of TIME_WAITs
I noticed something strange on my Hobbit server, though I'm not sure if it's a problem or not. There are about 140 TIME_WAITs from the local IP of the Hobbit server, and about 30 TIME_WAITs from localhost, all using port 1984. They look something like this:
0 0 1.2.3.4:42389 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42394 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42399 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42396 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42397 1.2.3.4:1984 TIME_WAIT
0 0 127.0.0.1:42310 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42309 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42315 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42322 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42320 127.0.0.1:1984 TIME_WAIT
There are also a few on various other ports: 0 0 127.0.0.1:42246 127.0.0.1:29021 TIME_WAIT 0 0 127.0.0.1:42301 127.0.0.1:29023 TIME_WAIT 0 0 127.0.0.1:42362 127.0.0.1:29017 TIME_WAIT 0 0 127.0.0.1:29030 127.0.0.1:42380 TIME_WAIT 0 0 127.0.0.1:42348 127.0.0.1:29019 TIME_WAIT
Any idea why there should be so many of them, particularly on the localhost address? And any idea if this looks abnormal or not?
What OS? Some operating systems allow network connections to hang around a bit after disconnection. On Solaris you can set a parameter to reduce the timeout period. Hobbit could trigger this because each client connects to the server, sends a data blast, and then disconnects.
From: Gary Baluha [mailto:gumby3203 at gmail.com]
Sent: Tuesday, January 29, 2008 8:58 AM
To: hobbit at hswn.dk
Subject: [hobbit] Large number of TIME_WAITs
I noticed something strange on my Hobbit server, though I'm notsure if it's a problem or not. There are about 140 TIME_WAITs from the local IP of the Hobbit server, and about 30 TIME_WAITs from localhost, all using port 1984. They look something like this:
0 0 1.2.3.4:42389 1.2.3.4:1984 TIME_WAIT
0 0 1.2.3.4:42394 1.2.3.4:1984 TIME_WAIT
0 0 1.2.3.4:42399 1.2.3.4:1984 TIME_WAIT
0 0 1.2.3.4:42396 1.2.3.4:1984 TIME_WAIT
0 0 1.2.3.4:42397 1.2.3.4:1984 TIME_WAIT
0 0 127.0.0.1:42310 127.0.0.1:1984TIME_WAIT 0 0 127.0.0.1:42309 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42315 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42322 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42320 127.0.0.1:1984 TIME_WAIT
There are also a few on various other ports:
0 0 127.0.0.1:42246 127.0.0.1:29021TIME_WAIT 0 0 127.0.0.1:42301 127.0.0.1:29023 TIME_WAIT 0 0 127.0.0.1:42362 127.0.0.1:29017 TIME_WAIT 0 0 127.0.0.1:29030 127.0.0.1:42380 TIME_WAIT 0 0 127.0.0.1:42348 127.0.0.1:29019 TIME_WAIT
Any idea why there should be so many of them, particularly onthe localhost address? And any idea if this looks abnormal or not?
Red Hat ES 4 Update 5. That's what I normally would think, but it doesn't make any sense that the source AND destination are both localhost or the local IP. It's as though the Hobbit server is trying to communicate with itself using network sockets. It would make more sense if either the source or destination was a client IP, but that isn't the case.
On Jan 29, 2008 10:15 AM, Hubbard, Greg L <greg.hubbard at eds.com> wrote:
What OS? Some operating systems allow network connections to hang around a bit after disconnection. On Solaris you can set a parameter to reduce the timeout period. Hobbit could trigger this because each client connects to the server, sends a data blast, and then disconnects.
*From:* Gary Baluha [mailto:gumby3203 at gmail.com] *Sent:* Tuesday, January 29, 2008 8:58 AM *To:* hobbit at hswn.dk *Subject:* [hobbit] Large number of TIME_WAITs
I noticed something strange on my Hobbit server, though I'm not sure if it's a problem or not. There are about 140 TIME_WAITs from the local IP of the Hobbit server, and about 30 TIME_WAITs from localhost, all using port 1984. They look something like this:
0 0 1.2.3.4:42389 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42394 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42399 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42396 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42397 1.2.3.4:1984 TIME_WAIT
0 0 127.0.0.1:42310 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42309 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42315 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42322 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42320 127.0.0.1:1984 TIME_WAIT
There are also a few on various other ports: 0 0 127.0.0.1:42246 127.0.0.1:29021 TIME_WAIT 0 0 127.0.0.1:42301 127.0.0.1:29023 TIME_WAIT 0 0 127.0.0.1:42362 127.0.0.1:29017 TIME_WAIT 0 0 127.0.0.1:29030 127.0.0.1:42380 TIME_WAIT 0 0 127.0.0.1:42348 127.0.0.1:29019 TIME_WAIT
Any idea why there should be so many of them, particularly on the localhost address? And any idea if this looks abnormal or not?
Wouldn't it make sense if you had a Hobbit client running on the same server as the Hobbit server?
=G=
From: Gary Baluha [mailto:gumby3203 at gmail.com] Sent: Tuesday, January 29, 2008 10:36 AM To: hobbit at hswn.dk Subject: Re: [hobbit] Large number of TIME_WAITs
Red Hat ES 4 Update 5. That's what I normally would think, but it doesn't make any sense that the source AND destination are both localhost or the local IP. It's as though the Hobbit server is trying to communicate with itself using network sockets. It would make more sense if either the source or destination was a client IP, but that isn't the case.
On Jan 29, 2008 10:15 AM, Hubbard, Greg L <greg.hubbard at eds.com> wrote:
What OS? Some operating systems allow network connections to hang around a bit after disconnection. On Solaris you can set a parameter to reduce the timeout period. Hobbit could trigger this because each client connects to the server, sends a data blast, and then disconnects.
From: Gary Baluha [mailto:gumby3203 at gmail.com]
Sent: Tuesday, January 29, 2008 8:58 AM
To: hobbit at hswn.dk
Subject: [hobbit] Large number of TIME_WAITs
I noticed something strange on my Hobbit server, though I'm notsure if it's a problem or not. There are about 140 TIME_WAITs from the local IP of the Hobbit server, and about 30 TIME_WAITs from localhost, all using port 1984. They look something like this:
0 0 1.2.3.4:42389 1.2.3.4:1984 TIME_WAIT
0 0 1.2.3.4:42394 1.2.3.4:1984 TIME_WAIT
0 0 1.2.3.4:42399 1.2.3.4:1984 TIME_WAIT
0 0 1.2.3.4:42396 1.2.3.4:1984 TIME_WAIT
0 0 1.2.3.4:42397 1.2.3.4:1984 TIME_WAIT
0 0 127.0.0.1:42310 127.0.0.1:1984TIME_WAIT 0 0 127.0.0.1:42309 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42315 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42322 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42320 127.0.0.1:1984 TIME_WAIT
There are also a few on various other ports:
0 0 127.0.0.1:42246 127.0.0.1:29021TIME_WAIT 0 0 127.0.0.1:42301 127.0.0.1:29023 TIME_WAIT 0 0 127.0.0.1:42362 127.0.0.1:29017 TIME_WAIT 0 0 127.0.0.1:29030 127.0.0.1:42380 TIME_WAIT 0 0 127.0.0.1:42348 127.0.0.1:29019 TIME_WAIT
Any idea why there should be so many of them, particularly onthe localhost address? And any idea if this looks abnormal or not?
Wouldn't it make sense if you had a Hobbit client running on the same server as the Hobbit server?
But that many TIME_WAITs?
=G=
*From:* Gary Baluha [mailto:gumby3203 at gmail.com] *Sent:* Tuesday, January 29, 2008 10:36 AM *To:* hobbit at hswn.dk *Subject:* Re: [hobbit] Large number of TIME_WAITs
Red Hat ES 4 Update 5. That's what I normally would think, but it doesn't make any sense that the source AND destination are both localhost or the local IP. It's as though the Hobbit server is trying to communicate with itself using network sockets. It would make more sense if either the source or destination was a client IP, but that isn't the case.
On Jan 29, 2008 10:15 AM, Hubbard, Greg L <greg.hubbard at eds.com> wrote:
What OS? Some operating systems allow network connections to hang around a bit after disconnection. On Solaris you can set a parameter to reduce the timeout period. Hobbit could trigger this because each client connects to the server, sends a data blast, and then disconnects.
*From:* Gary Baluha [mailto:gumby3203 at gmail.com] *Sent:* Tuesday, January 29, 2008 8:58 AM *To:* hobbit at hswn.dk *Subject:* [hobbit] Large number of TIME_WAITs
I noticed something strange on my Hobbit server, though I'm not sure if it's a problem or not. There are about 140 TIME_WAITs from the local IP of the Hobbit server, and about 30 TIME_WAITs from localhost, all using port 1984. They look something like this:
0 0 1.2.3.4:42389 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42394 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42399 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42396 1.2.3.4:1984 TIME_WAIT 0 0 1.2.3.4:42397 1.2.3.4:1984 TIME_WAIT
0 0 127.0.0.1:42310 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42309 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42315 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42322 127.0.0.1:1984 TIME_WAIT 0 0 127.0.0.1:42320 127.0.0.1:1984 TIME_WAIT
There are also a few on various other ports: 0 0 127.0.0.1:42246 127.0.0.1:29021 TIME_WAIT 0 0 127.0.0.1:42301 127.0.0.1:29023 TIME_WAIT 0 0 127.0.0.1:42362 127.0.0.1:29017 TIME_WAIT 0 0 127.0.0.1:29030 127.0.0.1:42380 TIME_WAIT 0 0 127.0.0.1:42348 127.0.0.1:29019 TIME_WAIT
Any idea why there should be so many of them, particularly on the localhost address? And any idea if this looks abnormal or not?
On Tuesday 29 January 2008 17:53:59 Gary Baluha wrote:
Wouldn't it make sense if you had a Hobbit client running on the same server as the Hobbit server?
But that many TIME_WAITs?
On a server receiving monitoring information and monitoring itself a total of ~ 150 hosts, I have ~ 500 hobbit ports in TIME_WAIT, and about 200 htto ports in TIME_WAIT.
More answers in-line (ug, thread re-assembly ...).
=G=
*From:* Gary Baluha [mailto:gumby3203 at gmail.com] *Sent:* Tuesday, January 29, 2008 10:36 AM *To:* hobbit at hswn.dk *Subject:* Re: [hobbit] Large number of TIME_WAITs
Red Hat ES 4 Update 5.
same on mine.
That's what I normally would think, but it doesn't make any sense that the source AND destination are both localhost or the local IP. It's as though the Hobbit server is trying to communicate with itself using network sockets. It would make more sense if either the source or destination was a client IP, but that isn't the case.
It is the case, bbtest-net connects to hobbitd. The client on the server also does. Any other server-side extensions may. bbgen may also (haven't checked, but I assume ...). The cgi's will probably also ...
On Jan 29, 2008 10:15 AM, Hubbard, Greg L <greg.hubbard at eds.com> wrote:
What OS? Some operating systems allow network connections to hang around a bit after disconnection. On Solaris you can set a parameter to reduce the timeout period.
On Linux, it is net.ipv4.tcp_fin_timeout, e.g.:
net.ipv4.tcp_fin_timeout = 30 in /etc/sysctl.conf, defaults to 60s.
Regards, Buchan
On Tue, Jan 29, 2008 at 10:36:17AM -0500, Gary Baluha wrote:
Red Hat ES 4 Update 5. That's what I normally would think, but it doesn't make any sense that the source AND destination are both localhost or the local IP. It's as though the Hobbit server is trying to communicate with itself using network sockets.
It makes perfect sense, actually. Various parts of Hobbit does talk to the hobbitd daemon - most of the webpages get their data this way, and all of the network test results are also reported using a network connection.
TIME_WAIT states aren't evil, just confusing if you don't know what they are for. They will clear themselves after a few seconds.
Regards, Henrik
On Jan 29, 2008 4:19 PM, Henrik Stoerner <henrik at hswn.dk> wrote:
On Tue, Jan 29, 2008 at 10:36:17AM -0500, Gary Baluha wrote:
Red Hat ES 4 Update 5. That's what I normally would think, but it doesn't make any sense that the source AND destination are both localhost or the local IP. It's as though the Hobbit server is trying to communicate with itself using network sockets.
It makes perfect sense, actually. Various parts of Hobbit does talk to the hobbitd daemon - most of the webpages get their data this way, and all of the network test results are also reported using a network connection.
TIME_WAIT states aren't evil, just confusing if you don't know what they are for. They will clear themselves after a few seconds.
Indeed. Thanks for everyone's input.
All,
I have a question about the client side of things. I havefigured out how to get the client installed on one box and have started it but I'm getting an error that I can't figure out. Any suggestions would be helpful. This is the error in the hobbitclient.log that I'm getting:
ld.so.1: hobbitd_client: fatal: libpcre.so.0: open failed: Nosuch file or directory
Killed
Any Ideas?HI.
This error is due to the problem that pcre is not installed in the box or the path of library or include path is mentioned.
Thanks & Regards,
Arunkumar
From: Snyder, Howard [mailto:HS3082 at att.com] Sent: Tuesday, January 29, 2008 9:10 PM To: hobbit at hswn.dk Subject: [hobbit] Client Question
All,
I have a question about the client side of things. I havefigured out how to get the client installed on one box and have started it but I'm getting an error that I can't figure out. Any suggestions would be helpful. This is the error in the hobbitclient.log that I'm getting:
ld.so.1: hobbitd_client: fatal: libpcre.so.0: open failed: Nosuch file or directory
Killed
Any Ideas?That is kinda what I thought. I have installed pcre twice and still get the error and no data to the server.
Thank you,
Howard Snyder
hs3082 at att.com
SR Network Control Engineer
Network Services
404-531-5325
This email, and any attachments, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. It is the property of Cingular Wireless and its Affiliates. If you are not the intended recipient of this email, you are hereby notified that any dissemination, distribution or copying of this email, any attachments thereto, and any use of the information contained is strictly prohibited. If you have received this email in error, please notify me at 404.531.5325 and permanently delete the original and any copy thereof.
For login requests and password resets, you will need to open a request on the "My Logins" website:
https://nslogins.edc.cingular.net/login.cfm?CFID=13246&CFTOKEN=41726264 <https://nslogins.edc.cingular.net/login.cfm?CFID=13246&CFTOKEN=41726264
For OSS problems:
DL-MNOC-OSS Adjunct
DL-MNOC-OSS Siemens
DL-MNOC-OSS Nortel
DL-MNOC-OSS Nokia
DL-MNOC-OSS Lucent
DL-MNOC-OSS Ericsson
From: Thangappandi, Arun (Ge Money, consultant) [mailto:arun.thangappandi at ge.com] Sent: Tuesday, January 29, 2008 10:53 AM To: hobbit at hswn.dk Subject: RE: [hobbit] Client Question
HI.
This error is due to the problem that pcre is not installed in the box or the path of library or include path is mentioned.
Thanks & Regards,
Arunkumar
From: Snyder, Howard [mailto:HS3082 at att.com] Sent: Tuesday, January 29, 2008 9:10 PM To: hobbit at hswn.dk Subject: [hobbit] Client Question
All,
I have a question about the client side of things. I havefigured out how to get the client installed on one box and have started it but I'm getting an error that I can't figure out. Any suggestions would be helpful. This is the error in the hobbitclient.log that I'm getting:
ld.so.1: hobbitd_client: fatal: libpcre.so.0: open failed: Nosuch file or directory
Killed
Any Ideas?What OS? If linux, you'll need to make sure the dev libraries are installed. If Solaris, you may need to add the lib path to the system lib path (man crle).
=G=
From: Snyder, Howard [mailto:HS3082 at att.com] Sent: Tuesday, January 29, 2008 12:00 PM To: hobbit at hswn.dk Subject: RE: [hobbit] Client Question
That is kinda what I thought. I have installed pcre twice and still get the error and no data to the server.
Thank you,
Howard Snyder
hs3082 at att.com
SR Network Control Engineer
Network Services
404-531-5325
This email, and any attachments, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. It is the property of Cingular Wireless and its Affiliates. If you are not the intended recipient of this email, you are hereby notified that any dissemination, distribution or copying of this email, any attachments thereto, and any use of the information contained is strictly prohibited. If you have received this email in error, please notify me at 404.531.5325 and permanently delete the original and any copy thereof.
For login requests and password resets, you will need to open a request on the "My Logins" website:
https://nslogins.edc.cingular.net/login.cfm?CFID=13246&CFTOKEN=41726264
For OSS problems:
DL-MNOC-OSS Adjunct
DL-MNOC-OSS Siemens
DL-MNOC-OSS Nortel
DL-MNOC-OSS Nokia
DL-MNOC-OSS Lucent
DL-MNOC-OSS Ericsson
From: Thangappandi, Arun (Ge Money, consultant) [mailto:arun.thangappandi at ge.com] Sent: Tuesday, January 29, 2008 10:53 AM To: hobbit at hswn.dk Subject: RE: [hobbit] Client Question
HI.
This error is due to the problem that pcre is not installed in the box or the path of library or include path is mentioned.
Thanks & Regards,
Arunkumar
From: Snyder, Howard [mailto:HS3082 at att.com] Sent: Tuesday, January 29, 2008 9:10 PM To: hobbit at hswn.dk Subject: [hobbit] Client Question
All,
I have a question about the client side of things. I havefigured out how to get the client installed on one box and have started it but I'm getting an error that I can't figure out. Any suggestions would be helpful. This is the error in the hobbitclient.log that I'm getting:
ld.so.1: hobbitd_client: fatal: libpcre.so.0: open failed: Nosuch file or directory
Killed
Any Ideas?
participants (7)
-
arun.thangappandi＠ge.com
-
bgmilne＠staff.telkomsa.net
-
Galen.Johnson＠sas.com
-
greg.hubbard＠eds.com
-
gumby3203＠gmail.com
-
henrik＠hswn.dk
-
HS3082＠att.com