Hi all, we are seeing a strange fping issue with Hobbit and I was
wondering if anyone may have seen the same.
We have changed the IP addresses on a large number of systems we are
monitoring due to moving them to a new location. We updated the bb-hosts file to have the new IPs, and DNS was also updated. However Hobbit occasionally falls back to testing the old IP for no apparent reason.
This happens quite regularly as well, as seen here:
Fri Nov 30 10:56:08 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:56:08_2007> 0:04:19
Fri Nov 30 10:48:31 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:48:31_2007> 0:07:37
Fri Nov 30 10:48:08 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:48:08_2007> 0:00:23
Fri Nov 30 10:42:27 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:42:27_2007> 0:05:41
Fri Nov 30 10:39:53 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:39:53_2007> 0:02:34
Fri Nov 30 10:36:49 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:36:49_2007> 0:03:04
Fri Nov 30 10:31:57 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:31:57_2007> 0:04:52
Fri Nov 30 10:25:27 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:25:27_2007> 0:06:30
Fri Nov 30 10:23:58 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:23:58_2007> 0:01:29
Fri Nov 30 10:19:19 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:19:19_2007> 0:04:39
Fri Nov 30 10:16:26 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:16:26_2007> 0:02:53
The conn status page when green shows the correct IP address, and
when red shows the wrong (old) IP address. Nothing in the bb-hosts or DNS change during this time.
I ran a "fping -AeC 1000 -i 5000 <host>" from the command line to
see if fping was switching IPs, and it does not seem to be, even when Hobbit shows red, the fping from the command line still his the new/current IP.
I've tried forcing a "testip" in bb-hosts to see if DNS was
involved, and with that forced it still changes in Hobbit.
Any suggestions would be much appreciated.
Thanks,
Shawn