I know that hobbit can monitor URLs, of course.
I have a need to monitor a constantly changing list of URLs, which could number in the hundreds or thousands.
Will hobbit scale to this level, or should I instead write something custom using wget, which can take a file containing a list of URLs as input?
Thanks for your suggestions, and thanks for hobbit.
| Dan |
On Wed, May 31, 2006 at 12:57:42PM -0700, Dan Simoes wrote:
I know that hobbit can monitor URLs, of course.
I have a need to monitor a constantly changing list of URLs, which could number in the hundreds or thousands.
Will hobbit scale to this level, or should I instead write something custom using wget, which can take a file containing a list of URLs as input?
I'm obviously biased, but I think you'll find Hobbit a lot more efficient at testing your URL's than any script using wget.
Now, the way Hobbit's URL testing reports results might not be what you want - it is after all designed to report data to a Hobbit server. I don't know what you plan to do with all of your test results ... But I think it should be possible to tweak the network test tool to report in a format suitable for what you need.
Regards, Henrik
Thanks for your reply Henrik. I'm all for avoiding reinventing the wheel :)
Do you have any suggestions on how to mate a rapidly changing list of URLs with the bb-hosts file? Also, I assume that hobbit, like bb, requires a restart whenever bb-hosts changes?
Finally, has anyone compiled a list of differences between bb and hobbit, sort of like "Hobbit for BB admins"? I've encountered some minor differences here and there, which is not unexpected.
| Dan |
On 5/31/06, Henrik Stoerner <henrik at hswn.dk> wrote:
On Wed, May 31, 2006 at 12:57:42PM -0700, Dan Simoes wrote:
I know that hobbit can monitor URLs, of course.
I have a need to monitor a constantly changing list of URLs, which could number in the hundreds or thousands.
Will hobbit scale to this level, or should I instead write something custom using wget, which can take a file containing a list of URLs as input?
I'm obviously biased, but I think you'll find Hobbit a lot more efficient at testing your URL's than any script using wget.
Now, the way Hobbit's URL testing reports results might not be what you want - it is after all designed to report data to a Hobbit server. I don't know what you plan to do with all of your test results ... But I think it should be possible to tweak the network test tool to report in a format suitable for what you need.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Wed, May 31, 2006 at 01:34:48PM -0700, Dan Simoes wrote:
Do you have any suggestions on how to mate a rapidly changing list of URLs with the bb-hosts file?
How rapid is "rapid" ? Once or twice a day, or several times a minute ?
My immediate thought is to put an "include urlhosts.txt" in the bb-hosts file, and then have some script which updates the "urlhosts.txt" file with your dynamic list of URL's and the "hosts" you want to map them to. I don't know if all of your URL's pertain to the same host(s), but if you have several hundred then you may want to split them up in some way that makes sense for your setup. Remember, "hosts" (entries in the bb-hosts file) need not be real, physical servers. They are really just entities that you can connect with a series of tests.
So come up with a script to generate the urlhosts.txt file in bb-hosts format, like 0.0.0.0 web1 # noconn http://web1.foo.com/ https://web1.foo.com/breakingnews/ 0.0.0.0 web2 # noconn http://....
Also, I assume that hobbit, like bb, requires a restart whenever bb-hosts changes?
Nope. Hobbit automatically reloads the bb-hosts file every 10 minutes, or when you send it a HUP signal.
Finally, has anyone compiled a list of differences between bb and hobbit, sort of like "Hobbit for BB admins"? I've encountered some minor differences here and there, which is not unexpected.
I guess there are some differences, but I've been embroiled in Hobbit for so long I've lost track of how BB did things. So someone else will have to answer that one.
Regards, Henrik
participants (2)
-
dan.simoes＠gmail.com
-
henrik＠hswn.dk