urlplus.pl http/https monitoring extension script
We are in the process of replacing several sitescope monitors with Hobbit, and I have found some of the built-in Hobbit http checks inadequate. As a result, I have been working on an extension script which at its core is essentially a wrapper for curl. However, it provides for more robust user-customization than is easily possible with Hobbit's built-in tests. It is still a work-in-progress, but I believe it has matured enough to be worth posting on The Shire. I wrote it using perl, but I have a feeling that some point in the future it may need to be converted to C (mostly for the speed). However, it seems to work pretty good for me now.
The main feature I required from this script was the addition of a per-host customizable timeout period. Some of the hosts I'm testing take up to a minute to return back any data, and this is considered normal. However, I didn't want to bump up the timeout for _all_ hosts. Admittedly, this script uses an additional configuration file and doesn't completely integrate with the bb-hosts file, but it works for me and helps keep the bb-hosts file clean. I'd be interested in what other people think of this script, which is the main reason I'm contributing it now. I have attached it to this post (I haven't yet subscribed to theshire-developers, so I'm not able to upload it myself yet).
I can take care of adding it to the Shire later tonight, Gary. If you would, fill out the following and email me offlist: http://www.trantor.org/theshire/doku.php/monitors:template.
=G=
From: Gary Baluha [mailto:gumby3203 at gmail.com] Sent: Monday, January 28, 2008 10:55 AM To: hobbit at hswn.dk; Theshire-announce at lists.sourceforge.net Subject: [hobbit] urlplus.pl http/https monitoring extension script
We are in the process of replacing several sitescope monitors with Hobbit, and I have found some of the built-in Hobbit http checks inadequate. As a result, I have been working on an extension script which at its core is essentially a wrapper for curl. However, it provides for more robust user-customization than is easily possible with Hobbit's built-in tests. It is still a work-in-progress, but I believe it has matured enough to be worth posting on The Shire. I wrote it using perl, but I have a feeling that some point in the future it may need to be converted to C (mostly for the speed). However, it seems to work pretty good for me now.
The main feature I required from this script was the addition of a per-host customizable timeout period. Some of the hosts I'm testing take up to a minute to return back any data, and this is considered normal. However, I didn't want to bump up the timeout for _all_ hosts. Admittedly, this script uses an additional configuration file and doesn't completely integrate with the bb-hosts file, but it works for me and helps keep the bb-hosts file clean. I'd be interested in what other people think of this script, which is the main reason I'm contributing it now. I have attached it to this post (I haven't yet subscribed to theshire-developers, so I'm not able to upload it myself yet).
Whoops, forgot to include the template information...
==== urlplus.pl ==== Summary
- Download: [[monitors:monitor|monitor]]
- Author: gumby3203 at gmail.com|Gary Baluha
- Compatible with: Linux, OpenBSD
- Last Update: 2008-01-28
\\--------------------------------------------------------------------
====== Monitor ======
^ Compatibility | Hobbit 4.2 | ^ Requirements | Perl, unix, curl | ^ Download | None | ^ Last Update | 2008-01-28 |
===== Description ===== Provides a slightly more robust form of URL content checking that Hobbit does out of the box. Per-host user-configuration is provided.
===== Installation ===== === Client side === N/A
=== Server side ===
- Copy urlplus.pl to $BBHOME/ext
- Create an new configuration file: $BBHOME/etc/cont-check.cfg
===== Source ===== ==== Monitor ==== <code> </code>
===== Known Bugs and Issues ===== None at this time
===== To Do ===== Need more testing on a larger variety of hosts, mostly for inclusion of new features
===== Credits =====
===== Changelog =====
- **2008-01-02**
- Initial creation
- **2008-01-28**
- Initial release to The Shire
On Jan 28, 2008 10:54 AM, Gary Baluha <gumby3203 at gmail.com> wrote:
We are in the process of replacing several sitescope monitors with Hobbit, and I have found some of the built-in Hobbit http checks inadequate. As a result, I have been working on an extension script which at its core is essentially a wrapper for curl. However, it provides for more robust user-customization than is easily possible with Hobbit's built-in tests. It is still a work-in-progress, but I believe it has matured enough to be worth posting on The Shire. I wrote it using perl, but I have a feeling that some point in the future it may need to be converted to C (mostly for the speed). However, it seems to work pretty good for me now.
The main feature I required from this script was the addition of a per-host customizable timeout period. Some of the hosts I'm testing take up to a minute to return back any data, and this is considered normal. However, I didn't want to bump up the timeout for _all_ hosts. Admittedly, this script uses an additional configuration file and doesn't completely integrate with the bb-hosts file, but it works for me and helps keep the bb-hosts file clean. I'd be interested in what other people think of this script, which is the main reason I'm contributing it now. I have attached it to this post (I haven't yet subscribed to theshire-developers, so I'm not able to upload it myself yet).
I've added it...I'll send out an official announcement when I get home tonight.
From: Gary Baluha [mailto:gumby3203 at gmail.com] Sent: Monday, January 28, 2008 11:07 AM To: hobbit at hswn.dk; Theshire-announce at lists.sourceforge.net Subject: [hobbit] Re: urlplus.pl http/https monitoring extension script
Whoops, forgot to include the template information...
==== urlplus.pl ====
Summary
Download: [[monitors:monitor|monitor]]
Author: gumby3203 at gmail.com|Gary Baluha
Compatible with: Linux, OpenBSD
Last Update: 2008-01-28
\\--------------------------------------------------------------------
====== Monitor ======
^ Compatibility | Hobbit 4.2 |
^ Requirements | Perl, unix, curl |
^ Download | None |
^ Last Update | 2008-01-28 |
===== Description =====
Provides a slightly more robust form of URL content checking that Hobbit does out of the box.
Per-host user-configuration is provided.
===== Installation =====
=== Client side ===
N/A
=== Server side ===
Copy urlplus.pl to $BBHOME/ext
Create an new configuration file: $BBHOME/etc/cont-check.cfg
===== Source =====
==== Monitor ====
<code>
</code>
===== Known Bugs and Issues =====
None at this time
===== To Do =====
Need more testing on a larger variety of hosts, mostly for inclusion of new features
===== Credits =====
===== Changelog =====
**2008-01-02**
- Initial creation
**2008-01-28**
- Initial release to The Shire
On Jan 28, 2008 10:54 AM, Gary Baluha <gumby3203 at gmail.com> wrote:
We are in the process of replacing several sitescope monitors with Hobbit, and I have found some of the built-in Hobbit http checks inadequate. As a result, I have been working on an extension script which at its core is essentially a wrapper for curl. However, it provides for more robust user-customization than is easily possible with Hobbit's built-in tests. It is still a work-in-progress, but I believe it has matured enough to be worth posting on The Shire. I wrote it using perl, but I have a feeling that some point in the future it may need to be converted to C (mostly for the speed). However, it seems to work pretty good for me now.
The main feature I required from this script was the addition of a per-host customizable timeout period. Some of the hosts I'm testing take up to a minute to return back any data, and this is considered normal. However, I didn't want to bump up the timeout for _all_ hosts. Admittedly, this script uses an additional configuration file and doesn't completely integrate with the bb-hosts file, but it works for me and helps keep the bb-hosts file clean. I'd be interested in what other people think of this script, which is the main reason I'm contributing it now. I have attached it to this post (I haven't yet subscribed to theshire-developers, so I'm not able to upload it myself yet).
On Jan 28, 2008 9:54 AM, Gary Baluha <gumby3203 at gmail.com> wrote:
We are in the process of replacing several sitescope monitors with Hobbit, and I have found some of the built-in Hobbit http checks inadequate. As a result, I have been working on an extension script which at its core is essentially a wrapper for curl. However, it provides for more robust user-customization than is easily possible with Hobbit's built-in tests. It is still a work-in-progress, but I believe it has matured enough to be worth posting on The Shire. I wrote it using perl, but I have a feeling that some point in the future it may need to be converted to C (mostly for the speed). However, it seems to work pretty good for me now.
The main feature I required from this script was the addition of a per-host customizable timeout period. Some of the hosts I'm testing take up to a minute to return back any data, and this is considered normal. However, I didn't want to bump up the timeout for _all_ hosts. Admittedly, this script uses an additional configuration file and doesn't completely integrate with the bb-hosts file, but it works for me and helps keep the bb-hosts file clean. I'd be interested in what other people think of this script, which is the main reason I'm contributing it now. I have attached it to this post (I haven't yet subscribed to theshire-developers, so I'm not able to upload it myself yet).
That looks like the kind of thing I've been doing in bash scripts using curl. I've thought about switching to a different language, but I don't know enough Perl to get started. Maybe I'll take another look... :)
Ralph Mitchell
That looks like the kind of thing I've been doing in bash scripts using curl. I've thought about switching to a different language, but I don't know enough Perl to get started. Maybe I'll take another look... :)
For the kind of work that this script is doing, there weren't really any reasons to require perl over shell script. I picked perl because I've been doing a lot of perl scripting lately, but also because it should be easier to convert the perl code to C at some point, if I feel that is necessary.
As far as speed goes, I've found there isn't much of a difference between perl and shell, though in certain cases I've seen shell script a little quicker. I'm sure the usual "your mileage may vary" quote is worth throwing out at this point ;-)
I'm not doing anything really perl-crazy with this script, and I think I have it pretty well commented, so it should be a decent reference if you want to get in to perl. Either way, I'd be interested in comparing notes in what our two scripts do, and possibly combining features from yours into this one.
On Jan 28, 2008 11:07 AM, Gary Baluha <gumby3203 at gmail.com> wrote:
That looks like the kind of thing I've been doing in bash scripts using
curl. I've thought about switching to a different language, but I don't know enough Perl to get started. Maybe I'll take another look... :)
For the kind of work that this script is doing, there weren't really any reasons to require perl over shell script. I picked perl because I've been doing a lot of perl scripting lately, but also because it should be easier to convert the perl code to C at some point, if I feel that is necessary.
As far as speed goes, I've found there isn't much of a difference between perl and shell, though in certain cases I've seen shell script a little quicker. I'm sure the usual "your mileage may vary" quote is worth throwing out at this point ;-)
Yeah, my mileage varies - I've got around 100 scripts running checks on 655 hosts (the number varies daily) generating over 2700 messages. I've got a fairly steady load average of around 6.5 on a single-cpu 733MHz DL380. I'm supposed to be getting a faster system, while at the same time the company keeps trying to "replace Hobbit/Big Brother with XXX", so I'm not holding my breath on that.
I'm not doing anything really perl-crazy with this script, and I think I
have it pretty well commented, so it should be a decent reference if you want to get in to perl. Either way, I'd be interested in comparing notes in what our two scripts do, and possibly combining features from yours into this one.
I think most of my scripts are fairly simple-minded - grab a page, grep for something, flag red/green - but some of them dig a lot deeper. Login, navigate through some pages, logout again. I won't name the worst one I have to deal with, but the first page does a redirect using a "meta refresh" tag, the next page does "self.location=/web/guest/home" in javascript, followed by "window.location=/c/portal/login" to get to the login page. After posting the login form, I get another "self.location" that takes me to a page containing another form that self-submits via onLoad in the <body> tag. After all that it's relatively trivial to extract the logout url to exit cleanly from the site. Loads of fun...
Ralph Mitchell
participants (3)
-
Galen.Johnson＠sas.com
-
gumby3203＠gmail.com
-
ralphmitchell＠gmail.com