Thank you very much, Ralph, you saved me a lot of time. I really owe you I think I am not the only one facing such need. Don't you think that your scripts can be placed in Xymonton for easy access?
Regards Ricardas
Ralph Mitchell <ralphmitchell at gmail.com> wrote on 25/05/2009 18:23:48:
It was titled "Hobbit content check script". Here's the body of the message, and the attachments:
Sorry I didn't get back to you earlier - I've been fighting with perl & SOAP, trying to get some status messages out of a bunch of load balancers. I think I finally got it now....
Attached (hopefully :) are a couple of files I tried to post to the list some time last year. There's a shell script that does a login to a Siteminder site, and a perl script evolved from something Daniel Stenberg wrote - he's the original author of curl. Daniel's very active on the curl mailing list, by the way, along with a bunch of other helpful folks, if it starts going weird on you.
The Siteminder login probably won't be much use to you as is, but it demonstrates a few things I've worked out over the years. Like, the standard options string to pass to curl, collecting messages along the way, and loading up the message string with a Manual Check url at the end for ops to try the check for themselves before waking people up...
You can put almost *anything* into the message string that gets sent to Hobbit. Right now I have scripts that check car rentals at travel sites for a particular company and emit a table of the current prices. Another script hits a cruise monitor and puts out a list using the &red and &green dots to mark cruise lines as reachable or not. You can push out font color and size changes, urls to saved pages, etc.
As far as cookies go, I generally just have curl save them whenever it finishes:
curl -b cookies -c cookies .....
If you do that you can go look at the cookies and see what came in. Other things become possible too, such as editing cookies in between curl calls. If you do keep cookies, though, the *first* thing you should do in the script is delete the cookie file. That's like clearing the browser cookie cache, and can avoid a lot of mysterious errors.
Finally, if you think you might spend some time on web site scripts, there's a Firefox extension that is absolutely invaluable - LiveHTTPheaders. It shows you *exactly* what the browser sends to the server, as well as what it gets back - not the page content, just the headers, but that's all you need to recreate the sequence. It even works twith secure sites. If you see something going out that you can't account for, the usual suspect is javascript. I've seen javascript that assembles a url on the fly from form elements, fiddles with some of the form elements, then posts the form. That kind of thing can be difficult to nail down, and almost impossible to replicate...
Ralph Mitchell 2009/5/25 Ricardas Vaitkus <Ricardas.Vaitkus at seb.lt> Hi Ralph
I tried to find your message in mail archive, but I couldn't. Can you tell me the title of your message, because by date I was not able to find it.
Thanks Ricardas
Ralph Mitchell <ralphmitchell at gmail.com> wrote on 22/05/2009 19:48:51:
I did a whole lot of that for my last employer. As far as I know they still use my scripts to run about 2500 checks on roughly 300 servers.
See my message on Jun 7, 2007 at 12:08 AM in the mail archive for details and a sample script.
Ralph Mitchell
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
[attachment "hobbitcontentcheckscript.zip" deleted by Ricardas Vaitkus/VBankas]