Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
I wrote something like that when I was with EDS. It grabs an image and compares the md5sum to the md5sum of a stored copy of the image. If it's different, sound the alarm. I can probably dredge that up this evening when I get home. In the meantime, it would not be too hard to reconstruct.
If it's just the image that isn't being updated, then grab a copy with curl:
curl -s -S -L -o image.jpg http://www.server.com/img.jpg
and md5sum it. Compare that to the md5sum of the last image you fetched. If different, you can then send a red report, otherwise send green.
If the page itself is not being update, just have the tcsh script insert a timestamp as a comment, then grab the page in much the same way and grep the timestamp from it. If the timestamp varies from real time by several minutes, sound the alarm.
Ralph Mitchell
On Fri, Jul 29, 2011 at 10:40 AM, Josh Luthman <josh at imaginenetworksllc.com>wrote:
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
On 29-07-2011 17:00, Ralph Mitchell wrote:
I wrote something like that when I was with EDS. It grabs an image and compares the md5sum to the md5sum of a stored copy of the image. If it's different, sound the alarm. I can probably dredge that up this evening when I get home. In the meantime, it would not be too hard to reconstruct.
Xymon has this built-in.
10.0.0.1 www # cont;http://www/img.jpg;#md5:83299f5516985377e3911fd95d0b0038
Regards, Henrik
Couldn't you monitor the image file modification date on the server?
Or have the script send a message to a log file on success and monitor in logs?
Paul Root - Engineer III - Qwest is becoming CenturyLink
From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Josh Luthman Sent: Friday, July 29, 2011 9:41 AM To: xymon at xymon.com Subject: [Xymon] Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
couple random thoughts: Would HTTP cache control headers help in this regard? You can poll a page via wget or some other mechanism and look at the page headers, that won't tell you if the image is old but if the page is regenerated on each image update it may work... If you have shell access to the server:/image.file you could test the mac times... Or, toss in some logic to catch the bugz :)
From: xymon-bounces at xymon.com [xymon-bounces at xymon.com] On Behalf Of Josh Luthman [josh at imaginenetworksllc.com] Sent: Friday, July 29, 2011 7:40 AM To: xymon at xymon.com Subject: [Xymon] Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
The page is set to expire immediately and usually the cache problems are the browser/client. At least in my experience.
On hobbit-clients.cfg for the host (it has proc tests already) I added:
FILE /var/www/foo/bar/output.png red mtime<1800 #go red if file older than 30 minutes
How do I get the client to send the details about this file? From reading the man page I need to specify client-local.cfg but I am not sure what goes in this. I'm pretty sure I want:
file:/var/www/foo/bar/output.png
Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box?
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Fri, Jul 29, 2011 at 11:58 AM, Tim McCloskey <tm at freedom.com> wrote:
couple random thoughts: Would HTTP cache control headers help in this regard? You can poll a page via wget or some other mechanism and look at the page headers, that won't tell you if the image is old but if the page is regenerated on each image update it may work... If you have shell access to the server:/image.file you could test the mac times... Or, toss in some logic to catch the bugz :)
From: xymon-bounces at xymon.com [xymon-bounces at xymon.com] On Behalf Of Josh Luthman [josh at imaginenetworksllc.com] Sent: Friday, July 29, 2011 7:40 AM To: xymon at xymon.com Subject: [Xymon] Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
If your system is setup to store the client data on the central server you can have something like the following in hobbit-clients.cfg.
HOST=somehost GROUP=optional-some-group FILE /some/path/tofile.ext SIZE<55M and SIZE>0 red
Client-local can take a host name too -- I think....
From: Josh Luthman [josh at imaginenetworksllc.com] Sent: Friday, July 29, 2011 11:24 AM To: Tim McCloskey Cc: xymon at xymon.com Subject: Re: [Xymon] Looking for "page too old" script
The page is set to expire immediately and usually the cache problems are the browser/client. At least in my experience.
On hobbit-clients.cfg for the host (it has proc tests already) I added:
FILE /var/www/foo/bar/output.png red mtime<1800 #go red if file older than 30 minutes
How do I get the client to send the details about this file? From reading the man page I need to specify client-local.cfg but I am not sure what goes in this. I'm pretty sure I want:
file:/var/www/foo/bar/output.png
Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box?
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Fri, Jul 29, 2011 at 11:58 AM, Tim McCloskey <tm at freedom.com<mailto:tm at freedom.com>> wrote: couple random thoughts: Would HTTP cache control headers help in this regard? You can poll a page via wget or some other mechanism and look at the page headers, that won't tell you if the image is old but if the page is regenerated on each image update it may work... If you have shell access to the server:/image.file you could test the mac times... Or, toss in some logic to catch the bugz :)
From: xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com> [xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>] On Behalf Of Josh Luthman [josh at imaginenetworksllc.com<mailto:josh at imaginenetworksllc.com>] Sent: Friday, July 29, 2011 7:40 AM To: xymon at xymon.com<mailto:xymon at xymon.com> Subject: [Xymon] Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman Office: 937-552-2340<tel:937-552-2340> Direct: 937-552-2343<tel:937-552-2343> 1100 Wayne St Suite 1337 Troy, OH 45373
Tim,
I have that right now. But I don't see how the server is going to know what the file meta data is unless it's sent to the server somehow.
At this time, probably about half an hour after I added it, the test is still white. I'm almost 100% sure hobbit-clients.cfg changes doesn't require a restart.
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Fri, Jul 29, 2011 at 2:34 PM, Tim McCloskey <tm at freedom.com> wrote:
If your system is setup to store the client data on the central server you can have something like the following in hobbit-clients.cfg.
HOST=somehost GROUP=optional-some-group FILE /some/path/tofile.ext SIZE<55M and SIZE>0 red
Client-local can take a host name too -- I think....
From: Josh Luthman [josh at imaginenetworksllc.com] Sent: Friday, July 29, 2011 11:24 AM To: Tim McCloskey Cc: xymon at xymon.com Subject: Re: [Xymon] Looking for "page too old" script
The page is set to expire immediately and usually the cache problems are the browser/client. At least in my experience.
On hobbit-clients.cfg for the host (it has proc tests already) I added:
FILE /var/www/foo/bar/output.png red mtime<1800 #go red if file older than 30 minutes
How do I get the client to send the details about this file? From reading the man page I need to specify client-local.cfg but I am not sure what goes in this. I'm pretty sure I want:
file:/var/www/foo/bar/output.png
Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box?
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Fri, Jul 29, 2011 at 11:58 AM, Tim McCloskey <tm at freedom.com<mailto: tm at freedom.com>> wrote: couple random thoughts: Would HTTP cache control headers help in this regard? You can poll a page via wget or some other mechanism and look at the page headers, that won't tell you if the image is old but if the page is regenerated on each image update it may work... If you have shell access to the server:/image.file you could test the mac times... Or, toss in some logic to catch the bugz :)
From: xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com> [ xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>] On Behalf Of Josh Luthman [josh at imaginenetworksllc.com<mailto:josh at imaginenetworksllc.com>] Sent: Friday, July 29, 2011 7:40 AM To: xymon at xymon.com<mailto:xymon at xymon.com> Subject: [Xymon] Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman Office: 937-552-2340<tel:937-552-2340> Direct: 937-552-2343<tel:937-552-2343> 1100 Wayne St Suite 1337 Troy, OH 45373
Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box?
Just took another look at a working config....here's a couple notes. You don't need the whole [linux] class, you can use a [hostname] in client-local.cfg ... [your-hostname] file:/var/www/foo/bar/output.png ...
The stanza above with a matching FILE directive from hobbit-clients.cfg (both server side) works for me, on 4.2.0.
If you have this setup - sounds like you do - and it's not working you can Try touching the logfetch.$foo.cfg and logfetch.$foo.status files in the tmp dir on the client (create them however you need, just make sure to chown them to your hobbit user).
You should not need to populate the files, just create them. Example file... cat someclienthostname:~hobbit/client/tmp/logfetch.someclienthostname.cfg log:/some/path/to/somefile:10240 file:/bin/su
Tim
Got it working! Thanks!!!
FILE /var/www/inxwireless/network/output.png red mtime<1800
This is correct. From what the documentation states, of which I may be understanding it incorrectly, this is backwards.
Initially I did mtime>1800 based on what I read on the man page. The test went red "File was modified 31 seconds ago - should be >1800"
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373
On Fri, Jul 29, 2011 at 4:31 PM, Tim McCloskey <tm at freedom.com> wrote:
Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box?
Just took another look at a working config....here's a couple notes. You don't need the whole [linux] class, you can use a [hostname] in client-local.cfg ... [your-hostname] file:/var/www/foo/bar/output.png ...
The stanza above with a matching FILE directive from hobbit-clients.cfg (both server side) works for me, on 4.2.0.
If you have this setup - sounds like you do - and it's not working you can Try touching the logfetch.$foo.cfg and logfetch.$foo.status files in the tmp dir on the client (create them however you need, just make sure to chown them to your hobbit user).
You should not need to populate the files, just create them. Example file... cat someclienthostname:~hobbit/client/tmp/logfetch.someclienthostname.cfg log:/some/path/to/somefile:10240 file:/bin/su
Tim
On 01-08-2011 17:46, Josh Luthman wrote:
Got it working! Thanks!!!
FILE /var/www/inxwireless/network/output.png red mtime<1800
This is correct. From what the documentation states, of which I may be understanding it incorrectly, this is backwards.
I must admit I had my doubts when coding this. But the idea is to read it as it is written: "File foo must have been modified LESS THAN 1800 seconds ago".
Regards, Henrik
The code makes more sense to me then the sentence explaining it in the man page. That's just me. Either way you only have two choices so it won't be a big mistake. On Aug 7, 2011 8:20 AM, "Henrik Størner" <henrik at hswn.dk> wrote:
On 01-08-2011 17:46, Josh Luthman wrote:
Got it working! Thanks!!!
FILE /var/www/inxwireless/network/output.png red mtime<1800
This is correct. From what the documentation states, of which I may be understanding it incorrectly, this is backwards.
I must admit I had my doubts when coding this. But the idea is to read it as it is written: "File foo must have been modified LESS THAN 1800 seconds ago".
Regards, Henrik
Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon
On 08/07/2011 01:53 PM, Josh Luthman wrote:
The code makes more sense to me then the sentence explaining it in the man page. That's just me. Either way you only have two choices so it won't be a big mistake.
On Aug 7, 2011 8:20 AM, "Henrik Størner" <henrik at hswn.dk <mailto:henrik at hswn.dk>> wrote:
On 01-08-2011 17:46, Josh Luthman wrote:
Got it working! Thanks!!!
FILE /var/www/inxwireless/network/output.png red mtime<1800
This is correct. From what the documentation states, of which I may be understanding it incorrectly, this is backwards.
I must admit I had my doubts when coding this. But the idea is to read it as it is written: "File foo must have been modified LESS THAN 1800 seconds ago".
Regards, Henrik
This would make sense if the file is being updated every 30 minutes. However, you may want to give it a 5 minute buffer to avoid timing false alarms, like "mtime<2100".
NB: I would have read this as "Turn red if the file is less than 30 minutes old" which would seem backwards but I'm reluctant to contradict the author of the program :-).
=G=
participants (6)
-
gjohnson＠trantor.org
-
henrik＠hswn.dk
-
josh＠imaginenetworksllc.com
-
Paul.Root＠qwest.com
-
ralphmitchell＠gmail.com
-
tm＠freedom.com