[hobbit] http timeout and content checking
I'm having a little problem converting some Sitescope web page checks into Hobbit content checks. In particular, I have several URLs of the form: https://some.machine.name:9212/monitoring/MonitorControlServlet?monitorName=... (names changed to protect the innocent)
The problem is, I can go to that URL manually, and it will show up fine...after about 20 seconds or so. The URL is slow to respond, because it is running a back-end check, and that check simply takes a while to complete. The result is that Hobbit is coming back saying that the website failed to respond after 10.x seconds, even though it would be fine if I waited another 10 seconds or so.
I would like to be able to increase the timeout, but only for URLs similar to above that are expected to take a while to respond. I don't want to increase this timeout across the board, since only a subset of all of the URL monitoring I am doing take so long to respond. Is there any way to do this?
Also on a related, though different note, are a few other tests that I can't get the "content" column to show up. These URLs are of the form:
https://username:password at some.machine.name:9212/monitoring/MonitorControlServlet?monitorName=com.company.common.util.monitoring.URLMonitor&url=https://some.machine.name:9212/LBListener/MessageReceiver&fullDisplay=true&isPost=true&ABCAction=AccountInfoRequest&ABCService=MemberManagement&payload=<request:AccountInfoRQ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:request=" com.company.accountrequest"><AccountInfo AccountCardNumber="4264292967999999" PortfolioID="177" SecurityInfo="true"/></request:AccountInfoRQ> (names changed to protect the innocent)
The http check is working fine, but it seems like hobbit is ignoring the fact that I have it set up as a content check. The bb-hosts line for this test looks like: 1.2.3.4 test-name # DESC:"Test description" noconn nosslcert cont;<URL above>;Success badhttp:1:2:3
I also just noticed that for these tests, bbtest is complaining "content-check, but no content-data in <bb-hosts line above>". However, if I manually go to the URL in a browser, it does indeed return "content-data", and it returns in a timely fashion (about 2 or 3 seconds). Any ideas?
On Dec 21, 2007 11:24 AM, Gary Baluha <gumby3203 at gmail.com> wrote:
I'm having a little problem converting some Sitescope web page checks into Hobbit content checks. In particular, I have several URLs of the form:
https://some.machine.name:9212/monitoring/MonitorControlServlet?monitorName=... (names changed to protect the innocent)
The problem is, I can go to that URL manually, and it will show up fine...after about 20 seconds or so. The URL is slow to respond, because it is running a back-end check, and that check simply takes a while to complete. The result is that Hobbit is coming back saying that the website failed to respond after 10.x seconds, even though it would be fine if I waited another 10 seconds or so.
I would like to be able to increase the timeout, but only for URLs similar to above that are expected to take a while to respond. I don't want to increase this timeout across the board, since only a subset of all of the URL monitoring I am doing take so long to respond. Is there any way to do this?
Also on a related, though different note, are a few other tests that I can't get the "content" column to show up. These URLs are of the form:
https://username:password at some.machine.name:9212/monitoring/MonitorControlServlet?monitorName=com.company.common.util.monitoring.URLMonitor&url=https://some.machine.name:9212/LBListener/MessageReceiver&fullDisplay=true&i... <request:AccountInfoRQ xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance" xmlns:request=" com.company.accountrequest"><AccountInfo AccountCardNumber="4264292967999999" PortfolioID="177" SecurityInfo="true"/></request:AccountInfoRQ> (names changed to protect the innocent)
The http check is working fine, but it seems like hobbit is ignoring the fact that I have it set up as a content check. The bb-hosts line for this test looks like: 1.2.3.4 test-name # DESC:"Test description" noconn nosslcert cont;<URL above>;Success badhttp:1:2:3
I also just noticed that for these tests, bbtest is complaining "content-check, but no content-data in <bb-hosts line above>". However, if I manually go to the URL in a browser, it does indeed return "content-data", and it returns in a timely fashion (about 2 or 3 seconds). Any ideas?
On Dec 21, 2007 11:24 AM, Gary Baluha <gumby3203 at gmail.com> wrote:
I'm having a little problem converting some Sitescope web page checks into Hobbit content checks. In particular, I have several URLs of the form:
https://some.machine.name:9212/monitoring/MonitorControlServlet?monitorName=... (names changed to protect the innocent)
The problem is, I can go to that URL manually, and it will show up fine...after about 20 seconds or so. The URL is slow to respond, because it is running a back-end check, and that check simply takes a while to complete. The result is that Hobbit is coming back saying that the website failed to respond after 10.x seconds, even though it would be fine if I waited another 10 seconds or so.
I would like to be able to increase the timeout, but only for URLs similar to above that are expected to take a while to respond. I don't want to increase this timeout across the board, since only a subset of all of the URL monitoring I am doing take so long to respond. Is there any way to do this?
Also on a related, though different note, are a few other tests that I can't get the "content" column to show up. These URLs are of the form:
https://username:password at some.machine.name:9212/monitoring/MonitorControlServlet?monitorName=com.company.common.util.monitoring.URLMonitor&url=https://some.machine.name:9212/LBListener/MessageReceiver&fullDisplay=true&i... <request:AccountInfoRQ xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance" xmlns:request=" com.company.accountrequest"><AccountInfo AccountCardNumber="4264292967999999" PortfolioID="177" SecurityInfo="true"/></request:AccountInfoRQ> (names changed to protect the innocent)
The http check is working fine, but it seems like hobbit is ignoring the fact that I have it set up as a content check. The bb-hosts line for this test looks like: 1.2.3.4 test-name # DESC:"Test description" noconn nosslcert cont;<URL above>;Success badhttp:1:2:3
I also just noticed that for these tests, bbtest is complaining "content-check, but no content-data in <bb-hosts line above>". However, if I manually go to the URL in a browser, it does indeed return "content-data", and it returns in a timely fashion (about 2 or 3 seconds). Any ideas?
Also on a related, though different note, are a few other tests that I can't get the "content" column to show up. These URLs are of the form:
https://username:password at some.machine.name:9212/monitoring/MonitorControlServlet?monitorName=com.company.common.util.monitoring.URLMonitor&url=https://some.machine.name:9212/LBListener/MessageReceiver&fullDisplay=true&i... <request:AccountInfoRQ xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance" xmlns:request=" com.company.accountrequest"><AccountInfo AccountCardNumber="4264292967999999" PortfolioID="177" SecurityInfo="true"/></request:AccountInfoRQ> (names changed to protect the innocent)
The http check is working fine, but it seems like hobbit is ignoring the fact that I have it set up as a content check. The bb-hosts line for this test looks like: 1.2.3.4 test-name # DESC:"Test description" noconn nosslcert cont;<URL above>;Success badhttp:1:2:3
I also just noticed that for these tests, bbtest is complaining "content-check, but no content-data in <bb-hosts line above>". However, if I manually go to the URL in a browser, it does indeed return "content-data", and it returns in a timely fashion (about 2 or 3 seconds). Any ideas?
I apologize if this is a duplicate email, but it looks like from my end that something didn't post correctly...
Anyway, I figured out why I'm getting this error. From the bb-network.logfile: 2007-12-24 23:14:28 content-check, but no content-data in 'cont; https://USERNAME:PASSWORD at HOSTNAME:9212/monitoring/MonitorControlServlet?monitorName=com.company.common.util.monitoring.URLMonitor&url=https://HOSTNAME:9212/LBListener/MessageReceiver&fullDisplay=true&isPost=true&TLSAction=hotelearningssummaryextract&TLSService=HotelEarnings&payload=<https://USERNAME:PASSWORD at hostname:9212/monitoring/MonitorControlServlet?monitorName=com.company.common.util.monitoring.URLMonitor&url=https://HOSTNAME:9212/LBListener/MessageReceiver&fullDisplay=true&isPost=true&TLSAction=hotelearningssummaryextract&TLSService=HotelEarnings&payload=><TLSBatchRQ><Parameters><Parameter'
So it appears the full URL from the bb-hosts file is getting truncated. The limit looks to be around 370 (or at least more than 256), and I can't find anywhere to override this, so it appears to be hard-coded without looking through the actual code.
Actually, it looks like it's not so much an issue with the max length for the URL, as it is for *any* line in bb-hosts. I started writing an extension script to do the content checking, and even that is getting the URL truncated. The problem appears that everything past 464 characters from the "#" in the bb-hosts file for any host is discarded.
As far as I can tell, there is no environment variable or setting that I can change in the config file to change this. Anyone else have any ideas?
On Dec 24, 2007 11:41 PM, Gary Baluha <gumby3203 at gmail.com> wrote:
Also on a related, though different note, are a few other tests that I can't get the "content" column to show up. These URLs are of the form:
https://username:password at some.machine.name:9212/monitoring/MonitorControlServlet?monitorName=com.company.common.util.monitoring.URLMonitor&url=https://some.machine.name:9212/LBListener/MessageReceiver&fullDisplay=true&i... <request:AccountInfoRQ xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance" xmlns:request=" com.company.accountrequest"><AccountInfo AccountCardNumber="4264292967999999" PortfolioID="177" SecurityInfo="true"/></request:AccountInfoRQ> (names changed to protect the innocent)
The http check is working fine, but it seems like hobbit is ignoring the fact that I have it set up as a content check. The bb-hosts line for this test looks like: 1.2.3.4 test-name # DESC:"Test description" noconn nosslcert cont;<URL above>;Success badhttp:1:2:3
I also just noticed that for these tests, bbtest is complaining "content-check, but no content-data in <bb-hosts line above>". However, if I manually go to the URL in a browser, it does indeed return "content-data", and it returns in a timely fashion (about 2 or 3 seconds). Any ideas?
I apologize if this is a duplicate email, but it looks like from my end that something didn't post correctly...
Anyway, I figured out why I'm getting this error. From the bb-network.logfile: 2007-12-24 23:14:28 content-check, but no content-data in 'cont;https://USERNAME:PASSWORD at HOSTNAME:9212/monitoring/MonitorControlServlet?monitorName=com.company.common.util.monitoring.URLMonitor&url=https://HOSTNAME:9212/LBListener/MessageReceiver&fullDisplay=true&isPost=tru...
<https://USERNAME:PASSWORD at hostname:9212/monitoring/MonitorControlServlet?monitorName=com.company.common.util.monitoring.URLMonitor&url=https://HOSTNAME:9212/LBListener/MessageReceiver&fullDisplay=true&isPost=true&TLSAction=hotelearningssummaryextract&TLSService=HotelEarnings&payload=><TLSBatchRQ><Parameters><Parameter'
So it appears the full URL from the bb-hosts file is getting truncated. The limit looks to be around 370 (or at least more than 256), and I can't find anywhere to override this, so it appears to be hard-coded without looking through the actual code.
On Wed, Jan 02, 2008 at 02:23:42PM -0500, Gary Baluha wrote:
Actually, it looks like it's not so much an issue with the max length for the URL, as it is for *any* line in bb-hosts. I started writing an extension script to do the content checking, and even that is getting the URL truncated. The problem appears that everything past 464 characters from the "#" in the bb-hosts file for any host is discarded.
It shouldn't do that, and I cannot trigger this with a simple test. Could you send me (off-list) the bb-hosts file you're using ?
Another simple check you can do: In the Hobbit source directory hobbit-4.2.0/lib there is a "loadhosts" binary - this is a debugging tool. You can run it with
cd hobbit-4.2.0 ./lib/loadhats /etc/hobbit/bb-hosts HOSTNAME
This should dump the tags found for the host HOSTNAME in bb-hosts. It would be interesting to see if the entries are also truncated there.
Do the URL's show up correctly on the "info" column page ?
Regards, Henrik
On Jan 3, 2008 6:11 AM, Henrik Stoerner <henrik at hswn.dk> wrote:
On Wed, Jan 02, 2008 at 02:23:42PM -0500, Gary Baluha wrote:
Actually, it looks like it's not so much an issue with the max length for the URL, as it is for *any* line in bb-hosts. I started writing an extension script to do the content checking, and even that is getting the URL truncated. The problem appears that everything past 464 characters from the "#" in the bb-hosts file for any host is discarded.
It shouldn't do that, and I cannot trigger this with a simple test. Could you send me (off-list) the bb-hosts file you're using ?
Just an update for the list...
With the debugging tool Henrik mentioned, the URL that is apparently getting truncated everywhere else is NOT truncated with the tool. I'm currently writing a URL content check extension script to both get around the truncation issue I'm having, as well as adding some content check features Hobbit doesn't have out-of-the-box, so ultimately it's not a big deal for me. However, I will work with Henrik to see if we can figure out where this issue is coming from.
participants (2)
-
gumby3203＠gmail.com
-
henrik＠hswn.dk