How about using mysqlreport? (http://hackmysql.com/mysqlreport)

It could run as cgi-bin script probably in the hobbit server itself or the client. It requires perl and the appropriate DBI modules.

I'm looking into incorporating replication slave monitoring into the script.

On 10/11/06, thomas.seglard.enata at cnp.fr <thomas.seglard.enata at cnp.fr> wrote:

Hi Francesco,

I'm sorry, I missed the new version and the query engine. I will try to look at it soon. Best regards,

Thomas

"Francesco Duranti" <fduranti at q8.it> a écrit sur 10/10/2006 21:16:24 :

...

You can use also dbcheck.pl to track the mysql database (connection and same stats as the normal sqlstatus script). The new version also has a custom query engine so it's possible to write to query to analyze and if someone know how to get more data out of sqlserver with query or sql command I can arrange some more checks for it (and graphs also...).

Francesco

...

-----Original Message----- From: thomas.seglard.enata at cnp.fr [mailto:thomas.seglard.enata at cnp.fr] Sent: Tuesday, October 10, 2006 5:09 PM To: hobbit at hswn.dk Subject: [hobbit] - MySQL monitoring

Hi,

I'm just asking you how do you monitor your MySQL databases ? I've setted up the bb-mysqlstatus.sh and modify it to support NCV format. Now I got graphs for queries/sec and slowqueries. That's cool but it's not very accurate... I'm using MySQL Administrator from http://dev.mysql.com/downloads/gui-tools/5.0.html and results are not the same. Did you write something to track insert, select, etc. from 'mysqladmin extended-status' ? Thanks for your help ! Sincerly,

Thomas

Ce message (et toutes ses pieces jointes eventuelles) est confidentiel et etabli a l'intention exclusive de ses destinataires. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, CNP Assurances et ses filiales declinent toute responsabilite au titre de ce message, s'il a ete altere, deforme ou falsifie.

---

This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. E-mails are susceptible to alteration. Neither CNP Assurances nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

Ce message (et toutes ses pieces jointes eventuelles) est confidentiel et etabli a l'intention exclusive de ses destinataires. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, CNP Assurances et ses filiales declinent toute responsabilite au titre de ce message, s'il a ete altere, deforme ou falsifie.

---

This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. E-mails are susceptible to alteration. Neither CNP Assurances nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

How long do you need to keep data? You can specify a date on trimhistory (see the man pages) and any log data before that date will be deleted.

thomas.seglard.enata at cnp.fr wrote:

Hi,

I got a space problem on my hobbit server (4.2.0-1). Actually, my data directory is approaching 40G of occupied space. I have no more space on my disk so I'm worrying myself a bit ! I've already done a './trimhistory --drop[svcs|logs]' but I earned just a few Mb... What can I remove from the data directory ?

Here is some figures and a picture :

8,6G ./data/histlogs 26G ./data/hostdata 35G ./data

Thomas

Ce message (et toutes ses pieces jointes eventuelles) est confidentiel et etabli a l'intention exclusive de ses destinataires. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, CNP Assurances et ses filiales declinent toute responsabilite au titre de ce message, s'il a ete altere, deforme ou falsifie.

---

This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. E-mails are susceptible to alteration. Neither CNP Assurances nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

-- Rich Smrcina VM Assist, Inc. Phone: 414-491-6001 Ans Service: 360-715-2467 rich.smrcina at vmassist.com

Catch the WAVV! http://www.wavv.org WAVV 2007 - Green Bay, WI - May 18-22, 2007

Can you send the command that you used?

thomas.seglard.enata at cnp.fr wrote:

I specified a date with trimhistory (1st june 2006) but it doesn't seem to remove files in some hostdata directories. I executed again trimhistory to remove all files/logs before the 1st of september this year. In spite of that, I can see some files before this date were not removed from the hostdata directory :

[hobbit at psi200 psp145b]$ ls -ltr |more total 2280296 -rw-rw-r-- 1 hobbit hobbit 123509 aoû 10 16:16 1155219302 -rw-rw-r-- 1 hobbit hobbit 122816 aoû 10 16:18 1155219426 -rw-rw-r-- 1 hobbit hobbit 121728 aoû 10 16:21 1155219550 -rw-rw-r-- 1 hobbit hobbit 122275 aoû 10 16:23 1155219673 -rw-rw-r-- 1 hobbit hobbit 122391 aoû 10 16:25 1155219791 [snip] -rw-rw-r-- 1 hobbit hobbit 277910 aoû 31 23:51 1157060958 -rw-rw-r-- 1 hobbit hobbit 277830 aoû 31 23:55 1157061259 -rw-rw-r-- 1 hobbit hobbit 280192 sep 1 00:05 1157061858 -rw-rw-r-- 1 hobbit hobbit 272742 sep 1 00:09 1157062159 -rw-rw-r-- 1 hobbit hobbit 270990 sep 1 00:15 1157062458 -rw-rw-r-- 1 hobbit hobbit 274434 sep 1 00:22 1157062759

Thanks for your help ! Sincerly,

Thomas

-- Rich Smrcina VM Assist, Inc. Phone: 414-491-6001 Ans Service: 360-715-2467 rich.smrcina at vmassist.com

Catch the WAVV! http://www.wavv.org WAVV 2007 - Green Bay, WI - May 18-22, 2007

I have a mysterious constellation.

With one exception showing historical logs works fine.

Only one service for one host error an occurs (Premature end of script headers). The same service for another host works.

If I add --debug in the bb-histlog.sh, it works (and shows "Trying header/footer file '/opt/hobbit/server/web/histlog_footer'").

So I played a litte with the hobbitsvc.c. If I comment if (tstamp) xfree(tstamp); in the function do_request, this one service for this one host works also!

My hobbit version is 4.2.0 with the all-in-one-Patch from Dec 15.

Michael

Henrik,

Is this intentional or a minor oversight?

Henrik Stoerner wrote:

On Mon, Oct 30, 2006 at 03:09:20PM +0100, thomas.seglard.enata at cnp.fr wrote:

...

Hi,

I got a space problem on my hobbit server (4.2.0-1). Actually, my data directory is approaching 40G of occupied space. I have no more space on my disk so I'm worrying myself a bit ! I've already done a './trimhistory --drop[svcs|logs]' but I earned just a few Mb... What can I remove from the data directory ?

Here is some figures and a picture :

8,6G ./data/histlogs 26G ./data/hostdata

trimhistory fails to remove data from the hostdata directory. Do a find ./data/hostdata -mtime +90 | xargs rm -f

to delete hostdata files older than 90 days.

Regards, Henrik

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

-- Rich Smrcina VM Assist, Inc. Phone: 414-491-6001 Ans Service: 360-715-2467 rich.smrcina at vmassist.com

Catch the WAVV! http://www.wavv.org WAVV 2007 - Green Bay, WI - May 18-22, 2007

Thanks for this info. I used this command and now my disk space is correct. Regards,

Thomas

henrik at hswn.dk (Henrik Stoerner) a écrit sur 30/10/2006 16:06:13 :

On Mon, Oct 30, 2006 at 03:09:20PM +0100, thomas.seglard.enata at cnp.fr wrote:

...

Hi,

I got a space problem on my hobbit server (4.2.0-1). Actually, my data directory is approaching 40G of occupied space. I have no more space on my disk so I'm worrying myself a bit ! I've already done a './trimhistory --drop[svcs|logs]' but I earned just a few Mb... What can I remove from the data directory ?

Here is some figures and a picture :

8,6G ./data/histlogs 26G ./data/hostdata

trimhistory fails to remove data from the hostdata directory. Do a find ./data/hostdata -mtime +90 | xargs rm -f

to delete hostdata files older than 90 days.

Regards, Henrik

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

Ce message (et toutes ses pieces jointes eventuelles) est confidentiel et etabli a l'intention exclusive de ses destinataires. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, CNP Assurances et ses filiales declinent toute responsabilite au titre de ce message, s'il a ete altere, deforme ou falsifie.

This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. E-mails are susceptible to alteration. Neither CNP Assurances nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.

There is a known problem when running trimhistory, the non-green page does not get updated properly. You will need to bounce Hobbit and it will start logging again.

thomas.seglard.enata at cnp.fr wrote:

Hi,

on my 'all non-green view', hobbit says : "No events received in last 240 minutes"... Though, it's not possible ! When I'm checking history button on some systems I can see events happened. So what's wrong ? I'm running Hobbit 4.2.0 with all-in-one patch on a Linux box. Best regards,

Thomas

-- Rich Smrcina VM Assist, Inc. Phone: 414-491-6001 Ans Service: 360-715-2467 rich.smrcina at vmassist.com

Catch the WAVV! http://www.wavv.org WAVV 2007 - Green Bay, WI - May 18-22, 2007

Hi,

I'm playing with linecount facility and it's really amazing. I will use it to trace connection numbers from applications and error messages from log files. I'm reading documentation on it for several hours (it's friday afternoon so I'm a bit slow :D) and don't figure how to 'grep' words from a set of files and graph the total number of matches. Here is an example of what I want to do.

My log files are within this directory /projets/mut/home/mutrec2/wls_ traitement/mut-domain2/logs/ . Then I would like to run this command and be able to graph result :

$ grep -i -E '(SEVERE|Error)' *log*|wc -l 98

Is there a way to do such thing ? Best regards,

Thomas

Ce message (et toutes ses pieces jointes eventuelles) est confidentiel et etabli a l'intention exclusive de ses destinataires. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, CNP Assurances et ses filiales declinent toute responsabilite au titre de ce message, s'il a ete altere, deforme ou falsifie.

This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. E-mails are susceptible to alteration. Neither CNP Assurances nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.

Hi,

I'm sorry to ask you this question again but does someone got Hobbit 4.2 running on AIX 4.2 ? I can't neither compile client with gcc nor IBM cc. Henrik made a patch to fix that but it doesn't work. Best regards,

Thomas

Ce message (et toutes ses pieces jointes eventuelles) est confidentiel et etabli a l'intention exclusive de ses destinataires. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, CNP Assurances et ses filiales declinent toute responsabilite au titre de ce message, s'il a ete altere, deforme ou falsifie.

This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. E-mails are susceptible to alteration. Neither CNP Assurances nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.

Yes hobbit can do this via POST form login. Check the HTTP cont tag.

thomas.seglard.enata at cnp.fr wrote:

Hello,

I'm asking here if I can use Hobbit as a "probe" to test Internet sites. I want to be able to trace requests and provide to customers response time of these requests. Basically, the 'cont' test of Hobbit is not sufficient 'cause you only test static pages. My main objective is to write some scenarios like : login with this login/pass / go to these links (trying to do some requests on Weblogic and database) / logout Does anybody have already set up scripts like that ? Thanks for help ! Sincerly,

Thomas

Ce message (et toutes ses pieces jointes eventuelles) est confidentiel et etabli a l'intention exclusive de ses destinataires. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, CNP Assurances et ses filiales declinent toute responsabilite au titre de ce message, s'il a ete altere, deforme ou falsifie.

---

This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. E-mails are susceptible to alteration. Neither CNP Assurances nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

On Fri, Dec 08, 2006 at 11:26:27AM +0100, thomas.seglard.enata at cnp.fr wrote:

I'm asking here if I can use Hobbit as a "probe" to test Internet sites. I want to be able to trace requests and provide to customers response time of these requests. Basically, the 'cont' test of Hobbit is not sufficient 'cause you only test static pages. My main objective is to write some scenarios like : login with this login/pass / go to these links (trying to do some requests on Weblogic and database) / logout

I've done it for one or two sites, and abandoned it because it is incredibly fragile when dealing with real-world websites. You're basically going to mimic the behaviour of a browser, but you probably don't have an engine that is fully capable of handling JavaScript, cookies, automated redirects and form submissions like a real browser does.

I did emulate a login to a web-based SAP system, and just the login sequence ended up doing some 20-30 HTTP requests, because you got bounced around webpages each of them setting a cookie, picking up a form element, modifying a URL etc.

If you must, then I'd suggest starting off with a network sniffer like Ethereal (it has a new name now, I think), and then doing a trace of all the requests that your browser performs. That will give you an idea of how much work is involved, from the number of requests that actually go out on the wire. Then you can start digging into each of the requests, and see if you can automate them using some scripting tool.

Regards, Henrik

On 12/8/06, Henrik Stoerner <henrik at hswn.dk> wrote:

I've done it for one or two sites, and abandoned it because it is incredibly fragile when dealing with real-world websites. You're basically going to mimic the behaviour of a browser, but you probably don't have an engine that is fully capable of handling JavaScript, cookies, automated redirects and form submissions like a real browser does.

Curl handles cookies, redirects that happen via Location headers, and form submissions, for both regular sites and secure sites.

My scripts all evolved under Big Brother, so they're all Bourne shell, using curl, grep, sed, head, tail, etc to process the pages. There's a perl script on the curl home page (http://curl.haxx.se) called formfind.pl, which pulls out form elements. I've got a hacked up copy of formfind.pl that hands me the form in a format suitable for posting back using curl.

I used to have a script that logged into an airline booking system, picked out flights, and went through the whole booking sequence, stopping right before it would have had to enter credit card details. That was about 19 steps, I think.

You don't necessarily have to be able to process javascript. A lot of forms just use it to verify the data, which you can ignore. I've run into a few that used javascript to modify the form variables and store the results into other form variables, but those aren't too hard to deal with once you understand what's happening.

Probably the worst thing I've had to deal with was a page that assembled a new url from bits of the form elements, then jumped to it by setting location.href=[newurl]. That kind of thing almost has to be hardwired, and as Henrik says, that has a tendency to be a bit fragile.

I think I've got an example script somewhere. If I can find it, I'll post it.

Ralph Mitchell

Thanks all for your answers and good advices. I'm going to write some scripts with curl and send results back to my customers. Ralph, if you can also post your script, it would be interesting to analyse it. Best regards,

Thomas (thomas.seglard.enata at cnp.fr)

"Ralph Mitchell" <ralphmitchell at gmail.com> a écrit sur 08/12/2006 15:19:38 :

On 12/8/06, Henrik Stoerner <henrik at hswn.dk> wrote:

...

I've done it for one or two sites, and abandoned it because it is incredibly fragile when dealing with real-world websites. You're basically going to mimic the behaviour of a browser, but you probably don't have an engine that is fully capable of handling JavaScript, cookies, automated redirects and form submissions like a real browser does.

Curl handles cookies, redirects that happen via Location headers, and form submissions, for both regular sites and secure sites.

My scripts all evolved under Big Brother, so they're all Bourne shell, using curl, grep, sed, head, tail, etc to process the pages. There's a perl script on the curl home page (http://curl.haxx.se) called formfind.pl, which pulls out form elements. I've got a hacked up copy of formfind.pl that hands me the form in a format suitable for posting back using curl.

I used to have a script that logged into an airline booking system, picked out flights, and went through the whole booking sequence, stopping right before it would have had to enter credit card details. That was about 19 steps, I think.

You don't necessarily have to be able to process javascript. A lot of forms just use it to verify the data, which you can ignore. I've run into a few that used javascript to modify the form variables and store the results into other form variables, but those aren't too hard to deal with once you understand what's happening.

Probably the worst thing I've had to deal with was a page that assembled a new url from bits of the form elements, then jumped to it by setting location.href=[newurl]. That kind of thing almost has to be hardwired, and as Henrik says, that has a tendency to be a bit fragile.

I think I've got an example script somewhere. If I can find it, I'll post it.

Ralph Mitchell

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk

Ce message (et toutes ses pieces jointes eventuelles) est confidentiel et etabli a l'intention exclusive de ses destinataires. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, CNP Assurances et ses filiales declinent toute responsabilite au titre de ce message, s'il a ete altere, deforme ou falsifie.

This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. E-mails are susceptible to alteration. Neither CNP Assurances nor any of its subsidiaries or affiliates shall be liable for the message if altered, changed or falsified.