Hi, Steve
For your two suggestions, I checked the source codes, there is a 100K limitation, so it doesn't help to introduce ignore if the Error is out of that range. So you may wonder why we output more than 100K in just 30 minutes, there are several reasons:
- It is our production server log, which is busy.
- We set level to Warning, which output more than Error level
- Some our codes did not set log level correctly, we're in the process of cleaning up it.
- If there's exception, we output whole thread log in INFO level, which is huge.
Anyway, I still think 100K in 30 minutes is a little small value for a busy site's log, I would like to remove this limitation and also keep cleaning up our logs.
Thanks for your suggestion, Samuel Cai
-----Original Message----- From: s_aiello at comcast.net [mailto:s_aiello at comcast.net] Sent: Thursday, July 24, 2008 8:41 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Bug Report: Critical error in log couldn't be sent to server every time
On Wednesday 23 July 2008, Samuel Cai wrote:
It really depends on what log level your application is logging at. If you are logging at 'INFO' level, then there will be alot of data to process. As you see, Hobbit implements a limit on how much log data it will parse. This is a good thing, at least in my opinion.
It all depends what is in your log... and why soo much data is being written. If they are all errors, well hobbit would be catching them telling you there are errors. Since this is not the case.. would guess your log has data other than errors.
Suggestions:
- tune your application log settings so that only errors are written.
- make use of the client-local.cfg log's setting of ignore. This will allow the hobbit client to identify what is an extraneous message, and ignore it. Per the man page:
The ignore PATTERN line (optional) defines lines in the logfile which are ignored entirely, i.e. they are stripped from the logfile data before sending it to the Hobbit server. It is used to remove completely unwanted "noise" entries from the logdata processed by Hobbit. "PATTERN" is a regular expression.
I hope this helps you, ~Steve
It's great to hear you guys, Hubbard and Steve, that you also find this is a limitation (more than a bug), not wrong in my configuration.
I was thinking to modify source codes before, but it might be difficult for me. I'll try your suggestions, thanks!
Samuel Cai
In my reply to your email, I said that this behavior "was a good thing". I do not find this to be a limitation at all. I offered you two possible solutions, were any of these applicable ?
The "limitation" really resides in whatever application is logging soo verbosely. Production level applications should have their logging limited as much as possible whenever possible, only logging indicators of errors. And whenever this isn't possible, make use of the IGNORE option.
~Steve
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk