histlog issue for large sites
Hi,
until now, i haven't did much cleaning in my histlogs, (i'll have to play with the trimhistory command)
basicly i would like to keep as much history as i can afford.
i have over 1 million files in my histlogs on my production bb server, and just realized that i'm over 50% of used inodes (which could become of probleme in less that a year in my case, soi'll plan to newfs that partition to have more inodes..)
henrik : have you thought of managing the histlogs another way ? (from my user point of view, i'ld need to have recent history (less that 30 days) quickly available; older history (over 30 days) is less important but i would still like to have it online...)
-- Olivier Beau
On Thu, Jul 21, 2005 at 12:29:48PM +0200, Olivier Beau wrote:
i have over 1 million files in my histlogs on my production bb server, and just realized that i'm over 50% of used inodes (which could become of probleme in less that a year in my case, soi'll plan to newfs that partition to have more inodes..)
henrik : have you thought of managing the histlogs another way ? (from my user point of view, i'ld need to have recent history (less that 30 days) quickly available; older history (over 30 days) is less important but i would still like to have it online...)
The files in the histlogs directory are only used when you drill down and view the detailed status from sometime in the past. Just to view the history page (with the overview of when it changed to red/green/blue...) only requires the files in the data/hist/ directory.
So no - I haven't though much about changing the way it's stored. What I currently do is to archive old histlogs/ files one month at a time - simply doing "cd .../histlogs; find . -name "*_Apr_*_2005" | cpio ..."
I have thought about changing the directory structure so it would be easier to split the histlogs by month - e.g. from
histlogs/hostname/testname/TIMESTAMP
to
histlogs/YEAR/MONTH/hostname/testname/TIMESTAMP
which would make it easier to handle. Since BB compatibility is no longer an objective in Hobbit, I can do that fairly easily.
Henrik
participants (2)
-
henrik@hswn.dk
-
olivier@qalpit.com