Hi,

Here is a solution in order to reduce I/O which is a lot consuming. I load all the rrd and hist and webserver (cgi-bin,www,web,secure-cgi)
into different tmpfs (http://en.wikipedia.org/wiki/TMPFS). This way everything is load into memory (cached).

free -k

          total       used       free     shared    buffers     cached

Mem: 6229400 5570816 658584 0 466896 4685444

I have 6G of memory and i set 2G for rrd and the other 2G for hist and
a very small one for the webserver 64M.

I check 2896 hosts. I have reduce my iowait% for 60-80% load to 0%. The webinterface is much faster.

I order to write data on the disk rsync data twice a day.

All of this does'nt execpt for having a local caching DNS daemon and
many other tunning parameters. You can also:

• reduce the timeout TCP/IP: echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout echo 1800 > /proc/sys/net/ipv4/tcp_keepalive_intvl echo 3000 > /proc/sys/net/ipv4/tcp_keepalive_time echo 0 > /proc/sys/net/ipv4/tcp_window_scaling echo 0 > /proc/sys/net/ipv4/tcp_sack echo 0 > /proc/sys/net/ipv4/tcp_timestamps

• increase TCP/IP buffer echo 262144 > /proc/sys/net/core/rmem_max echo 262144 > /proc/sys/net/core/wmem_max

• mount hobbit partition with noatime option and for more security
  nodev,noexec,nosuid

• unilimit cpu,mem,files for hobbit user

• increase inodes number on partition because of rrd and hist and
  clientdata directory.

• Controls the memory

Controls the default maxmimum size of a mesage queue

kernel.msgmax = 65536

Controls the maximum shared segment size, in bytes

kernel.shmmax = 4294967295

Controls the maximum number of shared memory segments, in pages

kernel.shmall = 268435456 kernel.shmmax = 536870912

Regards, KaYa

On 8/29/07, Henrik Stoerner <henrik (at) hswn.dk> wrote:

On Wed, Aug 29, 2007 at 04:14:58PM -0500, Jeff Newman wrote:

...

The server is a DL360 G3 with two 3ghz processors and 2gb of RAM.
It does only hobbit and cricket (10 or so routers, so not a huge hit)

I've noticed the following two things:

#1. I am constantly using 100% of my physical ram (no swap really)

Mem: 2075040k total, 2051928k used, 23112k free, 52172k buffers Swap: 0k total, 0k used, 0k free, 1744168k cached

is this normal? Not sure how to tell which process specifically is eating up the most RAM.

I think you're running Linux, right?

The "1744168k cached" normally goes in the "Mem:" line, not the "Swap:" line. And that's the clue to what's using your RAM: Linux uses available RAM as a variable-size disk cache; the disk cache grows until just about all of RAM is used. It usually makes sense, the cached data is a copy of data which is stored on disk, so when an application needs more memory, the diskcache memory can be freed instantly and allocated to the application.

Therefore, Linux boxes usually have very little "free" memory.

The "top" utility can sort processes by memory-usage. Also, "ps -vax" will tell you how much memory each process is using.

...

#2. I am consistantly running with 2-300 ports in TIME_WAIT
involving port 1984

netstat -an | grep TIME | grep 1984 | wc -l

253

Im not sure if this is normal either, nor am I sure if it is bad or not?

It's not unusual, and quite harmless. The TIME_WAIT state happens when a socket is closed; the operating system keeps the socket around for some time (usually 20-30 secs, cannot remember what the Linux default is) to make sure that all packets destined for this socket have been received (there might be some duplicated/retransmitted packets still in transit when the socket is closed). This makes sure that a new connection using the same port will not see packets from the old connection.

...

Here is the output from some of the "bb" buttons at the top of my hobbit page. If anything stands out, any help would be apprieciated.

DNS statistics:

hostnames resolved : 138

succesful : 134

failed : 4

DNS lookups completed 1188421854.099983
9.239654

Your DNS lookups are a bit slow - 9 seconds for 138 DNS lookups. Nothing critical, just a bit slower than I'd expect. Installing a local caching DNS daemon is a good way to eliminate this problem.

Regards, Henrik

To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe (at) hswn.dk