- Xymon - Xymon mailman web

monitoring xen machine status from Dom0
by tomg＠mcclatchyinteractive.com 04 Sep '07

04 Sep '07

Has anyone started thinking about or working on script(s) to monitor xen machines from the Dom0? It would be nice to have some of the data that xentop and other tools display in Hobbit and I would rather work with others on this list to develop the scripts rather than do everything on my own, if possible. I'm really new to Xen, so I still have a lot to learn. Tom -- Tom Georgoulias Sr. Systems Engineer McClatchy Interactive tomg at mcclatchyinteractive.com

2 3

patch to support new "st" vmstat field in RHEL5
by tomg＠mcclatchyinteractive.com 04 Sep '07

04 Sep '07

Red Hat Enterprise Linux 5 adds support for a 17th field in vmstat: "st" or %CPU stolen from a virtual machine. I'm using procps-3.2.7-8.1.el5, the latest production procps from Red Hat Network. I made a simple one line patch that adds this field to the default Linux OS layout, rather than create OS classes for the two linux distros that I know are supporting it (RHEL5 and Fedora 7). I'm guessing that this patch will collect and display a lot of NaN data for some distros, but I thought I'd pass it along. If there is a better way to handle this, let me know. Tom -- Tom Georgoulias Sr. Systems Engineer McClatchy Interactive tomg at mcclatchyinteractive.com

1 0

Problem since upgrading to 4.2.0 from 4.1 - "data flooding from"
by rob.macgregor＠gmail.com 04 Sep '07

04 Sep '07

I've just upgraded my Hobbit server from 4.1 to 4.2 (on a Gentoo 2006.1 host), having had a need for "Disabled until OK" :) The trouble that I now have is that reports from the client part of the server install are never making it through - all tests have turned purple and the log is full of "data flooding from 127.0.0.1, closing connection". I even reverted the client directory to the default from the installer, with no change. Pointers to how to further diagnose this, or even better resolve it, appreciated. -- Please keep list traffic on the list. Rob MacGregor Whoever fights monsters should see to it that in the process he doesn't become a monster. Friedrich Nietzsche

2 2

[hobbit] Hobbit Server performance
by xbgmsharp＠gmail.com 04 Sep '07

04 Sep '07

Hi, Here is a solution in order to reduce I/O which is a lot consuming. I load all the rrd and hist and webserver (cgi-bin,www,web,secure-cgi) into different tmpfs (http://en.wikipedia.org/wiki/TMPFS) This way everything is load into memory (cached). # free -k total used free shared buffers cached Mem: 6229400 5570816 658584 0 466896 4685444 I have 6G of memory and i set 2G for rrd and the other 2G for hist and a very small one for the webserver 64M. I check 2896 hosts. I have reduce my iowait% for 60-80% load to 0%. The webinterface is much faster. I order to write data on the disk rsync data twice a day. All of this does'nt execpt for having a local caching DNS daemon and many other tunning parameters. You can also: * reduce the timeout TCP/IP: echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout echo 1800 > /proc/sys/net/ipv4/tcp_keepalive_intvl echo 3000 > /proc/sys/net/ipv4/tcp_keepalive_time echo 0 > /proc/sys/net/ipv4/tcp_window_scaling echo 0 > /proc/sys/net/ipv4/tcp_sack echo 0 > /proc/sys/net/ipv4/tcp_timestamps * increase TCP/IP buffer echo 262144 > /proc/sys/net/core/rmem_max echo 262144 > /proc/sys/net/core/wmem_max * mount hobbit partition with noatime option and for more security nodev,noexec,nosuid * unilimit cpu,mem,files for hobbit user * increase inodes number on partition because of rrd and hist and clientdata directory. * Controls the memory # Controls the default maxmimum size of a mesage queue kernel.msgmax = 65536 # Controls the maximum shared segment size, in bytes kernel.shmmax = 4294967295 # Controls the maximum number of shared memory segments, in pages kernel.shmall = 268435456 kernel.shmmax = 536870912 Regards, KaYa On 8/29/07, Henrik Stoerner <henrik (at) hswn.dk> wrote: > On Wed, Aug 29, 2007 at 04:14:58PM -0500, Jeff Newman wrote: > > The server is a DL360 G3 with two 3ghz processors and 2gb of RAM. > It does only > > hobbit and cricket (10 or so routers, so not a huge hit) > > > > I've noticed the following two things: > > > > #1. I am constantly using 100% of my physical ram (no swap really) > > > > Mem: 2075040k total, 2051928k used, 23112k free, 52172k buffers > > Swap: 0k total, 0k used, 0k free, 1744168k cached > > > > is this normal? Not sure how to tell which process specifically is > > eating up the most RAM. > > I think you're running Linux, right? > > The "1744168k cached" normally goes in the "Mem:" line, not the "Swap:" > line. And that's the clue to what's using your RAM: Linux uses available > RAM as a variable-size disk cache; the disk cache grows until just about > all of RAM is used. It usually makes sense, the cached data is a copy of > data which is stored on disk, so when an application needs more memory, > the diskcache memory can be freed instantly and allocated to the > application. > > Therefore, Linux boxes usually have very little "free" memory. > > The "top" utility can sort processes by memory-usage. Also, "ps -vax" > will tell you how much memory each process is using. > > > > #2. I am consistantly running with 2-300 ports in TIME_WAIT > involving port 1984 > > > > # netstat -an | grep TIME | grep 1984 | wc -l > > 253 > > > > Im not sure if this is normal either, nor am I sure if it is bad or > > not? > > It's not unusual, and quite harmless. The TIME_WAIT state happens when a > socket is closed; the operating system keeps the socket around for some > time (usually 20-30 secs, cannot remember what the Linux default is) to > make sure that all packets destined for this socket have been received > (there might be some duplicated/retransmitted packets still in transit > when the socket is closed). This makes sure that a new connection using > the same port will not see packets from the old connection. > > > > Here is the output from some of the "bb" buttons at the top of my > > hobbit page. If anything stands out, any help would be apprieciated. > > > > DNS statistics: > > # hostnames resolved : 138 > > # succesful : 134 > > # failed : 4 > > DNS lookups completed 1188421854.099983 > 9.239654 > > Your DNS lookups are a bit slow - 9 seconds for 138 DNS lookups. Nothing > critical, just a bit slower than I'd expect. Installing a local caching > DNS daemon is a good way to eliminate this problem. > > > Regards, > Henrik > > > To unsubscribe from the hobbit list, send an e-mail to > hobbit-unsubscribe (at) hswn.dk > > >

2 1

New beta hobbit - rrd disk cache and graph display
by wxxx333＠gmail.com 04 Sep '07

04 Sep '07

Hi, As always kudos Henrik for the excellent work. I didn't start do play with the new hobbit beta version, who good improvements about the disk I/O limitations on huge load systems. I have a specific question about what's hobbit behavior when disk cache for rrd is active, and i request the graph for a specific metric. Will i not see the last (around) 30 minutes of info on the graph? Or is there some kind "trap" activated when i request a specific graph, who'll request that the info for that host/service should be written to the RRD before the graph is generated? Regards Werner

3 2

Msgs issue on solaris 10 and a dns issue on centos 4.4
by joshuakrause＠triad.rr.com 03 Sep '07

03 Sep '07

First Issue: I have a couple of solaris machines that range from 8-10 in versions. They all have the hobbit client installed on them and I am retrieving data back from all of them except for one. On that one I get all of the tests I am checking for except the messages. And I've looked on the server and there is a /var/adm/messages file and it has 644 permissions like all of the other Hobbit Clients I run but this one doesn't report anything back. Second Issue: I am monitoring 2 dns servers running CentOS 4.4. They are running the Hobbit Client and were working fine until I added a couple of DNS entries via webmin and had Webmin restart dns. After dns restarted and came back up hobbit has it marked as down and it hasn't come back up in 4 days. I have tried removing the hosts and making sure the logs are all removed. But nothing has worked.

1 0

rrd network traffic graphs
by joshuakrause＠triad.rr.com 03 Sep '07

03 Sep '07

I noticed that the network graphs on the trends column show up in bits or bytes how can i change them to show mb. So instead of 40000000.0 it wil show 40.0. As always thanks in advance

2 3

monitoring a file
by joshuakrause＠triad.rr.com 03 Sep '07

03 Sep '07

I am wanting to monitor a file to make sure it doesn't get any smaller than 500k. This is on a remote machine that has the client software setup on it. On the remote machine i put this entry in the localclient.cfg that looked like this: FILE /export/home/dncs/mystro/bfs/Compass_OOB/rt.grp.z SIZE<800 I am not getting anything to report back on the BBDISPLAY though. any ideas would be appreciated.

2 1

Disk usage modification
by joshuakrause＠triad.rr.com 02 Sep '07

02 Sep '07

I am trying to find out how to get the disk usage to display the df -h output instead of df -k. I've modified the hobbitserver.cfg and the hobbitclients.cfg where it had df -Pk to df -Ph and nothing has changed. Thanks, Josh

2 3

Disk usage display
by joshuakrause＠triad.rr.com 02 Sep '07

02 Sep '07

I am trying to find out how to get the disk usage to display the df -h output instead of df -k. I've modified the hobbitserver.cfg and the hobbitclients.cfg where it had df -Pk to df -Ph and nothing has changed. Thanks, Josh

1 0