October 2009 - Xymon - Xymon mailman web

[hobbit] performance help needed
by olivier＠audry.fr 27 Oct '09

27 Oct '09

hi all, do you have a lot of memory ? If yes you can create tmpfs for your rrd and sync the tmpfs every couple hours. We do it for a little hobbit server with 3500+ devices. For rrd hist and www dir. Regards Olivier AUDRY ----- Mail Original ----- De: "shea greg" <shea_greg at emc.com> À: bgmilne at staff.telkomsa.net, hobbit at hswn.dk Cc: "shea greg" <shea_greg at emc.com> Envoyé: Mardi 27 Octobre 2009 14h24:47 GMT +01:00 Amsterdam / Berlin / Berne / Rome / Stockholm / Vienne Objet: RE: [hobbit] performance help needed >>On Monday, 26 October 2009 20:55:15 shea_greg at emc.com wrote: >> Hi all, >> >> >> >> First off, sorry for the long post, I'm trying to supply as much data as >> possible for analysis. >> >> >> >> I have a single Hobbit server with approximately 3500 hosts, a mixture >> of windows and unix, some DB tests, >> >> some BEA tests and a few custom tests. I have over 70000 RRD files >> which seems to be causing Hobbit performance >> >> problems, most specifcally clock offset. I have a cron job that >> restarts Hobbit every 30 minutes otherwise the offset >> >> grows so large it eats all memory and OOM kill starts. NTP is fine, it >> seems to be the time it takes for Hobbit to process >> >> the client data. OS resides on RAID1 146GB drives SAS 15K RPM, second >> drive for RRDs is a single 300GB SAS 15K RPM. >> >> At the end is a graph showing the clock offset. What else can I try? > >Add more spindles. > >70 000 RRD files will result in a minimum of 233 IOPS (assuming they are all >being updated at 5-minute intervals). The EMC people I've spoken to say a 15k >FC disk shouldn't really be averaging much more than 180 IOPS, 15k SAS or 15k >SCSI wouldn't be any better. The 311 you seem to be doing isn't significant >overhead for the minumum of 233, so it is unlikely that any tuning will help. > >If you can't add spindles, you could look at the 4.3 branch, which has some >features that allow scaling out to more hosts, or streamlining RRD writes >(which may allow you to lose the clock offset, but will likely not reduce the >load average much). > >Regards, >Buchan Hi Buchan, Thanks for your response. I bounced around the idea of external storage, but even here at EMC there is a cost associated with external storage, that's why I tried the second drive. I've read about the enhancements in 4.3, but thought I should upgrade from RH 4.7 to RH 5.3 first (RH is the official supported Linux) as there were IO improvements in the kernel. I also tried a newer version of RRD 1.2.30 and 1.3.8. RRD 1.3.8 doesn't work Hobbit 4.2. On to the storage requisition process.... Thanks -Grs- Gregory R Shea EMC Corporation

1 0

performance help needed
by shea_greg＠emc.com 27 Oct '09

27 Oct '09

Hi all, First off, sorry for the long post, I'm trying to supply as much data as possible for analysis. I have a single Hobbit server with approximately 3500 hosts, a mixture of windows and unix, some DB tests, some BEA tests and a few custom tests. I have over 70000 RRD files which seems to be causing Hobbit performance problems, most specifcally clock offset. I have a cron job that restarts Hobbit every 30 minutes otherwise the offset grows so large it eats all memory and OOM kill starts. NTP is fine, it seems to be the time it takes for Hobbit to process the client data. OS resides on RAID1 146GB drives SAS 15K RPM, second drive for RRDs is a single 300GB SAS 15K RPM. At the end is a graph showing the clock offset. What else can I try? I moved the RRDs off to a separate drive hoping this would help, but the write per second is high. I've tried reducing read-ahead, mounting noatime,nodiratime, changing IO scheduling to deadline, nothing seems to help. Here's a sample output from iostat -xd 60 10: Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util sda 0.00 68.08 0.17 20.02 1.33 704.78 0.67 352.39 34.98 4.25 210.36 3.47 7.01 sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda2 0.00 68.08 0.17 20.02 1.33 704.78 0.67 352.39 34.98 4.25 210.36 3.47 7.01 sdb 0.00 674.60 1.53 311.04 12.27 7887.05 6.13 3943.52 25.27 24.50 78.38 1.91 59.70 sdb1 0.00 674.60 1.53 311.04 12.27 7887.05 6.13 3943.52 25.27 24.50 78.38 1.91 59.70 sdb2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-0 0.00 0.00 0.17 88.10 1.33 704.78 0.67 352.39 8.00 20.31 230.09 0.79 7.01 dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Drive sdb1 is housing the RRD files Memory seems fine: Memory Used Total Percentage Physical 7645M 7973M 95% Actual 4688M 7973M 58% Swap 64M 9983M 0% [hobbit at hobbitmon rrd]$ uname -a Linux hobbitmon 2.6.9-78.0.8.ELsmp #1 SMP Wed Nov 5 07:14:58 EST 2008 x86_64 x86_64 x86_64 GNU/Linux [hobbit at hobbitmon rrd]$ cat /etc/redhat-release Red Hat Enterprise Linux AS release 4 (Nahant Update 7) Output from bbgen: bbgen for Hobbit version 4.2.0 Statistics: Hosts : 3506 Status messages : 41934 Purple messages : 0 Pages : 171 Output from bbtest: bbtest-net version 4.2.0 SSL library : OpenSSL 0.9.7a Feb 19 2003 LDAP library: OpenLDAP 20213 Statistics: Hosts total : 3511 Hosts with no tests : 2390 Total test count : 1470 Status messages : 1596 Alert status msgs : 0 Transmissions : 18 DNS statistics: # hostnames resolved : 358 # succesful : 339 # failed : 19 # calls to dnsresolve : 530 TCP test statistics: # TCP tests total : 411 # HTTP tests : 161 # Simple TCP tests : 250 # Connection attempts : 411 # bytes written : 24722 # bytes read : 543706 TIME SPENT Event Starttime Duration bbtest-net startup 1256584823.382254 - Service definitions loaded 1256584823.383506 0.001252 Tests loaded 1256584823.468743 0.085237 DNS lookups completed 1256584828.565010 5.096267 Test engine setup completed 1256584828.572444 0.007434 TCP tests completed 1256584839.000192 10.427748 PING test completed (1082 hosts) 1256584881.612835 42.612643 PING test results sent 1256584890.617168 9.004333 Test result collection completed 1256584890.617453 0.000285 LDAP test engine setup completed 1256584890.617453 0.000000 LDAP tests executed 1256584890.617454 0.000001 LDAP tests result collection completed 1256584890.617455 0.000001 NTP tests executed 1256584894.477007 3.859552 RPC tests executed 1256584894.988810 0.511803 Test results transmitted 1256584895.016358 0.027548 bbtest-net completed 1256584895.018441 0.002083 TIME TOTAL 71.636187 Output for hobbitd: Statistics for Hobbit daemon Up since 26-Oct-2009 15:00:11 (0 days, 00:25:02) Incoming messages : 398039 - status : 367373 - combo : 5193 - page : 183 - summary : 75 - data : 15310 - client : 9595 - notes : 0 - enable : 0 - disable : 0 - ack : 0 - config : 0 - query : 50 - hobbitdboard : 63 - hobbitdlog : 180 - drop : 0 - rename : 0 - dummy : 5 - ping : 0 - notify : 0 - schedule : 1 - download : 0 - Bogus/Timeouts : 11 Incoming messages/sec : 262 (average last 300 seconds) status channel messages: 366410 (1 readers) stachg channel messages: 34214 (1 readers) page channel messages: 5600 (1 readers) data channel messages: 15310 (1 readers) notes channel messages: 0 (0 readers) enadis channel messages: 0 (0 readers) client channel messages: 9565 (1 readers) clichg channel messages: 17 (1 readers)

2 2

Monitoring Solaris Containers
by jkwade＠futurefrontiers.com 26 Oct '09

26 Oct '09

Hello, I'm just wondering, has anyone development a separate client for Solaris containers (zones). It would be nice if the same stats were displayed except that instead of seeing the global cpu and memory, you see the individual container cpu and memory utilization. I was just wondering if anyone had something out there? Thanks.James

3 2

Support of snmp direct ind zymon
by hobbit＠weiser.dk 26 Oct '09

26 Oct '09

Hi, I have looked a little around after some support direct ind xymon fore snmp devices. I know the Devmon but have some annoying bug in it... I have read som mails in the archive about snmp support, but have not found the overview of the function. Can someone help me to get an overview of futures and limits with the snmp? And thanks for at nice program, Henrik... Best regards, Kim

4 4

Notification problem when red->yellow->green ?
by nico＠crysto.org 26 Oct '09

26 Oct '09

Hello, When a red alert happens, then goes to yellow, then finally to green, there is no recovered notification. Is there a way to have a notification for this case please? Thanks. Cheers, Nico

4 5

dbcheck for RAC connections refused
by martin.flemming＠desy.de 26 Oct '09

26 Oct '09

Hi ! I want to use dbcheck.pl from http://sourceforge.net/projects/hobbit-perl-cl/ .. but unfortunatley something is going wrong, i get only "Connection Refused" Messages and no data .. :-( My config is below: bb-hosts: 0.0.0.0 atlas_tag_db # noconn dbcheck 0.0.0.0 atlast1 # noconn dbcheck 0.0.0.0 atlast2 # noconn dbcheck 0.0.0.0 atlast3 # noconn dbcheck 0.0.0.0 atlast4 # noconn dbcheck dbcheck.ini: oraclehome = /opt/products/oracle-client/10.2g/ username = XXXX password = XXXXXXXXXXXXXXXXXXXXXXX [atlas_tag_db] dbtype = Oracle oraclerac = yes hostname = lcg3d-a-v-4 # hostname of the rac instance n.2 sid = atlas_tag_db # sid of the rac instance n.2 port = 1521 # port of listener on rac instance n.2 username = XXXX password = XXXXXXXXXXXX [atlast1] dbtype = Oracle hostname = lcg3d-a-v-1 # hostname of the rac instance n.2 sid = atlast1 # sid of the rac instance n.2 port = 1521 # port of listener on rac instance n.2 username = XXXX password = XXXXXXXXXXX [atlast2] dbtype = Oracle hostname = lcg3d-a-v-2 # hostname of the rac instance n.2 sid = atlast2 # sid of the rac instance n.2 port = 1521 # port of listener on rac instance n.2 username = XXXXX password = XXXXXXXXXXXXXXX [atlast3] dbtype = Oracle hostname = lcg3d-a-v-3 # hostname of the rac instance n.2 sid = atlast3 # sid of the rac instance n.2 port = 1521 # port of listener on rac instance n.2 username = XXXXXX password = XXXXXXXXXXXXXXXX [atlast4] dbtype = Oracle hostname = lcg3d-a-v-4 # hostname of the rac instance n.2 sid = atlast4 # sid of the rac instance n.2 port = 1521 # port of listener on rac instance n.2 username = XXXXXXXX password = XXXXXXXXXXXXXXXXXXX It's a RAC-Cluster with the instances on virtual-xen-maschines ... The Xen-server for the virtual machines are running with oracle-(redhat)-enterprise-linux : Enterprise Linux Enterprise Linux Server release 5.2 (Carthage) Linux lcg3d-a-1.desy.de 2.6.18-92.el5xen #1 SMP Fri May 23 23:49:15 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux .. don't know, if that a important point for my problem ... The dbcheck-error-output is like bin/bbcmd ext/dbcheck.pl fast -dd 2009-07-29 21:09:50 Using default environment file /usr/lib/hobbit/server/etc/hobbitserver.cfg . . . Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::exec_local_cmd line 429 Wed Jul 29 21:09:50 2009:DEBUG:RESULT= 0, COMMAND= /usr/lib/hobbit/server/bin/bbhostgrep dbcheck ,VALUE=0.0.0.0 atlas_tag_db # dbcheck 0.0.0.0 atlast1 # dbcheck Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::exec_local_cmd line 429 Wed Jul 29 21:09:50 2009:DEBUG:RESULT= 0, COMMAND= /usr/lib/hobbit/server/bin/bbhostgrep dbcheck ,VALUE=0.0.0.0 atlas_tag_db # dbcheck 0.0.0.0 atlast1 # dbcheck . . . Wed Jul 29 21:09:50 2009:DEBUG: hostadress: atlast1 Wed Jul 29 21:09:50 2009:DEBUG: mode=1, tocheck=0, testmode=0 data0=uptime, data1=status+60 Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::check_test line 1048 Wed Jul 29 21:09:50 2009:DEBUG: mode=1, tocheck=0, testmode=0 data0=DBCheck, data1=status+60 Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::load_test_config line 1182 Wed Jul 29 21:09:50 2009:DEBUG: LINE=fulltest, EVENT=DBCheck, CHECK=status+60 Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::check_test line 1048 Wed Jul 29 21:09:50 2009:DEBUG: mode=1, tocheck=0, testmode=0 data0=ChkConn, data1=status+60 Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::load_test_config line 1182 Wed Jul 29 21:09:50 2009:DEBUG: LINE=conn, EVENT=ChkConn, CHECK=status+60 Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::check_test line 1048 Wed Jul 29 21:09:50 2009:DEBUG: mode=1, tocheck=1, testmode=0 data0=Audit, data1=notest Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::load_test_config line 1182 Wed Jul 29 21:09:50 2009:DEBUG: LINE=audit, EVENT=Audit, CHECK=notest Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::check_test line 1048 Wed Jul 29 21:09:50 2009:DEBUG: mode=1, tocheck=1, testmode=0 data0=TblSpace, data1=notest Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::load_test_config line 1182 Wed Jul 29 21:09:50 2009:DEBUG: LINE=tablespace, EVENT=TblSpace, CHECK=notest Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::check_test line 1048 Wed Jul 29 21:09:50 2009:DEBUG: mode=1, tocheck=1, testmode=0 data0=Extent, data1=notest Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::load_test_config line 1182 Wed Jul 29 21:09:50 2009:DEBUG: LINE=extent, EVENT=Extent, CHECK=notest Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::check_test line 1048 Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::check_test line 276 Wed Jul 29 21:09:50 2009:DEBUG: mode=1, tocheck=0, testmode=0 data0=uptime, data1=status+60 Wed Jul 29 21:09:50 2009:DEBUG: mode=1, tocheck=1, testmode=0 data0=HitCache, data1=notest Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::load_test_config line 1182 Wed Jul 29 21:09:50 2009:DEBUG: LINE=hitcache, EVENT=HitCache, CHECK=notest Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::check_test line 1048 Wed Jul 29 21:09:50 2009:DEBUG: LINE=unumber, EVENT=unumber, CHECK=status+60 Wed Jul 29 21:09:50 2009:ERROR on atlast1: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-1 # hostname of the rac instance n.2;port=1521 # port of listener on rac instance n.2;sid=atlast1 # sid of the rac instance n.2! Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::send_bb line 1109 Wed Jul 29 21:09:50 2009:DEBUG: TEST=conn, TYPE=status+60, EVENT=ChkConn, SENDTYPE=2, TESTLIVE=60, SUMREPTIME=0 Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::send_bb line 1109 Wed Jul 29 21:09:50 2009:DEBUG: BBDISPLAY: 131.169.56.65, hostname: atlast1, type: status+60, event: ChkConn, time: 00:00:00, color=red Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::send_bb line 1109 Wed Jul 29 21:09:50 2009:DEBUG: COMPLETE MESSAGE:status+60 atlast1.CkConn red Wed Jul 29 21:09:50 2009 Timeout connecting to DBI:Oracle:host=lcg3d-a-v-1 # hostname of the rac instance n.2;port=1521 # port of listener on rac instance n.2;sid=atlast1 # sid of the rac instance n.2! dbcheck.pl version 1.08 - column ChkConn lifetime 60, tested in ~ 00:00:00 (max 00:00:20) Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::send_bb line 339 Wed Jul 29 21:09:50 2009:DEBUG: TEST=unumber, TYPE=status+60, EVENT=unumber, SENDTYPE=2, TESTLIVE=60, SUMREPTIME=0 Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::send_bb line 339 Wed Jul 29 21:09:50 2009:DEBUG: BBDISPLAY: 131.169.56.65, hostname: atlast1, type: status+60, event: unumber, time: 00:00:00, color=clear Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::send_bb line 339 Wed Jul 29 21:09:50 2009:DEBUG: COMPLETE MESSAGE:status+60 atlast1.unumber clear Wed Jul 29 21:09:50 2009 Connection Check Failed dbcheck.pl version 1.08 - column unumber lifetime 60, tested in ~ 00:00:00 (max 00:00:20) Wed Jul 29 21:09:50 2009:ERROR on atlas_tag_db: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-4 # hostname of the rac instance n.2;port=1521 # port of listener on rac instance n.2;sid=atlas_tag_db # sid of the rac instance n.2! Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::send_bb line 1109 Wed Jul 29 21:09:50 2009:DEBUG: TEST=conn, TYPE=status+60, EVENT=ChkConn, SENDTYPE=2, TESTLIVE=60, SUMREPTIME=0 Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::send_bb line 1109 Wed Jul 29 21:09:50 2009:DEBUG: BBDISPLAY: 131.169.56.65, hostname: atlas_tag_db, type: status+60, event: ChkConn, time: 00:00:00, color=red Wed Jul 29 21:09:50 2009:CALL : Hobbit_fd_lib::send_bb line 1109 Wed Jul 29 21:09:50 2009:DEBUG: COMPLETE MESSAGE:status+60 atlas_tag_db.ChkConn red Wed Jul 29 21:09:50 2009 Timeout connecting to DBI:Oracle:host=lcg3d-a-v-4 # hostname of the rac instance n.2;port=1521 # port of listener on rac instance n.2;sid=atlas_tag_db # sid of the rac instance n.2! And status-page looks like Test Time = 00:00:01 Host Checked = 5 Status Msg = 60 Colors Colors clear 55 red 5 Events Events Audit 5 ChkConn 5 Extent 5 HitCache 5 InvObj 5 Locks 5 MemReq 5 RollBack 5 Session 5 TblSpace 5 unumber 5 uptime 5 Types Types status+60 60 Hosts Summary Hosts clear red Number Times atlas_tag_db 11 1 12 00:00:00 atlast1 11 1 12 00:00:00 atlast2 11 1 12 00:00:00 atlast3 11 1 12 00:00:00 atlast4 11 1 12 00:00:00 Hosts Summary Hosts Audit ChkConn Extent HitCache InvObj Locks MemReq RollBack Session TblSpace unumber uptime atlas_tag_db 1 1 1 1 1 1 1 1 1 1 1 1 atlast1 1 1 1 1 1 1 1 1 1 1 1 1 atlast2 1 1 1 1 1 1 1 1 1 1 1 1 atlast3 1 1 1 1 1 1 1 1 1 1 1 1 atlast4 1 1 1 1 1 1 1 1 1 1 1 1 Errors No General Errors atlas_tag_db Wed Jul 29 18:52:47 2009:ERROR: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-4 # hostname of the rac instance n.2;port=1521 # port of listener on rac instance n.2;sid=atlas_tag_db # sid of the rac instance n.2! atlast1 Wed Jul 29 18:52:47 2009:ERROR: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-1 # hostname of the rac instance n.2;port=1521 # port of listener on rac instance n.2;sid=atlast1 # sid of the rac instance n.2! atlast2 Wed Jul 29 18:52:47 2009:ERROR: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-2 # hostname of the rac instance n.2;port=1521 # port of listener on rac instance n.2;sid=atlast2 # sid of the rac instance n.2! atlast3 Wed Jul 29 18:52:47 2009:ERROR: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-3 # hostname of the rac instance n.2;port=1521 # port of listener on rac instance n.2;sid=atlast3 # sid of the rac instance n.2! atlast4 Wed Jul 29 18:52:47 2009:ERROR: Timeout connecting to DBI:Oracle:host=lcg3d-a-v-4 # hostname of the rac instance n.2;port=1521 # port of listener on rac instance n.2;sid=atlast4 # sid of the rac instance n.2! Warnings No General Warnings No Hosts Warnings dbcheck.pl version 1.08 - column dbcheck lifetime 60, tested in ~ 00:00:01 (max 00:02:00) I use the old and also the new version of dbcheck.pl ... Any idea why could my enviroment be damaged ? thanks & cheers martin

3 12

Tinh Do is out of the office.
by tinh.do＠continental-corporation.com 25 Oct '09

25 Oct '09

I will be out of the office starting 10/24/2009 and will not return until 11/02/2009. If you have any urgent needs please call me on my cell. I will respond to your message when I return.

1 0

FILES directory test not graphing
by pnixon＠gmail.com 24 Oct '09

24 Oct '09

Hey all, I did a search, but these are rather generic terms, so I'm hoping someone has seen this. When a directory is added to the FILES test, it doesn't graph properly, even though the data is appearing at the bottom of the graph image. Any suggestions on what to tweak, how to troubleshoot this? Thanks! --Patrick

3 8

Default templates in devmon distribution
by thomleavitt＠gmail.com 23 Oct '09

23 Oct '09

All, There are templates being distributed in the general template package with a sysdescr of "Linux" (see dell-perc, for example). This just caused me to waste way too much time at one point trying to figure out why my custom template was not being matched properly... once I walked away and came back, I immediately realized that devmon was matching my Netgear ReadyNAS 1100 and STM-300 to the "dell-perc" template (the empty RAID columns should have tipped me off right away). When looked, the spec file was matching on "Linux". So too, does the compaq-server spec file. I'm posting this as an awareness point for users, as I think it is kind of difficult to find any string more likely to be returned as part of a system description than "Linux". I also strongly recommend that devmon template authors use something else in their spec files to trigger device matches, and to think about the overall uniqueness of the strings they are using. Regards, Thomas Leavitt

1 0

Special bb-hosts file just for reporting
by hballinger＠heritage-healthcare.com 23 Oct '09

23 Oct '09

Is there any way that I can have a special customized bb-hosts file that I use just for generating an availability report? For example, I want to have a bb-hosts file that has friendly COMMENT tags, etc and that is grouped the way that our service catalog is grouped. This way I can generate an availability report that makes sense when demonstrating to the executive team. However, I want the layout of my normal monitoring system to be layed out logically in the way that makes sense to our IT team. Is there any way to run a command that will generate an availability report using a different bb-hosts file? Thanks! - Harold Ballinger IT Manager Heritage Healthcare, Inc. (888) 335-2620 | helpdesk (864) 224-3626 | office (864) 224-3093 | fax [cid:image001.jpg at 01CA5193.7895DC50] Visit our website: www.heritage-healthcare.com<http://www.heritage-healthcare.com/>

2 4