Dependency sytem in Hobbit
Hello all,
I am Etienne Marganne, a new comer in TI Automotive, a society which uses Hobbit as a monitoring tool, I will be in charge of Hobbit tool. We would like to enhance our monitoring tool in order to get more detailed informations through it.
One of the first thing we would like to do is to create a dependency system between monitored hosts across our network. For all the further discussion, please keep in mind that we have a very large network and two Hobbit servers where we would like to keep the same "hh-hosts" file.
The idea of the dependency system is the following one: once we have a host that fails on a network path in our network all the hosts further that one will be very likely unreachable. This will cause a lot of alerts to be triggered because of one failure. This is not interesting because our team will be flooded by those alerts. Therefore it could be useful to create dependencies so that those further devices on the same path will not trigger alerts (basically turning red).
There is a specific tag that can be used to do such a work, the "route" tag. In our case with two Hobbit servers, we would tune that with the "route_BBLOCATION" tag. Knowing that there are Hobbit clients on all the network nodes between the two endpoints, the simplest idea would be to list all the nodes in the description of those tags. This could work even if it would generate of job. A little bit further here, let's say that one node fails on a path, it is very probable that if you know that the following nodes are unreachable, you may not want to test them anymore (to forbid them to turn red).
Now think of a big network with a lot of redundancy, it is very probable that there will be indeed a lot of network paths between a server and one final host. If on one path one node fails, it does not mean that the final client is not reachable. But with the "route" tag, that client would be signal as unreachable since a member of the "route" has failed. This is not comfortable at all.
What we would like to know, or to get, if there is a way to get this dependency system work:
Hobbit Server A ---- 1 ---- 2 ---- 3 ---- 4 ---- Final Client
| |
| --------- 5 ------------6 ----------- |
There are two paths to reach the Final Client, one composed by 1, 2, 3, 4 and another one composed by 5, 6.
With the current "route" tag we would have such a list of nodes: route_HobbitServerA:1,2,3,4,5,6 then if 1 fails and the path 1-2-3-4 would not work anymore but the 5-6 one would still.
A good dependency system would to have such a thing: Final Client depends on 4, 4 depends on 3, 3 depends on 2, 2 depends on 1 but Final Clients also depends on 6 which depends on 5. More over this would also solve that kind of topology :
Hobbit Server A ---- 1 ---- 2 ---- 3 ---- 4 ---- Final Client
| | |
| --------- 5 ------------6 ----------- | Where
there is a link between 2 and 5, 2 and 6, 5 and 3, 6 and 3.
Maybe that something could be set up with the "depends" tag, however I do not know how the informations will propagate through the different dependencies done that tag.
Even further on the topic, the "route" tag performs only ping tests which does not seem enough to me. I would like to add cpu, disks, ... tests to the whole dependency system.
Thank you for your help and answers,
Regards,
Etienne Marganne
TI Automotive.
The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
Hi,
try the bbcombotest to add HA logic and make the depend tag point to the aggregated dot.
See bbcombotest.cfg
-- Charles Goyard - cgoyard at cvf.fr - (+33) 1 45 38 01 31
Hello, We often use Network-volumes for backup. Is there a way to use Hobbit to check this Harddisks for free space? At the moment I`ve mounted this into linux for checking free space.
This Network-HDD`s using also a linux OS, but I cannot install something.
Regards
M.Heinelt
-- 愛知県一宮市富士2-2-22 株式会社 ベガシステムズ TEL 0586-71-3903 FAX 0586-71-4071
Maik Heinelt ha scritto:
Hello, We often use Network-volumes for backup. Is there a way to use Hobbit to check this Harddisks for free space?
i have similar question: some of my linux server mount a nfs drive: can I include it in free space monitoring? by default the graphs don't show it.
thanks
Hello,
I'm not sure, but I think on linux machines Hobbit gathers information about free space on hard disks using the "df" command. Take a look at $BBHOME/client/bin, there are shell scripts for serveral OSes. Alter the df-command in "hobbitclient-linux.sh" by removing the "l" switch, to make it show all mounted file systems - also the network drives.
It should look something like this: "df -P -x none -x tmpfs -x shmfs -x unknown -x iso9660 |..."
Just give it a try... :)
-----Original Message----- From: Alessandro Tinivelli [mailto:alessandro.tinivelli at monrif.net] Sent: Monday, January 22, 2007 2:04 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Network-Volumes
Maik Heinelt ha scritto:
Hello, We often use Network-volumes for backup. Is there a way to use Hobbit to check this Harddisks for free space?
i have similar question: some of my linux server mount a nfs drive: can I include it in free space monitoring? by default the graphs don't show it.
thanks
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Etienne -- you are going to have to find someone to write that and add it to Hobbit. Path-based alarm suppression is one of the holy grails in the network management industry, and the reason it has not yet been solved is because it is a difficult problem. For small networks you can come up with a solution, but if you are using VLANs and WAN's and load balancers and all that other stuff it gets to be rather difficult.
There are many commercial software vendors that claim to have this problem solved -- but sometimes even their demos do not work. The little bit of dependency specification that you can put into Hobbit does indeed work, but not across the board.
GLH
From: Marganne, Etienne [mailto:emarganne at be.tiauto.com]
Sent: Friday, January 19, 2007 5:58 AM
To: hobbit at hswn.dk
Cc: henrik at hswn.dk
Subject: [hobbit] Dependency sytem in Hobbit
Hello all,
I am Etienne Marganne, a new comer in TI Automotive, a society
which uses Hobbit as a monitoring tool, I will be in charge of Hobbit tool. We would like to enhance our monitoring tool in order to get more detailed informations through it.
One of the first thing we would like to do is to create a
dependency system between monitored hosts across our network. For all the further discussion, please keep in mind that we have a very large network and two Hobbit servers where we would like to keep the same "hh-hosts" file.
The idea of the dependency system is the following one: once we
have a host that fails on a network path in our network all the hosts further that one will be very likely unreachable. This will cause a lot of alerts to be triggered because of one failure. This is not interesting because our team will be flooded by those alerts. Therefore it could be useful to create dependencies so that those further devices on the same path will not trigger alerts (basically turning red).
There is a specific tag that can be used to do such a work, the
"route" tag. In our case with two Hobbit servers, we would tune that with the "route_BBLOCATION" tag. Knowing that there are Hobbit clients on all the network nodes between the two endpoints, the simplest idea would be to list all the nodes in the description of those tags. This could work even if it would generate of job. A little bit further here, let's say that one node fails on a path, it is very probable that if you know that the following nodes are unreachable, you may not want to test them anymore (to forbid them to turn red).
Now think of a big network with a lot of redundancy, it is very
probable that there will be indeed a lot of network paths between a server and one final host. If on one path one node fails, it does not mean that the final client is not reachable. But with the "route" tag, that client would be signal as unreachable since a member of the "route" has failed. This is not comfortable at all.
What we would like to know, or to get, if there is a way to get
this dependency system work:
Hobbit Server A ---- 1 ---- 2 ---- 3 ---- 4 ---- Final Client
| |
| --------- 5 ------------6 ----------- |
There are two paths to reach the Final Client, one composed by
1, 2, 3, 4 and another one composed by 5, 6.
With the current "route" tag we would have such a list of nodes:
route_HobbitServerA:1,2,3,4,5,6 then if 1 fails and the path 1-2-3-4 would not work anymore but the 5-6 one would still.
A good dependency system would to have such a thing: Final
Client depends on 4, 4 depends on 3, 3 depends on 2, 2 depends on 1 but Final Clients also depends on 6 which depends on 5. More over this would also solve that kind of topology :
Hobbit Server A ---- 1 ---- 2 ---- 3 ---- 4 ---- Final Client
| | |
| --------- 5 ------------6 ----------- |
Where there is a link between 2 and 5, 2 and 6, 5 and 3, 6 and 3.
Maybe that something could be set up with the "depends" tag,
however I do not know how the informations will propagate through the different dependencies done that tag.
Even further on the topic, the "route" tag performs only ping
tests which does not seem enough to me. I would like to add cpu, disks, ... tests to the whole dependency system.
Thank you for your help and answers,
Regards,
Etienne Marganne
TI Automotive.
The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
On Friday 19 January 2007 15:56, Hubbard, Greg L wrote:
Etienne -- you are going to have to find someone to write that and add it to Hobbit. Path-based alarm suppression is one of the holy grails in the network management industry, and the reason it has not yet been solved is because it is a difficult problem. For small networks you can come up with a solution, but if you are using VLANs and WAN's and load balancers and all that other stuff it gets to be rather difficult.
There are many commercial software vendors that claim to have this problem solved -- but sometimes even their demos do not work. The little bit of dependency specification that you can put into Hobbit does indeed work, but not across the board. As fas I know, the hobbit _server_ can do this for the network tests. All network tests are done in 1 run. The data is processed and dependencies are calculated and errors are generated.
The problem is client checks. When an error is received, the hobbit server needs to check it's dependencies. But some of these dependencies are also client checks that are maybe not (yet) received. So the server can never know for sure if all dependencies are satisfied.
The only way I can imagine this to work is if all checks are send every 5 minutes. So, when a red check is received, the server waits 5 minutes untill all checks are updated and it can starts checking the dependencies.
But what with checks that are send every hour ? And do you really want to wait 5 minutes? s
Stef
I wrote about this some time back, the depends tag doesn't appear to work properly. You can specify a router (which is just another host) that things depend on, and that works. But a multi-level dependency (which depends seems to try to provide) would also be a good thing.
Stef Coene wrote:
On Friday 19 January 2007 15:56, Hubbard, Greg L wrote:
Etienne -- you are going to have to find someone to write that and add it to Hobbit. Path-based alarm suppression is one of the holy grails in the network management industry, and the reason it has not yet been solved is because it is a difficult problem. For small networks you can come up with a solution, but if you are using VLANs and WAN's and load balancers and all that other stuff it gets to be rather difficult.
There are many commercial software vendors that claim to have this problem solved -- but sometimes even their demos do not work. The little bit of dependency specification that you can put into Hobbit does indeed work, but not across the board. As fas I know, the hobbit _server_ can do this for the network tests. All network tests are done in 1 run. The data is processed and dependencies are calculated and errors are generated.
The problem is client checks. When an error is received, the hobbit server needs to check it's dependencies. But some of these dependencies are also client checks that are maybe not (yet) received. So the server can never know for sure if all dependencies are satisfied.
The only way I can imagine this to work is if all checks are send every 5 minutes. So, when a red check is received, the server waits 5 minutes untill all checks are updated and it can starts checking the dependencies.
But what with checks that are send every hour ? And do you really want to wait 5 minutes? s
Stef
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
-- Rich Smrcina VM Assist, Inc. Phone: 414-491-6001 Ans Service: 360-715-2467 rich.smrcina at vmassist.com
Catch the WAVV! http://www.wavv.org WAVV 2007 - Green Bay, WI - May 18-22, 2007
Ok, have just read this E-mail but I thought that if you did the route tag on a host it only went yellow (unreachable by proxy) if itself and a device in it's route entry failed (i.e. if the gateway fails but the host still responds to ping then it stays green) though Henrik would have to confirm this as I haven't had such a situation. Also I'm curious how the CPU, hard drive etc, dependencies would work? If CPU on host1 has a high load what effect would that have on host2? Unless of course a database was being hosted by host1...
Also can I ask, how large is this network? Henrik's is ~4000 or something ours is about 400-500.
Jason.
From: Hubbard, Greg L [mailto:greg.hubbard at eds.com] Sent: 19 January 2007 14:56 To: hobbit at hswn.dk Subject: RE: [hobbit] Dependency sytem in Hobbit
Etienne -- you are going to have to find someone to write that and add it to Hobbit. Path-based alarm suppression is one of the holy grails in the network management industry, and the reason it has not yet been solved is because it is a difficult problem. For small networks you can come up with a solution, but if you are using VLANs and WAN's and load balancers and all that other stuff it gets to be rather difficult.
There are many commercial software vendors that claim to have this problem solved -- but sometimes even their demos do not work. The little bit of dependency specification that you can put into Hobbit does indeed work, but not across the board.
GLH
From: Marganne, Etienne [mailto:emarganne at be.tiauto.com]
Sent: Friday, January 19, 2007 5:58 AM
To: hobbit at hswn.dk
Cc: henrik at hswn.dk
Subject: [hobbit] Dependency sytem in Hobbit
Hello all,
I am Etienne Marganne, a new comer in TI Automotive, a society
which uses Hobbit as a monitoring tool, I will be in charge of Hobbit tool. We would like to enhance our monitoring tool in order to get more detailed informations through it.
One of the first thing we would like to do is to create a
dependency system between monitored hosts across our network. For all the further discussion, please keep in mind that we have a very large network and two Hobbit servers where we would like to keep the same "hh-hosts" file.
The idea of the dependency system is the following one: once we
have a host that fails on a network path in our network all the hosts further that one will be very likely unreachable. This will cause a lot of alerts to be triggered because of one failure. This is not interesting because our team will be flooded by those alerts. Therefore it could be useful to create dependencies so that those further devices on the same path will not trigger alerts (basically turning red).
There is a specific tag that can be used to do such a work, the
"route" tag. In our case with two Hobbit servers, we would tune that with the "route_BBLOCATION" tag. Knowing that there are Hobbit clients on all the network nodes between the two endpoints, the simplest idea would be to list all the nodes in the description of those tags. This could work even if it would generate of job. A little bit further here, let's say that one node fails on a path, it is very probable that if you know that the following nodes are unreachable, you may not want to test them anymore (to forbid them to turn red).
Now think of a big network with a lot of redundancy, it is very
probable that there will be indeed a lot of network paths between a server and one final host. If on one path one node fails, it does not mean that the final client is not reachable. But with the "route" tag, that client would be signal as unreachable since a member of the "route" has failed. This is not comfortable at all.
What we would like to know, or to get, if there is a way to get
this dependency system work:
Hobbit Server A ---- 1 ---- 2 ---- 3 ---- 4 ---- Final Client
| |
| --------- 5 ------------6 ----------- |
There are two paths to reach the Final Client, one composed by
1, 2, 3, 4 and another one composed by 5, 6.
With the current "route" tag we would have such a list of nodes:
route_HobbitServerA:1,2,3,4,5,6 then if 1 fails and the path 1-2-3-4 would not work anymore but the 5-6 one would still.
A good dependency system would to have such a thing: Final
Client depends on 4, 4 depends on 3, 3 depends on 2, 2 depends on 1 but Final Clients also depends on 6 which depends on 5. More over this would also solve that kind of topology :
Hobbit Server A ---- 1 ---- 2 ---- 3 ---- 4 ---- Final Client
| | |
| --------- 5 ------------6 ----------- |
Where there is a link between 2 and 5, 2 and 6, 5 and 3, 6 and 3.
Maybe that something could be set up with the "depends" tag,
however I do not know how the informations will propagate through the different dependencies done that tag.
Even further on the topic, the "route" tag performs only ping
tests which does not seem enough to me. I would like to add cpu, disks, ... tests to the whole dependency system.
Thank you for your help and answers,
Regards,
Etienne Marganne
TI Automotive.
The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
participants (10)
-
alessandro.tinivelli@monrif.net
-
alessandro.tinivelli@monrifgroup.net
-
cgoyard@cvf.fr
-
emarganne@be.tiauto.com
-
greg.hubbard@eds.com
-
JasonAS_Jones@mentor.com
-
maik@vegasystems.com
-
rsmrcina@wi.rr.com
-
stef.coene@docum.org
-
wolfgang.wutz@siemensvdo.com