Has anyone thought about or implemented a configuration database backend for hobbit, primarily a database replacement for the hobbit-clients.cfg and hobbit-alerts.cfg to start....these are a couple that I would like to buils a webfront to and be able to give limited access to users so they could modify thresholds and alerting capabilities when needed.
I did this about 4 years back for Big Brother using Informix, I was able to setup thresholds and a replacement for the bb-host file inside a couple tables in a database. However back then Big Brother was mostly shell scripts and flat files for all its configuration. So adding hooks into it for talking to a databse was easy. With hobbit its all compiled code and Im not sure where to start, plus I don't have access to an Informix database or the API's for it anymore so Im looking at mysql and its api's for writing code in C.
I guess Im looking to see if anyone has started some development in this area?
Thanks Trent
Trent Melcher Unix/Linux Systems Administrator Securities America Financial Corporation (402) 399-9111 x2774 tmelcher at saionline.com Securities America, Inc., a Registered Broker/Dealer, Member NASD/SIPC. Advisory services offered through Securities America Advisors, Inc., an SEC Registered Investment Advisory .
On Thursday 07 December 2006 23:01, Trent Melcher wrote:
Has anyone thought about or implemented a configuration database backend for hobbit, primarily a database replacement for the hobbit-clients.cfg and hobbit-alerts.cfg to start....these are a couple that I would like to buils a webfront to and be able to give limited access to users so they could modify thresholds and alerting capabilities when needed.
I did this about 4 years back for Big Brother using Informix, I was able to setup thresholds and a replacement for the bb-host file inside a couple tables in a database. However back then Big Brother was mostly shell scripts and flat files for all its configuration. So adding hooks into it for talking to a databse was easy. With hobbit its all compiled code and Im not sure where to start, plus I don't have access to an Informix database or the API's for it anymore so Im looking at mysql and its api's for writing code in C. We are thinking about using mysql for storing trends and status messages.
For the configuration, I think mysql is overkill and makes it too complex. I love the simplicity of the config files. I'm more thinking about making a web-based frontend for the config files. The config files can be parsed as ini-files. And for parsing ini-files, perl is perfect. So writing a web-based frontend for the config files in perl, is not that hard.
Stef
I agree. I doubt that the efficiencies that might arise from a backend database would offset the extra complexity. Most Hobbit implementations are pretty small. The only reason why the host-file-like configuration is getting cumbersome is that so many features have crept into Hobbit that old folks like me can't remember them all...
GLH
-----Original Message----- From: Stef Coene [mailto:stef.coene at docum.org] Sent: Thursday, December 07, 2006 4:27 PM To: hobbit at hswn.dk Subject: Re: [hobbit] Configuration database backend.
On Thursday 07 December 2006 23:01, Trent Melcher wrote:
Has anyone thought about or implemented a configuration database backend for hobbit, primarily a database replacement for the hobbit-clients.cfg and hobbit-alerts.cfg to start....these are a couple that I would like to buils a webfront to and be able to give limited access to users so they could modify thresholds and alerting capabilities when needed.
I did this about 4 years back for Big Brother using Informix, I was able to setup thresholds and a replacement for the bb-host file inside
a couple tables in a database. However back then Big Brother was mostly shell scripts and flat files for all its configuration. So adding hooks into it for talking to a databse was easy. With hobbit its all compiled code and Im not sure where to start, plus I don't have access to an Informix database or the API's for it anymore so Im looking at mysql and its api's for writing code in C. We are thinking about using mysql for storing trends and status messages.
For the configuration, I think mysql is overkill and makes it too complex. I love the simplicity of the config files. I'm more thinking about making a web-based frontend for the config files. The config files can be parsed as ini-files. And for parsing ini-files, perl is perfect. So writing a web-based frontend for the config files in perl, is not that hard.
Stef
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Thu, Dec 07, 2006 at 11:27:08PM +0100, Stef Coene wrote:
On Thursday 07 December 2006 23:01, Trent Melcher wrote:
Has anyone thought about or implemented a configuration database backend for hobbit, primarily a database replacement for the hobbit-clients.cfg and hobbit-alerts.cfg to start....these are a couple that I would like to buils a webfront to and be able to give limited access to users so they could modify thresholds and alerting capabilities when needed.
I did this about 4 years back for Big Brother using Informix, I was able to setup thresholds and a replacement for the bb-host file inside a couple tables in a database. However back then Big Brother was mostly shell scripts and flat files for all its configuration. So adding hooks into it for talking to a databse was easy. With hobbit its all compiled code and Im not sure where to start, plus I don't have access to an Informix database or the API's for it anymore so Im looking at mysql and its api's for writing code in C.
We are thinking about using mysql for storing trends and status messages.
Be careful with the status messages, at least if you want to store the current status in a DB. Historical status messages - those that are currently stored in the histlogs/ directory - are OK, but for the current status log I think that the overhead associated with updating each status log in a DB every 5 minutes will be very high. And the number of times you'll actually be retrieving these data will probably be very small.
Copying the historical statuslogs (or the current ones) is very easy to do with an extra Hobbit worker module hanging off the "status" or "stachg" channels. You do not need to make any modifications to Hobbit itself - you basically run "hobbitd_channel --channel=stachg MYDBPROGRAM" and your program is fed all of the status messages on it's STDIN.
As for the trends data, I believe there are some patches to the rrdtool library that will let you store the RRD files in a DB. Check the rrd-users mailing list.
My personal favourite for a DB backend would be SQLite, since this eliminates all requirements for an external DB ressource. You don't want your monitoring to stop working when the DB becomes inaccessible.
Regards, Henrik
On Friday 08 December 2006 14:05, Henrik Stoerner wrote:
Be careful with the status messages, at least if you want to store the current status in a DB. Historical status messages - those that are currently stored in the histlogs/ directory - are OK, but for the current status log I think that the overhead associated with updating each status log in a DB every 5 minutes will be very high. And the number of times you'll actually be retrieving these data will probably be very small. Today I could have used such information. One of our servers went down, out of memory status. No way to know the process that did this.
Copying the historical statuslogs (or the current ones) is very easy to do with an extra Hobbit worker module hanging off the "status" or "stachg" channels. You do not need to make any modifications to Hobbit itself - you basically run "hobbitd_channel --channel=stachg MYDBPROGRAM" and your program is fed all of the status messages on it's STDIN. Is there any information about the channels? I know there is also a notes channel, but I couldn't find any (usefull) information how to use this. Same for the data channel. I want to "misuse" the hobbit communication to send extra information.
As for the trends data, I believe there are some patches to the rrdtool library that will let you store the RRD files in a DB. Check the rrd-users mailing list. Thx, I will take a look at it. Otherwise, patching create_and_update_rrd in do_rrd.c is not that hard so an external command can be called.
My personal favourite for a DB backend would be SQLite, since this eliminates all requirements for an external DB ressource. You don't want your monitoring to stop working when the DB becomes inaccessible. The same mysql database will be used for other stuff.
Stef
On Fri, Dec 08, 2006 at 04:32:59PM +0100, Stef Coene wrote:
On Friday 08 December 2006 14:05, Henrik Stoerner wrote:
Be careful with the status messages, at least if you want to store the current status in a DB. Historical status messages - those that are currently stored in the histlogs/ directory - are OK, but for the current status log I think that the overhead associated with updating each status log in a DB every 5 minutes will be very high. And the number of times you'll actually be retrieving these data will probably be very small. Today I could have used such information. One of our servers went down, out of memory status. No way to know the process that did this.
If it was running a Hobbit client, it should have saved the latest client message before it crashed, including the "ps" listing.
Copying the historical statuslogs (or the current ones) is very easy to do with an extra Hobbit worker module hanging off the "status" or "stachg" channels. You do not need to make any modifications to Hobbit itself - you basically run "hobbitd_channel --channel=stachg MYDBPROGRAM" and your program is fed all of the status messages on it's STDIN. Is there any information about the channels? I know there is also a notes channel, but I couldn't find any (usefull) information how to use this. Same for the data channel. I want to "misuse" the hobbit communication to send extra information.
It's a plain-text protocol. The easiest way to look at it is to run hobbitd_channel --channel=status cat and just watch the messages go by. Each message begins with a line @@status#1234|TIMESTAMP|HOSTNAME| and some more fields on the first line; then comes the text (if any) for the message - for a status message, this will be the raw message text. Finally, a line with "@@" marks the end of the message. It's pretty simple to work out the fields, except for the timestamps - there are timestamps for the time the message was received, the time of the last status change, the expiry time for ack's and disables etc. But for just storing it into a DB you probably don't need to care much about those. If you do, look at hobbitd/hobbitd.c and the posttochannel() routine which builds these messages. Regards, Henrik
On Friday 08 December 2006 21:45, Henrik Stoerner wrote:
If it was running a Hobbit client, it should have saved the latest client message before it crashed, including the "ps" listing. It saves the status when there is a change in the status. But I didn't had a proc monitor defined, so the status is white all the time. Even during the memory problem. So when the box was rebooted, the status was overwritten :(
It's a plain-text protocol. The easiest way to look at it is to run
hobbitd_channel --channel=status cat
and just watch the messages go by. Each message begins with a line
@@status#1234|TIMESTAMP|HOSTNAME|
and some more fields on the first line; then comes the text (if any) for the message - for a status message, this will be the raw message text. Finally, a line with "@@" marks the end of the message. It's pretty simple to work out the fields, except for the timestamps - there are timestamps for the time the message was received, the time of the last status change, the expiry time for ack's and disables etc. But for just storing it into a DB you probably don't need to care much about those. If you do, look at hobbitd/hobbitd.c and the posttochannel() routine which builds these messages. Thx, as I said, I will write down the stuff I "learn". What do you prefer? That I send you my information or that I update my "own" wiki? (I installed twiki some weeks ago and already created a hobbit web). I don't mind putting this twiki open for everybody so everyone can contribute.
Stef
On Fri, Dec 08, 2006 at 10:26:34PM +0100, Stef Coene wrote:
On Friday 08 December 2006 21:45, Henrik Stoerner wrote:
If it was running a Hobbit client, it should have saved the latest client message before it crashed, including the "ps" listing. It saves the status when there is a change in the status. But I didn't had a proc monitor defined, so the status is white all the time. Even during the memory problem. So when the box was rebooted, the status was overwritten :(
Even though the "proc" status doesn't change, the "memory" status would change. And a status that goes red triggers saving the full, raw client data (if you have the [hostdata] task enabled and you're running Hobbit 4.2.0).
If the memory status did go red, and you go to the detailed historical memory status webpage, you should find a "Client data" link at the bottom of the page - this is the data saved from when the memory status went red.
What do you prefer? That I send you my information or that I update my "own" wiki? (I installed twiki some weeks ago and already created a hobbit web).
I don't mind putting this twiki open for everybody so everyone can contribute.
Putting it on a Wiki is probably more efficient than having it go via me. There is a Hobbit wiki somewhere already, but I don't mind linking to wherever you decide to make the docs available.
Regards, Henrik
On Friday 08 December 2006 22:58, Henrik Stoerner wrote:
If the memory status did go red, and you go to the detailed historical memory status webpage, you should find a "Client data" link at the bottom of the page - this is the data saved from when the memory status went red. I didn't know that hobbit saves the full client log on a status change. I found the info. Thx.
What do you prefer? That I send you my information or that I update my "own" wiki? (I installed twiki some weeks ago and already created a hobbit web). I don't mind putting this twiki open for everybody so everyone can contribute.
Putting it on a Wiki is probably more efficient than having it go via me. There is a Hobbit wiki somewhere already, but I don't mind linking to wherever you decide to make the docs available. Ok, I will start updating my own twiki. I know there is the shire, but there is not much info on it. Anyway, I prefer to have my own documentation. This can later be merged with something else.
Stef
I have requested for unsubscribe my email list, but i am still getting lost email about hobbit. please take me off the list.
Thanks.
--- Henrik Stoerner <henrik at hswn.dk> wrote:
On Thu, Dec 07, 2006 at 11:27:08PM +0100, Stef Coene wrote:
On Thursday 07 December 2006 23:01, Trent Melcher wrote:
Has anyone thought about or implemented a configuration database backend for hobbit, primarily a database replacement for the hobbit-clients.cfg and hobbit-alerts.cfg to start....these are a couple that I would like to buils a webfront to and be able to give limited access to users so they could modify thresholds and alerting capabilities when needed.
I did this about 4 years back for Big Brother using Informix, I was able to setup thresholds and a replacement for the bb-host file inside a couple tables in a database. However back then Big Brother was mostly shell scripts and flat files for all its configuration. So adding hooks into it for talking to a databse was easy. With hobbit its all compiled code and Im not sure where to start, plus I don't have access to an Informix database or the API's for it anymore so Im looking at mysql and its api's for writing code in C.
We are thinking about using mysql for storing trends and status messages.
Be careful with the status messages, at least if you want to store the current status in a DB. Historical status messages - those that are currently stored in the histlogs/ directory - are OK, but for the current status log I think that the overhead associated with updating each status log in a DB every 5 minutes will be very high. And the number of times you'll actually be retrieving these data will probably be very small.
Copying the historical statuslogs (or the current ones) is very easy to do with an extra Hobbit worker module hanging off the "status" or "stachg" channels. You do not need to make any modifications to Hobbit itself - you basically run "hobbitd_channel --channel=stachg MYDBPROGRAM" and your program is fed all of the status messages on it's STDIN.
As for the trends data, I believe there are some patches to the rrdtool library that will let you store the RRD files in a DB. Check the rrd-users mailing list.
My personal favourite for a DB backend would be SQLite, since this eliminates all requirements for an external DB ressource. You don't want your monitoring to stop working when the DB becomes inaccessible.
Regards, Henrik
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Yahoo! Music Unlimited Access over 1 million songs. http://music.yahoo.com/unlimited
I would love to see something like this. Hobbit seems to be at that or close to that stage where something like a small db could really make things much more powerful. I would, however, encourage adopting something like what postfix does where many things are abstracted to the point where the app could use potentially any (reasonable) data store with no obtuse requirements about mandatory ones, outside of a config file to configure for a datastore.
Obviously, there would need to be the plain old config options for people who want reg configs, but sometimes reg configs don't work so well for a lot of systems with a lot of differences.
On 12/7/06, Trent Melcher <tmelcher at saionline.com> wrote:
Has anyone thought about or implemented a configuration database backend for hobbit, primarily a database replacement for the hobbit-clients.cfg and hobbit-alerts.cfg to start....these are a couple that I would like to buils a webfront to and be able to give limited access to users so they could modify thresholds and alerting capabilities when needed.
I did this about 4 years back for Big Brother using Informix, I was able to setup thresholds and a replacement for the bb-host file inside a couple tables in a database. However back then Big Brother was mostly shell scripts and flat files for all its configuration. So adding hooks into it for talking to a databse was easy. With hobbit its all compiled code and Im not sure where to start, plus I don't have access to an Informix database or the API's for it anymore so Im looking at mysql and its api's for writing code in C.
I guess Im looking to see if anyone has started some development in this area?
Thanks Trent
Trent Melcher Unix/Linux Systems Administrator Securities America Financial Corporation (402) 399-9111 x2774 tmelcher at saionline.com Securities America, Inc., a Registered Broker/Dealer, Member NASD/SIPC. Advisory services offered through Securities America Advisors, Inc., an SEC Registered Investment Advisory .
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
On Friday 08 December 2006 00:01, Trent Melcher wrote:
Has anyone thought about or implemented a configuration database backend for hobbit, primarily a database replacement for the hobbit-clients.cfg and hobbit-alerts.cfg to start....
I have been thinking about it ... but I think Henrik's concern is not to add a dependency on another tool to the monitoring system.
There are however ways of getting around this. For example, it may be possible to have a task that builds the configuration files from a database. You can then potentially have both advantages.
these are a couple that I would like to buils a webfront to and be able to give limited access to users so they could modify thresholds and alerting capabilities when needed.
For me the big reason isn't to ease configuration, but rather to leverage existing configuration information we have on our hosts. Our linux machines are deployed automatically based on configuration information in our config database, and includes IP addresses, function, hardware, site information etc, sufficient to deploy the machine ready for production in 10 minutes. Then, monitoring has to be added ....
I did this about 4 years back for Big Brother using Informix, I was able to setup thresholds and a replacement for the bb-host file inside a couple tables in a database. However back then Big Brother was mostly shell scripts and flat files for all its configuration. So adding hooks into it for talking to a databse was easy. With hobbit its all compiled code and Im not sure where to start, plus I don't have access to an Informix database or the API's for it anymore so Im looking at mysql and its api's for writing code in C.
This approach would probably mean that Hobbit is dependant on one of those flaky things it is supposed to monitor, or in the case of a power outage, one of those things that might not start up ...
I guess Im looking to see if anyone has started some development in this area?
I think there are potential advantages, but IMHO: -a means for providing a standard way of configuring external checks should be done first -Henrik needs to agree with the architecture
Since we already have a nice web interface to our configuration system (built with perl Catalyst) with authentication (against our LDAP server that everything authenticates against), session management, authorisation etc. already handled, I wouldn't really want to put user authentication in the same place.
Regards, Buchan
-- Buchan Milne ISP Systems Specialist - Monitoring/Authentication Team Leader B.Eng,RHCE(803004789010797),LPIC-2(LPI000074592)
On Fri, 8 Dec 2006, Buchan Milne wrote:
This approach would probably mean that Hobbit is dependant on one of those flaky things it is supposed to monitor, or in the case of a power outage, one of those things that might not start up ...
Our Windows client Mr Big pulls its configuration from a simple daemon called minicfg. It caches the configuration locally, so if minicfg is unavailable it simply uses the local copy.
Most clients can use this as their only configuration:
.config 10.0.4.5 27016
http://extranet.365-24.se/MrBig/
Ulric
On Fri, 2006-12-08 at 09:05 +0200, Buchan Milne wrote:
On Friday 08 December 2006 00:01, Trent Melcher wrote:
Has anyone thought about or implemented a configuration database backend for hobbit, primarily a database replacement for the hobbit-clients.cfg and hobbit-alerts.cfg to start....
I have been thinking about it ... but I think Henrik's concern is not to add a dependency on another tool to the monitoring system.
There are however ways of getting around this. For example, it may be possible to have a task that builds the configuration files from a database. You can then potentially have both advantages.
I do that currently with my snmp-capable devices. I have a mysql database that identifies the hostname, device type, and snmp authentication credentials (about half snmp v3, half community strings).
From that I generate mrtg configs and a bb-host file.
-- Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX Austin Energy http://www.austinenergy.com
On Fri, Dec 08, 2006 at 09:05:49AM +0200, Buchan Milne wrote:
For me the big reason isn't to ease configuration, but rather to leverage existing configuration information we have on our hosts. Our linux machines are deployed automatically based on configuration information in our config database, and includes IP addresses, function, hardware, site information etc, sufficient to deploy the machine ready for production in 10 minutes. Then, monitoring has to be added ....
Doing an automated extract of data from the DB and dumping that in bb-hosts format into a file or directory which is then included in the bb-hosts file is probably a fairly simple thing to setup.
I think there are potential advantages, but IMHO: -a means for providing a standard way of configuring external checks should be done first -Henrik needs to agree with the architecture
I'm all for GUI front-ends to build the configuration files, whether they use a DB backend or something else.
I'm very much against requiring a database to be running for Hobbit to work. The basic configuration-files (bb-hosts, hobbit-alerts.cfg and hobbit-clients.cfg) should be accessible without relying on any extra infrastructure components. Too often, those break just when you need your monitoring the most.
(For a real-life scenario, consider this: Someone accidentally hits the BIG RED "emergency power-off" button that cuts all power to your machine room, bypassing the UPS and diesel generators. 1200 servers turned off immediately, including Hobbit, databases, network switches, routers, fileservers etc. When power returned, the Hobbit servers did a quick filesystem check and were back online in less than 10 minutes (thanks to using journaling filesystems everywhere). All of the DB systems required manual intervention to get up and running, some of which took several hours to complete. Hobbit was the only system available for checking which servers were running and which ones weren't. True story).
So - using a DB for configuration is OK, as long as you dump the DB config into simple text-based files that can be used by Hobbit. If you want to use a DB backend for storing monitoring data that Hobbit collects, put it in an external Hobbit worker module, so Hobbit doesn't stop if the DB fails.
Regards, Henrik
On Friday 08 December 2006 14:21, Henrik Stoerner wrote:
I'm very much against requiring a database to be running for Hobbit to work. The basic configuration-files (bb-hosts, hobbit-alerts.cfg and hobbit-clients.cfg) should be accessible without relying on any extra infrastructure components. Too often, those break just when you need your monitoring the most. Totally agree. The idea was to create a web based frontend that stores the information "somewhere" (mysql or SQLite of xml) that generates the needed config files.
If I can convince some other people (one of them is my boss), you may expect lots of questions (and hopelly patches) the next few months ;)
The idea is to install a management server at our customers. This will be running hobbit and a mysql database. Hobbit will be used for real-time monitoring and short-term trending. The mysql database will be updated by hobbit and every night synced to a central server. This central server will be used for inventory and long-term trending. The monitoring is for AIX servers running Oracle software and Tivoli Storage Manager.
On my todo-list:
- web based driven configuration
- external collection of status messages and trending
- mis use of hobbit and data channels for communication -> can this be protected with ssl or something else ?
- lots and lots AIX and Oracle addons
- documentation of everything I'm trying to understand
If anyone has already something on this list or can/will help, please contact me.
My major problem is that I can not program in C. I know perl fairly well.
Stef
On Fri, Dec 08, 2006 at 04:52:02PM +0100, Stef Coene wrote:
On my todo-list:
- web based driven configuration
- external collection of status messages and trending
- mis use of hobbit and data channels for communication -> can this be protected with ssl or something else ?
Not as it is now. If you need that, I'd suggest installing something like OpenVPN and feed the communications through that - it's simple, easy to setup, and a reasonably efficient communications path.
My major problem is that I can not program in C. I know perl fairly well.
Why would that be a problem ? What you're planning to do can be done without touching any of the Hobbit code, you'll just be interfacing to the public interfaces (config files, hobbitd channels, the network interface on port 1984) and Hobbit doesn't care how you do that - whether it's from a Perl program, or a C program.
Of course, doing it in Perl means I cannot help you develop or maintain the code, since my abilities in Perl are probably even worse than your knowledge of C.
Regards, Henrik
On Friday 08 December 2006 21:50, Henrik Stoerner wrote:
On Fri, Dec 08, 2006 at 04:52:02PM +0100, Stef Coene wrote:
On my todo-list:
- web based driven configuration
- external collection of status messages and trending
- mis use of hobbit and data channels for communication -> can this be protected with ssl or something else ?
Not as it is now. If you need that, I'd suggest installing something like OpenVPN and feed the communications through that - it's simple, easy to setup, and a reasonably efficient communications path. Pff, my servers will be AIX boxes :( The best thing I can think off is ssh forwarding.
My major problem is that I can not program in C. I know perl fairly well. Why would that be a problem ? What you're planning to do can be done without touching any of the Hobbit code, you'll just be interfacing to the public interfaces (config files, hobbitd channels, the network interface on port 1984) and Hobbit doesn't care how you do that - whether it's from a Perl program, or a C program.
Of course, doing it in Perl means I cannot help you develop or maintain the code, since my abilities in Perl are probably even worse than your knowledge of C. That's why I wanted to this in C if possible. Stuff like ssl support for communation can be usefull for others.
Stef
On Fri, Dec 08, 2006 at 10:41:28PM +0100, Stef Coene wrote:
On Friday 08 December 2006 21:50, Henrik Stoerner wrote:
On Fri, Dec 08, 2006 at 04:52:02PM +0100, Stef Coene wrote:
On my todo-list:
- web based driven configuration
- external collection of status messages and trending
- mis use of hobbit and data channels for communication -> can this be protected with ssl or something else ?
Not as it is now. If you need that, I'd suggest installing something like OpenVPN and feed the communications through that - it's simple, easy to setup, and a reasonably efficient communications path. Pff, my servers will be AIX boxes :( The best thing I can think off is ssh forwarding.
Well, adding SSL support for the communication with the Hobbit network daemon has been on my list of "nice to have" features for some time. It seems more and more people are asking for it.
Regards, Henrik
Hi,
Stef Coene wrote :
On my todo-list:
- mis use of hobbit and data channels for communication
Here, we mis-use the "notes" channel to build inventory for hosts not yet monitored (we had to use the same ip/port as hobbit, because it's the only thing we can garantee is open on the firewalls). I adapted the "hobbitd_filestore" so it dumps the data, processed later by a database feeder. We don't need the notes channel because every status page has a link to a wiki page, used as a quick-n-dirty reference, aka knowlegde database". I consider adding "user1" and "user2" channels to hobbit, but at the time it works without further patching.
-- Charles Goyard - cgoyard at cvf.fr - (+33) 1 45 38 01 31
participants (10)
-
bgmilne＠staff.telkomsa.net
-
bigdan＠gmail.com
-
cgoyard＠cvf.fr
-
dan.mcdonald＠austinenergy.com
-
greg.hubbard＠eds.com
-
henrik＠hswn.dk
-
mukhi65＠yahoo.com
-
stef.coene＠docum.org
-
tmelcher＠saionline.com
-
ulric＠siag.nu