BigBrother migration: trending on the mem test
Hi All,
I am almost complete with my Proof-of-concept with out BigBrother to Hobbit migration. Yes, has been taking along time. There were a lot of BB customizations that needed porting.
Anyways. All my *NIX devices report into BB with their memory report as mem, not memory. So I have BigBrother & hobbit running in parallel, with a bbproxy sending bbclient data to both. One of my last issues is that I am missing memory trends for all my *nix servers. So is there anyway to tell the hobbit server to do memory trending on memory & mem reports ? I have tried a few things, but with no success.
If there is no way to configure hobbit to perform trend collection on memory & mem reports, I can just do a --extra-test option and script something. I have a few of those already. I was just curious if there was any inbuilt method.
Thanks, ~Steve
On Fri, Jul 13, 2007 at 08:34:32AM -0400, s_aiello at comcast.net wrote:
Anyways. All my *NIX devices report into BB with their memory report as mem, not memory. So I have BigBrother & hobbit running in parallel, with a bbproxy sending bbclient data to both. One of my last issues is that I am missing memory trends for all my *nix servers. So is there anyway to tell the hobbit server to do memory trending on memory & mem reports ? I have tried a few things, but with no success.
I think putting a "mem=memory" in the TEST2RRD setting and restarting the hobbitd_rrd processes (just kill them, they'll automatically restart) should do it.
Regards, Henrik
On Friday 13 July 2007 11:59, Henrik Stoerner wrote:
On Fri, Jul 13, 2007 at 08:34:32AM -0400, s_aiello at comcast.net wrote:
Anyways. All my *NIX devices report into BB with their memory report as mem, not memory. So I have BigBrother & hobbit running in parallel, with a bbproxy sending bbclient data to both. One of my last issues is that I am missing memory trends for all my *nix servers. So is there anyway to tell the hobbit server to do memory trending on memory & mem reports ? I have tried a few things, but with no success.
I think putting a "mem=memory" in the TEST2RRD setting and restarting the hobbitd_rrd processes (just kill them, they'll automatically restart) should do it.
That did not work. I will be checking to see if there is anything in the logs.
~Steve
Anyways. All my *NIX devices report into BB with their memory report as mem, not memory. So I have BigBrother & hobbit running in parallel, with a bbproxy sending bbclient data to both. One of my last issues is that I am missing memory trends for all my *nix servers. So is there anyway to tell the hobbit server to do memory trending on memory & mem reports ? I have tried a few things, but with no success.
I think putting a "mem=memory" in the TEST2RRD setting and restarting the hobbitd_rrd processes (just kill them, they'll automatically restart) should do it.
That did not work. I will be checking to see if there is anything in the logs.
Correction, that did not work for my Linux devices reporting in with a mem report, but it does work with my Solaris devices. I am assuming the output from the bb-memory version I am using, does not mesh with what Hobbit is set to trend on. So I will probably just need to send mem report to a modified memory-larrd script via the extra-scripts option. This will only be a short-time fix, until I can push hobbit clients out to replace the bb clients. The first step is to get the server upgraded, then the clients. Below I have have include the Linux/Sun memory output for reference:
Linux: green Sun Jul 15 09:56:28 EDT 2007 - Memory OK &green real 7926M / 7948M 99% &green swap 0M / 2047M 0%
Solaris: green Sun Jul 15 09:56:59 EDT 2007 - Memory OK Memory Used Total Percentage &green Physical 50378M 65536M 76% &clear Actual -1M 65536M 0% &green Swap 42658M 85191M 50%
Thanks for the guidance and help on this, ~Steve
I'm encountering an odd problem doing a rebuild of the Hobbit server. I'm using the same Solaris 8 system that I originally built my server on, however I no longer seem to be getting the pcre and gcc libs linked to my binaries, and they're entirely different sizes as well. Example:
Original hobbitsvc.cgi binary: Size: 1374336 $ldd hobbitsvc.cgi libpcre.so.0 => /usr/local/lib/libpcre.so.0 libresolv.so.2 => /usr/lib/libresolv.so.2 libsocket.so.1 => /usr/lib/libsocket.so.1 libnsl.so.1 => /usr/lib/libnsl.so.1 libc.so.1 => /usr/lib/libc.so.1 libgcc_s.so.1 => /usr/local/lib/libgcc_s.so.1 libdl.so.1 => /usr/lib/libdl.so.1 libmp.so.2 => /usr/lib/libmp.so.2 /usr/platform/SUNW,Sun-Fire-280R/lib/libc_psr.so.1
New hobbitsvc.cgi binary: Size: 532124 $ldd hobbitsvc.cgi libresolv.so.2 => /usr/lib/libresolv.so.2 libsocket.so.1 => /usr/lib/libsocket.so.1 libnsl.so.1 => /usr/lib/libnsl.so.1 libc.so.1 => /usr/lib/libc.so.1 libdl.so.1 => /usr/lib/libdl.so.1 libmp.so.2 => /usr/lib/libmp.so.2 /usr/platform/SUNW,Sun-Fire-280R/lib/libc_psr.so.1
I've tried numerous fixes. Set all the paths in LD_LIBRARY_PATH, added -R/usr/loca/lib to the Makefile for PCRE. Using the Sunfreeware SMCgcc 3.4.6 package with SMCmake installed also. I don't get any errors when running make or make install. I know it's actually using gcc:
gcc -g -O2 -Wall -Wno-unused -D_REENTRANT -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DSunOS -I. -I/build/hobbit-4.2.0/include -I/usr/local/include -o hobbitsvc.cgi hobbitsvc.o hobbitsvc-info.o hobbitsvc-trends.o ../lib/libbbgen.a -L/usr/local/lib -R/usr/local/lib -lpcre -lresolv -lsocket -lnsl
Am I missing something stupidly obvious here?
Have you tried updating your library path using crle?
=G=
-----Original Message----- From: Matthew Epp [mailto:matthew.epp at us.army.mil] Sent: Monday, July 16, 2007 2:22 PM To: hobbit at hswn.dk Subject: [hobbit] Missing libpcre and libgcc in compiled binaries
I'm encountering an odd problem doing a rebuild of the Hobbit server. I'm using the same Solaris 8 system that I originally built my server on, however I no longer seem to be getting the pcre and gcc libs linked to my binaries, and they're entirely different sizes as well. Example:
Original hobbitsvc.cgi binary: Size: 1374336 $ldd hobbitsvc.cgi libpcre.so.0 => /usr/local/lib/libpcre.so.0 libresolv.so.2 => /usr/lib/libresolv.so.2 libsocket.so.1 => /usr/lib/libsocket.so.1 libnsl.so.1 => /usr/lib/libnsl.so.1 libc.so.1 => /usr/lib/libc.so.1 libgcc_s.so.1 => /usr/local/lib/libgcc_s.so.1 libdl.so.1 => /usr/lib/libdl.so.1 libmp.so.2 => /usr/lib/libmp.so.2 /usr/platform/SUNW,Sun-Fire-280R/lib/libc_psr.so.1
New hobbitsvc.cgi binary: Size: 532124 $ldd hobbitsvc.cgi libresolv.so.2 => /usr/lib/libresolv.so.2 libsocket.so.1 => /usr/lib/libsocket.so.1 libnsl.so.1 => /usr/lib/libnsl.so.1 libc.so.1 => /usr/lib/libc.so.1 libdl.so.1 => /usr/lib/libdl.so.1 libmp.so.2 => /usr/lib/libmp.so.2 /usr/platform/SUNW,Sun-Fire-280R/lib/libc_psr.so.1
I've tried numerous fixes. Set all the paths in LD_LIBRARY_PATH, added -R/usr/loca/lib to the Makefile for PCRE. Using the Sunfreeware SMCgcc 3.4.6 package with SMCmake installed also. I don't get any errors when running make or make install. I know it's actually using gcc:
gcc -g -O2 -Wall -Wno-unused -D_REENTRANT -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DSunOS -I. -I/build/hobbit-4.2.0/include -I/usr/local/include -o hobbitsvc.cgi hobbitsvc.o hobbitsvc-info.o hobbitsvc-trends.o ../lib/libbbgen.a -L/usr/local/lib -R/usr/local/lib -lpcre -lresolv -lsocket -lnsl
Am I missing something stupidly obvious here?
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
I've thought about that, but the problem is I'd have to do that also on every system that runs the server, as we build our binaries on machines separate from everything else. Also, I've never had to modify with crle before and the existing servers are working just fine. I'm trying to figure out what has changed since I last completed a successful compile.
Galen Johnson wrote:
Have you tried updating your library path using crle?
=G=
-----Original Message----- From: Matthew Epp [mailto:matthew.epp at us.army.mil] Sent: Monday, July 16, 2007 2:22 PM To: hobbit at hswn.dk Subject: [hobbit] Missing libpcre and libgcc in compiled binaries
I'm encountering an odd problem doing a rebuild of the Hobbit server. I'm using the same Solaris 8 system that I originally built my server on, however I no longer seem to be getting the pcre and gcc libs linked to my binaries, and they're entirely different sizes as well. Example:
Original hobbitsvc.cgi binary: Size: 1374336 $ldd hobbitsvc.cgi libpcre.so.0 => /usr/local/lib/libpcre.so.0 libresolv.so.2 => /usr/lib/libresolv.so.2 libsocket.so.1 => /usr/lib/libsocket.so.1 libnsl.so.1 => /usr/lib/libnsl.so.1 libc.so.1 => /usr/lib/libc.so.1 libgcc_s.so.1 => /usr/local/lib/libgcc_s.so.1 libdl.so.1 => /usr/lib/libdl.so.1 libmp.so.2 => /usr/lib/libmp.so.2 /usr/platform/SUNW,Sun-Fire-280R/lib/libc_psr.so.1
New hobbitsvc.cgi binary: Size: 532124 $ldd hobbitsvc.cgi libresolv.so.2 => /usr/lib/libresolv.so.2 libsocket.so.1 => /usr/lib/libsocket.so.1 libnsl.so.1 => /usr/lib/libnsl.so.1 libc.so.1 => /usr/lib/libc.so.1 libdl.so.1 => /usr/lib/libdl.so.1 libmp.so.2 => /usr/lib/libmp.so.2 /usr/platform/SUNW,Sun-Fire-280R/lib/libc_psr.so.1
I've tried numerous fixes. Set all the paths in LD_LIBRARY_PATH, added -R/usr/loca/lib to the Makefile for PCRE. Using the Sunfreeware SMCgcc 3.4.6 package with SMCmake installed also. I don't get any errors when running make or make install. I know it's actually using gcc:
gcc -g -O2 -Wall -Wno-unused -D_REENTRANT -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DSunOS -I. -I/build/hobbit-4.2.0/include -I/usr/local/include -o hobbitsvc.cgi hobbitsvc.o hobbitsvc-info.o hobbitsvc-trends.o ../lib/libbbgen.a -L/usr/local/lib -R/usr/local/lib -lpcre -lresolv -lsocket -lnsl
Am I missing something stupidly obvious here?
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
Just for kicks I modified with crle on my build server, ran a make clean, make and make install. I still get the same binaries with missing lib links.
Galen Johnson wrote:
Have you tried updating your library path using crle?
Matthew Epp said the following, On 07/16/2007 01:21 PM:
I'm encountering an odd problem doing a rebuild of the Hobbit server. I'm using the same Solaris 8 system that I originally built my server on, however I no longer seem to be getting the pcre and gcc libs linked to my binaries, and they're entirely different sizes as well. Example:
When you ran configure did it have: Checking for PCRE ... Found PCRE include files in /usr/local/include Found PCRE libraries in /usr/local/lib
OR... in /usr/local/lib/ is there a libpcre.so or just a libpcre.a ? If its just a libpcre.a file its a static lib and it won't show up as being part of a dynamic dependancy as the code will be linked it directly.
I've tried numerous fixes. Set all the paths in LD_LIBRARY_PATH, added -R/usr/loca/lib to the Makefile for PCRE. Using the Sunfreeware SMCgcc 3.4.6 package with SMCmake installed also. I don't get any errors when running make or make install. I know it's actually using gcc:
gcc -g -O2 -Wall -Wno-unused -D_REENTRANT -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DSunOS -I. -I/build/hobbit-4.2.0/include -I/usr/local/include -o hobbitsvc.cgi hobbitsvc.o hobbitsvc-info.o hobbitsvc-trends.o ../lib/libbbgen.a -L/usr/local/lib -R/usr/local/lib -lpcre -lresolv -lsocket -lnsl
Am I missing something stupidly obvious here?
To unsubscribe from the hobbit list, send an e-mail to hobbit-unsubscribe at hswn.dk
sladewig wrote:
When you ran configure did it have: Checking for PCRE ... Found PCRE include files in /usr/local/include Found PCRE libraries in /usr/local/lib
Yes.
OR... in /usr/local/lib/ is there a libpcre.so or just a libpcre.a ? If its just a libpcre.a file its a static lib and it won't show up as being part of a dynamic dependancy as the code will be linked it directly.
Both. $ls -l /usr/local/lib|grep pcre -rw-r--r-- 1 root other 460740 Jul 12 2006 libpcre.a -rwxr-xr-x 1 root other 792 Jul 12 2006 libpcre.la -rwxr-xr-x 1 root other 433948 Jul 12 2006 libpcre.so.0.0.1 lrwxrwxrwx 1 root other 31 Jul 16 17:30 libpcre.so.1 -> /usr/local/lib/libpcre.so.0.0.1 -rw-r--r-- 1 root other 495576 Jul 12 2006 libpcrecpp.a -rwxr-xr-x 1 root other 1213 Jul 12 2006 libpcrecpp.la -rwxr-xr-x 1 root other 495492 Jul 12 2006 libpcrecpp.so.0.0.0 -rw-r--r-- 1 root other 15688 Jul 12 2006 libpcreposix.a -rwxr-xr-x 1 root other 853 Jul 12 2006 libpcreposix.la -rwxr-xr-x 1 root other 20988 Jul 12 2006 libpcreposix.so.0.0.0
So, you think it's seeing the libpcre.a and just building it statically? In that case, wouldn't the binary be slightly larger like the original build I had?
I found the issue. Very odd. For some reason, when I last built PCRE, it did not create symlinks for some of the files. These were missing: /usr/local/lib/libpcre.so -> libpcre.so.0.0.1 lrwxrwxrwx 1 root other 16 Jul 17 16:55 /usr/local/lib/libpcre.so.0 -> libpcre.so.0.0.1 -rwxr-xr-x 1 root other 433948 Jul 12 2006 /usr/local/lib/libpcreposix.so -> libpcreposix.so.0.0.0 lrwxrwxrwx 1 root other 21 Jul 17 16:56 /usr/local/lib/libpcreposix.so.0 -> libpcreposix.so.0.0.0
Once those were in place, the linker added in libpcre as well as libgcc as expected. Everything's working again. :) By the way, I was doing this so I could add a WIKI button to alert pages, which when clicked, does an auto-search in our internal wiki using that hostname and service as the keywords. Works great! One more thing I need to document and upload.
Matthew Epp said the following, On 07/17/2007 02:05 PM:
I found the issue. Very odd. For some reason, when I last built PCRE, it did not create symlinks for some of the files. These were missing: /usr/local/lib/libpcre.so -> libpcre.so.0.0.1 lrwxrwxrwx 1 root other 16 Jul 17 16:55 /usr/local/lib/libpcre.so.0 -> libpcre.so.0.0.1 -rwxr-xr-x 1 root other 433948 Jul 12 2006 /usr/local/lib/libpcreposix.so -> libpcreposix.so.0.0.0 lrwxrwxrwx 1 root other 21 Jul 17 16:56 /usr/local/lib/libpcreposix.so.0 -> libpcreposix.so.0.0.0
So it must have been using the static libs when it linked. If you still have the old binary around you can run nm on it and look for the references to the functions you are interested in. In this case pcre.
nm hobbitd |grep -i pcre [289] | 248184| 0|FUNC |GLOB |0 |UNDEF |pcre_compile [228] | 248196| 0|FUNC |GLOB |0 |UNDEF |pcre_exec [480] | 255612| 4|OBJT |GLOB |0 |22 |pcre_free
In this case the hobbitd binary has UNDEF for the routines with pcre. The system linker/loader would have to resolve these by using external libs such as the .so versions of the lib or the binary is a no-go. Any statically linked routines wouldn't have UNDEF.
-- steve
participants (5)
-
Galen.Johnson＠sas.com
-
henrik＠hswn.dk
-
matthew.epp＠us.army.mil
-
s_aiello＠comcast.net
-
sladewig＠bankinfo.com