On a related note, I see lots (sometimes dozens, depending on the client) of RRD files with names “ifstat.#.#.#.#.rrd”, where the ‘#’ are numbers, like this sampling:

-rw-r----- 1 xymon xymon 423K Mar 6 20:07 ifstat.47.147.0.12.rrd -rw-r----- 1 xymon xymon 423K Mar 6 20:12 ifstat.108.0.121.0.rrd -rw-r----- 1 xymon xymon 423K Mar 7 02:53 ifstat.73.68.58.9.rrd -rw-r----- 1 xymon xymon 423K Mar 7 09:23 ifstat.0.1.0.128.rrd -rw-r----- 1 xymon xymon 423K Mar 7 09:23 ifstat.0.116.111.32.rrd -rw-r----- 1 xymon xymon 423K Mar 7 09:23 ifstat.101.115.115.32.rrd

What are these numbers? I care because a host can have _lots_ of these files, and each currently takes up almost half a MB and we have thousands of hosts!

Thanks again for any help…

david

~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~ David Mills Systems Administrator Northrop Grumman (512) 595-1238 (mobile)

From: Mills,David (HHSC Contractor) Sent: Wednesday, March 28, 2018 9:54 AM To: Jeremy Laidman <jeremy at laidman.org> Cc: xymon at xymon.com Subject: RE: [Xymon] RRD files are too big

Brilliant, Jeremy. Thx for the assist!

~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~ David Mills Systems Administrator Northrop Grumman (512) 595-1238 (mobile)

From: Jeremy Laidman [mailto:jeremy at laidman.org] Sent: Wednesday, March 28, 2018 8:54 AM To: Mills,David (HHSC Contractor) <David.Mills at hhsc.state.tx.us<mailto:David.Mills at hhsc.state.tx.us>> Cc: xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] RRD files are too big

Forgot to cc the list.

On Wed., 28 Mar. 2018, 10:50 am Jeremy Laidman, <jeremy at laidman.org<mailto:jeremy at laidman.org>> wrote: I see the same thing.

Run "rrdtool info /path/to/file.rrd | grep cf" on both old and new RRD files. I'm guessing you will see many more consolidation functions in the newer RRD files. This may be due to an enhanced rrddefinitions.cfg file in the Terabithia package, apparently extended for the 4.3.13 release. The additional CFs are related to Holt-Winters prediction (HWPREDICT).

I suspect you could copy in the previous rrddefinitions.cfg file, delete an RRD file, and it'll get created without the extra CFs, and much smaller.

It may be possible to export, edit, import so as to not lose data.

J

On 28 March 2018 at 08:35, Mills,David (HHSC Contractor) <David.Mills at hhsc.state.tx.us<mailto:David.Mills at hhsc.state.tx.us>> wrote: Hey, Folks…

I posted on this issue a few months ago (cf http://lists.xymon.com/archive/2017-November/045048.html), but still need help.

I migrated / upgraded from Xymon 4.3.3 on a Solaris 10 host to a Linux host (RHEL 6) using the Terbithia binaries (4.3.28), and things have been doing well for the most part. Unfortunately the new RRD files are massively bigger than the original ones on the old server, causing a lot of space issues on the file system.

The old RRD files took up around 38KB each (avg.) but the new ones are fairly uniform at 400+KB each.

Can anyone please tell me how to get those big files down-sized back to something much closer to the old sizes? Or, perhaps, someone can point me at the parameters Xymon uses that determine the RRD file sizes?

FWIW, this is a representative listing of one host’s RRD files:

16:33:08 lwmp043:[xymon]/xymon-data/rrd> ls twiu003/ total 13M drwxr-xr-x 2 xymon xymon 4.0K Feb 26 12:14 ./ drwxr-xr-x 6284 xymon xymon 204K Mar 22 07:39 ../ -rw-r--r-- 1 xymon xymon 213K Mar 27 15:57 clock.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 15:52 disk,dev,odm.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 14:36 disk,dev,vx,dmp.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 14:31 disk,dev,vx,rdmp.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 16:12 disk,dpool.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 15:41 disk,etc,dfs,sharetab.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 15:57 disk,etc,svc,volatile.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 14:31 disk,export.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 16:17 disk,export,zones,cit032.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 15:01 disk,export,zones,clt031.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 14:41 disk,export,zones.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 15:16 disk,root.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 14:46 disk,tmp.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 14:56 disk,var.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 14:56 ifstat.aggr2.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 15:01 ifstat.aggr3.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 14:31 ifstat.e1000g3.rrd -rw-r--r-- 1 xymon xymon 423K Mar 27 14:31 ifstat.nxge0.rrd -rw-r--r-- 1 xymon xymon 213K Mar 27 15:01 la.rrd -rw-r--r-- 1 xymon xymon 213K Mar 27 15:11 memory.real.rrd -rw-r--r-- 1 xymon xymon 213K Mar 27 14:36 memory.swap.rrd -rw-r--r-- 1 xymon xymon 3.3M Mar 27 16:17 netstat.rrd -rw-r--r-- 1 xymon xymon 213K Mar 27 14:51 procs.rrd -rw-r--r-- 1 xymon xymon 844K Mar 27 15:53 sar.rrd -rw-r--r-- 1 xymon xymon 213K Mar 27 15:56 tcp.conn.rrd -rw-r--r-- 1 xymon xymon 213K Mar 27 15:26 users.rrd

TIA!

david

~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~ David Mills Systems Administrator Northrop Grumman (512) 595-1238 (mobile)

Xymon mailing list Xymon at xymon.com<mailto:Xymon at xymon.com> http://lists.xymon.com/mailman/listinfo/xymon

Thx, Shawn...

I agree that the naming suggests some unique connection info like an IP, but I don’t see any of our IPs embedded into those names. The snippet I offered earlier was from a Windows box. The following is an example from one of our Solaris 11 hosts. Note that the RRD files with "net<digit>" correspond to iface names net0, ... In the example that follows, the "vnetldc" probably refers to a virtual network interface tag since this host is part of a Solaris LDOM. The trailing hex digits probably have the same identifying function as the four dotted numbers in my original example, though I still don't know what they mean:

-rw-r--r-- 1 xymon xymon 423K Mar 28 11:12 ifstat.net0.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.net1.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.net2.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.net3.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:16 ifstat.net4.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:47 ifstat.phys.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:02 ifstat.vnetldc0x10.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:52 ifstat.vnetldc0x11.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 12:02 ifstat.vnetldc0x12.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:42 ifstat.vnetldc0x13.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.vnetldc0x14.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.vnetldc0x15.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:02 ifstat.vnetldc0x2.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 12:02 ifstat.vnetldc0x39.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 12:17 ifstat.vnetldc0x3a.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 12:07 ifstat.vnetldc0x3b.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:32 ifstat.vnetldc0x3c.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:32 ifstat.vnetldc0x3d.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.vnetldc0x3e.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:52 ifstat.vnetldc0x3f.rrd

~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~ David Mills Systems Administrator Northrop Grumman (512) 595-1238 (mobile)

-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Shawn Heisey Sent: Wednesday, March 28, 2018 11:03 AM To: xymon at xymon.com Subject: Re: [Xymon] RRD files are too big

On 3/28/2018 9:12 AM, Mills,David (HHSC Contractor) wrote:

On a related note, I see lots (sometimes dozens, depending on the client) of RRD files with names “ifstat.#.#.#.#.rrd”, where the ‘#’ are numbers, like this sampling:

-rw-r-----    1 xymon xymon 423K Mar  6 20:07 ifstat.47.147.0.12.rrd

-rw-r-----    1 xymon xymon 423K Mar  6 20:12 ifstat.108.0.121.0.rrd

-rw-r-----    1 xymon xymon 423K Mar  7 02:53 ifstat.73.68.58.9.rrd

-rw-r-----    1 xymon xymon 423K Mar  7 09:23 ifstat.0.1.0.128.rrd

-rw-r-----    1 xymon xymon 423K Mar  7 09:23 ifstat.0.116.111.32.rrd

-rw-r-----    1 xymon xymon 423K Mar  7 09:23 ifstat.101.115.115.32.rrd

Looks like the filenames are generated from information sent by the xymon client running on the host.  I see files with interface names on my Xymon server. These are from a client running on a SPARC Solaris system:

rrd/sahara/ifstat.bge1.rrd rrd/sahara/ifstat.bge3.rrd rrd/sahara/ifstat.bge2.rrd rrd/sahara/ifstat.bge0.rrd

I'm running 4.3.23, RPM package built from source.  These ifstat files are 38K in size.

I would have guessed that the numbers were IP addresses, but you've got two of them in that list starting with zero, and one ending with zero, so if those are IP addresses, there may be an invalid address configured somewhere.  And if they're not IP addresses, then I have no idea what they are, and you may need to look into the client running on the host.

Thanks, Shawn

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

The code that creates the RRD files is:

setupfn2("%s.%s.rrd", "ifstat", ifname);

So the format is "ifstat.<ifname>.rrd".

The weird numbers and names are what Xymon thinks are interface names (ifname). Xymon gets the interface names from the [ifstat] section of the client message. I suspect the parsing code for the OS you are using is not compatible with the format that is in the [ifstat] section. If you can provide the [ifstat] section of client message, and the OS or class specification (typically found at the top of the client message), I might be able to be more specific.

If you've changed a host from Solaris to Linux, but still have the OS defined as Solaris (or CLASS set in hosts.cfg), the wrong match expression will be applied (for a different OS), and you may get this effect.

Until you find a fix, a work-around might be to specify INTERFACES:eth\d+ (or whatever regular expression matches your interface names) in hosts.cfg.

On 29 March 2018 at 05:54, Galen Johnson <Galen.Johnson at sas.com> wrote:

I'm betting that you are correct that these are your interface names on that server. Running 'ifconfig -a' might help better identify what you're seeing, If they are virtual devices, they may be coming and going in such a way that Xymon sees them, and adds them but they are named something else the next run since I would expect the timestamps to be the same if they were all active.

=G=

---

From: Xymon <xymon-bounces at xymon.com> on behalf of Mills,David (HHSC Contractor) <David.Mills at hhsc.state.tx.us> Sent: Wednesday, March 28, 2018 1:28 PM To: Shawn Heisey; xymon at xymon.com Subject: Re: [Xymon] RRD files are too big

EXTERNAL

Thx, Shawn...

I agree that the naming suggests some unique connection info like an IP, but I don’t see any of our IPs embedded into those names. The snippet I offered earlier was from a Windows box. The following is an example from one of our Solaris 11 hosts. Note that the RRD files with "net<digit>" correspond to iface names net0, ... In the example that follows, the "vnetldc" probably refers to a virtual network interface tag since this host is part of a Solaris LDOM. The trailing hex digits probably have the same identifying function as the four dotted numbers in my original example, though I still don't know what they mean:

-rw-r--r-- 1 xymon xymon 423K Mar 28 11:12 ifstat.net0.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.net1.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.net2.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.net3.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:16 ifstat.net4.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:47 ifstat.phys.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:02 ifstat.vnetldc0x10.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:52 ifstat.vnetldc0x11.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 12:02 ifstat.vnetldc0x12.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:42 ifstat.vnetldc0x13.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.vnetldc0x14.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.vnetldc0x15.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:02 ifstat.vnetldc0x2.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 12:02 ifstat.vnetldc0x39.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 12:17 ifstat.vnetldc0x3a.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 12:07 ifstat.vnetldc0x3b.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:32 ifstat.vnetldc0x3c.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:32 ifstat.vnetldc0x3d.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 10:12 ifstat.vnetldc0x3e.rrd -rw-r--r-- 1 xymon xymon 423K Mar 28 11:52 ifstat.vnetldc0x3f.rrd

~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~ David Mills Systems Administrator Northrop Grumman (512) 595-1238 (mobile)

-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Shawn Heisey Sent: Wednesday, March 28, 2018 11:03 AM To: xymon at xymon.com Subject: Re: [Xymon] RRD files are too big

On 3/28/2018 9:12 AM, Mills,David (HHSC Contractor) wrote:

...

On a related note, I see lots (sometimes dozens, depending on the client) of RRD files with names “ifstat.#.#.#.#.rrd”, where the ‘#’ are numbers, like this sampling:

-rw-r----- 1 xymon xymon 423K Mar 6 20:07 ifstat.47.147.0.12.rrd

-rw-r----- 1 xymon xymon 423K Mar 6 20:12 ifstat.108.0.121.0.rrd

-rw-r----- 1 xymon xymon 423K Mar 7 02:53 ifstat.73.68.58.9.rrd

-rw-r----- 1 xymon xymon 423K Mar 7 09:23 ifstat.0.1.0.128.rrd

-rw-r----- 1 xymon xymon 423K Mar 7 09:23 ifstat.0.116.111.32.rrd

-rw-r----- 1 xymon xymon 423K Mar 7 09:23 ifstat.101.115.115.32.rrd

Looks like the filenames are generated from information sent by the xymon client running on the host. I see files with interface names on my Xymon server. These are from a client running on a SPARC Solaris system:

rrd/sahara/ifstat.bge1.rrd rrd/sahara/ifstat.bge3.rrd rrd/sahara/ifstat.bge2.rrd rrd/sahara/ifstat.bge0.rrd

I'm running 4.3.23, RPM package built from source. These ifstat files are 38K in size.

I would have guessed that the numbers were IP addresses, but you've got two of them in that list starting with zero, and one ending with zero, so if those are IP addresses, there may be an invalid address configured somewhere. And if they're not IP addresses, then I have no idea what they are, and you may need to look into the client running on the host.

Thanks, Shawn

---

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

---

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

---

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon