On Thursday 03 August 2006 18:12, Brodie, Kent wrote:
Buchan- you should really capture the incoming data segment while simultaneously watching the rrd-data.log file; that way you can 100% let Henrik know which data chunks are causing RRD issues.
@@data#579958|1154700491.686221|10.10.9.32||shemali.telkomsa.net|proccounts data shemali,telkomsa,net.proccounts cron:1 @@ @@data#579959|1154700491.686591|10.10.9.32||shemali.telkomsa.net|portcounts data shemali,telkomsa,net.portcounts ssh:0 ldapclient:7 hobbitclient:0 @@ @@data#579960|1154700491.686716|10.10.9.32||shemali.telkomsa.net|netstat data shemali,telkomsa,net.netstat linux Ip: 934146872 total packets received 0 forwarded 0 incoming packets discarded 934079142 incoming packets delivered 1014866720 requests sent out Icmp: 2857 ICMP messages received 0 input ICMP message failed. ICMP input histogram: destination unreachable: 79 echo requests: 2778 2856 ICMP messages sent 0 ICMP messages failed ICMP output histogram: destination unreachable: 78 echo replies: 2778 Tcp: 15978 active connections openings 12472 passive connection openings 0 failed connection attempts 236 connection resets received 14 connections established 933371889 segments received 1014788334 segments send out 662519 segments retransmited 0 bad segments received. 24 resets sent Udp: 686592 packets received 78 packets to unknown port received. 0 packet receive errors 75535 packets sent TcpExt: 1 invalid SYN cookies received 116 packets pruned from receive queue because of socket buffer overrun ArpFilter: 0 19067 TCP sockets finished time wait in fast timer 7114110 delayed acks sent 5809 delayed acks further delayed because of locked socket Quick ack mode was activated 708 times 66719 packets directly queued to recvmsg prequeue. 1788 packets directly received from backlog 3286186 packets directly received from prequeue 752163521 packets header predicted 9728 packets header predicted and directly queued to user TCPPureAcks: 3672858 TCPHPAcks: 746701951 TCPRenoRecovery: 0 TCPSackRecovery: 125111 TCPSACKReneging: 0 TCPFACKReorder: 0 TCPSACKReorder: 0 TCPRenoReorder: 0 TCPTSReorder: 1 TCPFullUndo: 1 TCPPartialUndo: 9 TCPDSACKUndo: 0 TCPLossUndo: 381 TCPLoss: 505690 TCPLostRetransmit: 494 TCPRenoFailures: 0 TCPSackFailures: 1472 TCPLossFailures: 0 TCPFastRetrans: 519174 TCPForwardRetrans: 132661 TCPSlowStartRetrans: 2590 TCPTimeouts: 2034 TCPRenoRecoveryFail: 0 TCPSackRecoveryFail: 4542 TCPSchedulerFailed: 5 TCPRcvCollapsed: 13111 TCPDSACKOldSent: 708 TCPDSACKOfoSent: 0 TCPDSACKRecv: 1103 TCPDSACKOfoRecv: 0 TCPAbortOnSyn: 0 TCPAbortOnData: 5 TCPAbortOnClose: 10 TCPAbortOnMemory: 0 TCPAbortOnTimeout: 2 TCPAbortOnLinger: 0 TCPAbortFailed: 0 TCPMemoryPressures: 0 @@ @@data#579961|1154700491.687361|10.10.9.32||shemali.telkomsa.net|ifstat data shemali,telkomsa,net.ifstat linux bond0 Link encap:Ethernet HWaddr 00:12:79:D7:D7:1B inet6 addr: fe80::212:79ff:fed7:d71b/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:935050018 errors:0 dropped:0 overruns:0 frame:0 TX packets:1014881892 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:815414722 (777.6 MiB) TX bytes:3419943727 (3.1 GiB) bond0.4 Link encap:Ethernet HWaddr 00:12:79:D7:D7:1B inet addr:192.168.211.40 Bcast:192.168.211.255 Mask:255.255.255.0 inet6 addr: fe80::212:79ff:fed7:d71b/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:934813237 errors:0 dropped:0 overruns:0 frame:0 TX packets:1014856688 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1705163146 (1.5 GiB) TX bytes:3889683012 (3.6 GiB) bond0.4:4 Link encap:Ethernet HWaddr 00:12:79:D7:D7:1B inet addr:192.168.211.204 Bcast:192.168.211.255 Mask:255.255.255.0 UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 bond0.54 Link encap:Ethernet HWaddr 00:12:79:D7:D7:1B inet addr:192.168.12.17 Bcast:192.168.15.255 Mask:255.255.252.0 inet6 addr: fe80::212:79ff:fed7:d71b/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:25654 errors:0 dropped:0 overruns:0 frame:0 TX packets:55 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2296065 (2.1 MiB) TX bytes:2478 (2.4 KiB) bond0.106 Link encap:Ethernet HWaddr 00:12:79:D7:D7:1B inet addr:192.168.106.69 Bcast:192.168.106.255 Mask:255.255.255.0 inet6 addr: fe80::212:79ff:fed7:d71b/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:211968 errors:0 dropped:0 overruns:0 frame:0 TX packets:27252 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:11974449 (11.4 MiB) TX bytes:2811952 (2.6 MiB) eth0 Link encap:Ethernet HWaddr 00:12:79:D7:D7:1B inet6 addr: fe80::212:79ff:fed7:d71b/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:934187736 errors:0 dropped:0 overruns:0 frame:0 TX packets:1014881888 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:751314594 (716.5 MiB) TX bytes:3419943441 (3.1 GiB) Interrupt:201 eth1 Link encap:Ethernet HWaddr 00:12:79:D7:D7:1B inet6 addr: fe80::212:79ff:fed7:d71b/64 Scope:Link UP BROADCAST RUNNING NOARP SLAVE MULTICAST MTU:1500 Metric:1 RX packets:862282 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:64100128 (61.1 MiB) TX bytes:286 (286.0 b) Interrupt:209 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:216 errors:0 dropped:0 overruns:0 frame:0 TX packets:216 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:71610 (69.9 KiB) TX bytes:71610 (69.9 KiB) @@ @@data#579962|1154700491.688080|10.10.9.32||shemali.telkomsa.net|vmstat data shemali,telkomsa,net.vmstat linux 0 2 1200 788904 1772 774328 0 0 3009 2047 8106 22738 0 18 55 27 @@ Regards, Buchan -- Buchan Milne ISP Systems Specialist B.Eng,RHCE(803004789010797),LPIC-2(LPI000074592)