Jay Paroline

The saga continues. Today while checking up on one of our servers, I noticed that it’s load average was insanely low. It’s one of our “biggest” servers, so it typically deals with a large percentage of our traffic. Load averages over 10 are quite normal, and it was seeing a load average of 2-3.

root@RHL039:~# tail /var/log/messages
Apr 28 10:59:56 RHL039 kernel: printk: 9445 messages suppressed.
Apr 28 10:59:56 RHL039 kernel: ip_conntrack: table full, dropping packet.
Apr 28 11:00:01 RHL039 kernel: printk: 9756 messages suppressed.
Apr 28 11:00:01 RHL039 kernel: ip_conntrack: table full, dropping packet.
Apr 28 11:00:06 RHL039 kernel: printk: 6111 messages suppressed.
Apr 28 11:00:06 RHL039 kernel: ip_conntrack: table full, dropping packet.
Apr 28 11:00:11 RHL039 kernel: printk: 3900 messages suppressed.
Apr 28 11:00:11 RHL039 kernel: ip_conntrack: table full, dropping packet.
Apr 28 11:00:16 RHL039 kernel: printk: 2063 messages suppressed.
Apr 28 11:00:16 RHL039 kernel: ip_conntrack: table full, dropping packet.

root@RHL039:~# cat /proc/sys/net/ipv4/ip_conntrack_max
800000
root@RHL039:~# cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
799995


When the server drops packets like this, things seem randomly and intermittently broken, so we apologize for that. I upped the limit (again) to get things working, but we still don’t know why so many connections are being held open in the first place.