[FS#3366] router keeps running out of memory, have to power cycle it periodically, need help figuring out why.

OpenWrt Bugs openwrt-bugs at lists.openwrt.org
Mon Jan 4 06:08:59 EST 2021


The following task has been changed.  The changes are listed below.  For full information about what has changed, visit the URL and click the History tab.

FS#3366 - router keeps running out of memory, have to power cycle it periodically, need help figuring out why.
User who did this: Baptiste Jonglez (bjonglez)
Task details edited:
Supply the following if possible:
 - Device problem occurs on 
netgear r6350
 - Software versions of OpenWrt/LEDE release, packages, etc.
OpenWrt 19.07.4 r11208-ce6496d796 / LuCI openwrt-19.07 branch git-20.247.75781-0d0ab01
I installed ocserv too but I don't think its the problem
 - Steps to reproduce
Just let router run, maybe will stay up for a week and then at some point gui tells me can't fork,  I can't login to router with ssh and I have to power cycle it.

I added swap space on a USB stick to try to keep it from locking up.
I've ssh'ed into and run the following to try to figure out what's going on.

I expected to find a process growing but nothing looks obvious to me.   I just see free memory keep going down, and I see a bit of swap space used.

Any suggestions on what else I could do to figure out what's going on with the memory usage?

looking at meminfo it shows about ~100M used but the numbers don't seem to add up to that.

while sleep 60
  top -b -n 1 | head -n 10
  ps w | awk '$3 > 4000'
  free -m
  cat /proc/meminfo
  printf "\n"

Here's what 10 days looks like:

Wed Sep 30 00:56:16 EDT 2020
 00:56:16 up 10 days,  2:38,  load average: 0.00, 0.02, 0.00
Mem: 100276K used, 23496K free, 1076K shrd, 1832K buff, 4600K cached
CPU:   0% usr   0% sys   0% nic 100% idle   0% io   0% irq   0% sirq
Load average: 0.01 0.02 0.00 2/71 27832
27831  7096 root     R     1224   1%   0% top -b -n 1
 2175  2174 root     S     4912   4%   0% ocserv-sm
 2174     1 root     S     4740   4%   0% ocserv-main
  931     1 root     S     2324   2%   0% /sbin/rpcd -s /var/run/ubus.sock -t 30
 2001     1 root     S     1784   1%   0% /usr/sbin/hostapd -s -P /var/run/wifi-phy0.pid -B /var/run/hostapd-phy0.conf
 1077     1 root     S     1748   1%   0% /sbin/netifd
 2174 root      4740 S    ocserv-main
 2175 root      4912 S    ocserv-sm
              total        used        free      shared  buff/cache   available
Mem:         123772       93796       23544        1076        6432        2604
Swap:        262140        1024      261116
MemTotal:         123772 kB
MemFree:           23500 kB
MemAvailable:       2560 kB
Buffers:            1832 kB
Cached:             4600 kB
SwapCached:          232 kB
Active:             5184 kB
Inactive:           3364 kB
Active(anon):       1120 kB
Inactive(anon):     2072 kB
Active(file):       4064 kB
Inactive(file):     1292 kB
Unevictable:           0 kB
Mlocked:               0 kB
HighTotal:             0 kB
HighFree:              0 kB
LowTotal:         123772 kB
LowFree:           23500 kB
SwapTotal:        262140 kB
SwapFree:         261116 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:          1960 kB
Mapped:             2136 kB
Shmem:              1076 kB
Slab:              11756 kB
SReclaimable:       1912 kB
SUnreclaim:         9844 kB
KernelStack:         624 kB
PageTables:          308 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      324024 kB
Committed_AS:       6680 kB
VmallocTotal:    1040376 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB


More information can be found at the following URL:

You are receiving this message because you have requested it from the Flyspray bugtracking system.  If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.

More information about the openwrt-bugs mailing list