[FS#3366] router keeps running out of memory, have to power cycle it periodically, need help figuring out why.

[OpenWrt Bugs]

THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.

A new Flyspray task has been opened.  Details are below. 

User who did this - wrighrc (wrighrc) 

Attached to Project - OpenWrt/LEDE Project
Summary - router keeps running out of memory, have to power cycle it periodically, need help figuring out why.
Task Type - Bug Report
Category - Base system
Status - Unconfirmed
Assigned To - 
Operating System - All
Severity - Medium
Priority - Very Low
Reported Version - openwrt-19.07
Due in Version - Undecided
Due Date - Undecided
Details - Supply the following if possible:
 - Device problem occurs on 
netgear r6350
 - Software versions of OpenWrt/LEDE release, packages, etc.
OpenWrt 19.07.4 r11208-ce6496d796 / LuCI openwrt-19.07 branch git-20.247.75781-0d0ab01
I installed ocserv too but I don't think its the problem
 - Steps to reproduce
Just let router run, maybe will stay up for a week and then at some point gui tells me can't fork,  I can't login to router with ssh and I have to power cycle it.

I added swap space on a USB stick to try to keep it from locking up.
I've ssh'ed into and run the following to try to figure out what's going on.

I expected to find a process growing but nothing looks obvious to me.   I just see free memory keep going down, and I see a bit of swap space used.

Any suggestions on what else I could do to figure out what's going on with the memory usage?

looking at meminfo it shows about ~100M used but the numbers don't seem to add up to that.

while sleep 60
do
  date
  uptime
  top -b -n 1 | head -n 10
  ps w | awk '$3 > 4000'
  free -m
  cat /proc/meminfo
  printf "\n"
done

Here's what 10 days looks like:

Wed Sep 30 00:56:16 EDT 2020
 00:56:16 up 10 days,  2:38,  load average: 0.00, 0.02, 0.00
Mem: 100276K used, 23496K free, 1076K shrd, 1832K buff, 4600K cached
CPU:   0% usr   0% sys   0% nic 100% idle   0% io   0% irq   0% sirq
Load average: 0.01 0.02 0.00 2/71 27832
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
27831  7096 root     R     1224   1%   0% top -b -n 1
 2175  2174 root     S     4912   4%   0% ocserv-sm
 2174     1 root     S     4740   4%   0% ocserv-main
  931     1 root     S     2324   2%   0% /sbin/rpcd -s /var/run/ubus.sock -t 30
 2001     1 root     S     1784   1%   0% /usr/sbin/hostapd -s -P /var/run/wifi-phy0.pid -B /var/run/hostapd-phy0.conf
 1077     1 root     S     1748   1%   0% /sbin/netifd
  PID USER       VSZ STAT COMMAND
 2174 root      4740 S    ocserv-main
 2175 root      4912 S    ocserv-sm
              total        used        free      shared  buff/cache   available
Mem:         123772       93796       23544        1076        6432        2604
Swap:        262140        1024      261116
MemTotal:         123772 kB
MemFree:           23500 kB
MemAvailable:       2560 kB
Buffers:            1832 kB
Cached:             4600 kB
SwapCached:          232 kB
Active:             5184 kB
Inactive:           3364 kB
Active(anon):       1120 kB
Inactive(anon):     2072 kB
Active(file):       4064 kB
Inactive(file):     1292 kB
Unevictable:           0 kB
Mlocked:               0 kB
HighTotal:             0 kB
HighFree:              0 kB
LowTotal:         123772 kB
LowFree:           23500 kB
SwapTotal:        262140 kB
SwapFree:         261116 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:          1960 kB
Mapped:             2136 kB
Shmem:              1076 kB
Slab:              11756 kB
SReclaimable:       1912 kB
SUnreclaim:         9844 kB
KernelStack:         624 kB
PageTables:          308 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      324024 kB
Committed_AS:       6680 kB
VmallocTotal:    1040376 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB




 

More information can be found at the following URL:
https://bugs.openwrt.org/index.php?do=details&task_id=3366

You are receiving this message because you have requested it from the Flyspray bugtracking system.  If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.