[FS#2053] Regular crashes of ath10k-ct driver on ZyXEL NBG6817

Wed Nov 4 02:19:06 EST 2020

THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.

The following task has a new comment added:

FS#2053 - Regular crashes of ath10k-ct driver on ZyXEL NBG6817
User who did this - Shane (digitalcircuit)

----------
I have run into what appears to be a similar issue with the NBG6817 as well - it seems to have gotten a bit worse since updating to 19.07.4 with the new ath10k firmware (less automatic reboots, more lockups).

SQM QoS is enabled, 3 virtual WiFi networks (primary, guest, open), with VLANs over Ethernet to another access point (an RT-AC68U running FreshTomato).

With UDP syslog messages sent to another machine on my network, I've managed to capture a reboot (better), and a hang (requires manual hard reboot to recover):

**Versions**

ath10k-firmware-qca9984-ct - 2020-07-02-1
kmod-ath - 4.14.195+4.19.137-1-2
kmod-ath10k-ct - 4.14.195+2019-09-09-5e8cd86f-1
OpenWrt 19.07.4 r11208-ce6496d796 / LuCI openwrt-19.07 branch git-20.247.75781-0d0ab01

**Kernel panic and reboot**

Oct 14 15:49:09 OpenWRT kernel: [497781.448640] Unable to handle kernel paging request at virtual address dd79f4c0
Oct 14 15:49:09 OpenWRT kernel: [497781.448664] pgd = d3624000
Oct 14 15:49:09 OpenWRT kernel: [497781.454739] [dd79f4c0] *pgd=5f61141e(bad)
Oct 14 15:49:09 OpenWRT kernel: [497781.457524] Internal error: Oops: 8000000d [#1] SMP ARM
Oct 14 15:49:09 OpenWRT kernel: [497781.461686] Modules linked in: pppoe ppp_async ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_netlink nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE c
fg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_commen
t xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nfnetlink nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw
nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache iptable_raw iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat sch_cake nf_conntrack sch_tbf sch_ingress sch_htb sch_hfsc em_u32
Oct 14 15:49:09 OpenWRT kernel: [497781.515843]  cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred ledtrig_usbport nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter
ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 ifb tun usb_storage leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_of_simple ohci_platform ohci_hcd phy_qcom_dwc3 ahci ehci_platform sd_mod ahci_platform libahci
_platform libahci libata scsi_mod ehci_hcd gpio_button_hotplug ext4 mbcache jbd2 crc32c_generic
Oct 14 15:49:09 OpenWRT kernel: [497781.558730] CPU: 1 PID: 30328 Comm: kworker/1:1 Not tainted 4.14.195 #0
Oct 14 15:49:09 OpenWRT kernel: [497781.580960] Hardware name: Generic DT based system
Oct 14 15:49:09 OpenWRT kernel: [497781.587655] Workqueue: events 0xc0654ce0
Oct 14 15:49:09 OpenWRT kernel: [497781.592241] task: d8153700 task.stack: c8ab4000
Oct 14 15:49:09 OpenWRT kernel: [497781.596409] pc : []    lr : []    psr: a0000113
Oct 14 15:49:09 OpenWRT kernel: [497781.601098] sp : c8ab5e5c  ip : 00000002  fp : dd79f4e8
Oct 14 15:49:09 OpenWRT kernel: [497781.607433] r10: 66d16940  r9 : 16e36000  r8 : dd79f240
Oct 14 15:49:09 OpenWRT kernel: [497781.612729] r7 : ddc12010  r6 : 00000000  r5 : c050e0e4  r4 : c05ad3d0
Oct 14 15:49:09 OpenWRT kernel: [497781.618025] r3 : 00000002  r2 : 00000001  r1 : 00000000  r0 : 00000000
Oct 14 15:49:09 OpenWRT kernel: [497781.624363] Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
Oct 14 15:49:09 OpenWRT kernel: [497781.630961] Control: 10c5787d  Table: 5562406a  DAC: 00000051
Oct 14 15:49:09 OpenWRT kernel: [497781.638251] Process kworker/1:1 (pid: 30328, stack limit = 0xc8ab4210)
Oct 14 15:49:09 OpenWRT kernel: [497781.644067] Stack: (0xc8ab5e5c to 0xc8ab6000)
Oct 14 15:49:09 OpenWRT kernel: [497781.650578] 5e40:                                                                c05adda8
Oct 14 15:49:09 OpenWRT kernel: [497781.655104] 5e60: c0a61004 00000000 dcc65000 dd79f268 66d16940 16e36000 00009c40 dcc6d400
Oct 14 15:49:09 OpenWRT kernel: [497781.663354] 5e80: 66d16940 dcc65100 dcc6511c 00000005 dcc65140 00000000 ddc18540 c0655c2c
Oct 14 15:49:09 OpenWRT kernel: [497781.671599] 5ea0: 00000000 c065566c 00009c40 dcc6d400 00000000 c0a61004 00000000 00000005
Oct 14 15:49:09 OpenWRT kernel: [497781.679845] 5ec0: 001a5248 00000000 ddc18540 c0651d5c 00000002 0005dc00 001a5248 00000021
Oct 14 15:49:09 OpenWRT kernel: [497781.688092] 5ee0: dcc63380 dcc6d400 dcc5ecc0 dcc63200 dcc63200 dcc5ecc0 00000040 c0654118
Oct 14 15:49:09 OpenWRT kernel: [497781.696339] 5f00: dcc633b8 00000000 dcc63384 dcc6d400 c0a1f0bc 00000000 00000040 c0654d18
Oct 14 15:49:09 OpenWRT kernel: [497781.704585] 5f20: dcc633b8 c4c9db80 ddc18540 ddc1b400 00000000 c033703c ddc18558 ffffe000
Oct 14 15:49:09 OpenWRT kernel: [497781.712832] 5f40: c4c9db80 ddc18540 c4c9db98 ddc18558 ffffe000 c0a02d00 00000008 c0337538
Oct 14 15:49:09 OpenWRT kernel: [497781.721078] 5f60: cca7bebc c8177f40 c8ab4000 c75eaac0 cca7bebc c8177f5c c4c9db80 c03371f4
Oct 14 15:49:09 OpenWRT kernel: [497781.729324] 5f80: 00000000 c033d1d8 00000000 c75eaac0 c033d090 00000000 00000000 00000000
Oct 14 15:49:09 OpenWRT kernel: [497781.737572] 5fa0: 00000000 00000000 00000000 c0307c48 00000000 00000000 00000000 00000000
Oct 14 15:49:09 OpenWRT kernel: [497781.745817] 5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Oct 14 15:49:09 OpenWRT kernel: [497781.754063] 5fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
Oct 14 15:49:09 OpenWRT kernel: [497781.762305] Function entered at [] from []
Oct 14 15:49:09 OpenWRT kernel: [497781.770551] Code: 000ed1f2 00000000 00000000 00000000 (dd79f440)
Oct 14 15:49:09 OpenWRT kernel: [497781.776361] ---[ end trace db7165d341c15beb ]---
Oct 14 15:49:09 OpenWRT kernel: [497781.782606] Kernel panic - not syncing: Fatal exception
Oct 14 15:49:09 OpenWRT kernel: [497781.787388] CPU0: stopping
Oct 14 15:49:09 OpenWRT kernel: [497781.792764] CPU: 0 PID: 1058 Comm: netifd Tainted: G      D         4.14.195 #0
Oct 14 15:49:09 OpenWRT kernel: [497781.795283] Hardware name: Generic DT based system
Oct 14 15:49:09 OpenWRT kernel: [497781.802921] Function entered at [] from []
Oct 14 15:49:09 OpenWRT kernel: [497781.807521] Function entered at [] from []
Oct 14 15:49:09 OpenWRT kernel: [497781.813426] Function entered at [] from []
Oct 14 15:49:09 OpenWRT kernel: [497781.819326] Function entered at [] from []
Oct 14 15:49:09 OpenWRT kernel: [497781.825229] Function entered at [] from []
Oct 14 15:49:09 OpenWRT kernel: [497781.831132] Exception stack(0xdc2ebfb0 to 0xdc2ebff8)
Oct 14 15:49:09 OpenWRT kernel: [497781.837039] bfa0:                                     be8e45d4 00000078 00000058 b6ef0954
Oct 14 15:49:09 OpenWRT kernel: [497781.842257] bfc0: be8e4998 b6ef0953 be8e4a88 ffffffff b6ef0953 00000000 00000000 00000000
Oct 14 15:49:09 OpenWRT kernel: [497781.850502] bfe0: 00012889 be8e4580 b6ea3e28 b6ecfb58 20000010 ffffffff
Oct 14 15:49:09 OpenWRT kernel: [497781.858743] Rebooting in 3 seconds..

**Device freeze, no automatic recovery (device stopped responding after all this)**

Oct 15 16:01:06 OpenWRT kernel: [87095.946889] INFO: rcu_sched detected stalls on CPUs/tasks:
Oct 15 16:01:06 OpenWRT kernel: [87095.946925] #0111-...: (2 ticks this GP) idle=30e/140000000000001/0 softirq=1165976/1165976 fqs=1050
Oct 15 16:01:06 OpenWRT kernel: [87095.951267] #011(detected by 0, t=2102 jiffies, g=814833, c=814832, q=221)
Oct 15 16:01:06 OpenWRT kernel: [87095.960209] Sending NMI from CPU 0 to CPUs 1:
Oct 15 16:01:06 OpenWRT kernel: [87105.969246] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87105.969292] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 2, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87105.975373] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87105.982754] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87105.990031] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 2, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87105.997310] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87106.004540] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87106.011899] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 2, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87106.019182] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87106.026415] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87106.033768] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 2, skipped old beacon
Oct 15 16:01:06 OpenWRT kernel: [87106.041078] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
[...trimmed many repeating similar messages...]
Oct 15 16:01:07 OpenWRT kernel: [87107.294455] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Oct 15 16:01:07 OpenWRT kernel: [87107.301829] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 2, skipped old beacon
Oct 15 16:01:07 OpenWRT kernel: [87107.309114] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Oct 15 16:01:07 OpenWRT kernel: [87107.316334] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Oct 15 16:01:07 OpenWRT kernel: [87107.327832] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Oct 15 16:01:07 OpenWRT kernel: [87107.330995] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Oct 15 16:01:07 OpenWRT kernel: [87107.338440] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Oct 15 16:01:07 OpenWRT kernel: [87107.345502] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 2, skipped old beacon
Oct 15 16:01:07 OpenWRT kernel: [87107.354378] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon

----

If I should file a new issue instead, share more details, etc, just let me know!  I cannot yet reliably reproduce this issue, but it seems to happen every 2-14 days.
----------

More information can be found at the following URL:
https://bugs.openwrt.org/index.php?do=details&task_id=2053#comment8974

You are receiving this message because you have requested it from the Flyspray bugtracking system.  If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.