[OpenWrt-Devel] [linux-mips] [openwrt] [sc16is7xx] [bitbang I2C] Any suggestions on board reboot caused by "sched: RT throttling activated"?
xbing6 at gmail.com
Sun Oct 22 20:18:20 EDT 2017
Any suggestions, thanks.
On 10/19/2017 01:40 PM, Xuebing Wang wrote:
> Hi community,
> Any suggestions on how should I proceed with my issue?
> Description of my issue:
> - My board kernel prints: "sched: RT throttling activated", then
> reboot (there is no back-trace in UART console).
> - I have sc16is752 (I2C based UART) on my board.
> - As Atheros AR9331 does not have dedicated I2C controller, I am
> using bitbang I2C.
> - This is likely related to below line of sc16is752 driver
> ret = devm_request_threaded_irq(dev, irq, NULL, sc16is7xx_ist,
> - './ps -eo pid,nice,policy,cmd' confirms that above is a realtime
> kernel thread (with SCHED_FIFO scheduling policy)
> - Occurrence rate is about once or several times a day, after I
> stress I2C/UART traffic.
> - Occurrence rate is much lower if I do *NOT* stress I2C/UART traffic.
> My setup:
> - Atheros AR9331 chip, which is MIPS32 24kc
> - OpenWRT tag 15.05
> - Linux kernel v3.18.29
> - Not preempt kernel (CONFIG_PREEMPT_NONE=y)
> - As AR9331 does not have dedicated I2C controller, I am using
> bitbang I2C as below:
> - source code of sc16is752 driver sc16is7xx.c is here:
> Question 1): There seems a fundamental question about OpenWRT mips
> linux kernel?
> - OpenWRT tag 15.05
> - In OpenWRT file target/linux/ar71xx/Makefile, there is CPU_TYPE=34kc
> - Atheros AR9331 datasheet says it is 24kc (rather than 34kc)
> - With CPU_TYPE=34kc, it uses gcc options "-mips32r2 -mtune=34kc"
> Question 2): I have no idea what causes the reboot? It is a chip
> hardware reboot, or function similar to emergency_restart() is called
> somewhere in the kernel? Kernel console only shows "sched: RT
> throttling activated" and reboot, there is no back-trace.
> If I enable preempt kernel (CONFIG_PREEMPT=y), "RT throttling
> activated" still happens, and there is no reboot.
> - After "RT throttling activated" happens, I use "top -d 1" to check
> CPU usage, I observed at least once sc16is752 threaded-irq handler
> kernel thread occupies 95% of CPU for some seconds (not sure how many
> seconds), then sc16is752 threaded-irq handler CPU usage drops to
> normal, and the whole OpenWRT/Linux system is normal.
> - I did observe once that sc16is752 threaded-irq handler CPU usage
> stays at 95% forever, it seems like a dead-lock inside sc16is752
> threaded-irq handler.
> - It seems a dead-lock similar (in some way?) to below patch:
> Question 3) Is there any explanation why board does not reboot when I
> enable preempt kernel (CONFIG_PREEMPT=y)?
> Question 4) I am not sure if this is related to MIPS32r2 34kc (with
> 9-stage pipeline multi-threaded processor core), while Atheros AR9331
> is 24kc (with 8-stage pipeline processor core)?
> I have a temporary work-around:
> - In bitbang I2C driver, use hrtimer-backed usleep_range() (rather
> than busywait udelay()), then this issue is gone (verified by stress
> - As I do not know how long it takes to arm hrtimer on AR9331
> platform, I use usleep_range(10us, 20us), this greatly reduces bitbang
> I2C throughput, and reduces sc16is752 (I2C based UART) baudrate.
> Xuebing Wang
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
More information about the openwrt-devel