[OpenWrt-Devel] [RFC] lantiq: SMP interrupts and ethernet driver backport from vanilla v5

[Hauke Mehrtens]

Hi Petr,

Thank you for the tests.

On 2/3/19 9:44 AM, Petr Cvek wrote:
> I did an exhaustive ethernet measuring. Parameters: no wifi, no running
> dsl connection, 1 cable (in 1G port with external phy), no USB, almost
> vanilla openwrt (changes for rootfs on NFS, kernel command line and fix
> for uwc2 USB).
> 
> run.sh
> A script which runs and logs measurements into a file named by its first
> parameter (or just the unix epoch time). You need to run "iperf3 -s" on
> lantiq manually.

I am surprised that "iperf3 -c 10.0.0.80" has the double speed of
"iperf3 -c 10.0.0.80 -R" isn't this the same stream just in the
different direction?

> *_a and *_b files
> Main iperf3 measurements. It seems the schedulers migrates processes
> between VPEs (in every configuration). There are sometimes differences
> of ~20Mbit/s. To isolate that one would probably have to compile and use
> cgroups.
> 
> *_netcat
> Running netcat pipe with pipeview in server/client and TCP/UDP mode.
> More useful to measure the changes of the process affinity.
> 
> vanilla*
> Default (slowest) openwrt code.
> 
> backported*
> Ethernet driver with full napi polling, used from vanilla kernel v5.
> Newest patch in the attachment too.
> 
> dma_burst*
> The change in size of DMA burst, from default to (probably) 4 words.
> 
> smp_icu_different_vpes*
> ICU with SMP interrupts support. Forcing the IRQ SMP affinity to
> different VPE for TX and RX interrupt. Process jumping between VPE
> should have the smallest impact here.
> 
> smp_icu_same_vpe*
> Both TX and RX interrupts are on the same VPE (forced on CPU0). Probably
> the biggest impact from process jumping between VPEs (not checked).
> 
> everything*
> SMP ICU, DMA burst and "backported" ethernet driver together.
> 
> 
> Notes:
> 1) The backported driver seems to not working on some machines (I would
> assume everything different from 4x 1G ports). You can still patch the
> polling and interrupt routines to match the vanilla kernel. A help why
> it doesn't boot would be appreciated (are your machine using multiple
> eth%d interfaces, how does dmesg look like, ...).
> 
> 2) The vanilla openwrt driver declares TX2 interrupt and HW. There is no
> traffic on this interrupt (on DMA channel probably too?). Does anybody
> have some documentation how this work? Are there more DMA interfaces
> from/to which send the data?

In the SoC there are 2 internal GPHYs and some *MII connections to
external PHYs. They are all connected to the GSWIP inside of the SoC, on
the CPU port normally a special tag is added to every packet which
indicates from which LAN port it was received on. This CPU port is
connected to the PPE and the PPE is connected to two DMA channels which
send the packets to the MIPS CPU running Linux. There are two ports
between the PPA and the CPU, the upstream driver only uses one of these
ports as we let the PPE stay in the default configuration. We could
later use it to put high prio packets on a special channel or to put the
DSL packets onto the other channel.

> 3) The second ICU devicetree node doesn't need to be deleted for
> uniprocessor ICU usage.

When the Voice firmware is used the second VPE is used for this FW and
not for Linux then the ICU should only use one block.

> Feel free to make your own tests and sorry for hardcoded IPs :-D

I would like to use the upstream driver also in OpenWrt, I am working on
bridge offloading for the upstream DSA driver.  OpenWrt still misses a
script which can convert the common swconfig configuration into a bridge
configuration if such a script would be there I would like to switch to
the DSA driver with kernel 4.19.
Therefore I am not sure how efficient it is to invest so much time into
the driver currently used in OpenWrt.

> 
> Petr
> 
> Dne 30. 01. 19 v 11:38 Petr Cvek napsal(a):
>> Hello,
>>
>> I discovered the lantiq xrx200 lacks support for interrupts on secondary
>> VPE, and I've managed to add this functionality to the kernel, the
>> second icu controller lives on base address 0x1f880300 and works in the
>> same way as first. Tested with 4.14.93 kernel, untested patches in the
>> attachment or on openwrt forum:
>>
>> https://forum.openwrt.org/t/xrx200-irq-balancing-between-vpes/29732/11
>>
>> My second submission is a backported xrx200 ethernet driver from vanilla
>> kernel v5 patched with switch and phy functions. Using kernel napi
>> polling seems to highly increase the throughput of the network.
>>
>> Trying to initially increase the throughput of the network I've found
>> there is an inefficiently setting of DMA burst size. My patch sets this
>> value to a higher value.
>>
>> These two patches were tested with 4.14.93 kernel too, there is forum
>> thread:
>>
>> https://forum.openwrt.org/t/how-can-we-make-the-lantiq-xrx200-devices-faster/9724/30
>>
>> RFC for any parallel development info (next version of ethernet driver
>> will be based on vanilla kernel version?), ideas, testing ...
>>
>> best regards,
>> Petr Cvek
>>
>>
>> _______________________________________________
>> openwrt-devel mailing list
>> openwrt-devel at lists.openwrt.org
>> https://lists.openwrt.org/mailman/listinfo/openwrt-devel


_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel