[OpenWrt-Devel] automated testing brainstorming

Daniel Golle daniel.golle at gmail.com
Tue Jan 4 09:28:54 CET 2011


i'm day-dreaming about an automated testing environment for OpenWrt
for some days. automated testing with JUnit and ruby-unit has made my
life much better and the time-loss when developing on OpenWrt without
decent automated testing can be very frustrating.
sure, everybody can just setup a bunch of scripts to test-through
whatever, but sharing the test-cases and would be great.
therefore we need a common language to specify test-cases. i want to
share the ideas i had so far. i'd be glad to hear your opinion and
anything your feel being related to this idea.

why?
- increase quality
- have better ways to find, reproduce and communicate information about bugs
- keep us busy with developing, less time spending doing repetitive
things and waiting for results

how?
- have a test-host connected to one or more OpenWrt-based test-clients
(via serial and eventually using a reset-switch)
- let the host control and record what happens on the clients
- maybe each openwrt package can come with test-cases?
- maybe we can have a test-spec for each hardware target?
- possibility to specify multi-device-test cases, like have one be an
wifi ap and the other be wifi client
- have quantitative metrics and reports for some tests (like wifi
data-throughput)
- in case of a non-satisfying test-result, a bug-report can be
generate which, after approval, would be pushed into the bug-tracker

test-cases could be organized in an object-oriented fashion, i'm
thinking of an abstract testing-framework running on the test-host
which processes YAML or XML files holding the information of the tests
to be executed. i'd implement that in Ruby using ruby-unit.

dreams...
- have a hardware test-farm
- have a web-interface like tinderbox.mozilla.org

please share whatever comes up in you!


More information about the openwrt-devel mailing list