Add a few notes about how testing works in U-Boot. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Teddy Reed <teddy.reed@gmail.com>master
parent
022885cb9c
commit
f6349c3c4c
@ -0,0 +1,82 @@ |
||||
Testing in U-Boot |
||||
================= |
||||
|
||||
U-Boot has a large amount of code. This file describes how this code is |
||||
tested and what tests you should write when adding a new feature. |
||||
|
||||
|
||||
Sandbox |
||||
------- |
||||
U-Boot can be built as a user-space application (e.g. for Linux). This |
||||
allows test to be executed without needing target hardware. The 'sandbox' |
||||
target provides this feature and it is widely used in tests. |
||||
|
||||
|
||||
Pytest Suite |
||||
------------ |
||||
|
||||
Many tests are available using the pytest suite, in test/py. This can run |
||||
either on sandbox or on real hardware. It relies on the U-Boot console to |
||||
inject test commands and check the result. It is slower to run than C code, |
||||
but provides the ability to unify lots of test and summarise their results. |
||||
|
||||
You can run the tests on sandbox with: |
||||
|
||||
./test/py/test.py --bd sandbox --build |
||||
|
||||
This will produce HTML output in build-sandbox/test-log.html |
||||
|
||||
See test/py/README.md for more information about the pytest suite. |
||||
|
||||
|
||||
tbot |
||||
---- |
||||
|
||||
Tbot provides a way to execute tests on target hardware. It is intended for |
||||
trying out both U-Boot and Linux (and potentially other software) on a |
||||
number of boards automatically. It can be used to create a continuous test |
||||
environment. See tools/tbot/README for more information. |
||||
|
||||
|
||||
Ad-hoc tests |
||||
------------ |
||||
|
||||
There are several ad-hoc tests which run outside the pytest environment: |
||||
|
||||
test/fs - File system test (shell script) |
||||
test/image - FIT and lagacy image tests (shell script and Python) |
||||
test/stdint - A test that stdint.h can be used in U-Boot (shell script) |
||||
trace - Test for the tracing feature (shell script) |
||||
vboot - Test for verified boot (shell script) |
||||
|
||||
The above should be converted to run as part of the pytest suite. |
||||
|
||||
|
||||
When to write tests |
||||
------------------- |
||||
|
||||
If you add code to U-Boot without a test you are taking a risk. Even if you |
||||
perform thorough manual testing at the time of submission, it may break when |
||||
future changes are made to U-Boot. It may even break when applied to mainline, |
||||
if other changes interact with it. A good mindset is that untested code |
||||
probably doesn't work and should be deleted. |
||||
|
||||
You can assume that the Pytest suite will be run before patches are accepted |
||||
to mainline, so this provides protection against future breakage. |
||||
|
||||
On the other hand there is quite a bit of code that is not covered with tests, |
||||
or is covered sparingly. So here are some suggestions: |
||||
|
||||
- If you are adding a new uclass, add a sandbox driver and a test that uses it |
||||
- If you are modifying code covered by an existing test, add a new test case |
||||
to cover your changes |
||||
- If the code you are modifying has not tests, consider writing one. Even a |
||||
very basic test is useful, and may be picked up and enhanced by others. It |
||||
is much easier to add onto a test - writing a new large test can seem |
||||
daunting to most contributors. |
||||
|
||||
|
||||
Future work |
||||
----------- |
||||
|
||||
Converting existing shell scripts into pytest tests. |
Loading…
Reference in new issue