Enhance the UMS test to optionally mount a partition and read/write a file
to it, validating that the content written and read back are identical.
This enhancement is backwards-compatible; old boardenv contents that don't
define the new configuration data will cause the test code to perform as
before.
test/ums/ is deleted since the Python test now performs the same testing
that it did.
The code is also re-written to make use of the recently added utility
module, and split it up into nested functions so the overall logic of
the test process can be followed more easily without the details
cluttering the code.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Acked-by: Simon Glass <sjg@chromium.org>
Add various common utility functions. These will be used by a forthcoming
re-written UMS test, and a brand-new DFU test.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Sometimes it's useful to run shell commands and ignore any errors. One
example might be cleanup logic; if a test-case experiences an error, the
cleanup logic might experience an error too, and we don't want that error
to mask the original error, so we want to ignore the subsequent error.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Write a note to the log file when a test sends CTRL-C to U-Boot. This
makes it easier to follow what's happening in the logs, especially since
U-Boot doesn't echo the character back to its output, so there's no other
signal of what's going on.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tests may fail for a number of reasons, and in particular for reasons
other than a timeout waiting for U-Boot to print expected data. If the
last operation that a failed test performs is not waiting for U-Boot to
print something, then any trailing output from U-Boot during that test's
operation will not be logged as part of that test, but rather either
along with the next test, or even thrown away, potentiall hiding clues
re: the test failure reason.
Solve this by explicitly draining (and hence logging) the U-Boot output
in the case of failed tests.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Prior to this change, U-Boot was lazilly (re-)spawned if/when a test
attempted to interact with it, and no active connection existed. This
approach was simple, yet had the disadvantage that U-Boot might be
spawned in the middle of a test function, e.g. after the test had already
performed actions such as creating data files, etc. In that case, this
could cause the log to contain the sequence (1) some test logs, (2)
U-Boot's boot process, (3) the rest of that test's logs. This isn't
optimally readable. This issue will affect the upcoming DFU and enhanced
UMS tests.
This change converts u_boot_console to be a function-scoped fixture, so
that pytest attempts to re-create the object for each test invocation.
This allows the fixture factory function to ensure that U-Boot is spawned
prior to every test. In practice, the same object is returned each time
so there is essentially no additional overhead due to this change.
This allows us to remove:
- The explicit ensure_spawned() call from test_sleep, since the core now
ensures that the spawn happens before the test code is executed.
- The laxy calls to ensure_spawned() in the u_boot_console_*
implementations.
The one downside is that test_env's "state_ttest_env" fixture must be
converted to a function-scoped fixture too, since a module-scoped fixture
cannot use a function-scoped fixture. To avoid overhead, we use the same
trick of returning the same object each time.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Currently, Spawn.expect() imposes its timeout solely upon receipt of new
data, not on its overall operation. In theory, this could cause the
timeout not to fire if U-Boot continually generated output that did not
match the expected patterns.
Fix the code to additionally impose a timeout on overall operation, which
is the intended mode of operation.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
Acked-by: Simon Glass <sjg@chromium.org>
Execute "sleep", and validate that it sleeps for approximately the correct
amount of time.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
This test invokes the "ums" command in U-Boot, and validates that a USB
storage device is enumerated on the test host system, and can be read
from.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Migrate all most tests from command_ut.c into the Python test system.
This allows the tests to be run against any U-Boot binary that supports
the if command (i.e. where hush is enabled) without requiring that
binary to be permanently bloated with the code from command_ut.
Some tests in command_ut.c can only be executed from C code, since they
test internal (more unit-level) features of various U-Boot APIs. The
migrated tests can all operate directly from the U-Boot console.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
This tests whether the following features of the U-Boot shell:
- Execution of a directly entered command.
- Compound commands (; delimiter).
- Quoting of arguments containing spaces.
- Executing commands from environment variables.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
This tests whether md/mw work, and affect each-other.
Command repeat is also tested.
test/cmd_repeat.sh is removed, since the new Python-based test does
everything it used to.
Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
This tests basic environment variable functionality.
Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Test the sandbox port's implementation of the reset command and SIGHUP
handling. These should both cause the U-Boot process to exit gracefully.
Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach
to the U-Boot console, run the entire script of tests against it, and
summarize the results. Advantages of this approach are:
- Testing is performed in the same way a user or script would interact
with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
It is asserted that writing test-related code in Python is simpler and
more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.
A few simple tests are provided as examples. Soon, we should convert as
many as possible of the other tests in test/* and test/cmd_ut.c too.
The hook scripts, relay control utilities, and udev rules I use for my
own HW setup are published at https://github.com/swarren/uboot-test-hooks.
See README.md for more details!
Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Michal Simek <michal.simek@xilinx.com>
Tested-by: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org> #v3
Stephen Warren
Stephen Warren