Run oeqa Tests in QEMU¶
This guide covers running the ToloMEO runtime test suite (tlm_systemd, tlm_platform, tlm_cloud) and the on-target
ptests against a QEMU image locally, under TCG (no /dev/kvm required).
Prerequisites¶
- The devcontainer (provides
kasand the disk toolssfdisk,mkfs.ext4,mksquashfs). - Device credentials at
certs/agent.env, consumed by thetlm_cloudchecks. - Network access from the container, needed only by the
tlm_cloudreachability probe.
Build the Test Image¶
Build image-devel. The image itself carries ptest-runner and tolomeo-qemu-netcfg; the kas-ci/testimage.yml
fragment is needed only when running the tests, where it enables the testimage task and sets TEST_SUITES:
The boot artifacts land in build/tmp-glibc/deploy/images/tolomeo-qemux86-64/. For a from-scratch build (empty local
sstate, remote mirror only), run make clean-sstate first.
The checkout also applies the kas/patches/poky/ patches, which backport resulttool junit from oe-core to scarthgap.
Provision the Second QEMU Disk¶
The image boots with a second virtio disk (GPT A/B rootfs slots, data, and a certs partition) plus a shared folder.
Provision them at the build TMPDIR, flashing certs/agent.env onto the certs partition:
This creates build/tmp-glibc/qemu-disk.img and build/tmp-glibc/shared/.
Run the Tests¶
Run the tests with either of two equivalent approaches.
Option A — In-Build Task¶
Run the testimage task; it boots QEMU and runs the fragment's TEST_SUITES:
kas shell kas/tolomeo-qemux86-64_tolomeo-devel_image-devel.yml:kas-ci/testimage.yml \
-c 'bitbake image-devel -c testimage'
Option B — Standalone Runner¶
Boot the already-built image with runqemu and run oe-test against it — no BitBake. Pass the deploy directory; the
suite list is optional and defaults to the full set:
meta-tolomeo-qemu/scripts/run-oe-test.sh \
build/tmp-glibc/deploy/images/tolomeo-qemux86-64 \
ping ssh ptest tlm_systemd tlm_platform tlm_cloud
A successful run ends with:
runtime - OK - All required tests passed (successes=..., skipped=0, failures=0, errors=0)
[run-oe-test] oe-test exit: 0
Know What the Suites Cover¶
| Suite | Checks |
|---|---|
ping, ssh |
Reachability and SSH into the guest |
ptest |
On-target shell ptests (ota-monitor, system-update-postboot, swupdate, hwrevision) |
tlm_systemd |
Per-unit enable-state and ordering (including start after connhex-edge-agent); nothing failed |
tlm_platform |
Platform mounts and overlays (/app, /data, /certs, /shared, overlayfs-etc) |
tlm_cloud |
network-online.target reached, connhex-edge-agent active, /certs/agent.env present, egress |
Read the Reports¶
Both approaches write a machine-readable testresults.json (same oeqa schema) plus detailed logs — no extra flags
needed. Archive the testresults.json, and the log directory for detail.
Option A — In-Build Task¶
| Artifact | Path |
|---|---|
| JSON report | build/tmp-glibc/log/oeqa/testresults.json (per-image copy under image-devel/) |
| Detailed logs (SSH command log, QEMU boot log) | build/tmp-glibc/work/tolomeo_qemux86_64-tolomeo-linux/image-devel/*/testimage/ |
Option B — Standalone Runner¶
| Artifact | Path |
|---|---|
| JSON report | runtime-results/testresults.json, relative to the directory you ran the script from |
| Console / detail log | runtime-results.log and timestamped runtime-results-<YYYYMMDDhhmmss>.log, same place |
Option B accumulates runs
runtime-results/testresults.json keeps every run — each adds a top-level key
runtime_image-devel_tolomeo-qemux86-64_<timestamp>. Delete runtime-results/ first for a single-run report.
Both files share the oeqa schema, so resulttool reads either — for a summary, or to diff two runs:
sources/poky/scripts/resulttool report build/tmp-glibc/log/oeqa/ # Option A
sources/poky/scripts/resulttool report runtime-results/ # Option B
sources/poky/scripts/resulttool regression <baseline-dir> <new-dir> # diff two runs
JUnit Report¶
resulttool junit converts a testresults.json to JUnit XML, the format CI publishes as the test report. It writes
junit.xml next to the JSON unless -j names another path; -a also writes the ptest logs to a test-logs/
subfolder and references them from the XML:
sources/poky/scripts/resulttool junit build/tmp-glibc/log/oeqa/testresults.json # Option A
sources/poky/scripts/resulttool junit runtime-results/testresults.json # Option B
sources/poky/scripts/resulttool junit build/tmp-glibc/log/oeqa/testresults.json -j junit.xml -a
The command prints a summary line of the form total -> tests: …, success: …, error: …, failures: …, skipped: ….
Troubleshooting¶
The boot times out before the login prompt¶
TCG boots are slow. Raise BOOT_TIMEOUT for run-oe-test.sh, or TEST_QEMUBOOT_TIMEOUT for the in-build task.
The cloud checks skip or the credential check fails¶
certs/agent.env is missing or empty. The reachability probe is opt-in and non-fatal (it skips without egress), but
tlm_cloud still checks that /certs/agent.env is present — re-run the provisioning step with a valid file.
resulttool has no junit subcommand¶
The sources/poky checkout predates the kas/patches/poky/ backport, or kas skipped the patches because the checkout
has local changes. Stash or discard the changes in sources/poky, then check out again:
kas checkout kas/tolomeo-qemux86-64_tolomeo-devel_image-devel.yml
test -f sources/poky/scripts/lib/resulttool/junit.py && echo "resulttool junit available"
runqemu cannot find a QEMU binary¶
The build tree is incomplete, so the native QEMU under recipe-sysroot-native is absent. Rebuild the image (first
step) so the native tools exist.
See also
- Test Updates in QEMU — trigger OTA and USB updates against a QEMU instance
- Run in Standalone Mode — run a ToloMEO image in QEMU without the devcontainer