## deqp-runner

This is a tool for parallelizing a VK-GL-CTS or dEQP run across the
CPUs in a system, collecting the results, and comparing them to the
baseline for the device.  It is geared toward driver developers and CI
systems looking for fast turnaround of dEQP results (it is not a valid
way to produce a VK/GL conformance result).

### Installing

```
apt-get install cargo
cargo install deqp-runner
```

### Running

Example run:

```
deqp-runner run \
      --deqp ~/src/VK-GL-CTS-build/modules/gles2/deqp-gles2 \
      --caselist ~/src/VK-GL-CTS/external/openglcts/data/mustpass/gles/aosp_mustpass/3.2.6.x/gles2-master.txt \
      --output new-run  \
      --baseline last-run/failures.csv \
      -- \
      --deqp-surface-width=256 --deqp-surface-height=256 \
      --deqp-surface-type=pbuffer \
      --deqp-gl-config-name=rgba8888d24s8ms0 \
      --deqp-visibility=hidden
```

You will get summary output as tests are run and results collected,
with failing tests logged to stderr with references of where to find
the associated caselists (and the full QPA file next to them).  The
failing tests will also be extracted to testname.qpa which you can run
dEQP's testlog-to-xml on for visualizing the failure.

Unexpectedly failing tests will be automatically rerun to detect flaky
tests, which will be treated as success of the pipeline (this is
useful in CI, to keep rare and uncategorized flakes from impacting
other developers).  Additionally, you can pass a --flakes argument to
a list of regexes of known flaky tests, to reduce the chance of
spurious failures in a CI pipeline.

### Piglit support

deqp-runner now contains experimental support for running piglit tests.  An
example run might look like:

```
piglit-runner run \
      --output new-run \
      --piglit-folder ~/src/piglit \
      --profile quick \
      --baseline last-run/failures.csv \
      --skips piglit-soft-skips.txt \
      --process-isolation
```

It has some known issues:

- No piglit subtest result tracking (includes shader_tests in from no_isolation)
- No fast skipping based on extensions enabled

### Cross building for your embedded device

Add the following to ~/.cargo/config:

```
[target.armv7-unknown-linux-gnueabihf]
linker = "arm-linux-gnueabihf-gcc"

[target.aarch64-unknown-linux-gnu]
linker = "aarch64-linux-gnu-gcc"
```

And set up the new toolchain and build:

```
rustup target add aarch64-unknown-linux-gnu
cargo build --release --target aarch64-unknown-linux-gnu deqp-runner
scp target/aarch64-unknown-linux-gnu/release/deqp-runner device:bin/
```

### License

Licensed under the MIT license
   ([LICENSE](LICENSE) or http://opensource.org/licenses/MIT)
