gpg-lacre/doc/testing.md

52 lines
2.1 KiB
Markdown
Raw Normal View History

2022-01-07 12:10:30 +01:00
# Testing
First tests have been set up to cover GPG Mailgate with at least basic test
that would be easy to run. The tests are called "end-to-end", meaning that we
2022-01-07 12:10:30 +01:00
feed some input to GPG Mailgate and inspect the output.
## Running tests
2022-04-23 13:13:57 +02:00
To run tests, use command `make test`.
2022-01-07 12:10:30 +01:00
There are 4 types of tests:
2022-04-23 13:13:57 +02:00
* `make e2etest` -- they cover a complete Lacre flow, from feeding it with
2022-04-23 13:13:57 +02:00
an email to accepting its encrypted form;
* `make daemontest` -- similar to the original `e2etest` goal, but tests the
behaviour of the Lacre Daemon, i.e. key part of the Advanced Content
Filter.
2022-04-23 13:13:57 +02:00
* `make unittest` -- just small tests of small units of code;
* `make crontest` -- execute cron job with a SQLite database.
2022-04-23 13:13:57 +02:00
E2E tests (`make e2etest`) should produce some helpful logs, so inspect
contents of `test/logs` directory if something goes wrong.
2022-01-07 12:10:30 +01:00
If your system's Python binary isn't found in your `$PATH` or you want to use
a specific binary, use make's macro overriding: `make test
PYTHON=/path/to/python`.
2022-01-07 12:10:30 +01:00
## Key building blocks
- *Test Script* (`test/e2e_test.py`) and *Daemon Test Script*
(`test/daemon_test.py`) that orchestrate the other components. They perform
test cases described in the *Test Configuration*. They spawn *Test Mail
Relay* and *GPG Mailgate* in appropriate order.
- *Test Mail Relay* (`test/relay.py`), a simplistic mail daemon that only
supports the happy path. It accepts a mail message and prints it to
2022-01-07 12:10:30 +01:00
stdandard output.
- *Test Configuration* (`test/e2e.ini`) specifies test cases: their input,
expected results and helpful documentation. It also specifies the port that
2022-01-07 12:10:30 +01:00
the *Test Mail Relay* should listen on.
## Limitations
Currently tests only check if the message has been encrypted, without
verifying that the correct key has been used. That's because we don't know
(yet) how to have a reproducible encrypted message. Option
2022-01-07 12:10:30 +01:00
`--faked-system-time` wasn't enough to produce identical output.
## Troubleshooting
When things go wrong, be sure to study `test/logs/e2e.log` and
`test/logs/gpg-mailgate.log` files -- they contain some useful information.