Commit graph

56 commits

Author SHA1 Message Date
Piotr F. Mieszkowski 56101b86c0 Clean up the code after refactor
- Clean up PGP/MIME flow by using API instead of explicit/manual generation of
  headers.
- Fix E2E test configuration for PGP/MIME case.
- Add first lacre.core unit tests.
- Add another Contract Test.
2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski d2ed4a9cee Make test reports more readable 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 94e22caf8e When encryption fails, revert to cleartext delivery
When GnuPG refuses to encrypt a message (e.g. when key has expired), record
information about the failure and send to logs, then deliver cleartext.  This
way we won't bounce email that could be delivered without encryption.

Also: add more E2E tests.
2023-05-08 22:16:41 +02:00
Piotr F. Mieszkowski ffffa6a364 Correct sample message file name 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b242edf098 Add a more sophisticated UTF-8 test case
Add a new test message to verify Lacre's behaviour when processing
UTF-8 messages with text in two different scripts (latin-based and cyrillic).

Also: log Content-Transfer-Encoding when logging headers is enabled.
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 4da4019321 Log message defects and optionally some non-PII headers 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b6bd36a460 Use bytes instead of str to hold message bodies
smtplib.SMTP expects ASCII-only message bodies when message body is provided
as a 'str'.  If we pass a 'bytes', we need to choose encoding earlier and we
do this by calling 'as_bytes' on messages with SMTP policy, which takes care
of formatting the body properly.

As a result, ISO-8859-x messages are converted to Quoted Printable and UTF-8
messages are Base64-encoded.

Testing this behaviour is tricky, because we use the same SMTP client to send
test data.  For this reason, test code has become a bit ugly, but it does
exactly what we need.
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 0fac54a29a Add UTF-8 and ISO-8859-2 test cases, make tests more reliable 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 037a527c44 Add a test case with non-ASCII message content 2023-05-08 22:14:24 +02:00
Wiktor Kwapisiewicz 46bb125684
Add missing test case for add_key and delete_key 2022-12-23 13:49:43 +01:00
Piotr F. Mieszkowski 588b447e69 Add basic unit tests for GnuPG module
- Cover basic GnuPG functionality with tests (confirm_key, public_keys).
- Add a test public key file.
- Fix resource leak by closing streams opened by Popen.
2022-12-14 21:09:46 +01:00
Piotr F. Mieszkowski 53378b516e Add a test for message with PGP inline markers that's not encrypted
If a user mentions PGP markers inside their message, we should not classify it
as already encrypted.
2022-10-22 21:29:59 +02:00
Piotr F. Mieszkowski b4f30d7e8f Add test clear text input message with PGP markers
It's possible to trick Lacre by sending PGP markers in message body, causing
it to classify that message being already encrypted.  This test case is used
to reproduce this scenario.
2022-10-22 20:54:08 +02:00
Piotr F. Mieszkowski b91501d3dd Convert all test input messages to CRLF line endings 2022-10-22 20:52:14 +02:00
Piotr F. Mieszkowski 540ca2adf3 Improve logging
- Report processing time in milliseconds.
- Use module names in log messages instead of file-names without extensions.
2022-10-20 21:56:01 +02:00
Piotr F. Mieszkowski 9696b7e997 Separate key-cache and key-loader
Extract key-loading code to a dedicated class KeyRing in lacre.keyring module.
KeyCache only keeps a static map of identities, making it safe to use in
asynchronous context (and race condition resistant).
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 9f3ad49f14 Rename lacre.keycache to lacre.keyring
This will better reflect the fact we're doing more than just caching.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski d7e4947afd Add cache validity configuration parameter
Also, log basic information in KeyCache and provide load() and reload()
operations to make daemon's code cleaner.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 5f601fa50c Implement a basic KeyCache 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 07263d5afa Reformat tests 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 8963eee47f Reformat GnuPG module 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski a5bcf2d9b2 Make daemon E2E tests use configured parameters 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 414f1d5921 Implement E2E tests for lacre.daemon
- Add a dedicated configuration file for lacre.daemon.

- Implement test/daemon_test.py like test/e2e_test.py, to automate the
following procedure:

    1. Start up lacre.daemon.
    2. For each test case, send test message to the daemon and verify that the
       output received by test/utils/relay.py contains expected pattern.

- Simplify Makefile.

- Fix indentation here and there.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski a131cd66d3 Move different parts of Lacre tests to subdirectories
In particular, move:

- test utilities to test/utils;
- unit tests to test/modules.

Also: start implementing the Lacre daemon test (just a stub for now).
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 4c844384e3 Implement a bare minimum of advanced filtering
- Forward messages without encryption.

- Include a simple test setup in the Makefile.

- Add a test to send a test message to the daemon.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 881a8d1756 Add GnuPG encryption support for addresses with delimiters
If a user registers their key for address alice@example.com but receives a
message sent to alice+something@example.com, this message should be encrypted
as well.

- Implement delimiter support for GnuPG encryption.

- Add E2E test case for a clear text message delivered to an address with
delimiter.

- Fix minor bug: wrong configuration parameter was retrieved when logging
information about enc_domain_keymap being active.
2022-06-08 21:20:58 +02:00
Piotr F. Mieszkowski c86c620668 Extract delimiter support, add unit tests
Also: fix recursive call to get_cert_for_email.
2022-06-07 22:14:32 +02:00
Piotr F. Mieszkowski 46be24670c Fix charset resolution in Content-Type parser 2022-06-01 23:44:41 +02:00
Piotr F. Mieszkowski 4c6fdc52ec Check mandatory config early, add tests
Also: extend failover logging configuration with file-based handler to make
sure that the user gets _some_ logs even if they do not configure Lacre at
all.
2022-05-31 22:09:10 +02:00
Piotr F. Mieszkowski 3bcc1151e5 Add E2E case: a user with a key and PGP/MIME configured
- Add a new test input message for a new test identity, test scenario
  configuration and a test key.

- While retrieving message payload, determine charset based on the
  Content-Type header.  When missing, default to UTF-8.

- Use more comprehensible variables names.

- Adjust logging levels.
2022-05-30 00:49:40 +02:00
Piotr F. Mieszkowski 707fc96234
Add more contract tests
- Verify that Message.get_payload() returns str, unless passed decode=True,
when it returns bytes.

- Verify that RawConfigParser returns str.
2022-05-25 22:13:40 +02:00
Piotr F. Mieszkowski 558872d9d0 Start documenting dependency contracts with unit tests
Implement some unit tests for 'email' package so we know precisely how this
package behaves.
2022-05-16 20:57:12 +02:00
Piotr F. Mieszkowski 8f0d8f4933 Add newlines to key material 2022-05-06 20:40:16 +02:00
Piotr F. Mieszkowski 75ccfb0850 Use logging module
- Replace custom logging code with calls to logging module.
- Use logging.config to provide configuration parameters.

To make Lacre's logging more flexible, use fileConfig from logging.config to
set up all parameters.  If the configuration file is missing, use dictConfig
with hardcoded reasonable defaults.
2022-05-06 19:39:56 +02:00
Piotr F. Mieszkowski d90b50f7e7 Extract config, separate logging, split into smaller functions
- Move configuration-processing code to a separate module (lacre.config) and
  provide a simple API to access configuration parameters.
- Prepare to use builtin logging module to log diagnostic data.
- Rework the configuration-processing file to make it cleaner.
- Log additional information while processing configuration.
- Reorder functions.
2022-05-06 19:39:56 +02:00
Piotr F. Mieszkowski a82ff9f3f1 Use actual keys (RSA and Ed25519) in cron-test 2022-04-24 10:42:51 +02:00
Piotr F. Mieszkowski ffc53b935a Finish migration to SQLAlchemy and automate testing cron.py 2022-04-23 13:08:40 +02:00
Piotr F. Mieszkowski 9e17726e39 Use f-strings for formatting 2022-02-06 00:27:20 +01:00
Piotr F. Mieszkowski a201265f87 Rework how E2E tests are executed
Tests kept breaking (not failing) randomly with "Broken pipe" errors.  Rework
how processes are spawned to make sure that it doesn't break them again.
2022-01-19 21:57:46 +01:00
Piotr F. Mieszkowski c81c6e6e0d Remove hardcoded python3.8 path
- Let the user overwrite Python binary name while calling make.

- Use environment variable set by make to instruct e2e_test.py which binary it
should call to execute Python code.
2022-01-10 20:22:17 +01:00
Piotr F. Mieszkowski 5a8d2c0108 Add E2E cases: signed cleartext and multipart/encrypted 2022-01-10 19:48:44 +01:00
Piotr F. Mieszkowski 435528de43 Add an E2E test case with an already encrypted message 2022-01-10 19:32:46 +01:00
Piotr F. Mieszkowski 24f0c86d4f Tidy up tests
- Makefile: add 'unittest' to .PHONY targets.
- Remove unnecessary #! line from e2e_test.py.
- Extract Python path to test/e2e.ini file.
2022-01-10 18:39:29 +01:00
Piotr F. Mieszkowski b2a01c15b0 Fix auto-migrated code
- Use b'' (byte strings) where appropriate.

- Fix indentation.

- Replace python2.x references with python3.x.
2022-01-10 18:21:40 +01:00
Piotr F. Mieszkowski 5f02223ec7 Perform automatic migration to Python 3.x
Use lib2to3 automatic migration tool provided by Python 2.x to convert
codebase to new idioms.

Command line:

find . -type f -name '*.py' \
    -exec python2.7 -m lib2to3 \
    -f all -f idioms -f buffer -f set_literal -f ws_comma -w \
    '{}' '+'
2022-01-10 18:21:20 +01:00
Piotr F. Mieszkowski 3b9f714cdb Ignore random_seed
Do not keep test/keyhome/random_seed in repository.
2022-01-10 17:47:23 +01:00
Piotr F. Mieszkowski 1002da78fa Clean up test messages
Remove unused msgout files, fix addresses in msgin files.
2022-01-09 22:00:06 +01:00
Piotr F. Mieszkowski 2cf60dec40 Add unit tests for GnuPG command-line generator
Extract a function to calculate GPG commands to be executed and cover it with
unit tests.
2022-01-09 21:37:14 +01:00
Piotr F. Mieszkowski fc2779ef7d Improve test code structure
- Move things to configuration file where appropriate (logging format, etc.).

- Rework execute_e2e_test signature to simplify it and get rid of keyword
arguments.

- Simplify output.

- Include a header comment in configuration file.
2022-01-08 13:42:23 +01:00
Piotr F. Mieszkowski 01377f4dd2 Keep test/certs directory 2022-01-07 12:03:38 +01:00