Commit Graph

412 Commits

Author SHA1 Message Date
Piotr F. Mieszkowski 0cdbf5ba7d Add emoji test 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski c08d66ac80 Migrate daemon E2E tests to unittest framework 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 518b823b5c Fix simple filter: pass policy to as_bytes()
Also: adjust expected test output because it's now Base64-encoded.
2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 459779bea6 Add more test cases 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 61cf50effe Fix MIME content sub-type handling for non-plain text messages 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski bc92d7a31c [e2e_test] Identify test cases by iteracting config sections 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski da0ffb4a51 Add a test message with Emoji 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 34e8b6a4eb Move the last key-related function to keyring module 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 285f5dbf18 Don't overwrite CTE
We rely on Content Manager to select the right Content-Transfer-Encoding.
2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 1acb330c02 Rewrite e2e_test to use unittest framework 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski fdd11dba14 Log more information about FS events while reloading keys 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 682de14630 Split the code into smaller modules
Introduce modules:
- lacre.transport - for actual delivery via SMTP
- lacre.smime - to take care of S/MIME stuff

Implement lacre.transport.SendFrom class that does a almost exactly the same
thing as the original send_msg function, but without using global variable to
store original message sender.
2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski ff6e0bfbdd Move recipient-processing code to a dedicated module 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 5f5b374f84 Unify send_msg, add more type hints 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 67e6df17fb Move success flag before 'try' 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 0da169ae61 Log exceptions from failover delivery 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski f4e21217c2 [simple-filter] Retry delivery recoding text parts 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 1f1fe1dadb Add lots of log messages 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 8a42f3fea1 Improve error-handling for simple filter and test relay 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 8def4b40dc Correct sample message in contract tests 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski e595e8baf4 Fix contract tests 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski a5f79c1ae7 Wrap recipient lists
Instead of passing pairs of lists (emails and keys) separately, implement a
class RecipientList to wrap such pair of lists.
2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski c5e788b2a0 Add more contract tests for email module 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 12b7c3394d Make test relay slightly cleaner 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski b2bd6a9926 Encapsulate recipient lists
Implement RecipientList class with:
- recipient (email) list,
- key (identity) list.

Cover with basic unit test.
2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 86a3e0031b Adjust gpg-mailgate.py script to use SMTPUTF8 policy 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 54ee9d9875 Add minor E2E test improvements 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski 603a88489e Polish the code 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski ffd5f08ad9 Make PGP message recognition more thorough 2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski d342f206de Handle messages as EmailMessage
In the daemon, specify policy as SMTPUTF8.  That sets the deafult message type
to EmailMessage.

EmailMessage class is richer, including support for Content Managers, giving
it the capability to properly handle textual data and its encodings.

Also: add another contract test.
2023-05-08 22:17:02 +02:00
Piotr F. Mieszkowski ace2ce6b06 Add more email-parsing contract tests 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 509aac6de3 Improve command-generation, logging and readability 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski ea8b246538 Clean up PGP/MIME flow
- Use MIMEPart instead of Message when encrypting in PGP/MIME mode.

- Wrap text/plain messages in MIMEPart, instead of manipulating payloads
  manually.

- Add a test for wrapping.
2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 765637fd3a Fix logging initialisation by fixing import order 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 1a3ce89ce5 Always encrypt decoded payload 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 3c8b792203 Fix line-discarding bug 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 56101b86c0 Clean up the code after refactor
- Clean up PGP/MIME flow by using API instead of explicit/manual generation of
  headers.
- Fix E2E test configuration for PGP/MIME case.
- Add first lacre.core unit tests.
- Add another Contract Test.
2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 27b07e672d Rework PGP/MIME flow 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 5e408259c0 Start using Content Manager
Also:
- Pass text to Popen in GnuPG (used to be bytes).
- Make is_payload_pgp_inline type-agnostic (str / bytes).
2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski d2ed4a9cee Make test reports more readable 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 94e22caf8e When encryption fails, revert to cleartext delivery
When GnuPG refuses to encrypt a message (e.g. when key has expired), record
information about the failure and send to logs, then deliver cleartext.  This
way we won't bounce email that could be delivered without encryption.

Also: add more E2E tests.
2023-05-08 22:16:41 +02:00
Piotr F. Mieszkowski 1cdca1d06d Log up to 2,5kB of message headers 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 3c1544e423 [daemon] Parse Envelope.original_content 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski ffffa6a364 Correct sample message file name 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b242edf098 Add a more sophisticated UTF-8 test case
Add a new test message to verify Lacre's behaviour when processing
UTF-8 messages with text in two different scripts (latin-based and cyrillic).

Also: log Content-Transfer-Encoding when logging headers is enabled.
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b94123e83e Use SMTPUTF8 policy, add more debug logging 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 4da4019321 Log message defects and optionally some non-PII headers 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b6bd36a460 Use bytes instead of str to hold message bodies
smtplib.SMTP expects ASCII-only message bodies when message body is provided
as a 'str'.  If we pass a 'bytes', we need to choose encoding earlier and we
do this by calling 'as_bytes' on messages with SMTP policy, which takes care
of formatting the body properly.

As a result, ISO-8859-x messages are converted to Quoted Printable and UTF-8
messages are Base64-encoded.

Testing this behaviour is tricky, because we use the same SMTP client to send
test data.  For this reason, test code has become a bit ugly, but it does
exactly what we need.
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 0fac54a29a Add UTF-8 and ISO-8859-2 test cases, make tests more reliable 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 037a527c44 Add a test case with non-ASCII message content 2023-05-08 22:14:24 +02:00