When we fail to encrypt a message, we log a message with a complete traceback.
The goal is to ensure that when diagnosing it later, we have complete
information.
We want to avoid deserialising message contents, because Python's email module
might produce different representation than the MUA sending original message.
The result would be a transformed message, which could mean broken message in
certain conditions.
- Introduce exceptions to be raised upon transient and permanent delivery
failures, as specified by SMTP RFC. Depending on type of failure, return
either 451 or 554 reply code.
- When serialising a message, treat ValueError as a serialisation issue (and
try again to deliver in cleartext).
1. Log the full traceback only once for each error (when we bounce the
message).
2. Use 451 response code on processing failure.
3. Disable decoding message contents as we operate on raw data anyway.
When we know we need to bounce a message and [daemon]log_headers is enabled,
we record up to 2.5kB of message headers at ERROR level. This could help
diagnosing issues later.
Also: no longer record MIME Type, Charset and Content-Transfer-Encoding, as
the issues related to these properties no longer occur.
When we fail to produce byte representation of the email message being
processed, we may end up bouncing a message. An example of such case would be
a message with a Message-Id header that Python's email parser library cannot
process.
In such cases, just take whatever original content we have received and pass
it to the destination without touching it to minimise any chances of breaking
the overall flow.
- Introduce '[daemon]bounce_on_keys_missing' option to let the admin decide if
they want Lacre to deliver cleartext message when identity database is
unreachable or throws exceptions. It defaults to 'no'.
- In IdentityRepository, use option mentioned above to decide what to do when
an exception is caught.
Introduce modules:
- lacre.transport - for actual delivery via SMTP
- lacre.smime - to take care of S/MIME stuff
Implement lacre.transport.SendFrom class that does a almost exactly the same
thing as the original send_msg function, but without using global variable to
store original message sender.
In the daemon, specify policy as SMTPUTF8. That sets the deafult message type
to EmailMessage.
EmailMessage class is richer, including support for Content Managers, giving
it the capability to properly handle textual data and its encodings.
Also: add another contract test.
When GnuPG refuses to encrypt a message (e.g. when key has expired), record
information about the failure and send to logs, then deliver cleartext. This
way we won't bounce email that could be delivered without encryption.
Also: add more E2E tests.
Add a new test message to verify Lacre's behaviour when processing
UTF-8 messages with text in two different scripts (latin-based and cyrillic).
Also: log Content-Transfer-Encoding when logging headers is enabled.
smtplib.SMTP expects ASCII-only message bodies when message body is provided
as a 'str'. If we pass a 'bytes', we need to choose encoding earlier and we
do this by calling 'as_bytes' on messages with SMTP policy, which takes care
of formatting the body properly.
As a result, ISO-8859-x messages are converted to Quoted Printable and UTF-8
messages are Base64-encoded.
Testing this behaviour is tricky, because we use the same SMTP client to send
test data. For this reason, test code has become a bit ugly, but it does
exactly what we need.
Expose a new parameter: [daemon]max_data_bytes, to limit Lacre's memory
usage and allow processing of messages larger than 32MB (which is the
default limit).
Function asyncio.run creates a new event loop each time it's called and
executes coroutine in that new loop. However, we want all our coroutines to
be executed from the same event loop, so we acquire a loop when lacre.daemon
starts and then use it to execute them later.
See: Disroot/gpg-lacre#109
Extract key-loading code to a dedicated class KeyRing in lacre.keyring module.
KeyCache only keeps a static map of identities, making it safe to use in
asynchronous context (and race condition resistant).
Subscribe to FS events from keyring directory using Python Watchdog and when a
modification is observed, reload the key cache.
Since we may receive more than one event about a single modification, keep
directory's last modification to recognise 'false positives'.
Use [default]cache_refresh_minutes configuration parameter to define periods
between cache reloads. After this number of minutes cache will be reloaded.
- Polish implementation of mail operations (lacre/mailop.py). Add two
strategies: InlineOpenPGPEncrypt and MimeOpenPGPEncrypt, to support two modes
of OpenPGP encryption.
- In delivery_plan, only use those strategies that actually make sense with
the recipients we'd got.
- Add flag_enabled predicate (lacre/config.py) to make configuration checks
easier / simpler.
- Handle TypeError errors in Advanced Filter, indicating a delivery failure
when they appear.
- Add type hints to some of the functions.
- Add a "mailop" module to define mail operations. Each should inherit from
MailOperation class (which just defines the contract).
- Make lacre.mailgate.delivery_plan always return KeepIntact strategy to have
a daemon that just forwards messages without modifying them.
- Add sample configuration.
- Include daemon configuration in mandatory parameter check.
