Commit Graph

357 Commits

Author SHA1 Message Date
Piotr F. Mieszkowski da53ed3821 Log message defects and optionally some non-PII headers 2022-11-27 14:10:13 +01:00
Piotr F. Mieszkowski 51863dda2b Use bytes instead of str to hold message bodies
smtplib.SMTP expects ASCII-only message bodies when message body is provided
as a 'str'.  If we pass a 'bytes', we need to choose encoding earlier and we
do this by calling 'as_bytes' on messages with SMTP policy, which takes care
of formatting the body properly.

As a result, ISO-8859-x messages are converted to Quoted Printable and UTF-8
messages are Base64-encoded.

Testing this behaviour is tricky, because we use the same SMTP client to send
test data.  For this reason, test code has become a bit ugly, but it does
exactly what we need.
2022-11-03 22:37:21 +01:00
Piotr F. Mieszkowski ae574a701b Add UTF-8 and ISO-8859-2 test cases, make tests more reliable 2022-11-03 08:42:55 +01:00
Piotr F. Mieszkowski a344056cca Add a test case with non-ASCII message content 2022-10-31 20:48:07 +01:00
Piotr F. Mieszkowski 95069c06a7 Add config parameter to set DATA size limit
Expose a new parameter: [daemon]max_data_bytes, to limit Lacre's memory
usage and allow processing of messages larger than 32MB (which is the
default limit).
2022-10-29 20:35:21 +02:00
Piotr F. Mieszkowski 91969dbfbb Re-use the same Event Loop for each coroutine
Function asyncio.run creates a new event loop each time it's called and
executes coroutine in that new loop.  However, we want all our coroutines to
be executed from the same event loop, so we acquire a loop when lacre.daemon
starts and then use it to execute them later.

See: Disroot/gpg-lacre#109
2022-10-29 16:54:58 +02:00
pfm 9f1c4db49d Merge pull request 'doc-updates' (#108) from doc-updates into main
Reviewed-on: #108
2022-10-26 17:21:41 +00:00
Piotr F. Mieszkowski 86b0cd335e Replace UTF-8 non-breaking space with a plain space 2022-10-26 19:20:47 +02:00
Piotr F. Mieszkowski 6f379709f3 Update README to explicitly state that Lacre is in beta state 2022-10-26 19:19:24 +02:00
pfm dab882550e Merge pull request 'improved formatting' (#107) from EmanuelLoos/gpg-lacre:main into main
Reviewed-on: #107
2022-10-26 16:57:35 +00:00
EmanuelLoos 68c09c0eb1 improved formatting
Just some small Markdown syntax corrections.
2022-10-26 16:57:35 +00:00
pfm c3cc37bf56 Merge pull request 'Improve code quality' (#103) from rc2-improvements into main
Reviewed-on: #103
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-26 16:49:04 +00:00
Piotr F. Mieszkowski 18c790f986 Only reload on pubring.kbx file modifications 2022-10-23 13:51:42 +02:00
Piotr F. Mieszkowski d39cadb9aa Use proper name in webgate-cron.py logs 2022-10-23 13:51:10 +02:00
Piotr F. Mieszkowski 53378b516e Add a test for message with PGP inline markers that's not encrypted
If a user mentions PGP markers inside their message, we should not classify it
as already encrypted.
2022-10-22 21:29:59 +02:00
Piotr F. Mieszkowski b4f30d7e8f Add test clear text input message with PGP markers
It's possible to trick Lacre by sending PGP markers in message body, causing
it to classify that message being already encrypted.  This test case is used
to reproduce this scenario.
2022-10-22 20:54:08 +02:00
Piotr F. Mieszkowski b91501d3dd Convert all test input messages to CRLF line endings 2022-10-22 20:52:14 +02:00
Piotr F. Mieszkowski fc85cdb841 Rework PGP-Inline verification/recognition 2022-10-22 19:58:16 +02:00
Piotr F. Mieszkowski ba7978b4a6 Make webgate-cron code more readable 2022-10-22 14:33:20 +02:00
Piotr F. Mieszkowski 2ac26c09ce Simplify code, improve log entries, add comments 2022-10-22 11:23:17 +02:00
Piotr F. Mieszkowski 00289759a3 Add aiosmtpd dependency 2022-10-22 11:23:11 +02:00
Piotr F. Mieszkowski 8f8d9dc1b6 Rename mailgate.py to core.py 2022-10-22 11:23:04 +02:00
pfm 4bdbd0febb Merge pull request 'Improve logging' (#101) from 100-logging-improvements into main
Reviewed-on: #101
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-21 06:30:23 +00:00
Piotr F. Mieszkowski 99e939bb4e Remove too verbose debug logs, implement repr() for KeyCache 2022-10-20 22:27:34 +02:00
Piotr F. Mieszkowski 540ca2adf3 Improve logging
- Report processing time in milliseconds.
- Use module names in log messages instead of file-names without extensions.
2022-10-20 21:56:01 +02:00
pfm 9aa1c3732c Merge pull request 'Adjust cron tests after recent FE decoupling' (#99) from cron-tests into main
Reviewed-on: #99
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-19 19:09:38 +00:00
Piotr F. Mieszkowski a82d9f96b3 Fix indentation and some warnings 2022-10-19 21:04:25 +02:00
Piotr F. Mieszkowski fe49d985ec Adjust cron tests after recent FE decoupling 2022-10-19 20:54:40 +02:00
pfm 13636bfddd Merge pull request 'Implement Advanced Content Filter' (#97) from daemon into main
Reviewed-on: #97
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-19 18:48:39 +00:00
Piotr F. Mieszkowski 5f8c94673d Add basic daemon documentation 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 641253b3ec Make key-loading async, remove unused parameter 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 9696b7e997 Separate key-cache and key-loader
Extract key-loading code to a dedicated class KeyRing in lacre.keyring module.
KeyCache only keeps a static map of identities, making it safe to use in
asynchronous context (and race condition resistant).
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 9f3ad49f14 Rename lacre.keycache to lacre.keyring
This will better reflect the fact we're doing more than just caching.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski acbb2ab776 Document dependencies
Add requirements.txt file with versions of dependencies known to work well.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 2da97a5a9a Reformat code, add doc comments 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski eb0d5a1326 Reload keyring on filesystem events
Subscribe to FS events from keyring directory using Python Watchdog and when a
modification is observed, reload the key cache.

Since we may receive more than one event about a single modification, keep
directory's last modification to recognise 'false positives'.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 386c23f9f8 Document dependencies on the logging module 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 1db0a09fa5 Log processing time for successful deliveries 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski a85b7b7a43 Reload key cache only if keyring dir was modified 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski f5cff3292a Reload key cache periodically
Use [default]cache_refresh_minutes configuration parameter to define periods
between cache reloads.  After this number of minutes cache will be reloaded.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski d7e4947afd Add cache validity configuration parameter
Also, log basic information in KeyCache and provide load() and reload()
operations to make daemon's code cleaner.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 5f601fa50c Implement a basic KeyCache 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 07263d5afa Reformat tests 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski c41df63e42 Reorder tests to avoid interferences 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski a2eeaeee9d Implement Advanced Filter flow for cleartext and OpenPGP
- Polish implementation of mail operations (lacre/mailop.py).  Add two
strategies: InlineOpenPGPEncrypt and MimeOpenPGPEncrypt, to support two modes
of OpenPGP encryption.

- In delivery_plan, only use those strategies that actually make sense with
the recipients we'd got.

- Add flag_enabled predicate (lacre/config.py) to make configuration checks
easier / simpler.

- Handle TypeError errors in Advanced Filter, indicating a delivery failure
when they appear.

- Add type hints to some of the functions.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski ce6a0c5466 Continue refactoring
- Add more encryption strategies.
- Replace tuples (email + key) with dedicated objects.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 8963eee47f Reformat GnuPG module 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski a5bcf2d9b2 Make daemon E2E tests use configured parameters 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski d01865d21c Refactor into smaller functions and objects 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski ddcef93abb Fix a bug introduced by refactoring, clean up code
- Fix certificate retrieval.

- Store recipients within MailOperation objects.

- Log more information.

- Fix some warnings.
2022-10-19 18:36:23 +00:00