Commit Graph

520 Commits

Author SHA1 Message Date
Piotr F. Mieszkowski 3c1544e423 [daemon] Parse Envelope.original_content 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski ffffa6a364 Correct sample message file name 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b242edf098 Add a more sophisticated UTF-8 test case
Add a new test message to verify Lacre's behaviour when processing
UTF-8 messages with text in two different scripts (latin-based and cyrillic).

Also: log Content-Transfer-Encoding when logging headers is enabled.
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b94123e83e Use SMTPUTF8 policy, add more debug logging 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 4da4019321 Log message defects and optionally some non-PII headers 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b6bd36a460 Use bytes instead of str to hold message bodies
smtplib.SMTP expects ASCII-only message bodies when message body is provided
as a 'str'.  If we pass a 'bytes', we need to choose encoding earlier and we
do this by calling 'as_bytes' on messages with SMTP policy, which takes care
of formatting the body properly.

As a result, ISO-8859-x messages are converted to Quoted Printable and UTF-8
messages are Base64-encoded.

Testing this behaviour is tricky, because we use the same SMTP client to send
test data.  For this reason, test code has become a bit ugly, but it does
exactly what we need.
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 0fac54a29a Add UTF-8 and ISO-8859-2 test cases, make tests more reliable 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 037a527c44 Add a test case with non-ASCII message content 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 5eb687f0cd Add config parameter to set DATA size limit
Expose a new parameter: [daemon]max_data_bytes, to limit Lacre's memory
usage and allow processing of messages larger than 32MB (which is the
default limit).
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski f6bf86c533 Re-use the same Event Loop for each coroutine
Function asyncio.run creates a new event loop each time it's called and
executes coroutine in that new loop.  However, we want all our coroutines to
be executed from the same event loop, so we acquire a loop when lacre.daemon
starts and then use it to execute them later.

See: Disroot/gpg-lacre#109
2023-05-08 22:14:24 +02:00
pfm 609a81adcd Merge pull request 'Remove documentation and sample config for decryption' (#121) from doc-updates into main
Reviewed-on: #121
2023-03-11 11:33:44 +00:00
Piotr F. Mieszkowski 37335de329 Remove documentation and sample config for decryption
We no longer support decrypting emails, so:

- Remove parameters from exmaple configuration file.
- Remove relevant section from installation instructions (INSTALL.md).
2023-03-11 12:30:24 +01:00
pfm 7389dc8fde Merge pull request 'Add missing test case for `add_key` and `delete_key`' (#116) from wiktor/gpg-lacre:add-test-for-add-delete-key into main
Reviewed-on: #116
Reviewed-by: pfm <pfm@no-reply@disroot.org>
2023-02-16 18:04:17 +00:00
Wiktor Kwapisiewicz 46bb125684
Add missing test case for `add_key` and `delete_key` 2022-12-23 13:49:43 +01:00
pfm 2215886850 Merge pull request 'Add basic unit tests for GnuPG module' (#114) from gnupg-tests into main
Reviewed-on: #114
2022-12-14 20:29:01 +00:00
Piotr F. Mieszkowski 588b447e69 Add basic unit tests for GnuPG module
- Cover basic GnuPG functionality with tests (confirm_key, public_keys).
- Add a test public key file.
- Fix resource leak by closing streams opened by Popen.
2022-12-14 21:09:46 +01:00
pfm 5c3fecfb85 Merge pull request 'Formatting corrections and whitespace character fixes' (#112) from Onnayaku/gpg-lacre:doc-fixes into main
Reviewed-on: #112
Reviewed-by: pfm <pfm@no-reply@disroot.org>
2022-12-14 19:32:40 +00:00
Onnayaku c7c49977c9
Change 'Indented code blocks' to 'Fenced code blocks' in adv-filt.md 2022-11-01 23:53:37 +01:00
Onnayaku 2c6adfe335
Minor formatting corrections and whitespace character fixes in adv-filt.md testing.md INSTALL.md and README.md 2022-11-01 21:35:48 +01:00
pfm 9f1c4db49d Merge pull request 'doc-updates' (#108) from doc-updates into main
Reviewed-on: #108
2022-10-26 17:21:41 +00:00
Piotr F. Mieszkowski 86b0cd335e Replace UTF-8 non-breaking space with a plain space 2022-10-26 19:20:47 +02:00
Piotr F. Mieszkowski 6f379709f3 Update README to explicitly state that Lacre is in beta state 2022-10-26 19:19:24 +02:00
pfm dab882550e Merge pull request 'improved formatting' (#107) from EmanuelLoos/gpg-lacre:main into main
Reviewed-on: #107
2022-10-26 16:57:35 +00:00
EmanuelLoos 68c09c0eb1 improved formatting
Just some small Markdown syntax corrections.
2022-10-26 16:57:35 +00:00
pfm c3cc37bf56 Merge pull request 'Improve code quality' (#103) from rc2-improvements into main
Reviewed-on: #103
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-26 16:49:04 +00:00
Piotr F. Mieszkowski 18c790f986 Only reload on pubring.kbx file modifications 2022-10-23 13:51:42 +02:00
Piotr F. Mieszkowski d39cadb9aa Use proper name in webgate-cron.py logs 2022-10-23 13:51:10 +02:00
Piotr F. Mieszkowski 53378b516e Add a test for message with PGP inline markers that's not encrypted
If a user mentions PGP markers inside their message, we should not classify it
as already encrypted.
2022-10-22 21:29:59 +02:00
Piotr F. Mieszkowski b4f30d7e8f Add test clear text input message with PGP markers
It's possible to trick Lacre by sending PGP markers in message body, causing
it to classify that message being already encrypted.  This test case is used
to reproduce this scenario.
2022-10-22 20:54:08 +02:00
Piotr F. Mieszkowski b91501d3dd Convert all test input messages to CRLF line endings 2022-10-22 20:52:14 +02:00
Piotr F. Mieszkowski fc85cdb841 Rework PGP-Inline verification/recognition 2022-10-22 19:58:16 +02:00
Piotr F. Mieszkowski ba7978b4a6 Make webgate-cron code more readable 2022-10-22 14:33:20 +02:00
Piotr F. Mieszkowski 2ac26c09ce Simplify code, improve log entries, add comments 2022-10-22 11:23:17 +02:00
Piotr F. Mieszkowski 00289759a3 Add aiosmtpd dependency 2022-10-22 11:23:11 +02:00
Piotr F. Mieszkowski 8f8d9dc1b6 Rename mailgate.py to core.py 2022-10-22 11:23:04 +02:00
pfm 4bdbd0febb Merge pull request 'Improve logging' (#101) from 100-logging-improvements into main
Reviewed-on: #101
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-21 06:30:23 +00:00
Piotr F. Mieszkowski 99e939bb4e Remove too verbose debug logs, implement repr() for KeyCache 2022-10-20 22:27:34 +02:00
Piotr F. Mieszkowski 540ca2adf3 Improve logging
- Report processing time in milliseconds.
- Use module names in log messages instead of file-names without extensions.
2022-10-20 21:56:01 +02:00
pfm 9aa1c3732c Merge pull request 'Adjust cron tests after recent FE decoupling' (#99) from cron-tests into main
Reviewed-on: #99
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-19 19:09:38 +00:00
Piotr F. Mieszkowski a82d9f96b3 Fix indentation and some warnings 2022-10-19 21:04:25 +02:00
Piotr F. Mieszkowski fe49d985ec Adjust cron tests after recent FE decoupling 2022-10-19 20:54:40 +02:00
pfm 13636bfddd Merge pull request 'Implement Advanced Content Filter' (#97) from daemon into main
Reviewed-on: #97
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-19 18:48:39 +00:00
Piotr F. Mieszkowski 5f8c94673d Add basic daemon documentation 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 641253b3ec Make key-loading async, remove unused parameter 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 9696b7e997 Separate key-cache and key-loader
Extract key-loading code to a dedicated class KeyRing in lacre.keyring module.
KeyCache only keeps a static map of identities, making it safe to use in
asynchronous context (and race condition resistant).
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 9f3ad49f14 Rename lacre.keycache to lacre.keyring
This will better reflect the fact we're doing more than just caching.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski acbb2ab776 Document dependencies
Add requirements.txt file with versions of dependencies known to work well.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 2da97a5a9a Reformat code, add doc comments 2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski eb0d5a1326 Reload keyring on filesystem events
Subscribe to FS events from keyring directory using Python Watchdog and when a
modification is observed, reload the key cache.

Since we may receive more than one event about a single modification, keep
directory's last modification to recognise 'false positives'.
2022-10-19 18:36:23 +00:00
Piotr F. Mieszkowski 386c23f9f8 Document dependencies on the logging module 2022-10-19 18:36:23 +00:00