Commit Graph

529 Commits

Author SHA1 Message Date
Piotr F. Mieszkowski 765637fd3a Fix logging initialisation by fixing import order 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 1a3ce89ce5 Always encrypt decoded payload 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 3c8b792203 Fix line-discarding bug 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 56101b86c0 Clean up the code after refactor
- Clean up PGP/MIME flow by using API instead of explicit/manual generation of
  headers.
- Fix E2E test configuration for PGP/MIME case.
- Add first lacre.core unit tests.
- Add another Contract Test.
2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 27b07e672d Rework PGP/MIME flow 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 5e408259c0 Start using Content Manager
Also:
- Pass text to Popen in GnuPG (used to be bytes).
- Make is_payload_pgp_inline type-agnostic (str / bytes).
2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski d2ed4a9cee Make test reports more readable 2023-05-08 22:17:01 +02:00
Piotr F. Mieszkowski 94e22caf8e When encryption fails, revert to cleartext delivery
When GnuPG refuses to encrypt a message (e.g. when key has expired), record
information about the failure and send to logs, then deliver cleartext.  This
way we won't bounce email that could be delivered without encryption.

Also: add more E2E tests.
2023-05-08 22:16:41 +02:00
Piotr F. Mieszkowski 1cdca1d06d Log up to 2,5kB of message headers 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 3c1544e423 [daemon] Parse Envelope.original_content 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski ffffa6a364 Correct sample message file name 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b242edf098 Add a more sophisticated UTF-8 test case
Add a new test message to verify Lacre's behaviour when processing
UTF-8 messages with text in two different scripts (latin-based and cyrillic).

Also: log Content-Transfer-Encoding when logging headers is enabled.
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b94123e83e Use SMTPUTF8 policy, add more debug logging 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 4da4019321 Log message defects and optionally some non-PII headers 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski b6bd36a460 Use bytes instead of str to hold message bodies
smtplib.SMTP expects ASCII-only message bodies when message body is provided
as a 'str'.  If we pass a 'bytes', we need to choose encoding earlier and we
do this by calling 'as_bytes' on messages with SMTP policy, which takes care
of formatting the body properly.

As a result, ISO-8859-x messages are converted to Quoted Printable and UTF-8
messages are Base64-encoded.

Testing this behaviour is tricky, because we use the same SMTP client to send
test data.  For this reason, test code has become a bit ugly, but it does
exactly what we need.
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 0fac54a29a Add UTF-8 and ISO-8859-2 test cases, make tests more reliable 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 037a527c44 Add a test case with non-ASCII message content 2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski 5eb687f0cd Add config parameter to set DATA size limit
Expose a new parameter: [daemon]max_data_bytes, to limit Lacre's memory
usage and allow processing of messages larger than 32MB (which is the
default limit).
2023-05-08 22:14:24 +02:00
Piotr F. Mieszkowski f6bf86c533 Re-use the same Event Loop for each coroutine
Function asyncio.run creates a new event loop each time it's called and
executes coroutine in that new loop.  However, we want all our coroutines to
be executed from the same event loop, so we acquire a loop when lacre.daemon
starts and then use it to execute them later.

See: Disroot/gpg-lacre#109
2023-05-08 22:14:24 +02:00
pfm 609a81adcd Merge pull request 'Remove documentation and sample config for decryption' (#121) from doc-updates into main
Reviewed-on: #121
2023-03-11 11:33:44 +00:00
Piotr F. Mieszkowski 37335de329 Remove documentation and sample config for decryption
We no longer support decrypting emails, so:

- Remove parameters from exmaple configuration file.
- Remove relevant section from installation instructions (INSTALL.md).
2023-03-11 12:30:24 +01:00
pfm 7389dc8fde Merge pull request 'Add missing test case for `add_key` and `delete_key`' (#116) from wiktor/gpg-lacre:add-test-for-add-delete-key into main
Reviewed-on: #116
Reviewed-by: pfm <pfm@no-reply@disroot.org>
2023-02-16 18:04:17 +00:00
Wiktor Kwapisiewicz 46bb125684
Add missing test case for `add_key` and `delete_key` 2022-12-23 13:49:43 +01:00
pfm 2215886850 Merge pull request 'Add basic unit tests for GnuPG module' (#114) from gnupg-tests into main
Reviewed-on: #114
2022-12-14 20:29:01 +00:00
Piotr F. Mieszkowski 588b447e69 Add basic unit tests for GnuPG module
- Cover basic GnuPG functionality with tests (confirm_key, public_keys).
- Add a test public key file.
- Fix resource leak by closing streams opened by Popen.
2022-12-14 21:09:46 +01:00
pfm 5c3fecfb85 Merge pull request 'Formatting corrections and whitespace character fixes' (#112) from Onnayaku/gpg-lacre:doc-fixes into main
Reviewed-on: #112
Reviewed-by: pfm <pfm@no-reply@disroot.org>
2022-12-14 19:32:40 +00:00
Onnayaku c7c49977c9
Change 'Indented code blocks' to 'Fenced code blocks' in adv-filt.md 2022-11-01 23:53:37 +01:00
Onnayaku 2c6adfe335
Minor formatting corrections and whitespace character fixes in adv-filt.md testing.md INSTALL.md and README.md 2022-11-01 21:35:48 +01:00
pfm 9f1c4db49d Merge pull request 'doc-updates' (#108) from doc-updates into main
Reviewed-on: #108
2022-10-26 17:21:41 +00:00
Piotr F. Mieszkowski 86b0cd335e Replace UTF-8 non-breaking space with a plain space 2022-10-26 19:20:47 +02:00
Piotr F. Mieszkowski 6f379709f3 Update README to explicitly state that Lacre is in beta state 2022-10-26 19:19:24 +02:00
pfm dab882550e Merge pull request 'improved formatting' (#107) from EmanuelLoos/gpg-lacre:main into main
Reviewed-on: #107
2022-10-26 16:57:35 +00:00
EmanuelLoos 68c09c0eb1 improved formatting
Just some small Markdown syntax corrections.
2022-10-26 16:57:35 +00:00
pfm c3cc37bf56 Merge pull request 'Improve code quality' (#103) from rc2-improvements into main
Reviewed-on: #103
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-26 16:49:04 +00:00
Piotr F. Mieszkowski 18c790f986 Only reload on pubring.kbx file modifications 2022-10-23 13:51:42 +02:00
Piotr F. Mieszkowski d39cadb9aa Use proper name in webgate-cron.py logs 2022-10-23 13:51:10 +02:00
Piotr F. Mieszkowski 53378b516e Add a test for message with PGP inline markers that's not encrypted
If a user mentions PGP markers inside their message, we should not classify it
as already encrypted.
2022-10-22 21:29:59 +02:00
Piotr F. Mieszkowski b4f30d7e8f Add test clear text input message with PGP markers
It's possible to trick Lacre by sending PGP markers in message body, causing
it to classify that message being already encrypted.  This test case is used
to reproduce this scenario.
2022-10-22 20:54:08 +02:00
Piotr F. Mieszkowski b91501d3dd Convert all test input messages to CRLF line endings 2022-10-22 20:52:14 +02:00
Piotr F. Mieszkowski fc85cdb841 Rework PGP-Inline verification/recognition 2022-10-22 19:58:16 +02:00
Piotr F. Mieszkowski ba7978b4a6 Make webgate-cron code more readable 2022-10-22 14:33:20 +02:00
Piotr F. Mieszkowski 2ac26c09ce Simplify code, improve log entries, add comments 2022-10-22 11:23:17 +02:00
Piotr F. Mieszkowski 00289759a3 Add aiosmtpd dependency 2022-10-22 11:23:11 +02:00
Piotr F. Mieszkowski 8f8d9dc1b6 Rename mailgate.py to core.py 2022-10-22 11:23:04 +02:00
pfm 4bdbd0febb Merge pull request 'Improve logging' (#101) from 100-logging-improvements into main
Reviewed-on: #101
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-21 06:30:23 +00:00
Piotr F. Mieszkowski 99e939bb4e Remove too verbose debug logs, implement repr() for KeyCache 2022-10-20 22:27:34 +02:00
Piotr F. Mieszkowski 540ca2adf3 Improve logging
- Report processing time in milliseconds.
- Use module names in log messages instead of file-names without extensions.
2022-10-20 21:56:01 +02:00
pfm 9aa1c3732c Merge pull request 'Adjust cron tests after recent FE decoupling' (#99) from cron-tests into main
Reviewed-on: #99
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2022-10-19 19:09:38 +00:00
Piotr F. Mieszkowski a82d9f96b3 Fix indentation and some warnings 2022-10-19 21:04:25 +02:00
Piotr F. Mieszkowski fe49d985ec Adjust cron tests after recent FE decoupling 2022-10-19 20:54:40 +02:00