Disroot/gpg-lacre
8
15
Fork 4
Code Issues 37 Pull requests 1 Releases 3 Wiki Activity

Fix encoding issues #123

Merged
pfm merged 55 commits from post-test-fixes into main 2023-05-11 22:22:24 +02:00
Conversation 1 Commits 55 Files changed 30 +1605 -680
pfm commented 2023-05-11 21:50:56 +02:00
Collaborator
Copy link
  • Use SMTPUTF8 email policy consistently.
  • Rework E2E tests to use unittest framework.
  • Improve overall message handling.
  • Add more E2E tests.
- Use SMTPUTF8 email policy consistently. - Rework E2E tests to use unittest framework. - Improve overall message handling. - Add more E2E tests.
pfm added 55 commits 2023-05-11 21:50:57 +02:00 
Function asyncio.run creates a new event loop each time it's called and
executes coroutine in that new loop.  However, we want all our coroutines to
be executed from the same event loop, so we acquire a loop when lacre.daemon
starts and then use it to execute them later.

See: Disroot/gpg-lacre#109
Expose a new parameter: [daemon]max_data_bytes, to limit Lacre's memory
usage and allow processing of messages larger than 32MB (which is the
default limit).
smtplib.SMTP expects ASCII-only message bodies when message body is provided
as a 'str'.  If we pass a 'bytes', we need to choose encoding earlier and we
do this by calling 'as_bytes' on messages with SMTP policy, which takes care
of formatting the body properly.

As a result, ISO-8859-x messages are converted to Quoted Printable and UTF-8
messages are Base64-encoded.

Testing this behaviour is tricky, because we use the same SMTP client to send
test data.  For this reason, test code has become a bit ugly, but it does
exactly what we need.
Add a new test message to verify Lacre's behaviour when processing
UTF-8 messages with text in two different scripts (latin-based and cyrillic).

Also: log Content-Transfer-Encoding when logging headers is enabled.
When GnuPG refuses to encrypt a message (e.g. when key has expired), record
information about the failure and send to logs, then deliver cleartext.  This
way we won't bounce email that could be delivered without encryption.

Also: add more E2E tests.
Also:
- Pass text to Popen in GnuPG (used to be bytes).
- Make is_payload_pgp_inline type-agnostic (str / bytes).
- Clean up PGP/MIME flow by using API instead of explicit/manual generation of
  headers.
- Fix E2E test configuration for PGP/MIME case.
- Add first lacre.core unit tests.
- Add another Contract Test.
- Use MIMEPart instead of Message when encrypting in PGP/MIME mode.

- Wrap text/plain messages in MIMEPart, instead of manipulating payloads
  manually.

- Add a test for wrapping.
In the daemon, specify policy as SMTPUTF8.  That sets the deafult message type
to EmailMessage.

EmailMessage class is richer, including support for Content Managers, giving
it the capability to properly handle textual data and its encodings.

Also: add another contract test.
Implement RecipientList class with:
- recipient (email) list,
- key (identity) list.

Cover with basic unit test.
Instead of passing pairs of lists (emails and keys) separately, implement a
class RecipientList to wrap such pair of lists.
Introduce modules:
- lacre.transport - for actual delivery via SMTP
- lacre.smime - to take care of S/MIME stuff

Implement lacre.transport.SendFrom class that does a almost exactly the same
thing as the original send_msg function, but without using global variable to
store original message sender.
We rely on Content Manager to select the right Content-Transfer-Encoding.
Also: adjust expected test output because it's now Base64-encoded.
- Remove unused code.
- Add docstrings.
pfm changed title from WIP: Fix encoding issues to Fix encoding issues 2023-05-11 22:19:10 +02:00
pfm commented 2023-05-11 22:21:21 +02:00
Author
Collaborator
Copy link

Fixes #110.

Fixes #110.
pfm merged commit 07fb8d6ae8 into main 2023-05-11 22:22:24 +02:00
pfm deleted branch post-test-fixes 2023-05-11 22:22:24 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
ANSIBLE

   
BUG

Something that's not working properly

  
CODE

   
DEVELOPMENT

   
DOCUMENTATION

Documentation

  
FEEDBACK

Suggestions and comments

  
FIX

A solution or patch for an issue

  
HOWTOs

Guides and tutorials

  
IDEA

Potential improvements and things to consider when time allows

  
INFRA

   
ISSUE

A problem

  
MAILSERVER

   
RESEARCH

   
TESTS

   
To-Be-Reviewed

We should discuss these issues and decide what to do about them. They are candidates to be closed without further action or their scope might change.

  
WEB

   
WEBSITE

Lacre website related

No labels
ANSIBLE
BUG
CODE
DEVELOPMENT
DOCUMENTATION
FEEDBACK
FIX
HOWTOs
IDEA
INFRA
ISSUE
MAILSERVER
RESEARCH
TESTS
To-Be-Reviewed
WEB
WEBSITE
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Disroot/gpg-lacre#123
No description provided.
Delete branch "post-test-fixes"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?