Commit graph

540 commits

Author SHA1 Message Date
93002c4910
Log encryption failure tracebacks
When we fail to encrypt a message, we log a message with a complete traceback.
The goal is to ensure that when diagnosing it later, we have complete
information.
2024-09-13 12:22:17 +02:00
pfm
c1c4bd1573 Merge pull request 'Log key cache expiry dates' (#156) from bugfix/key-queue-ttl-logs into develop
Reviewed-on: #156
2024-09-13 12:01:00 +02:00
e01d5f6feb
Log key cache expiry dates 2024-08-28 22:07:19 +02:00
pfm
64b401fb66 Merge pull request 'Parse [database]max_queue_hours as integer, handle invalid format' (#155) from bugfix/key-queue-ttl into develop
Reviewed-on: #155
Reviewed-by: muppeth <muppeth@no-reply@disroot.org>
2024-08-27 21:13:10 +02:00
1bec99afaf
Parse [database]max_queue_hours as integer, handle invalid format 2024-08-27 18:01:20 +02:00
pfm
58a7117bb9 Merge pull request 'Improve error handling' (#152) from error-handling into main
Reviewed-on: #152
2024-08-23 14:30:00 +02:00
ba23eeb8b3
Update expected cleartext output
Lacre no longer transforms cleartext payloads.
2024-08-23 14:16:28 +02:00
e643ce7722
Do not touch original payload in KeepIntact mode 2024-08-23 14:16:28 +02:00
8b5d924321
Implement LazyMessage, a wrapper for original contents
We want to avoid deserialising message contents, because Python's email module
might produce different representation than the MUA sending original message.
The result would be a transformed message, which could mean broken message in
certain conditions.
2024-08-23 14:16:28 +02:00
b6155ade96
Improve tests
- Always default to 'python' if PYTHON env. var. unset.
- Enable SQLAlchemy warnings in daemon tests.
- Commit changes in schema initialisation script.
2024-08-23 14:16:28 +02:00
4977185ba1
Add header-only parser and minor test improvements 2024-08-23 14:16:27 +02:00
474d20f32b
Add contract tests for headers-only parsing of email messages 2024-08-23 14:16:27 +02:00
95b1396ee0
Upgrade SQLAlchemy, enable warning capturing 2024-08-23 14:16:27 +02:00
c92b9aed80
Only expire KeyConfirmationQueue items that haven't been confirmed 2024-08-23 14:16:27 +02:00
abd3f923fb
Extract expiry calculation from KeyConfirmationQueue 2024-08-23 14:16:27 +02:00
e6619a660f
When cleaning up after cron-job, remove expired queue items
- Implement KeyConfirmationQueue.delete_expired_queue_items to delete items
older than a given number of hours.

- Introduce configuration parameter to specify maximum number of hours.  It
defaults to 1 hour.

- Update documentation to explain that we never assign ST_TO_BE_DELETED.
2024-08-23 14:16:27 +02:00
ce595971e1
Document key lifecycle 2024-08-23 14:16:27 +02:00
09d7a498df
Improve delivery error-handling
- Introduce exceptions to be raised upon transient and permanent delivery
failures, as specified by SMTP RFC.  Depending on type of failure, return
either 451 or 554 reply code.

- When serialising a message, treat ValueError as a serialisation issue (and
try again to deliver in cleartext).
2024-08-23 14:16:27 +02:00
8955cf6c9d
Improve error-handling and error-reporting
1. Log the full traceback only once for each error (when we bounce the
message).

2. Use 451 response code on processing failure.

3. Disable decoding message contents as we operate on raw data anyway.
2024-08-23 14:16:27 +02:00
0ef3012e33
Fix a typo in bouncing, wrap UnicodeEncodeError in custom exception 2024-08-23 14:16:27 +02:00
pfm
d3f1aa3a02 Merge pull request 'Improve error handling' (#146) from error-handling into main
Reviewed-on: #146
2024-03-03 08:35:03 +00:00
e28864074c
Log exception and traceback when we fail-over to cleartext 2024-03-03 09:26:50 +01:00
0ec1bc3056
Set default values for non-nullable lacre_keys columns 2024-03-03 09:25:14 +01:00
110ea885f2
Deliver cleartext if Unicode encoding or message serialisation fail 2024-03-02 18:36:41 +01:00
3138864d32
Include exception in ExecutionTimeLogger log record
Also: cover ExecutionTimeLogger with a unit test.
2024-03-02 18:06:51 +01:00
pfm
1d8acc8eb8 Merge pull request '142_lacreadm' (#145) from 142_lacreadm into main
Reviewed-on: #145
2024-03-01 19:50:10 +00:00
8006b96df2 Rename lacreadm wrapper, mention it in documentation 2024-03-01 19:47:10 +00:00
f80e4ecb9e Implement a very thin wrapper around 'python -m lacre.admin' 2024-03-01 19:47:10 +00:00
676ff47933 Don't ignore 'bin' directory 2024-03-01 19:47:10 +00:00
f1c135850c lacre.admin: Report misconfiguration
Also: log more info when the daemon starts.
2024-03-01 19:47:10 +00:00
pfm
ccfaa39501 Merge pull request 'Fix unencrypted delivery' (#144) from v0.2_unencrypted-delivery-fix into main
Reviewed-on: #144
2024-03-01 19:34:19 +00:00
7806d8c32a
Log message headers on a hard error
When we know we need to bounce a message and [daemon]log_headers is enabled,
we record up to 2.5kB of message headers at ERROR level.  This could help
diagnosing issues later.

Also: no longer record MIME Type, Charset and Content-Transfer-Encoding, as
the issues related to these properties no longer occur.
2024-03-01 20:28:51 +01:00
04ca103494
Fix unencrypted delivery in case of message generation failure
When we fail to produce byte representation of the email message being
processed, we may end up bouncing a message.  An example of such case would be
a message with a Message-Id header that Python's email parser library cannot
process.

In such cases, just take whatever original content we have received and pass
it to the destination without touching it to minimise any chances of breaking
the overall flow.
2024-03-01 20:14:09 +01:00
pfm
d75ded751e Merge pull request 'Rename GPG-Mailgate to Lacre' (#138) from 81_rename-to-lacre into main
Reviewed-on: #138
2024-02-23 07:35:21 +00:00
f601080e87 lacre.admin: Add more 'queue' documentation, clean up 2024-02-23 08:33:47 +01:00
80c25f6d2e lacre.admin: Document new sub-command 'database' 2024-02-23 08:25:52 +01:00
aa2eb604d4 lacre.admin: Add a sub-command to manipulate database schema
- It supports option '-i' to initialise the schema.
- It logs a warning-level record of the schema manipulation.
2024-02-21 21:10:49 +01:00
f7e6708949 Adjust lacre.dbschema to reflect original schema.sql
- Set nullability of columns.
- Set up primary keys and auto-increment where necessary.
- Add missing 'lacre_locks' table.
- Implement a function to create tables.
2024-02-21 21:10:49 +01:00
be615df6e4 Split webgate-cron.py into small functions 2024-01-21 11:28:46 +01:00
bfd3541b18 Retrieve data from db result before returning from Context Manager
SQLAlchemy's connection is a Context Manager and if we return a result from
code wrapped in a Context Manager, its cursor might already be closed.
2024-01-20 18:52:47 +01:00
8d2bf403a7 Add lacre.admin queue --list option, log query parameters 2024-01-16 20:33:23 +01:00
55a369df83 Add debug sqlalchemy logs (disabled by default) 2024-01-08 22:45:59 +01:00
cd67b0934e Unify configuration requirements 2024-01-08 22:19:10 +01:00
276e0d0cd4 Use one config for cron and daemon tests 2024-01-07 21:52:52 +01:00
bc2fc53416 Update gitignore: generated config, project name 2024-01-06 15:46:33 +01:00
260a3f3e9c Configure pooling for cron tests 2024-01-06 15:26:18 +01:00
a943b50adb Update test configuration after renaming 2024-01-06 15:06:36 +01:00
a98ff611ee Continue renaming: config files, tests, docs 2024-01-06 14:45:09 +01:00
ad3a54fcd7 Rename GPG-Mailgate to Lacre
Update naming in documentation and the source code.
2024-01-06 14:34:54 +01:00
7208f66527 Improve simple filter structure 2024-01-05 22:21:20 +01:00