Commit graph

145 commits

Author SHA1 Message Date
273ebc7510
Update exception message format 2024-09-13 12:23:44 +02:00
93002c4910
Log encryption failure tracebacks
When we fail to encrypt a message, we log a message with a complete traceback.
The goal is to ensure that when diagnosing it later, we have complete
information.
2024-09-13 12:22:17 +02:00
e01d5f6feb
Log key cache expiry dates 2024-08-28 22:07:19 +02:00
1bec99afaf
Parse [database]max_queue_hours as integer, handle invalid format 2024-08-27 18:01:20 +02:00
e643ce7722
Do not touch original payload in KeepIntact mode 2024-08-23 14:16:28 +02:00
8b5d924321
Implement LazyMessage, a wrapper for original contents
We want to avoid deserialising message contents, because Python's email module
might produce different representation than the MUA sending original message.
The result would be a transformed message, which could mean broken message in
certain conditions.
2024-08-23 14:16:28 +02:00
4977185ba1
Add header-only parser and minor test improvements 2024-08-23 14:16:27 +02:00
95b1396ee0
Upgrade SQLAlchemy, enable warning capturing 2024-08-23 14:16:27 +02:00
c92b9aed80
Only expire KeyConfirmationQueue items that haven't been confirmed 2024-08-23 14:16:27 +02:00
abd3f923fb
Extract expiry calculation from KeyConfirmationQueue 2024-08-23 14:16:27 +02:00
e6619a660f
When cleaning up after cron-job, remove expired queue items
- Implement KeyConfirmationQueue.delete_expired_queue_items to delete items
older than a given number of hours.

- Introduce configuration parameter to specify maximum number of hours.  It
defaults to 1 hour.

- Update documentation to explain that we never assign ST_TO_BE_DELETED.
2024-08-23 14:16:27 +02:00
09d7a498df
Improve delivery error-handling
- Introduce exceptions to be raised upon transient and permanent delivery
failures, as specified by SMTP RFC.  Depending on type of failure, return
either 451 or 554 reply code.

- When serialising a message, treat ValueError as a serialisation issue (and
try again to deliver in cleartext).
2024-08-23 14:16:27 +02:00
8955cf6c9d
Improve error-handling and error-reporting
1. Log the full traceback only once for each error (when we bounce the
message).

2. Use 451 response code on processing failure.

3. Disable decoding message contents as we operate on raw data anyway.
2024-08-23 14:16:27 +02:00
0ef3012e33
Fix a typo in bouncing, wrap UnicodeEncodeError in custom exception 2024-08-23 14:16:27 +02:00
e28864074c
Log exception and traceback when we fail-over to cleartext 2024-03-03 09:26:50 +01:00
0ec1bc3056
Set default values for non-nullable lacre_keys columns 2024-03-03 09:25:14 +01:00
110ea885f2
Deliver cleartext if Unicode encoding or message serialisation fail 2024-03-02 18:36:41 +01:00
3138864d32
Include exception in ExecutionTimeLogger log record
Also: cover ExecutionTimeLogger with a unit test.
2024-03-02 18:06:51 +01:00
f1c135850c lacre.admin: Report misconfiguration
Also: log more info when the daemon starts.
2024-03-01 19:47:10 +00:00
7806d8c32a
Log message headers on a hard error
When we know we need to bounce a message and [daemon]log_headers is enabled,
we record up to 2.5kB of message headers at ERROR level.  This could help
diagnosing issues later.

Also: no longer record MIME Type, Charset and Content-Transfer-Encoding, as
the issues related to these properties no longer occur.
2024-03-01 20:28:51 +01:00
04ca103494
Fix unencrypted delivery in case of message generation failure
When we fail to produce byte representation of the email message being
processed, we may end up bouncing a message.  An example of such case would be
a message with a Message-Id header that Python's email parser library cannot
process.

In such cases, just take whatever original content we have received and pass
it to the destination without touching it to minimise any chances of breaking
the overall flow.
2024-03-01 20:14:09 +01:00
aa2eb604d4 lacre.admin: Add a sub-command to manipulate database schema
- It supports option '-i' to initialise the schema.
- It logs a warning-level record of the schema manipulation.
2024-02-21 21:10:49 +01:00
f7e6708949 Adjust lacre.dbschema to reflect original schema.sql
- Set nullability of columns.
- Set up primary keys and auto-increment where necessary.
- Add missing 'lacre_locks' table.
- Implement a function to create tables.
2024-02-21 21:10:49 +01:00
bfd3541b18 Retrieve data from db result before returning from Context Manager
SQLAlchemy's connection is a Context Manager and if we return a result from
code wrapped in a Context Manager, its cursor might already be closed.
2024-01-20 18:52:47 +01:00
8d2bf403a7 Add lacre.admin queue --list option, log query parameters 2024-01-16 20:33:23 +01:00
55a369df83 Add debug sqlalchemy logs (disabled by default) 2024-01-08 22:45:59 +01:00
cd67b0934e Unify configuration requirements 2024-01-08 22:19:10 +01:00
ad3a54fcd7 Rename GPG-Mailgate to Lacre
Update naming in documentation and the source code.
2024-01-06 14:34:54 +01:00
8f8f081d28 Fix key-removal condition, improve logging 2024-01-04 19:45:25 +01:00
07539a97d3 Improve logging
- Don't re-configure lacre.notify logger.
- Issue more DEBUG logs when deleting keys.
2023-12-20 23:03:04 +01:00
41b7535412 Add more logging, add --delete option to admin queue sub-command 2023-12-19 09:02:42 +01:00
9b5d578985 lacre.config: Make both enums case-insensitive
Also: use PGPStyle in lacre.core.
2023-12-17 20:42:57 +01:00
ff429c93e6 Convert pooling parameters to integers 2023-12-17 14:12:52 +01:00
90da933bf9 Make disconnect handling configuration explicit
- Provide a new reuqired parameter: [database]pooling_mode and use it during
  SQLAlchemy engine initialisation.

- Update tests and configuration (including sample configuration).

- Adjust repository unit test to load config during setup.

- Pass an engine instance to repository constructors instead of connections.
  Engine keeps a connection pool and we rely on it.
2023-12-17 14:03:20 +01:00
86cc27e918 lacre.repositories: Configure SQLAlchemy connection pooling
Provide 3 new configuration parameters in database section:

- max_connection_age --- number of seconds before an idle connection is
  "recycled", i.e. replaced with a new one;

- pool_size --- number of simultaneous connections kept in the pool;

- max_overflow --- maximum number of simultaneous connections we could make to
  the database.

Update sample config, including links to documentation.
2023-12-16 23:32:27 +01:00
e8d0d248b3 lacre.repositories: Add missing import 2023-12-10 21:39:59 +01:00
8cc1136a90 lacre.daemon: When keys can't be loaded, fail gracefully
- Introduce '[daemon]bounce_on_keys_missing' option to let the admin decide if
  they want Lacre to deliver cleartext message when identity database is
  unreachable or throws exceptions.  It defaults to 'no'.

- In IdentityRepository, use option mentioned above to decide what to do when
  an exception is caught.
2023-12-10 21:27:05 +01:00
fe2c0cbf76 Fix unprintable exception issue 2023-12-09 20:57:09 +01:00
75c48282b0 Rework encryption exception handling
Also: remove misleading comment about message.defects.
2023-12-09 20:38:46 +01:00
fc08813bdc Improve unencryptable message logs 2023-12-09 19:48:20 +01:00
d51c675881 lacre.admin: Make import -r option a flag (Boolean) 2023-12-05 21:51:35 +01:00
abaf8820d7 lacre.admin: Add -r / --reload option to import command
With -r option, import command will first remove all identities and then load
them again from pubring.kbx.
2023-12-05 21:49:23 +01:00
94d0a62766 Identity removal: execute prepared DELETE 2023-12-05 21:33:19 +01:00
cc1bacbe3d Move some imports to lacre.notify 2023-12-05 21:13:02 +01:00
0d852bc279 lacre.repositories: Fix IdentityRepository existence predicate 2023-12-04 22:57:43 +01:00
052551072e Change table prefix from 'gpgmw' to 'lacre' 2023-12-02 20:02:59 +01:00
0975ce3a69 lacre.admin: Handle database exceptions 2023-11-26 19:52:58 +01:00
b44bd7b150 lacre.admin: Implement identity import, fix identity list
- Let the user specify a directory, using the one from configuration by
  default.

- If user requested identity list without a specific email, list all.  Drop
  support for '-a' option.
2023-11-26 18:30:25 +01:00
97c4f9f14a lacre.repositories: Fix naming after refactoring 2023-11-25 16:09:23 +01:00
626fce5f2c lacre.admin: Implement 'identities' sub-command 2023-11-25 16:08:54 +01:00