Just did a few more fixes to index files. Do they help with anyone's problems?
- ACL plugin didn't work unless control dir was separate from maildir
- More index file handling fixes
I think we're quite near v1.0 now.
* ACL plugin + Maildir: Moved dovecot-acl file from control directory
to maildir. To prevent accidents caused by this change, Dovecot
kills itself if it finds dovecot-acl file from the control directory.
* When opening a maildir, check if tmp/'s atime is over 8h old. If it
is, delete files in it with ctime older than 36h. However if
atime - ctime > 36h, it means that there's nothing to be deleted and
the scanning isn't done. We update atime ourself if filesystem is
mounted with noatime.
* base_dir doesn't need to be group-readable, don't force it.
* mail_read_mmaped setting is deprecated and possibly broken. It's now
removed from dovecot-example.conf, but it still works for now.
* Removed also umask setting from dovecot-example.conf since currently
it doesn't do what it's supposed to.
+ Authentication cache caches now also userdb data.
+ Added mail_log plugin to log various mail operations. Currently it
logs mail copies, deletions, expunges and mailbox deletions.
- dict quota: messages=n parameter actually changed storage limit.
- A lot of fixes to handling index files. This should fix almost all
of the problems ever reported.
- LDAP: auth_bind=yes was more or less broken.
- Saved mails and dovecot-keywords file didn't set the group from
dovecot-shared file.
- Fixed potential assert-crash while searching messages
- Fixed some crashes with invalid X-UID headers in mboxes
- If you didn't have a namespace with empty prefix, giving STATUS
command for a non-existing namespace caused the connection to give
"NO Unknown namespace" errors for all the future commands.
So, compile socketmap support in unconditionally (as the Perl scripts
are actually only examples of socketmap functionality; any language can be
used in reality). Remove socketmap related OPTIONs completely.
* Now compatible with SunOS
* Correctly handle international character sets in email From
Plus changes from Debian:
* Document more options in config.py
* Flesh out manual page
* Check exit status of sendmail, and die if it fails
* Use fcntl even on non-Sunos-Unix
2006-11-01 08:19 Oswald Buddenhagen <ossi@users.sf.net>
* src/: drv_imap.c, mbsync.1:
- un-document "Host imaps:[...]" syntax and introduce new option UseIMAPS
instead
- apply ted's patch to support UseIMAPS in conjunction with Tunnel
- document that SSLv2 is No Good (TM)
2006-11-01 08:14 Oswald Buddenhagen <ossi@users.sf.net>
* src/drv_imap.c:
- move assigning default port to the place of use
towards the viewing and safe modification of the contents in a qmail
queue.
qmqtool was designed with Michele Beltrame's "qmHandle" in mind,
however no source code from qmHandle was used within qmqtool.
The list of changes since version 6.2.5.5 is too large to mention here.
The new version provides a fix for the vulnerability reported in the
fetchmail-SA-2006-02.txt advisory.
Based on package provided in PR 33155, fix some pkglint warnings.
sendymail allows you to send e-mails through Yahoo! Mail, using your
favorite email client. sendymail works as a gateway between Yahoo!
Mail web interface and your e-mail program, and can be used in
conjunction with fetchyahoo or YoSucker to emulate a POP3/SMTP-like
interface.
Based on PR 33091, made pkglint clean and update.
FetchYahoo is a Perl script that downloads mail from a Yahoo! account
to a local mail spool. It is meant to replace fetchmail for people
using Yahoo! mail since Yahoo!'s POP service no longer free. It
downloads messages to a local mail spool, including all parts and
attachments. It then deletes messages unless requested not to. It can
also optionally forward messages to a specified e-mail address and
repeat with a given interval.
Exim version 4.66
-----------------
PH/01 Two more bugs that were introduced by 4.64/PH/07, in addition to the one
fixed by 4.65/MH/01 (is this a record?) are fixed:
(i) An empty string was always treated as zero by the numeric comparison
operators. This behaviour has been restored.
(ii) It is documented that the numeric comparison operators always treat
their arguments as decimal numbers. This was broken in that numbers
starting with 0 were being interpreted as octal.
While fixing these problems I realized that there was another issue that
hadn't been noticed. Values of message_size_limit (both the global option
and the transport option) were treated as octal if they started with 0.
The documentation was vague. These values are now always treated as
decimal, and I will make that clear in the documentation.
Exim version 4.65
-----------------
TK/01 Disable default definition of HAVE_LINUX_SENDFILE. Clashes with
Linux large file support (_FILE_OFFSET_BITS=64) on older glibc
versions. (#438)
MH/01 Don't check that the operands of numeric comparison operators are
integers when their expansion is in "skipping" mode (fixes bug
introduced by 4.64-PH/07).
PH/01 If a system filter or a router generates more than SHRT_MAX (32767)
child addresses, Exim now panics and dies. Previously, because the count
is held in a short int, deliveries were likely to be lost. As such a
large number of recipients for a single message is ridiculous
(performance will be very, very poor), I have chosen to impose a limit
rather than extend the field.
Exim version 4.64
-----------------
TK/01 Bugzilla #401. Fix DK spooling code so that it can overwrite a
leftover -K file (the existence of which was triggered by #402).
While we were at it, introduced process PID as part of the -K
filename. This should rule out race conditions when creating
these files.
TK/02 Bugzilla #402. Apply patch from Simon Arlott, speeding up DK signing
processing considerably. Previous code took too long for large mails,
triggering a timeout which in turn triggers #401.
TK/03 Introduced HAVE_LINUX_SENDFILE to os.h-Linux. Currently only used
in the DK code in transports.c. sendfile() is not really portable,
hence the _LINUX specificness.
TF/01 In the add_headers option to the mail command in an Exim filter,
there was a bug that Exim would claim a syntax error in any
header after the first one which had an odd number of characters
in the field name.
PH/01 If a server that rejects MAIL FROM:<> was the target of a sender
callout verification, Exim cached a "reject" for the entire domain. This
is correct for most verifications, but it is not correct for a recipient
verification with use_sender or use_postmaster set, because in that case
the callout does not use MAIL FROM:<>. Exim now distinguishes the special
case of MAIL FROM:<> rejection from other early rejections (e.g.
rejection of HELO). When verifying a recipient using a non-null MAIL
address, the cache is ignored if it shows MAIL FROM:<> rejection.
Whatever the result of the callout, the value of the domain cache is
left unchanged (for any other kind of callout, getting as far as trying
RCPT means that the domain itself is ok).
PH/02 Tidied a number of unused variable and signed/unsigned warnings that
gcc 4.1.1 threw up.
PH/03 On Solaris, an unexpectedly close socket (dropped connection) can
manifest itself as EPIPE rather than ECONNECT. When tidying away a
session, the daemon ignores ECONNECT errors and logs others; it now
ignores EPIPE as well.
PH/04 Applied Nico Erfurth's refactoring patch to tidy up mime.c
(quoted-printable decoding).
PH/05 Applied Nico Erfurth's refactoring patch to tidy up spool_mbox.c, and
later the small subsequent patch to fix an introduced bug.
PH/06 Installed the latest Cygwin Makefile from the Cygwin maintainer.
PH/07 There was no check for overflow in expansions such as ${if >{1}{4096M}}.
PH/08 An error is now given if message_size_limit is specified negative.
PH/09 Applied and tidied up Jakob Hirsch's patch for allowing ACL variables
to be given (somewhat) arbitrary names.
JJ/01 exipick 20060919.0, allow for arbitrary acl_ variables introduced
in 4.64-PH/09.
JJ/02 exipick 20060919.0, --show-vars args can now be regular expressions,
miscellaneous code fixes
PH/10 Added the log_reject_target ACL modifier to specify where to log
rejections.
PH/11 Callouts were setting the name used for EHLO/HELO from $smtp_active_
hostname. This is wrong, because it relates to the incoming message (and
probably the interface on which it is arriving) and not to the outgoing
callout (which could be using a different interface). This has been
changed to use the value of the helo_data option from the smtp transport
instead - this is what is used when a message is actually being sent. If
there is no remote transport (possible with a router that sets up host
addresses), $smtp_active_hostname is used.
PH/12 Installed Andrey Panin's patch to add a dovecot authenticator. Various
tweaks were necessary in order to get it to work (see also 21 below):
(a) The code assumed that strncpy() returns a negative number on buffer
overflow, which isn't the case. Replaced with Exim's string_format()
function.
(b) There were several signed/unsigned issues. I just did the minimum
hacking in of casts. There is scope for a larger refactoring.
(c) The code used strcasecmp() which is not a standard C function.
Replaced with Exim's strcmpic() function.
(d) The code set only $1; it now sets $auth1 as well.
(e) A simple test gave the error "authentication client didn't specify
service in request". It would seem that Dovecot has changed its
interface. Fortunately there's a specification; I followed it and
changed what the client sends and it appears to be working now.
PH/13 Added $message_headers_raw to provide the headers without RFC 2047
decoding.
PH/14 Corrected misleading output from -bv when -v was also used. Suppose the
address A is aliased to B and C, where B exists and C does not. Without
-v the output is "A verified" because verification stops after a
successful redirection if more than one address is generated. However,
with -v the child addresses are also verified. Exim was outputting "A
failed to verify" and then showing the successful verification for C,
with its parentage. It now outputs "B failed to verify", showing B's
parentage before showing the successful verification of C.
PH/15 Applied Michael Deutschmann's patch to allow DNS black list processing to
look up a TXT record in a specific list after matching in a combined
list.
PH/16 It seems that the options setting for the resolver (RES_DEFNAMES and
RES_DNSRCH) can affect the behaviour of gethostbyname() and friends when
they consult the DNS. I had assumed they would set it the way they
wanted; and indeed my experiments on Linux seem to show that in some
cases they do (I could influence IPv6 lookups but not IPv4 lookups).
To be on the safe side, however, I have now made the interface to
host_find_byname() similar to host_find_bydns(), with an argument
containing the DNS resolver options. The host_find_byname() function now
sets these options at its start, just as host_find_bydns() does. The smtp
transport options dns_qualify_single and dns_search_parents are passed to
host_find_byname() when gethostbyname=TRUE in this transport. Other uses
of host_find_byname() use the default settings of RES_DEFNAMES
(qualify_single) but not RES_DNSRCH (search_parents).
PH/17 Applied (a modified version of) Nico Erfurth's patch to make
spool_read_header() do less string testing, by means of a preliminary
switch on the second character of optional "-foo" lines. (This is
overdue, caused by the large number of possibilities that now exist.
Originally there were few.) While I was there, I also converted the
str(n)cmp tests so they don't re-test the leading "-" and the first
character, in the hope this might squeeze out yet more improvement.
PH/18 Two problems with "group" syntax in header lines when verifying: (1) The
flag allowing group syntax was set by the header_syntax check but not
turned off, possible causing trouble later; (2) The flag was not being
set at all for the header_verify test, causing "group"-style headers to
be rejected. I have now set it in this case, and also caused header_
verify to ignore an empty address taken from a group. While doing this, I
came across some other cases where the code for allowing group syntax
while scanning a header line wasn't quite right (mostly, not resetting
the flag correctly in the right place). These bugs could have caused
trouble for malformed header lines. I hope it is now all correct.
PH/19 The functions {pwcheck,saslauthd}_verify_password() are always called
with the "reply" argument non-NULL. The code, however (which originally
came from elsewhere) had *some* tests for NULL when it wrote to *reply,
but it didn't always do it. This confused somebody who was copying the
code for some other use. I have removed all the tests.
PH/20 It was discovered that the GnuTLS code had support for RSA_EXPORT, a
feature that was used to support insecure browsers during the U.S. crypto
embargo. It requires special client support, and Exim is probably the
only MTA that supported it -- and would never use it because real RSA is
always available. This code has been removed, because it had the bad
effect of slowing Exim down by computing (never used) parameters for the
RSA_EXPORT functionality.
PH/21 On the advice of Timo Sirainen, added a check to the dovecot
authenticator to fail if there's a tab character in the incoming data
(there should never be unless someone is messing about, as it's supposed
to be base64-encoded). Also added, on Timo's advice, the "secured" option
if the connection is using TLS or if the remote IP is the same as the
local IP, and the "valid-client-cert option" if a client certificate has
been verified.
PH/22 As suggested by Dennis Davis, added a server_condition option to *all*
authenticators. This can be used for authorization after authentication
succeeds. (In the case of plaintext, it servers for both authentication
and authorization.)
PH/23 Testing for tls_required and lost_connection in a retry rule didn't work
if any retry times were supplied.
PH/24 Exim crashed if verify=helo was activated during an incoming -bs
connection, where there is no client IP address to check. In this
situation, the verify now always succeeds.
PH/25 Applied John Jetmore's -Mset patch.
PH/26 Added -bem to be like -Mset, but loading a message from a file.
PH/27 In a string expansion for a processed (not raw) header when multiple
headers of the same name were present, leading whitespace was being
removed from all of them, but trailing whitespace was being removed only
from the last one. Now trailing whitespace is removed from each header
before concatenation. Completely empty headers in a concatenation (as
before) are ignored.
PH/28 Fixed bug in backwards-compatibility feature of PH/09 (thanks to John
Jetmore). It would have mis-read ACL variables from pre-4.61 spool files.
PH/29 [Removed. This was a change that I later backed out, and forgot to
correct the ChangeLog entry (that I had efficiently created) before
committing the later change.]
PH/30 Exim was sometimes attempting to deliver messages that had suffered
address errors (4xx response to RCPT) over the same connection as other
messages routed to the same hosts. Such deliveries are always "forced",
so retry times are not inspected. This resulted in far too many retries
for the affected addresses. The effect occurred only when there were more
hosts than the hosts_max_try setting in the smtp transport when it had
the 4xx errors. Those hosts that it had tried were not added to the list
of hosts for which the message was waiting, so if all were tried, there
was no problem. Two fixes have been applied:
(i) If there are any address or message errors in an SMTP delivery, none
of the hosts (tried or untried) are now added to the list of hosts
for which the message is waiting, so the message should not be a
candidate for sending over the same connection that was used for a
successful delivery of some other message. This seems entirely
reasonable: after all the message is NOT "waiting for some host".
This is so "obvious" that I'm not sure why it wasn't done
previously. Hope I haven't missed anything, but it can't do any
harm, as the worst effect is to miss an optimization.
(ii) If, despite (i), such a delivery is accidentally attempted, the
routing retry time is respected, so at least it doesn't keep
hammering the server.
PH/31 Installed Andrew Findlay's patch to close the writing end of the socket
in ${readsocket because some servers need this prod.
PH/32 Added some extra debug output when updating a wait-xxx database.
PH/33 The hint "could be header name not terminated by colon", which has been
given for certain expansion errors for a long time, was not being given
for the ${if def:h_colon_omitted{... case.
PH/34 The spec says: "With one important exception, whenever a domain list is
being scanned, $domain contains the subject domain." There was at least
one case where this was not true.
PH/35 The error "getsockname() failed: connection reset by peer" was being
written to the panic log as well as the main log, but it isn't really
panic-worthy as it just means the connection died rather early on. I have
removed the panic log writing for the ECONNRESET error when getsockname()
fails.
PH/36 After a 4xx response to a RCPT error, that address was delayed (in queue
runs only) independently of the message's sender address. This meant
that, if the 4xx error was in fact related to the sender, a different
message to the same recipient with a different sender could confuse
things. In particualar, this can happen when sending to a greylisting
server, but other circumstances could also provoke similar problems.
I have changed the default so that the retry time for these errors is now
based a combination of the sender and recipient addresses. This change
can be overridden by setting address_retry_include_sender=false in the
smtp transport.
PH/37 For LMTP over TCP/IP (the smtp transport), error responses from the
remote server are returned as part of bounce messages. This was not
happening for LMTP over a pipe (the lmtp transport), but now it is the
same for both kinds of LMTP.
PH/38 Despite being documented as not happening, Exim was rewriting addresses
in header lines that were in fact CNAMEs. This is no longer the case.
PH/39 If -R or -S was given with -q<time>, the effect of -R or -S was ignored,
and queue runs started by the daemon processed all messages. This has
been fixed so that -R and -S can now usefully be given with -q<time>.
PH/40 Import PCRE release 6.7 (fixes some bugs).
PH/41 Add bitwise logical operations to eval (courtesy Brad Jorsch).
PH/42 Give an error if -q is specified more than once.
PH/43 Renamed the variables $interface_address and $interface_port as
$received_ip_address and $received_port, to make it clear that these
values apply to message reception, and not to the outgoing interface when
a message is delivered. (The old names remain recognized, of course.)
PH/44 There was no timeout on the connect() call when using a Unix domain
socket in the ${readsocket expansion. There now is.
PH/45 Applied a modified version of Brad Jorsch's patch to allow "message" to
be meaningful with "accept".
SC/01 Eximstats V1.43
Bug fix for V1.42 with -h0 specified. Spotted by Chris Lear.
SC/02 Eximstats V1.44
Use a glob alias rather than an array ref in the generated
parser. This improves both readability and performance.
SC/03 Eximstats V1.45 (Marco Gaiarin / Steve Campbell)
Collect SpamAssassin and rejection statistics.
Don't display local sender or destination tables unless
there is data to show.
Added average volumes into the top table text output.
SC/04 Eximstats V1.46
Collect data on the number of addresses (recipients)
as well as the number of messages.
SC/05 Eximstats V1.47
Added 'Message too big' to the list of mail rejection
reasons (thanks to Marco Gaiarin).
SC/06 Eximstats V1.48
Mainlog lines which have GMT offsets and are too short to
have a flag are now skipped.
SC/07 Eximstats V1.49 (Alain Williams)
Added the -emptyok flag.
SC/08 Eximstats V1.50
Fixes for obtaining the IP address from reject messages.
JJ/03 exipick.20061117.2, made header handling as similar to exim as possible
(added [br]h_ prefixes, implemented RFC2047 decoding. Fixed
whitesspace changes from 4.64-PH/27
JJ/04 exipick.20061117.2, fixed format and added $message_headers_raw to
match 4.64-PH/13
JJ/05 exipick.20061117.2, bug fixes (error out sooner when invalid criteria
are found, allow negative numbers in numeric criteria)
JJ/06 exipick.20061117.2, added new $message_body_missing variable
JJ/07 exipick.20061117.2, added $received_ip_address and $received_port
to match changes made in 4.64-PH/43
PH/46 Applied Jori Hamalainen's patch to add features to exiqsumm.
PH/47 Put in an explicit test for a DNS lookup of an address record where the
"domain" is actually an IP address, and force a failure. This locks out
those revolvers/nameservers that support "A-for-A" lookups, in
contravention of the specifications.
PH/48 When a host name was looked up from an IP address, and the subsequent
forward lookup of the name timed out, the host name was left in
$sender_host_name, contrary to the specification.
PH/49 Although default lookup types such as lsearch* or cdb*@ have always been
restricted to single-key lookups, Exim was not diagnosing an error if
* or *@ was used with a query-style lookup.
PH/50 Increased the value of DH_BITS in tls-gnu.c from 768 to 1024.
MH/01 local_scan ABI version incremented to 1.1. It should have been updated
long ago, but noone interested enough thought of it. Let's just say that
the "1.1" means that there are some new functions that weren't there at
some point in the past.
PH/51 Error processing for expansion failure of helo_data from an smtp
transport during callout processing was broken.
PH/52 Applied John Jetmore's patch to allow tls-on-connect and STARTTLS to be
tested/used via the -bh/-bhc/-bs options.
PH/53 Added missing "#include <time.h>" to pcre/pcretest.c (this was a PCRE
bug, fixed in subsequent PCRE releases).
PH/54 Applied Robert Bannocks' patch to avoid a problem with references that
arises when using the Solaris LDAP libraries (but not with OpenLDAP).
PH/55 Check for a ridiculously long file name in exim_dbmbuild.
INSTALLATION_DIRS, as well as all occurrences of ${PREFIX}/man with
${PREFIX}/${PKGMANDIR}.
Fixes PR 35265, although I did not use the patch provided therein.
If you've had problems with getting errors about index files
sometimes being corrupted, please try if this release fixes it. If
you've reported any bugs that this release hasn't fixed, please
report them again so I know they still didn't get fixed and that I
didn't forget them.
* IMAP: When trying to fetch an already expunged message, Dovecot used
to just disconnect client. Now it instead replies with dummy NIL
data.
* Priority numbers in plugin names have changed. If you're installing
from source, you should delete the existing plugin files before
installing the new ones, otherwise you'll get errors.
* Maildir: We're using rename() to move files from tmp/ to new/ now.
See http://wiki.dovecot.org/MailboxFormat/Maildir -> "Issues with
the specification" for reasoning why this is safe. This makes saving
mails faster, and also makes Dovecot usable with Mac OS X's HFS+
(after you also set dotlock_use_excl=yes, see below).
+ Added dotlock_use_excl setting. If enabled, dotlocks are created
directly using O_EXCL flag, instead of by creating a temporary file
which is hardlinked. O_EXCL is faster, but may not work with NFS.
+ If Dovecot crashes with Linux or Solaris, it'll log a
"Raw backtrace". It's worse than gdb's backtrace, but better than
nothing.
+ Added maildir_copy_preserve_filename=yes setting.
+ Added a lazy-expunge plugin to allow users to unexpunge their mails.
+ maildir quota: Added ignore setting to maildir quota, which allows
ignoring quota in Trash mailbox.
+ dict quota: If dictionary doesn't yet contain the quota, calculate
it by going through all the mails in all the mailboxes.
+ login_log_format_elements: Added %a=local port and %b=remote port
+ Added -i and -o options to rawlog to restrict logging only to
input or output.
- Doing a STATUS command for a selected mailbox (not a recommended
IMAP client behavior) caused Dovecot to sync the mailbox silently.
This could have lost eg. EXPUNGE events from clients, causing them
to use wrong sequence numbers.
- deliver was treating boolean settings set to "no" as if they were
"yes" (they were supposed to be commented out for "no")
- Running "dovecot" with -a or -n option while Dovecot was running
deleted all authentication sockets, which caused all the future
logins to fail.
- maildir: RENAME and DELETE didn't touch control directory if it was
different from maildir or index dir.
- We treated internal userdb lookup errors as "user unknown" errors.
In such situations this caused deliver to think the user didn't
exist and the mail get bounced.
- pam: Setting cache_key crashed
- shared maildir: dovecot-keywords file's mode wasn't taken from
dovecot-shared file.
- dovecotpw wasn't working with PowerPC
greetdelay introduces a small delay before an SMTP greeting. It can
also optionally enforce RFC 2821's recommendation that SMTP clients
not send any commands before receiving the greeting message.
With a delay of 30s it has done me a world of good.
* Update qregex patch (PR pkg/34760) to 20060423:
- qregex adds the matched regex pattern to its log entries if the
LOGREGEX environment variable is set.
* Update realrcptto patch to 20061210:
- Logging uses substdio_puts() and substdio_flush() instead of
substdio_putsflush(). This makes log entries less likely to be
interleaved. Thanks to Matthew Dempsky for finding this.
- For QMAILRRTENYALL, use error code 554 after DATA, not 550.
Thanks to ... sorry, I lost track of who found this.
- Log stat() errors for .qmail files. Thanks to Chris Bensend for
suggesting this.
* Update tls-smtpauth combined patch to 20060105. TLS changes:
- bug: qmail-remote loops on malformed server response (B. Shupp,
A. Meltzer)
- no STARTTLS advertised when control/servercert.pem absent (Jason
Haar)
- control/notlshosts (Albert Weichselbraun)
- control/tlshosts/exhaustivelist
- scripts honor conf-users (Sven Verdoolaege)
- strerror declaration in tls.c compile problem (Renato Botelho,
Bill Shupp)
- chown uid.gid deprecated, should be uid:gid (Bill Shupp)
SMTP AUTH changes:
- includes the evaluation of the 'Auth' and the 'Size' parameter
in the 'Mail From:' command.
- uses DJB functions to copy FDs.
- corrects some minor mistakes displaying the 'Auth' userid.
- uses keyword "ESMTPA" in Received header in case of authentication
to comply with RFC 3848.
pkgsrc changes:
* Note SPECIAL_PERMS on qmail-queue binary (from dsainty@).
- Fixed text files to not overwrite the "text/top" and "text/bottom"
files unconditionally. Also rewrote places where the filename was
duplicated in the tags.
- (Un)subscribe requests initiated and confirmed by a moderator are now
marked in the Log as "+mod" or "-mod". This is accomplished by the
addition of another pair of subscribe/unsubscribe confirmation
commands ("rc.cookie" and "wc.cookie") to ezmlm-manage to
differentiate between moderated (un)subscribe requests and
(un)subscribe requests iniated and confirmed by a moderator.
2.3.0 provides the new printing function and new mail notification by
the tray icon. 2.3.0 also includes various usability improvements.
In Win32 version, the included GTK+ library has been updated to the
latest 2.10.6, and it introduces the improvements of usability and
bugfixes.
