I've finally read all the mails in the mailing list and in my INBOX. If
I haven't replied to some of your mail, please resend it.
Remember that since 1.0.rc9 release dovecot.index.cache files will get
rebuilt in 64bit systems, and it's probably better to delete them
manually so you don't get errors in log files.
There are only a couple of issues left in my v1.0-TODO list:
- Master process appears to be leaking log fds with kqueue. Could
someone again give me access to a system where this happens?
- Login process problems. How well does it work now? Hopefully well
enough that v1.0 could be released.
- LDAP authentication is leaking memory? Can anyone confirm this? Even
better, can someone figure out what exactly is leaking? :) Not a v1.0
blocker though.
I think v1.0 will be released once no-one has reported any major
problems for a Dovecot release in 2-4 weeks. I think login process
handling is the only potentially major problem left.
There are a few patches from people that I haven't forgotten, but I've
decided not to put them into v1.0 anymore:
- Filesystem quota group. I don't think it's that important feature,
and it might break something.
- HFS+ hardlink avoiding
- Managesieve
- vmailmgr support
And finally the changes in this release:
* When matching allowed_nets IPs, convert IPv6-mapped-IPv4 addresses
to actual IPv4 addresses first.
+ IMAP: Try to avoid sending duplicate/useless message flag updates
+ Added support for non-plaintext authentication for vpopmail if it
returns plaintext passwords. Based on patch by Remi Gacogne.
+ Added %D modified to return "sub.domain.org" as
"sub,dc=domain,dc=org" (for LDAP queries). Patch by Andrey Panin.
- rc9 broke cache files in 64bit systems
- deliver works now with mail_chroot
- auth cache didn't work properly with multiple passdbs
- Fixes to handling CRLF linefeeds in mboxes.
Most importantly this should fix the login process problems that people
have been reporting. There were also some bugs in the proxying feature.
Also note the 64bit change in dovecot.index.cache files. Unless you
delete dovecot.index.cache files manually, you'll these kind of error
messages into your logs:
Error: Corrupted index cache file ...dovecot.index.cache:
registered field date.sent size changed
They'll get fixed automatically of course, but it might be a bit
annoying to see them.
* 64bit systems: dovecot.index.cache file will be rebuilt because
some time fields have been changed from 64bit fields to 32bit
fields. Now the same cache file can be used in both 32bit and
64bit systems without it being rebuilt.
* Added libmysqlclient workaround to conflicting sha1_result symbol,
which caused Dovecot to fail logging into MySQL.
+ dovecot.index.cache file opening is delayed until it's actually
needed. This reduces disk accesses a bit with eg. STATUS commands.
+ auth_cache: Try to handle changing passwords automatically: If
password verification fails, but the last one had succeeded, don't
use the cache. This works only with plaintext auth.
- dovecot.index.cache: We didn't properly detect if some fields were
different length than we expected, which caused assert crashes
- Lots of fixes to login/master process handling
- mbox: Fixed a bug causing "X-IMAPbase uid-last unexpectedly lost
in mbox file" errors, and possibly others.
I've still over 200 mails unread in the mailing list, and important
things left in TODO. This release is an improvement over rc7 anyway,
hopefully I'll have time to fix the rest soon.
* GSSAPI: Changed POP3 service name to "pop", which is what the
standard says
* "mbox:/var/mail/%u" no longer works as the mail location. You'll
have to specify the mail root explicitly, just like the examples
always have: "mbox:~/mail:INBOX=/var/mail/%u"
+ SHA1, LDAP-MD5, PLAIN-MD5, PLAIN-MD4: The password can be now either
hex or base64 encoded. The encoding is detected automatically based
on the password string length.
+ Allow running only Dovecot master and dovecot-auth processes with
protocols=none setting
+ deliver: -f <envelope sender> parameter can be used to set mbox
From_-line's sender address
+ deliver: Log all mail saves and failures
+ Tru64 SIA passdb support. Patch by Simon L Jackson.
- INBOX was listed twice in mailbox list if namespace prefix was used
- INBOX-prefixed namespaces were a bit broken
- kqueue: Fix 100% CPU usage
- deliver: Duplicate storage was a bit broken
- dictionary code was broken (ie. dict quota)
- SIGHUP caused crashes sometimes
> Can everyone now agree that there are no more hangs? :)
>
> * Require that Dovecot master process's version number matches the
> child process's, unless version_ignore=yes. Usually it's an
> accidental installation problem if the version numbers don't match.
> * Maildir: Create maildirfolder file when creating new maildirs.
>
> + ldap+prefetch: Use global uid/gid settings if LDAP query doesn't
> return them
> + %variables: Negative offsets count from the end of the string.
> Patch by Johannes Berg.
> - kqueue ioloop code rewrite
> - notify=kqueue might have caused connection hangs sometimes
> - deliver: If message body contained a valid mbox From_ line, it
> and the rest of the message was skipped.
> - mbox: We got into infinite loops if trying to open a 2 byte sized
> file as mbox.
> - Don't crash with ssl_disable=yes
> - quota plugin caused compiling problems with some OSes
> - mbox: After saving a mail to a synced mbox, we lost the sync which
> caused worse performance
>
> I think my v1.0 TODO list is:
>
> - avoid duplicate flag change notifications, or in case the message is
> also expunged don't bother notifying its flag changes at all
> (shouldn't be hard)
> - HFS+ avoid-hardlinks patch?
> - 32bit -> 64bit upgrade still doesn't work without assert-crashing:
> mail-cache-transaction.c: line 709 (mail_cache_add): assertion failed:
> (fixed_size == (unsigned int)-1 || fixed_size == data_size)
> - Courier-compatible INBOX. namespace gives "invalid namespace" errors
>
> I'm still not sure about the last one though. Could someone give me
> specific commands that clients send that causes it?
>
> As for the 32bit -> 64bit upgrade fix, could someone give me access to a
> x86-64 machine for a while to test it out?
pkgsrc option for kqueue support. Hence, remove --with-ioloop=best again, and
enable the "kqueue" option by default on *BSD platforms (as already suggested
by grant when he first added the option). The pkg's default behaviour does
not change, so don't bump PKGREVISION.
I/O loop method was changed from kqueue(2) to poll(2) in 1.0rc2, which has
been reported to cause problems on *BSD. --with-ioloop=best selects kqueue
on platforms that implement it (*BSD), and poll on others. You can use
"dovecot --build-options" to see what your binary has been compiled with.
- Use (the newly defined) SSLDIR so we use .../certs & .../private
rather than .../certs/certs and .../certs/private
- Update ssl_cert_file & ssl_key_file in example dovecot.conf to match SSLDIR
- Update mkcert.sh to also match SSLDIR
Back to rc1's SSL proxying code with some improvements, which hopefully
now makes everyone happy.
I'm seeing all kinds of problems with namespace prefixes and LIST code.
I guess it would be important to fix it before v1.0 since it makes
upgrades from other servers easier..
I also tried looking into the "Unknown namespace" problems that happens
if you try to create only one namespace with "INBOX." prefix. I couldn't
really figure out what causes those errors. Could someone show me what
commands the client tries to use which causes it?
* Removed login_max_logging_users setting since it was somewhat weird
in how it worked. Added login_max_connections to replace it with
login_process_per_connection=no, and with =yes its functionality
is now within login_max_processes_count.
+ Added --with-linux-quota configure option to specify which Linux
quota version to use, in case it's not correct in sys/quota.h.
Usually used as --with-linux-quota=2
+ acl plugins: If .DEFAULT file exists in global ACL root directory,
use it as the default ACLs for all mailboxes.
- Fixes to login process handling, especially with
login_process_per_connection=no.
- Back to the original SSL proxy code but with one small fix, which
hopefully fixes the occational hangs with it
- Several fixes to handling LIST command more correctly.
- SSL connections hanged sometimes, especially when saving messages.
- mbox: Mail bodies were saved with CR+LF linefeeds
- Mail forwarding was broken with deliver/Sieve
- dbox fixes. Might actually be usable now.
- Index file handling fixes with keywords
- Cache file was incorrectly used in some situations, which probably
caused problems sometimes.
- Maildir++ quota: Don't count "." and ".." directory sizes to quota.
After rewriting maildirsize file keep its fd open so that we can
later update it. Patch by Alexander Zagrebin
* disable_plaintext_auth=yes: Removed hardcoded 127.* and ::1 IP
checks. Now we just assume that the connection is secure if the
local IP matches the remote IP address.
* SSL code rewrite which hopefully makes it work better than before.
Seems to work correctly, but if you suddently have trouble with SSL
connections this is likely the reason.
+ verbose_ssl=yes: Log also SSL alerts and BIO errors
- If namespace's location field wasn't set, the default location
was supposed to be used but it wasn't.
- When copying ssl-parameters.dat file from /var/lib to /var/run its
permissions went wrong if it couldn't be copied with hard linking.
- Fixed filesystem quota plugin to work with BSDs.
- Maildir: Saving mails didn't work if quota plugin was enabled (again)
- Maildir: Messages' received time wasn't saved properly when
saving/copying multiple messages at a time. Also if using quota
plugin the S= size was only set for the first saved file, and even
that was wrong.
- passdb passwd-file: Don't require valid uid/gid fields if file
isn't also being used as a userdb.
- PostgreSQL: Handle failures better so that there won't be
"invalid fd" errors in logs.
- Don't try to expunge messages if the mailbox is read-only. It'll
just cause our index files to go out of sync with the real
mailbox and cause errors.
- ANONYMOUS authentication mechanism couldn't work because
anonymous_username setting wasn't passed from master process.
* PAM: If user's password is expired, give "Password expired" error
message to the user. Now actually working thanks to Vaidas Pilkauskas
* Relicensed dovecot-auth, lib-sql and lib-ntlm to MIT license. See
COPYING file for more information.
* Abuse prevention: When creating a mailbox, limit the number of
hierarchies (up to 20) and the length of the mailbox name within
a hierarchy (up to 200 characters).
* mbox: If saved mail doesn't end with LF, add it ourself so that the
mails always have one empty line before the next From-line.
+ Added --with-statedir configure option which defaults to
$localstatedir/lib/dovecot. ssl-parameters.dat is permanently
stored in that directory and is copied to login_dirs from there.
+ IMAP: Support SASL-IR extension (SASL initial response)
+ Support initial SASL response with LOGIN mechanism. Patch by Anders
Karlsson
+ Added PLAIN-MD4 password scheme. Patch by Andrey Panin.
+ Added support for XFS disk quotas. Patch by Pawel Jarosz
+ If another process deletes the opened mailbox, try to handle it
without writing errors to log file. Handles the most common cases.
+ Added TLS support for LDAP if the library supports it.
- SEARCH command was more or less broken with OR and NOT conditions
- Dovecot corrupted mbox files which had CR+LF linefeeds in headers
- MySQL code could have crashed while escaping strings
- MD4 code with NTLM authentication was broken with 64bit systems.
Patch by Andrey Panin
- Plugin loading was broken in some OSes (eg. FreeBSD)
- Several fixes to handling empty values in configuration file
- Several fixes to dictionary quota backend and dict server.
Also changed how they're configured.
- deliver: Fixed plugin handling settings
- mbox_min_index_size handling was somewhat broken
- passdb passwd-file: extra_args field wasn't read unless the file
was also used as userdb.
C++ and Fortran by default, but doing so does not bomb configure or build,
so there's no harm in the pkgsrc warnings.)
Because this changes the dependencies on systems using pkgsrc-supplied gcc,
bump PKGREVISION. <sigh>
Fixes a lot of bugs. The next release will be the first "release
candidate" instead of a beta.
* PAM: Don't call pam_setcred() unless setcred=yes PAM passdb
argument was given.
* Moved around settings in dovecot-example.conf to be in more logical
groups.
+ Local delivery agent (deliver binary) works again.
+ LDAP: Added support for SASL binding. Patch by Geert Jansen
+ ssl_verify_client_cert: Check CRLs. If auth_verbose=yes, log
invalid sent certificates. If verbose_ssl=yes, log even the valid
certificates. When using the username from the certificate, use
CommonName. Based on patch by HenkJan Wolthuis
+ PAM: Set PAM_TTY which is needed by some PAM plugins
+ dovecot --exec-mail ext <binary path> can now be used to start
binaries which want dovecot.conf to be read, for example the
convert-tool.
- Expunging needed to be done twice if client used STORE +FLAGS.SILENT
command to set the \Deleted flags
- Added sql_escape_string() to lib-sql API and use it instead of
normal \-escaping.
- ACL plugin fixes
- DIGEST-MD5: Trying to use subsequent authentication crashed
dovecot-auth.
- Fetching BODY when BODYSTRUCTURE was already cached caused the
reply to be broken in some cases
- Lots of fixes for index file handling
- dbox fixes and changes
- mbox syncing broke if some extraneous/broken headers were removed
(eg. extra X-IMAPbase headers in mails)
- Running Dovecot from inetd work now properly with POP3
- Quota plugin fixes for calculating the quota correctly
changes since 1.0beta7:
* Fixed a security hole with mbox: "1 LIST .. *" command could
list all directories and files under the mbox root directory, so
if your mails were stored in eg. /var/mail/%u/ directory, the
command would list everything under /var/mail.
+ Unless nfs_check=no or mmap_disable=yes, check for the first login
if the user's index directory exists in NFS mount. If so, refuse to
run. This is done only on first login to avoid constant extra
overhead.
+ If we have plugins set and imap_capability unset, figure out the
IMAP capabilities automatically by running imap binary at startup.
The generated capability list isn't updated until Dovecot is
restarted completely, so if you add or remove IMAP plugins you
should restart. If you have problems related to this, set
imap_capabilities setting manually to work around it.
+ Added auth_username_format setting
- pop3_lock_session setting wasn't really working
- Lots of fixes related to quota handling. It's still not working
perfectly though.
- Lots of index handling fixes, especially with mmap_disable=yes
- Maildir: saving mails could have sometimes caused "Append with UID
n, but next_uid = m" errors
- flock() locking never timeouted because ignoring SIGALRM caused the
system call just to be restarted when SIGALRM occurred (probably not
with all OSes though?)
- kqueue: Fixed "Unrecognized event". Patch by Vaclav Haisman
+ Added shutdown_clients setting to control if existing imap/pop3 processes
should be killed when master is.
- Master login fixes, PLAIN authentication was still broken..
v1.0.beta6 2006-04-12
* The login and master usernames were reversed when using
master_user_separator (now the order is UW-IMAP compatible).
* Killing dovecot master process now kills all IMAP and POP3
processes also.
+ -a parameter to dovecot prints now all settings that Dovecot uses.
-n prints all settings that are different from defaults.
+ Added pop3_lock_session setting
+ %M modifier returns string's MD5 sum. Patch by Ben Winslow
- PLAIN SASL authentication wasn't working properly, causing failed
logins with some clients (broken in beta4)
- Fixes to Maildir++ quota, should actually work now
- Don't crash if passwd-file has entries without passwords
(eg. deny=yes databases)
- Fixed prefetch userdb to work nicely with other userdbs
- If master process runs out of file descriptors, don't go to
infinite loop (unlikely to have happened unless the OS's default
fd limit was too low)
- Fixed non-plaintext password lookups from LDAP. Patch by Lior Okman
- %U modifier was actually lowercasing the string. Patch by
Ben Winslow
which were fixed again in beta5.
patch-ac and patch-ad were taken from CVS and are not needed anymore.
Changes in Dovecot 1.0beta4:
* Changed the default lock_method back to fcntl. Apparently flock
gives problems with some systems.
* mbox: mailboxes beginning with '.' are now also listed
* Replaced mail_use_modules and mail_modules settings with mail_plugins
and mail_plugin_dir. Now instead of loading all plugins from the
directory, you'll have to give a list of plugins to load. If the
plugin couldn't be loaded, the process exits instead of just
ignoring the problem (this is important with ACL plugin).
+ Added support for "master users" who can log in as other people.
The master username can be given either in authorization ID
string with SASL PLAIN mechanism or by setting
auth_master_user_separator and giving it within the normal username
string.
+ Added ACL plugin with ACL file backend. This however doesn't mean
that there yet exists a proper shared folder support. If master user
logged in as someone else, the ACLs are checked as the master user.
+ Added some Dovecot extensions to checkpassword passdb, see ChangeLog
+ Updated passwd-file format to allow specifying any key=value fields
+ Maildir++ quota support and several quota fixes
+ passdb supporting extra fields: Added "allow_nets" option which takes
a comma separated list of IPs/networks where to allow user to log in.
+ NFS: Handle ESTALE errors the best way we can
+ IMAP now writes to log when client disconnects
+ In shared mailboxes (if dovecot-shared file exists) \Seen flags are
now kept only in index files, so as long as each user has a separate
index file they have separate \Seen flags.
- Fixes to DIGEST-MD5 realm handling so it works with more clients
- BODYSTRUCTURE -> BODY conversion from cache file was broken with
mails containing message/rfc822 parts.
- Fixed several memory leaks
- We could have sent client FETCH notifications about messages before
telling about them with EXISTS
- Compiling fixes for Solaris and some other OSes
- Fixed problem with internal timeout handling code, which caused eg.
outlook-idle workaround to break.
- If /dev/urandom didn't exist, we didn't seed OpenSSL's random number
generator properly. Patch by Vilmos Nebehaj.
- Maildir: Recent flags weren't always immediately removed from mails
when mailbox was opened.
- Several changes to SSL proxying code, hopefully making it work
better.
Changes in Dovecot 1.0beta5:
- Beta4's SSL proxying rewrite worked worse than I thought.
Reverted it back to original code.
- Filesystem quota plugin now looks up the mount path correctly.
(we've patched them). Wanted to wait with this for dovecot1.0beta4 but this
takes longer than I thought. Not worth bumping PKGREVISION, IMO.
Ok with tv.
default at the next version bump?)
- set SSL_{CFLAGS,LIBS} when calling configure script so they are
found correctly on Solaris, and make the configure script do what it
claims and ignore pkg-config when these are set.
- add lib-sql Makefile patch from Dovecot CVS so this builds correctly
when no SQL auth support is built.
as was done in wip/dovecot-nightly. That comment is misleading with
pkgsrc, since those paths are not what pkgsrc encodes into the conf file.
Bump PKGREVISION (unfortunate, but there will be another beta pretty
soon anyway).
* Dotlock code changed to timeout faster in some situations when
the lock file is old.
+ Added support for loading SQL drivers dynamically (see INSTALL file for how
to build them)
+ Keywords are stored to dboxes, and other dbox improvements.
+ dict-sql could actually work now, making quota-in-sql-database possibly
working now (not fully tested)
+ Added mail storage conversion plugin to convert automatically from one
mailbox format to another while user logs in. Doesn't preserve UIDVALIDITY/
UIDs though.
+ Added plugin { .. } section to dovecot.conf for passing parameters to
plugins (see dovecot-example.conf).
+ Added ssl-build-param binary which is used to generate ssl-parameters.dat.
Main dovecot binary doesn't anymore link to SSL libraries, and this also
makes the process title be clearer about why the process is eating all the
CPU.
- Fix building without OpenSSL
- Fixed memory leak in MySQL driver
- Fixes to checkpassword
- Broken Content-Length header could have broken mbox opening
- Fixed potential hangs after APPEND command
- Fixed potential crashes in dovecot-auth and imap/pop3-login
- zlib plugin now links with -lz so it could actually work
- kqueue fixes by Vaclav Haisman
Also, change the package's name to 1.0beta3 (from 1.0b2), since "beta" is what
pkgsrc recognizes according to pkg_info(1). (sorry tv, I thought it was "b".)
correct a mistake in my previous commit:
- add dependency on zlib
- add option for GNU TLS as an alternative to OpenSSL
- drop SASL option, it's no longer supported
- explicitly disable some options with --disable-*
- "s,/usr/pkg,@PREFIX@," in patch-ab -- I accidentally diffed it after the SUBST stage
- change SUBST_STAGE from post-patch to pre-configure to avoid this in the future
- bump PKGREVISION for all this
Thanks to tv@.
recommended by the Dovecot author (the 0.99.x series are deprecated).
Major changes:
v1.0.beta2 2006-01-22 Timo Sirainen <tss@iki.fi>
+ Added SQLite support. Patch by Jakob Hirsch.
+ Added auth_debug_passwords setting. If it's not enabled, hide all
password strings from logs.
+ Added mail_cache_min_mail_count and mbox_min_index_size settings
which can be used to make Dovecot do less disk writes in small
mailboxes where they don't benefit that much.
+ Added --build-ssl-parameters parameter to dovecot binary
- SSL parameters were being regenerated every 10 minutes, although not
with all systems.
- Fixed dovecot-auth crashing at startup. Happened only with some
specific compilers.
- base_dir was supposed to be set world-readable, not world-writable
v1.0.beta1 2006-01-16 Timo Sirainen <tss@iki.fi>
* Almost a complete rewrite since 0.99.x, but some of the major
changes are:
+ Index file code rewritten to do less disk I/O, wait locks less and in
generate be smarter. They also support being in clustered filesystems
and NFS support is mostly working also.
+ Mail caching is smarter. Only the data that client requests is
cached. Before Dovecot opened and cached all mails when mailbox was
opened the first time, which was slow.
+ Mbox handling code rewritten to be much faster, safer and correct
+ New authentication mechanisms: APOP, GSSAPI, LOGIN, NTLM and RPA.
+ LDAP supports authentication binds
+ Authentication server can cache password database lookups
+ Support for multiple authentication databases
+ Namespace configuration
+ Dovecot works with shared
- Add an option for sqlite support.
- Take over maintainership.
All suggested (and ok'ed) by xtraeme.
backslashes anymore. A single backslash is enough. Changed the
definition in all affected packages. For those that are not caught, an
additional check is placed into bsd.pkginstall.mk.
bump PKGREVISION since this is a change from the previous behaviour,
where PAM support was autodetected. it must now be explicitly enabled
via PKG_OPTIONS.dovecot.
And always is defined as share/examples/rc.d
which was the default before.
This rc.d scripts are not automatically added to PLISTs now also.
So add to each corresponding PLIST as required.
This was discussed on tech-pkg in late January and late April.
Todo: remove the RCD_SCRIPTS_EXAMPLEDIR uses in MESSAGES and elsewhere
and remove the RCD_SCRIPTS_EXAMPLEDIR itself.
Changes:
o Message address fields are now parsed differently, fixing some
issues with spaces. Affects only clients which use FETCH ENVELOPE
command.
o Message MIME parser was somewhat broken with missing MIME boundaries
o mbox: Don't allow X-UID headers in mails to override the UIDs we
would otherwise set. Too large values can break some clients and
cause other trouble.
o passwd-file userdb wasn't working
o PAM crashed with 64bit systems
o non-SSL inetd startup wasn't working
o If UID FETCH notices and skips an expunged message, don't return
a NO reply. It's not needed and only makes clients give error
messages.
