amavisd-new-2.6.2 release notes
MAIN NEW FEATURES SUMMARY
- bounce killer: improved detection of nonstandard bounces;
- bounces to be killed no longer waste SpamAssassin time;
- tool to convert dkim-filter keysfile into amavisd configuration;
- compatibility with SpamAssassin 3.3 (CVS head) regained;
- rewritten and expanded documentation section on DKIM signing and
verification in amavisd-new-docs.html;
COMPATIBILITY WITH 2.6.1
- apart from small differences in logging and notifications, the
version 2.6.2 is compatible with 2.6.1, with its configuration file
and its environment;
- virus scanner entries were updated (as described below, most notably by
adding a regexp flag m), so be sure to update existing configuration file;
updated virus scanner entries can be used with 2.6.1 too;
- the %sql_clause default has changed in detail (see below), if its value
is overridden in a configuration file the setting may need updating;
See full release notes:
http://www.ijs.si/software/amavisd/release-notes.txt
Based on maintainer update request in PR 39196.
There are a lot of changes and some incompatabilities with 2.5.3
(current version in pkgsrc) particularly as respects SQL schema.
Consult vendor's releases notes for more detail:
http://www.ijs.si/software/amavisd/release-notes.txt
From the amavisd-new-2.5.3 release notes:
BUG FIXES
- fix parsing a SMTP status response from MTA when releasing from a
quarantine, when a MTA response did not include an enhanced status
code (RFC 3463) (such as with old versions of Postfix); a parsing
failure resulted in attribute "setreply=450 4.5.0 Unexpected:..."
in an AM.PDP protocol response, even though a release was successful;
reported by Ron Miller, John M. Kupski, investigated by Tony Caduto
and Jeremy Fowler;
- change parsing of addresses in From, To, and Cc header fields, avoiding
complex Perl regular expressions which could crash a process on certain
degenerate cases of these header fields; thanks for detailed problem
reports to Carsten Lührs and Attila Nagy;
- completely rewritten parsing of Received header field to work around a
Perl regular expression problem which could crash a process on certain
degenerate cases of mail header fields; problem reported by Thomas Gelf;
- harden to some extent regular expressions in parse_message_id to cope
better with degenerate cases of header fields carrying message-id;
- sanitize 8-bit characters in In-Reply-To and References header fields
before using them in Pen Pals SQL lookups to avoid UTF-8 errors like:
penpals_check FAILED: sql exec: err=7, 22021, DBD::Pg::st execute failed:
ERROR: invalid byte sequence for encoding "UTF8": 0xd864
- when turning an infection report into a spam report, avoid adding newly
discovered virus names (i.e. fraud names) to a cached list if these names
are already listed; previously the list would just grow on each passage
through a cache, leading to unsightly long lists of spam tests in a
report; based on a patch by Henrik Krohns;
- fix diagnostics when an invalid command line argument is given;
OTHER
- reduce log clutter when certain Perl modules are loaded late, i.e. after
chrooting and daemonizing, but still before a fork; now only issue one
log entry by a parent process: "extra modules loaded after daemonizing: ";
- slightly relax mail address syntax in subroutine split_address;
- fetch additional information (tags) from SpamAssassin: TESTS, ASN,
ASNCIDR, DKIMDOMAIN and DKIMIDENTITY, making them available through a macro
'supplementary_info' (if a version of SpamAssassin in use provides them);
- updated DKIM section in amavisd-new-docs.html, removing the historical
DomainKeys milter from examples;
- declared a dummy subroutine dkim_key() and new dummy configuration
variables @dkim_signature_options_bysender_maps, %signed_header_fields,
$reputation_factor, @signer_reputation_maps and $sql_partition_tag, members
of policy banks, in preparation for 2.6.0 - declared now for improved
downgrade compatibility of 2.6.0 configuration files, if need arises.
BUG FIXES
- in a milter setup log_id was left undefined, which resulted in log lines
without id, and a SQL constraint violation "Column 'am_id' cannot be null"
when logging to SQL was enabled. The bug was introduced in 2.5.1;
problem reported by Martin Svensson;
- suppress a quarantining attempt if the message also needs to be archived
to the same location (same sql key or same local filename);
reported by Wazir Shpoon;
- adjust $socketname in amavisd-release to match its default counterpart
in amavisd (i.e. /var/amavis/amavisd.sock); reported by Stanley Appel;
And more... please review the Changelog file.
This release adds checking of a number of archive members to improve
protection from runaway dearchivers.
It fixes SQL quarantining of mail with a null sender, and recognizes
PostgreSQL error S8006.
Parsing of invalid header has been improved.
Calling 'finish' on a SA message object was added.
A nonstandard SMTP status code 254 is no longer used, and enforcing
of option 8BITMIME is avoid even on 8-bit contents.
Checking of eval status was improved to recognize additional failure
modes.
Disabling of MIME decoding and invoking of a file(1) utility has been
made possible. An AV entry for ArcaVir was added.
Patch provided by MAINTAINER, Julian Dunn in PR 35578.
---------------------------------------------------------------------------
January 30, 2007
amavisd-new-2.4.5 release notes
SECURITY
- Recommended version of Convert::UUlib is 1.08 or higher
to avoid processing of uninitialized data containing 'random' garbage.
Note that a security hole in uulib which comes with Convert::UUlib 1.04
and older is now (as of 2006-12-05) known to be exploitable:
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2005-1349
credits to Jean-Sebastien Guay-Leroux;
- p0f-analyzer.pl will no longer reply to queries coming from low-numbered
UDP ports below 1024 or from nfsd port 2049, and will ignore queries
with nonce longer than 1024 character or containing characters outside
of \040-\177 range to limit its usefulness as a potential reflector
for an attacker from internal networks.
INCOMPATIBLE CHANGE WITH 2.4.4
- p0f-analyzer.pl now only binds to a loopback interface by default, instead
of to all interfaces; change $bind_addr in p0f-analyzer.pl to '0.0.0.0'
if p0f-analyzer.pl is running on a different host from amavisd or from
other querying clients; suggested by Shaun T. Erickson and Mario Liehr;
BUG FIXES
- let p0f-analyzer.pl exit when a pipe on stdin is closed (e.g. when p0f
is killed or crashes), instead of entering a tight loop; reported by
Justin Piszcz and Henrik Krohns;
- hard-blacklisting no longer skips quarantining when
$spam_quarantine_cutoff_level is undefined (or is an empty string);
- restart timer after Sophie times out; previously the next attempt
would run with no time limit; reported by Nick Leverton and
Nicklas Bondesson;
- fixed AM.PDP code to always provide smtp-quoted form in angle brackets
in delrcpt and addrcpt attributes of a response, i.e. in the same form
as was received in sender and recipient attributes;
- fix error reporting in open_on_specific_fd when POSIX::dup2 fails;
thanks to Chris (decoder);
- fix signal handling in read_snmp_variables() and register_proc(),
a signal could previously get lost (not re-signaled) if it occurred
within these subroutines;
- fixed get_body_digest which incorrectly determined 7- or 8-bitness
of mail header and body, setting body_type incorrectly (with only
cosmetic ill-effects);
- AM.PDP protocol: ensure proper address form is used in server response
attributes 'delrcpt' and 'addrcpt': the same form should be used as
in 'sender' and 'recipient' attributes. The attribute value syntax is
specified in RFC 2821 as 'Reverse-path' (i.e. smtp-quoted form, enclosed
in <>); previously enclosing angle brackets were missing in a server reply;
- documentation - amavisd.conf-default incorrectly stated that a default
value for $prepend_header_fields_hdridx is 1; actually the default is 0
as correctly indicated in release notes; reported by Jo Rhett;
OTHER
- qmail interfacing notice:
MTA timeout for waiting on results from amavisd should be longer than
$child_timeout (8 minutes by default) with some margin, setting MTA timeout
to 15 or 20 minutes is usual. With qmail however the QMQP code in qmail
has hard-coded timeouts set, 10 seconds for connect and 60 seconds for
read/write. If amavisd processing takes longer than 60 seconds, the MTA
drops connection and retries later, yet amavisd continues processing
and eventually delivers a mail (with each MTA retry), causing repeated
deliveries of the same message. The following patch by Eric Huss on
the www.qmail.org page: http://www.ehuss.org/qmail/qmqpc-timeout.tar.gz
should be applied to qmail when interfacing it to a post-queue content
filter. Problem researched by Nicklas Bondesson;
- better timeout handling in interface code to daemonized virus scanners
like clamd, Sophie, Trophie: allow short time (10 s) for connect and
for sending a request, then allow normal (long) time to collect results;
keep evidence of the initial deadline on retries;
- prefer '7bit' as Content-Transfer-Encoding when attaching original message
or its headers (message/rfc822 or text/rfc822-headers) to DSN or to a
defanged mail, and only specify '8bit' when necessary;
- remove protecting the $ and @ characters in second argument
of a regexp selector macro, it is unnecessary and confusing;
- sanitize Message-ID and Resent-Message-ID header field bodies in
macros %m, %r and header_field by providing angle brackets if missing
to facilitate log parsing (angle brackets are RFC 2822 required syntax
and are semantically not part of a message id);
- updated $map_full_type_to_short_type_re to avoid mapping file(1) result
'MS-DOS executable (built-in)' to types 'exe-ms' and 'exe'; the file(1)
utility generously declares any text file starting with LZ to be a
'MS-DOS executable (built-in)'; thanks to Noel Jones, Jakob Curdes
and Clifton Royston for troubleshooting;
- add X-Spam-* header fields to quarantined mail if spam score is at or
above tag_level. Previously message needed to be recognized as spammy
or spam (tag2 or kill level) in order to receive spam header fields
in quarantined copy. This also makes it more consistent with adding
such header fields to passed mail; suggested by Michael Gaskins;
- add X-Amavis-OS-Fingerprint header field to quarantined mail;
- header field X-Spam-Score in a passed or quarantined mail now reflects
score boost even when SA score is unknown (e.g. when SA was not called),
and reflects white and blacklisting by pushing score to 0 or 64, to
make it consistent with a bar size in X-Spam-Level header field;
- resignal "timed out" after (almost) every eval {} which has no subsequent
call to prolong_timer() to ensure we do not continue running with
disabled timer. Exceptions are DESTROY and END handlers, and code which
handles timer in some other way (e.g. by keeping evidence of a deadline);
- for the purpose of looking up client IP address in @mynetworks_maps,
treat unknown/unavailable IP address as 0.0.0.0; this allows treating
directly submitted mail on the MTA host (not submitted through SMTP) as
coming from IP address 0.0.0.0 (i.e. "This" Network - according to RFC 1700);
Note that this is indistinguishable from other reasons when IP address
is not made available to amavisd, e.g. when smtp_send_xforward_command
option in Postfix smtp service is not enabled, which is why the default
setting of @mynetworks does not include a 0.0.0.0/8 network to prevent
falsely loading a MYNETS policy bank.
One should add 0.0.0.0/8 to a @mynetworks list only when XFORWARD is known
to work and if some software on the MTA host is submitting its mail to MTA
directly, e.g. through a sendmail command, and MYNETS policy bank loading
is needed for proper processing of such mail;
- report a more informative message when a file(1) utility fails to produce
useful results: joins exit status with a parsing report into one message;
thanks to Andres, whose file(1) utility was crashing with SEGV;
- consistency: rearrange implicitly adding $X_HEADER_TAG to a hash
%allowed_added_header_fields so that it is possible to turn off
insertion of $X_HEADER_TAG header field by turning off associated key in
%allowed_added_header_fields even when $X_HEADER_TAG is explicitly defined;
- let %allowed_added_header_fields also control insertion of header fields
into quarantined message;
- amavisd-nanny now displays a title line indicating the semantics of columns;
- Courier patch: ensure the information is stored to newly introduced
recip_addr_smtp and sender_smtp object attributes, which are needed
to preserve pristine address forms for DSN and ORCPT use and for logging;
a patch by Martin Orr;
- qmqpqq (qmail): ensure the information is stored to newly introduced
recip_addr_smtp and sender_smtp object attributes;
- qmail patch now activates line-by-line sending to qmail to avoid qmail bug
('bare LF' reported when CR and LF are separated by a TCP packet boundary);
- tighten a regexp on matching a p0f fingerprint for Windows XP to avoid
matching 'Windows XP SP1+, 2000 SP3'; suggested by Michael Scheidell;
- updated AV entry for CentralCommand Vexira (vascan):
removed hard-coded option '--vdb'; by Brian Wong;
- internal: move code dealing with a SA call to a dedicated
subroutine call_spamassassin;
- internal: provide new routines to collect scalar and structured results
from a subprocess (collect_results, collect_results_structured) and
take advantage of them in decoding, in AV and in dspam interface routines,
unifying code and providing results size sanity limit and consistent
killing of runaway external programs;
- experimental: taking advantage of the above, make it possible to run SA in
a spawned process, requested by setting a new config variable $sa_spawned
to true (it is off by default); benefits are that a mainstream child process
can not be brought down by potential processing problems in SA or its
external modules, and timeouts are handled cleanly by a calling process;
downside is an increase of process count (worst case: doubled), with
corresponding increase in memory footprint, plus about 20 .. 30 ms
of additional processing time for each call to SA;
- added a tuning tip on buffer sizes to README.sql for MySQL with InnoDB,
by Wayne Smith;
- updated URL of Sophie AV scanner;
---------------------------------------------------------------------------
June 27, 2006
amavisd-new-2.4.2 release notes
SUMMARY OF CHANGES:
- new feature: "pen pals soft-whitelisting" lowers spam score of received
replies to a message previously sent by a local user to this address;
- new feature: added command line options to override certain configuration
settings from a config file, see below;
- documentation bug fixes, especially on the use of SQL data type TIMESTAMP;
- zoo decoder interface routine can now use utility unzoo(1) or zoo(1);
---------------------------------------------------------------------------
May 8, 2006
amavisd-new-2.4.1 release notes
INCOMPATIBLE CHANGE WITH 2.4.0:
- notification templates incompatibility with 2.4.0 (but not with versions
2.3.3 or older): major contents category numbers are renumbered due to a
newly inserted category CC_SPAMMY; it affects the use of macro ccat_maj
in templates (one field added), and only affect users which provide
non-default templates based on 2.4.0 templates; older templates (2.3.3
or earlier) are unaffected as they do not use macro ccat_maj;
---------------------------------------------------------------------------
April 3, 2006
amavisd-new-2.4.0 release notes
The most important changes since 2.3.3 at a glance:
Delivery status notifications (DSN) are now supported, both as a SMTP
protocol extension and in notifications. Header fields like X-Amavis
and X-Spam are now prepended to mail header for DomainKeys compatibility.
Configuration variables can be chosen based on mail contents category,
which is now represented explicitly. A built-in macro expander is enhanced,
providing new macros and call types. Added support for passive operating
system fingerprinting with the use of p0f, supplying collected information
as a header field to SpamAssassin. Provide compatibility with Net::Server
0.91 and later.
patch provided by eggert at macvaerk dot dtu dot dk
in PR 31127
changes:
Version 2.3.3 is a maintenance release over 2.3.2. Besides fixing known
problems and providing some optimizations, no new features were added.
If using SpamAssassin older than 3.1, an upgrade of either SA to 3.1,
or an upgrade of amavisd-new to 2.3.3 is recommended.
- privacy: add a safety fuse / workaround around calls to SA to detect
SA's failure (in SA versions before 3.1) to catch a failed exec() in a
forked process, which could produce runaway process clones. See SA bug
report #4370. An incident of a mail copy being delivered to unrelated
recipient reported by Joel Nimety;
- privacy: turn warning into a fatal error when a quarantine ID of a message
requested for a quarantine release does not match the requested mail_id;
- security: require minimal version 1.35 of Compress::Zlib to avoid
vulnerability in the zlib compression library;
- the dsn_cutoff_level should have been ignored if undefined according to
documentation, but was not, causing DSN to be suppressed regardless of
spam level; discovered by Gary V;
- ensure the banned check is not performed if all recipients agree
it is not needed, even in presence of $banned_namepath_re;
undesired behaviour (not strictly incorrect) reported by Joel Nimety;
- missing import of lookup_ip_acl in module Amavis::In::AMCL caused
failure in sendmail milter setup when using the new AM.PDP protocol;
reported by Mic And;
- document and explicitly define handling of syntactically invalid IP address
in lookup_ip_acl: it matches a zero-length-mask net, a constant lookup table,
or a hash entry with an undef key, but no other entries in IP lookup tables;
syntactically invalid IP addresses are now logged;
- fix parsing if IPv6 address in $notify_method and $forward_method in case
of dynamic destination override (the use of '*' in method fields);
- check during startup that $myhostname is a fully qualified domain name
(or 'localhost', if you must), and abort if it isn't, otherwise a non-FQDN
can end up in places where RFC 2822 does not allow it; if uname(3) does not
provide a FQDN, then an assignment to $myhostname must be done explicitly
in amavisd.conf;
- when quarantining to a single file in mbox format the 'From ...' line
needs an English date, regardless of current locale; fixed by globally
setting locale LC_TIME to "C";
- pass on the parameter BODY=8BITMIME on MAIL FROM when submitting to MTA
when original message reception indicated it is needed (RFC 1652).
Note that mail forwarding may now fail if the feeding MTA requests
BODY=8BITMIME SMTP service extension (or just passes data with msb set),
but the MTA on the output side does not allow the use of the BODY parameter
in SMTP. In case of Postfix this may only happen when receiving service
on port 10025 is misconfigured and does not announce ESMTP capability
and support for the SMTP service extension 8BITMIME;
- RFC 2554 requires auth_param to be xtext-encoded addr-spec (no angle
brackets) or "<>", not the xtext-encoded addr-spec enclosed in angle
brackets (when specifying submitter during authentication); fixed;
- apply some sanity limit on collected bad-header samples to ensure that
a grossly broken mail does not unnecessarily fill up memory;
- when sending recipient warnings for viruses, banned files, or bad headers,
recipient address must not be rfc2822-quoted twice; fixed;
- fix interpretation of $defang_all to really imply all; previously it only
affected clean messages;
- in quarantined mail the reported spam score in X-Spam-Status header field
now includes maximum of all by-recipient score boosts (less surprising
when soft-whitelisting through @score_sender_maps is in use); suggested
by Mike Cappella and Gary V;
- when a policy delegation protocol attribute "request" is not "AM.PDP"
(perhaps it is a Postfix policy delegation request) don't attempt to find
and open a mail file;
- do_ascii and do_unarj: set environment variable TMPDIR or a command line
temporary directory option to "$tempdir/parts" instead of $TEMPBASE
to minimize possible pollution of top level directory;
- don't abort even if amavisd.conf returns undef as a final value,
as long as there are no errors reading or interpreting it;
- if during 'amavisd stop' or 'amavisd reload' the old running daemon does
not go away for one minute after sending it a SIGTERM, use a bigger
hammer and send it a SIGKILL; suggested by Sven Riedel;
- extend LDAP lookups to allow multiple search attributes (multiple
occurrences of %m in a query); a patch by Michael Hall (and a similar
one by Matthias Bandemer);
- LDAP lookup on an empty envelope address (e.g. a null return path)
adds another lookup key "<>", as it is difficult if not impossible
to have LDAP attributes with empty string as a value; by Michael Hall;
- LDAP.schema: drop "MUST ( mail )" from objectclass 'amavisAccount';
suggested by Michael Hall;
- updated comments and documentation, most notably the README.chroot;
- contributed file Macintosh.tar.gz updated by Dale Walsh;
COMPATIBILITY
- replaced 'hits=' with 'score=' in inserted X-Spam-Status header field
(and in some internal log entries) for compatibility with a changed
default in SpamAssassin 3.1;
- insert X-Spam-Score header field for compatibility with SA (previously
insertion of this header field was commented-out because the information
is redundant, as the score already appears in X-Spam-Status);
OPTIMIZATION
- speed up sending a mail header or full defanged (rewritten) mail over SMTP
back to MTA by a factor of 4 by buffering header fields into large chunks
to avoid bottleneck in Net::Cmd::datasend, which has lots of overhead for
line-by-line writes. Previously slow writes mostly affected mail messages
with extreme header lengths (such as results of a broken mail loop), or
when delivering defanged messages, particularly at sites with large MTA
mail size limits, sometimes to a point of exceeding timeout limits;
reported by Dominik Weber and Ralf Hildebrandt;
- move subroutine lookup_ip_acl() and associated ip_to_vec() into its own
dedicated new package Amavis::Lookup::IP; provide a constructor to pre-parse
IP lookup tables to speed up IP lookups in lookup_ip_acl; prepare pre-parsed
commonly used IP lookup tables (@mynetworks_maps, @publicnetworks_maps,
@inet_acl);
- optimized reading loop in SMTP DATA state, receiving data is now about
35% faster when mail size limit is not enforced (which is a default);
no speedup when mail size limit _is_ enforced;
- cache results of evaluated macros during a single call to expand(),
as macro calls often come in pairs, like: [?%e||\[%e\] ]
or [? %#T ||, Tests: [%T|,]]; together with the above optimization in
pre-parsed IP lookups it shaves off 25% of time in preparing main log entry;
- set locale LC_TIME to "C" globally, avoid changing and restoring locale
for every log write and when generating RFC2822 timestamps;
- added an optimization note in README.sql about indexes and about
SELECT count(*) in MySQL with InnoDB; investigation by Paolo Cravero;
---------------------------------------------------------------------------
June 29, 2005
amavisd-new-2.3.2 release notes
INCOMPATIBILITY with 2.3.1 and earlier versions:
If running amavisd daemon in chroot please note:
Each child process now opens its own syslog connection or a file descriptor
to a log file, and no longer inherits a connection from its parent.
When running in chroot jail and logging to syslog, the syslog client
routines need syslogd socket to be present in the chroot subtree to be
able to establish a connection with syslogd, otherwise logging output
may be lost. Additional syslogd sockets (to be made available in the
jail) may be requested from the syslogd daemon, see its documentation.
This requirement is equivalent to the requirement of chrooted Postfix
services (see Postfix documentation file BASIC_CONFIGURATION_README).
BUG FIXES since 2.3.1:
- do not enforce $MAXFILES limit during top-level MIME decoding to avoid
tempfailing mail; MIME parts are still counted, so a limit exceeded may
still be reported during subsequent decoding, but this is handled more
gracefully and does not cause preserved temporary directories to be left
behind; reported by Marcin Lemanski; suggested by Stephane Lentz and
Robert LeBlanc (noted in the 2.0 release notes);
- use recv() instead of read() to get results from daemonized virus scanners
in an attempt to avoid a bogus Perl I/O status on some Linux installations
(reported by Sander Steffann); we now get a meaningful status codes like
ECONNRESET instead of a bogus EBADF (Bad file descriptor);
- ignore status ECONNRESET when reading results of a daemonized virus scanner
from a socket, specific to some Linux versions; thanks to Sander Steffann
for the initial report and extensive help in debugging the Perl problem;
- run_av and other similar code sections: replace line-by-line reads by
block-by-block reads wherever possible to avoid inappropriate status report
EBADF (Bad file descriptor) caused by Perl I/O bug when last line is not
terminated by a newline. The problem was affecting reading response from
some command line virus checkers; reported by Sander Steffann;
- ignore status EAGAIN when reading results on a pipe from a forked process;
the status EAGAIN seems to be an artifact of Perl I/O on some installations;
reported by several people to cause problems on FreeBSD with Perl 5.8.7
(but Perl 5.8.6 is fine); thanks to Bart Matterne for testing and feedback;
- allow one level of indirection when collecting %needed_protocols;
global setting $protocol='COURIER' did not work, a workaround was needed
with previous version, e.g.: $policy_bank{'QMQPqq'}={protocol=>'QMQPqq'};
reported by Nicklas Bondesson and Martin Orr;
- fix a bug (introduced with 2.3.0) in Courier and QMQPqq setups, where global
information about processed message wasn't always reset and could leak
into processing of a subsequent message; reported by Nicklas Bondesson;
- SQL: fix arguments in calls to last_insert_id(), failing under PostgreSQL
(MySQL didn't mind); pointed out by Henrik Krohns;
- if module SAVI is loaded, insist it is version 0.30 or later;
incompatibility with earlier versions reported by Andrzej Kukula;
- make use of the new Net::Server 0.88 hook run_n_children_hook() to
reload SAVI database; removes a need to apply SAVI patch to Net::Server;
the Net::Server hook was suggested by Paul B. Henson and others,
and incorporated into Net::Server 0.88 by Paul Seamons;
- reopen log file or syslog connection in each child process to make it use
its own file descriptor; also minimizes transients when syslogd is restarted
and its socket re-created, as reported by Les Ault. When running in chroot
please make sure a syslogd socket is also available in the chroot jail,
see README.chroot for syslogd options (and BASIC_CONFIGURATION_README
in Postfix documentation for the Postfix equivalent);
- close log file or syslog in forked process before exec, just to play nicely;
- do_lha: fix extracting archive member filename in case of broken archive
or empty name (avoid interpreting creation date as a file name);
do not increment OpsDecByLha counter for empty archives, which are
most likely not lha archives at all;
- obey $final_bad_header_destiny D_DISCARD or D_REJECT even for messages
with bad headers from mailing lists or with a null envelope sender (DSN);
previously such messages were passed; undesired behaviour reported
by Cami Sardinha.
Such messages are still let through with $final_bad_header_destiny set to
D_BOUNCE, as otherwise they will be lost because a bounce is suppressed
for null sender messages and for mail from mailing list. This behaviour
is retained for backwards compatibility, but may need to be reconsidered.
- fix regexp for extracting am_id from amavis-milter helper program requests;
- if fork/exec fails, try to commit suicide in forked process with
POSIX::_exit(1) first, before trying kill('KILL',$$) as a last resort;
- updated $log_templ example in amavisd.conf-sample to match the default;
pointed out by Gary V;
- further reduce a couple of more frequent Perl warnings about the use of
uninitialized values in expressions;
- pre-load additional Perl modules required by SA 3.1 plugins;
- require minimal versions of modules: Time::HiRes 1.49, Archive::Zip 1.14;
- replaced nonexistent variable @sa_spam_modifies_subj_maps by
@spam_modifies_subj_maps in commented-out example in amavisd.conf-sample;
noticed by Joachim Schoenberg;
LDAP CHANGES by Michael Hall:
All the LDAP changes are transparent to the user.
- rewritten some of the code similar to the restructuring of the SQL code
in version amavisd-new-2.3.0. A new package Amavisd::LDAP::Connection was
added which is a LDAP connection object, and the old connection-related code
in Amavis::Lookup::LDAP has been moved to the new package. Amavisd-new will
now try to reconnect (once) while processing a message, similar to SQL;
- added the ability to specify a '%d' (domain) token in the LDAP base DN;
based on idea from Alexander Wittig;
- updated default LDAP port based on whether SSL/TLS is being used or not;
based on idea from Timo Veith;
- updated the search code to query for multiple records and return the results
sorted in 'make_query_keys' order versus doing a query for each key.
As a result performance is enhanced, and the tweaks 'ldap_get_all', and
'use_query_keys' (recently added) are no longer applicable or needed
and have been removed;
- improved LDAP error reporting and misc changes to multivalued attributes;
- documentation changes (amavisd.conf-default, README.lookups);
MINOR IMPROVEMENTS:
- macro %c (commonly used in a log template) reports spam score no longer
as a single number, but as an explicit sum of a SA score and a by-sender
boost score (from @score_sender_maps) when boost score is nonzero;
suggested by Ed Walker;
- enhancement to amavisd-release: if its only command line argument is '-',
then read arguments from stdin, one release request per line, ignoring empty
lines; input lines have the same format as command line arguments, i.e.:
mail_file
mail_file secret_id
mail_file secret_id alt_recip1 alt_recip2 ...
- better handle cases where a persistent temporary file email.txt
as prepared by the SMTP server module gets replaced as a result
of some user program modification (e.g. when invoking altermime);
problems reported by Dinesh Shah and Leonardo Rodrigues;
Here's an excerpt from the rather long RELEASE_NOTES included in the
distribution:
QUICK OVERVIEW:
Provides more flexible configuration of decoders. Allows recipients to have
individual banning rules. Assigns a long-term unique id to each message,
reducing clashes and facilitating retrieval of information. The daemon can
store information to a SQL database for logging, reporting and quarantine
retrieval, optionally storing entire message to a SQL database. File-based
quarantine can disperse files to 62 subdirectories. Provides a quarantine
release mechanism. Reconnects to SQL if connection is broken. Can skip
quarantining high-score spam. Compatibility with IPv6-enabled Postfix
is improved.
SECURITY:
- require minimal version 1.05 of Convert::UUlib to avoid a known security
problem in the underlying uulib (likely to be exploitable);
Release notes:
December 22, 2004
amavisd-new-2.2.1 release notes
SECURITY:
- add support for the pax(1) archive decoder, which can handle tar/cpio/pax
archives (including legacy format variants). Due to limitations in cpio
(and in Archive::Tar), for security reasons it is preferred to decode
such archives with pax and no longer with cpio; please add a line:
$pax = 'pax';
to amavisd.conf and verify that the program pax is installed on the system
(and in the jail if running in chroot);
- perform additional tests at startup time on the proper protection
of the configuration file;
- add file name extensions wmf, emf and grp to the example list of
banned extension, according to recent Microsoft security bulletins;
suggested by Stephane Lentz;
- introduces 'clean but inconclusive' av scanner result to avoid a specialized
or quick partial av scanner like jpeg checker to claim mail is clean
when all other general purpose av scanners fail (see below);
INCOMPATIBILITY:
- removed some legacy $*_ldap variables, as they are no longer needed;
These variables were still declared but ignored in 2.2.0 for compatibility
with older amavisd.conf files. Such variables need to be removed from
the amavisd.conf if they are still present there from older versions,
otherwise Perl will complain with 'Global symbol ... requires explicit
package name";
OTHER FIXES:
- files_to_scan and decompose_mail are now able to remove unexpected
directories which may have been left behind by some failed decoding
and were causing temporary failures and mail delivery retries;
error recovery problem after failed unarj reported by Ralf Hildebrandt;
- error recovery code in files_to_scan and rmdir_recursively now tries to
change protection on directories and files, and retry if the first attempt
to access them fails because of denied permission;
- pre-load some additional Perl modules needed by SA when running in chroot;
- add module Net::LDAP::Search to a list of pre-fetched modules;
omission pointed out by Paul Jacobson;
- when quarantining is disabled by keeping $QUARANTINEDIR undefined,
the log entry and administrator notification message inappropriately
suggested that mail was quarantined, which in fact (appropriately)
it was not. Setting $QUARANTINEDIR='' did work as expected.
Reported by Sascha Lucas;
- avoid the use of Encode::is_utf8 due to a Perl bug (still present in 5.8.5)
where Encode::is_utf8 on tainted utf8 character string produces false;
- modify safe_encode() to guarantee the result is a string of octets,
not a string of UTF-8 characters; it saves some unnecessary work in
further processing and keeps MIME::Entity from UTF swamp when running
in chroot; problem pointed out by Branko F. Gracnar;
- avoid braindead Perl default where an empty regexp implies the last
successfully matched regexp, which (if not being very careful) brings in
some completely unrelated last-executed regular expression;
- change kill 'TERM' into kill 'KILL' when a forked process within run_command
and run_command_consumer gets into deep trouble, to avoid exit handlers
being invoked in the subprocess (which could lead to two processes trying
to clean the same set of temporary files);
- in an old sendmail setup using the amavis(.c) helper program without
LDA arguments, avoid inappropriate warning:
"WARN: no recips left (forgot to set $forward_method=undef using milter?)
and return status 0 instead of 99 when message is to be blocked, as the
helper program amavis(.c) does not recognize status 99 in this situation
and inappropriately passed it on to sendmail; reported by The Mindflayer;
- the @bypass_header_checks_maps is now able to also bypass the bad header
checks as provided by MIME::Parser; inconsitency reported by CRivera;
- avoid some Perl warning messages; thanks to Bill Landry;
CHANGES AND MINOR NEW FEATURES:
- add configuration variable @newvirus_admin_maps (and $newvirus_admin,
along with corresponding SQL field 'newvirus_admin') which works like
the existing @virus_admin_maps (and $virus_admin), except that it sends
virus administrator notification to specified e-mail address only for newly
encountered viruses which have not yet been encountered since the amavisd
startup. It makes use of by-virusname counters in the SNMP counters
database. If more than one child process starts working on infected
message containing a not-yet-accounted-for virus, there might be more
than one 'first time' notification, this is not a malfunction. Both
the @newvirus_admin_maps and the @virus_admin_maps may be enabled,
each (possibly both) would receive their notifications as appropriate.
A useful setting is to globally enable only the new virus notifications,
and additionally enable _all_ administrator notifications for internally
originating mail only (by the use of policy banks);
- provide separate configuration variables @banned_admin_maps and
@bad_header_admin_maps, along with corresponding SQL fields
'banned_admin' and 'bad_header_admin'; their function was previously
covered by @virus_admin_maps, which now only still controls administrator
notifications in case of viruses;
- introduces 'clean but inconclusive' av scanner result to avoid a specialized
or quick partial av scanner like jpeg checker to claim mail is clean
when all other general purpose av scanners fail:
in av scanner entries (lists @av_scanners and @av_scanners_backup) give
an extended meaning to undefined fourth argument (the 'match for clean'
list or regexp). The interpretation of the fourth argument is now:
4. an array ref of av scanner exit status values, or a regexp (to be
matched against scanner output), indicating NO VIRUSES found;
a special case is a value undef, which does not claim file to be clean
(i.e. it never matches, similar to []), but suppresses a failure warning;
to be used when the result is inconclusive (useful for specialized and
quick partial scanners such as jpeg checker);
Also modified example jpeg checker entry in amavisd.conf accordingly.
- NOD32 av scanner: changed @av_scanners entry to match the new version
of the scanner; thanks to Nejc Skoberne;
- added @av_scanners entry for File::Scan;
- when preparing a SQL SELECT clause for white/blacklisting lookup,
take into account a relative position of ? and %k in the
$sql_select_white_black_list template to improve flexibility
of specifying the clause; suggested by Matt Petteys;
- reduce the log level of some more common and harmless log messages;
- macro %p and the log entry now reports full policy bank path,
not just the last loaded policy bank name;
- added LDAP attributes amavisWarnVirusRecip, amavisWarnBannedRecip,
and amavisWarnBadHeaderRecip; by Joel Nimety and Michael Hall;
- renamed LDAP attribute name amavisSpamModifiesSubject to
amavisSpamModifiesSubj in order to match the documented LDAP schema;
noticed by Kees Bos, patch by Michael Hall;
- add support for ripOLE decoder, which attempt to extract embedded documents
from MS OLE documents (MS Office) (http://www.pldaniels.com/ripole/,
by Paul L Daniels)); ripOLE is still experimental/alpha code;
To be make amavisd-new find the installed program 'ripole', add the:
$ripole = 'ripole';
to the amavisd.conf; suggested by David Wilson and Noel Jones;
- allow multiple occurrences of command line option: -c config_file
and execute the provided configuration files one after the other;
based on a subset of functionality provided as a patch by Davor Ocelic;
- a slight improvement (in default $map_full_type_to_short_type_re)
in classifying mpeg and some other multimedia files;
- several minor code cleanups;
- add a recommendation by Daniel J McDonald to a documentation file INSTALL:
If different UID is preferred for an AV scanner, a solution for
ClamAV is to add user clamav to the amavis group, and then add
AllowSupplementaryGroups to clamd.conf;
- enclosed a simple demonstrational Perl program amavis.pl, which is
functionally much like the amavis.c helper program, but talks the new
AM.PDP protocol with the amavisd daemon. See README.protocol for the
description of AM.PDP protocol. To be placed in amavisd.conf:
$protocol='AM.PDP'; $unix_socketname='/var/amavis/amavisd.sock';
Usage: amavis.pl sender recip1 recip2 ... < message.txt
- documentation updates;
New in 2.2.0:
MIME traversal now includes MIME container parts (e.g. multipart/*,
message/*), making them visible to banned rules. This version
preserves original zip archives for virus scanners if the archive
contains any zero-length members. New short types 'dll' and 'empty'
makes blocking recent viruses more flexible, including their
unsuccessful propagation attempts. It recognizes standard Unix
archives and unpacks Debian binary packages. The LDAP modules were
rewritten. The handling of double errors was improved. This version
supports mail size limits and Mail::ClamAV 0.12. A new AV entry
'check-jpeg' can test JPEG images for validity.
New in 2.1.2:
This release fixes (hard) blacklisting and whitelisting on static
lookup tables, which was failing to match any sender. The 'neutral'
sender notification, which was joining the Subject and the Message-ID
header fields in some situations, has been fixed. The signal and
error handling in code sections holding BDB locks is now more
thorough. A new %e macro is provided that evaluates to a best guess
of the originator IP address collected from the Received trace.
New in 2.1.1:
The default use of $banned_filename_re, which was lost in 2.1.0,
was added back. A fix was made for inappropriate log entry in SQL
whitelisting, complaining about unexpected wb field value. Missing
import of &ca was added to the amavisd-new-courier.patch. A default
directory is now provided with delivery method "bsmtp", if not
specified. The Mail::SpamAssassin::Plugin::Hashcash module is
pre-loaded with SA 3.0.0, and Mail::SpamAssassin::SpamCopURI with
URI::* is loaded for older SA versions. Small enhancements were
made to amavisd-nanny.
New in 2.1.0:
The use of BerkeleyDB is now optional. The configuration files were
cleaned up, and a small new amavisd-nanny utility that shows the
status of all child processes and checks for vanished or stale
processes was included. Two important bugfixes were made in the
ACL and SQL lookup code along with numerous other fixes and small
improvements. Users of 2.0 should upgrade to this release.
amavisd-new is an interface between message transfer agent (MTA) and
one or more content checkers, e.g. virus scanners, SpamAssassin, etc.
It is a performance-enhanced and feature-enriched version of amavisd
(which in turn is a daemonized version of AMaViS or amavis-perl).
amavisd-new is normally positioned at or near a central mailer, not
necessarily where user's mailboxes and final delivery takes place. If
you are looking for fully per-user configurable and/or low-message-rate
solution to be placed at the final stage of mail delivery (e.g. called
from procmail), there may be other solutions more appropriate for your
needs.
Package created and maintained by Julian Dunn in pkgsrc-wip.
obache
reed
xtraeme
recht
cube
agc
wiz
jlam