An abridged upstream changelog follows...
---------------------------------------------------------------------------
October 25, 2014
amavisd-new-2.10.1 release notes
BUG FIXES
- fixed a missing import of mail_addr_idn_to_ascii() and idn_to_utf8()
when SQL is in use, resulting in:
Undefined subroutine &Amavis::Out::SQL::Log::mail_addr_idn_to_ascii
Undefined subroutine &Amavis::Out::SQL::Log::idn_to_utf8
OTHER
- avoid warnings issued by perl 5.21.5:
Negative repeat count does nothing at ./amavisd line 17218
Missing argument in sprintf at ./amavisd line 3678
- updated documentation in 2.10.0 release notes (new macros);
---------------------------------------------------------------------------
October 22, 2014
amavisd-new-2.10.0 release notes
COMPATIBILITY
- New requirement: perl module Net::LibIDN needs to be installed.
- Uses a perl module File::LibMagic if installed, instead of spawning
a file(1) utility.
- Default log templates and notification templates have changed
in details
- International domain names (IDN) encoded in ASCII-compatible encoding
found in e-mail addresses and in Message-ID header field will be decoded
to Unicode for presentation purposes (syslog, JSON structured log,
notifications). This decoding does not affect a mail message itself.
- Logging via syslog expects that syslogd (or equivalent) will not
clobber UTF-8 octets.
- Third party log parsers may need updating to accept logs with Unicode
characters in UTF-8 encoding.
- A SMTP response to an EHLO command will now announce SMTPUTF8 capability
by default.
BUG FIXES
- releasing a message from an SQL quarantine was broken in version 2.9.1
due to introduction of parent_mail_id()
and Gionatan Danti;
- if checking of a message was aborted prematurely (like due to a timeout
or some fatal error), JSON log could receive a copy of a previous
log entry;
- prevent non-ASCII non-UTF-8 octets from reaching a JSON log/report
(which produced an invalid JSON object and Elasticsearch complaining);
- allow SMTP commands MAIL FROM and RCPT TO to accept options without
values, as allowed by the RFC 5321 syntax;
- in delivery status notification (DSN) the field Received-From-MTA
specified 'smtp' as mta-name-type, instead of a 'dns' as prescribed
in RFC 3464;
- releasing from a quarantine left envelope sender address as '<>'
instead of using the address found in a Return-Path header field
of a quarantined message
- avoid failure in os_fingerprint or in smtp forwarding in certain cases
where the $os_fingerprint_method or $forward_method or $notify_method
uses an asterisk in place of a host IP address or port number.
- files LDAP.ldif and LDAP.schema: added a missing attribute
amavisDisclaimerOptions to objectClass
NEW FEATURES
- added support for Internationalized Email:
- added support for Internationalized Domain Names (IDN) according
to IDNA (RFC 5890, RFC 5891; RFC 3490);
- delivery notifications and admin notifications now show the following
information encoded as UTF-8 (which is a default $bdy_encoding) in the
plain text part of the message: IDN domain names in sender and recipient
mail addresses and Message-ID are first decoded to Unicode, Subject and
author display names are MIME-decoded;
- 'amavisd showkeys' and 'amavisd testkeys' can now deal with IDN
(international domain names): domain names in DNS zone comments
end up as UTF-8, DNS labels are in ASCII (A-labels); domain names in
calls to dkim_key() may be specified either as UTF-8 or in ASCII (ACE);
- new macro 'mail_addr_decode' takes an e-mail address as a string of
octets, where a local part may be encoded as UTF-8, and the domain part
may be an international domain name (IDN) consisting either of U-labels
or A-labels or NR-LDH labels.
- new macro 'mail_addr_decode_octets' is like 'mail_addr_decode', except
that the result is a string of octets, only valid as UTF-8 if the
provided address was a valid UTF-8 (garbage-in/garbage-out);
- new macro 'header_field_octets' is like 'header_field', except that
a result is a string of octets in UTF-8 encoding, suitable for a log
template;
- new macro 'ip_proto_trace_all' expands into a list of information
items from a Received header trace;
- new macro 'ip_proto_trace_public' is like ip_proto_trace_all, except
that entries with non-public IP address are excluded from the list;
- new macro 'protocol' evaluates to a protocol name by which a message
was received by amavisd, according to RFC 3848 ("Transmission Types
Registration") and "Mail Transmission Types" / "WITH protocol types"
IANA registration
- new macro 'client_protocol' expands into a protocol name by which
a message was received from a client by MTA; the information is passed
from MTA to amavisd through XFORWARD PROTO SMTP protocol extension or
through AM.PDP (milter); typical values are 'ESMTP' or 'SMTP';
- use a perl module File::LibMagic when available, instead of spawning
a file(1) utility for classifying contents of mail parts.
OTHER
- RFC 6533: recognize a MIME type 'message/global' as similar
to 'message/rfc822', and 'message/global-headers' as similar
to 'text/rfc822-headers' where appropriate (e.g. in bounce killer);
- header validity check now distinguishes 'non-ASCII and invalid UTF-8'
from 'non-ASCII but valid UTF-8' characters in a mail header section.
- ORCPT attribute in SMTP 'RCPT TO' command now accepts the original
recipient mail address in any of these encodings: utf-8-address,
utf-8-addr-unitext, utf-8-addr-xtext, or as a legacy xtext,
as required by RFC 6533;
- updated do_cabextract (extraction of Microsoft cabinet .cab archives)
to recognize a slightly changed output of cabextract version 1.2;
- adjusted some timeouts to leave more reserve for later stages of
mail processing and forwarding;
- prefer sanitizing/protecting control characters as hex code (like \x7F)
instead of octal (like \177) (e.g. in logging and DSN);
---------------------------------------------------------------------------
June 27, 2014
amavisd-new-2.9.1 release notes
BUG FIXES
- when a $final_bad_header_destiny is set to D_BOUNCE and a mail
message was a DSN or coming from a mailing list, spam checks were
inappropriately skipped (logged as: "bypassing of spam checks,
message will be blocked anyway", which was not true as the blocking
was overruled at a later stage);
- perl 5.20.0 may occasionally report:
"Malformed UTF-8 character (unexpected end of string)"
due to a bug in perl [perl #122148]. Treat this error as non-fatal
and only log a warning in two such cases: when SpamAssassin is called
as a spawned subprocess ($sa_spawned=1) and is returning a result
to a parent process, or when constructing a structured report.
OTHER
- updated decoding of RAR archives to recognize a changed format in output
of 'unrar' utility version 5;
- avoid tempfailing a message if a redis server is down, just log the
error and carry on;
- some minor logging changes to facilitate troubleshooting;
---------------------------------------------------------------------------
May 9, 2014
amavisd-new-2.9.0 release notes
COMPATIBILITY
This version drops dependency on a Perl module Redis, and makes
dependencies on modules Convert::TNEF and Convert::UUlib truly optional.
The following change may affect third-party log parsers:
To facilitate forensic log analysis and troubleshooting, log entries
'FWD from' and 'SEND from' at level 1 now carry one additional
prefixed information field which is the unique internal mail_id of
the message, possibly followed by a parent_mail_id in parenthesis,
e.g.:
(00525-02) XE9xnQYjrWyd FWD from <...> -> <...>, ...
(00495-02) v1pyIOMQkUYD(CIcqao-vCDO9) SEND from <...> -> <...>, ...
NEW FEATURES SUMMARY
- structured log/reporting to a Redis server in JSON format;
- IP address reputation (uses a Redis server);
- added two minor content categories to the major ccat CC_UNCHECKED
(encrypted (=1) and over-limits/mail-bomb (=2) );
- introduced a by-recipient setting %final_destiny_maps_by_ccat.
BUG FIXES
- fixed "Insecure dependency in sprintf" in Sophos SAVI av-scanner,
- fixed the interface code to virus scanners Sophie, Trophie and fpscand,
where a time-out on a long-running virus scan would leave a connection
to the virus scanner open and a late response from a scanner to a
previous request could be interpreted as a result of the current scan;
- fixed a bug in transforming an IPv6 alternative form IP address into
a preferred form. One effect of this bug was declaring an IPv4-mapped
IPv6 address as syntactically incorrect;
- if SQL logging was disabled a pen pals feature was non-functional even
when a Redis storage back-end was available and collecting data; now
pen pals is fully functional with a Redis database back-end and no SQL;
- provided our own Redis client code, avoiding Redis CPAN module bugs,
its slowness and non-support for IPv6.
- fixed a regexp in parsing wildcarded signing domain in a DKIM key
declaration and in a wildcarded sender pattern of signing options.
- dropped hard-coded dependency on modules Convert::TNEF and Convert::UUlib.
The Convert::TNEF was made optional in amavisd-new-2.8.0, but the
program still failed if the module could not be loaded at startup.
- import a missing do_log_safe() in Amavis::LDAP::Connection to avoid
a warning: _WARN: \t(in cleanup)
Undefined subroutine &Amavis::LDAP::Connection::do_log_safe
called at (eval 101) line 76 during global destruction;
- at startup amavis may try to find a decoder for 7z and zip extensions
twice;
- fixed the amavisd-new-courier.patch which resulted in two instances
of sub post_bind_hook(). Only tested for syntax.
NEW FEATURES
- Structured logging/reporting in JSON format is now available through
a redis server.
- IP address reputation
- Added support for decompressing LZ4 streams in mail attachments when
an external utility lz4c is available and the 'file' utility recognizes
such streams (probably since version file-5.17). Default settings
of @decoders and $map_full_type_to_short_type_re now recognize LZ4;
if these settings are replaced by a configuration file, the config
file needs to be updated to include the new entry.
- Added two minor content categories to the major ccat CC_UNCHECKED
to allow distinguishing between reasons of decoders failure.
- introduced a setting %final_destiny_maps_by_ccat, which makes it
possible to specify by-recipient final destiny for each contents
category, e.g. use D_REJECT on spam to some users, and D_BOUNCE or
D_DISCARD or D_PASS for others. Introduced mostly for completeness.
- added a setting $outbound_disclaimers_only. When set to true and
disclaimers are enabled, it will only allow adding disclaimers
to non-local recipients. For backward compatibility the default
value is false (undef)
- the $recipient_delimiter setting can now hold a multi-character string,
specifying all characters that can delimit an address extension from
a base e-mail address. Previously this setting was restricted to a
single character (typically a '+' or a '-').
- added macros report_json and rot13 (to be used in a log template):
* the macro 'report_json' expands to a JSON representation of a
structured log event;
* the macro 'rot13' replaces a string in its argument with an obfuscated
string where letters are shifted by 13 positions of an English
alphabet (a popular variant of a Caesar cipher to conceal spoilers);
this may serve to (poorly) hide strings such as mail Subject or
an e-mail address from casual browsing of a log;
OTHER
- dropped dependency on a CPAN module Redis, implementing our own
client-side redis protocol implementation (Amavis::TinyRedis).
It is faster and smaller, and supports opening sessions with a
redis server over IPv6 (or over IPv4 or over a Unix socket).
- store only essential information for pen pals operation to a Redis
storage back-end to save memory on a database server; information on
inbound messages is no longer stored there, i.e. only information on
originating messages is kept;
- more informative logging of pen pals query results when using a Redis
storage back-end. The redis support code (Lua and protocol handling)
was largely rewritten for efficiency since amavisd-new 2.8.1.
- added LDAP attribute amavisDisclaimerOptions 1.3.6.1.4.1.15312.2.2.1.47
to LDAP.schema;
- reduced EDNS payload size from 1240 bytes to a conservative default
of 1220 bytes when calling Mail::DKIM verifier;
- optimization: filter for public IP addresses from a Received trace
only once;
- added one digit of precision in the TIMING log report to reported small
elapsed times (below 5 ms);
- in a milter setup (AM.PDP) the log-id wasn't unique; adding a request
sequence number to it;
- avoid writing a notification to stdout about a warm reload for the benefit
of a cron job;
- reduced log level on some of the less useful log messages in a milter
setup;
- documentation README.sql-mysql: added "CREATE INDEX msgs_idx_mail_id..."
with a note on an InnoDB requirement for a foreign key;
---------------------------------------------------------------------------
June 28, 2013
amavisd-new-2.8.1 release notes
COMPATIBILITY
- when 0MQ (a.k.a. ZeroMQ) is used between Amavis components as an
internal messaging protocol, make sure to replace all 0MQ-enabled
Amavis components on upgrading amavisd, as the internal protocol
has changed slightly, taking advantage of 0MQ multi-part messages
for better performance. Affected programs are: amavis-services,
amavisd-status, amavisd-snmp-subagent-zmq, and amavisd.
- amavisd is compatible with perl 5.18.0 and with SpamAssassin 3.4.0
BUG FIXES
- fixed a bug in the SMTP client code, where the final SMTP status did
not reflect a failure status of a DATA command from a back-end MTA.
This caused a reception of a mail message to be confirmed but a message
was then lost, as it could not be passed to a back-end MTA. The bug
went unnoticed for years, as the commonly used MTAs normally reject
either at the MAIL FROM, at RCPT TO, or at the data-dot stage, but not
at the DATA command.
- fixed calling an external spam scanner DSPAM or Bogofilter, which
failed with a message:
auto-learning with spam scanner ... failed: error running program
- if a configuration file path as given through a command line option -c
or as an argument to include_config_files() was not an absolute path,
and that file contained an error, the do() would search the @INC list
for alternative files of the same name, and reported an unrelated error
(typically: No such file or directory) instead of reporting the true
reason for a failure;
- fixed a regular expression in amavisd.conf for an 'Avast!' AV entry
to properly extract a virus name;
- added LDAP errors LOCAL_ERROR and OPERATIONS_ERROR to the set of
expected error conditions which lets amavisd retry the failed
operation;
NEW FEATURES SUMMARY
- new Redis storage for the "pen pals" feature;
- improved IPv6 support;
- support for p0f v3;
- new macros ip_trace_all and ip_trace_public;
- amavisd-status now shows a bar graph display
of the number of active processes;
- the timing report log entry can show CPU usage
at log level 2 if a module Unix::Getrusage is available;
- improved IPv6 support: p0f-analyzer.pl can now communicate with amavisd
processes over an INET6 socket (or over an INET or UNIX socket as before).
- rewritten p0f-analyzer.pl to support a newer p0f v3 output format,
while still recognizing an older p0f v2 output format;
- improved IPv6 support: program amavisd-snmp-subagent-zmq can now
attach as an AgentX to a Net-SNMP daemon snmpd over an INET6 socket;
- improved IPv6 support: program amavisd-submit can now submit a mail
message to amavisd over an INET6 socket;
- a macro W can now produce a list of all virus scanners invoked,
along with a list of virus names each scanner detected;
- new macros ip_trace_all and ip_trace_public;
- templates for administrator notifications, recipient notifications,
and sender notifications now use macro 'ip_trace_all' instead of macros
'e' and 't' in order to report the full 'received' trace, not just the
first hop;
- macro supplementary_info recognizes new arguments: VERSION, SUBVERSION,
and RULESVERSION, providing additional information from SpamAssassin
correspond to equivalent SpamAssassin tags;
- a new command line option -X allows controlling some exotic features,
useful for example in debugging or automatic testing.
- a configuration setting $sa_debug may now specify a comma-separated
list of SpamAssassin debug facilities, complementing a similar method
of specifying these facilities through a command line option -d.
- pass the size of an original mail body as a 'supplementary attribute'
to SpamAssassin for the benefit of a 'check_body_length' eval rule
(new with SpamAssassin 3.4.0).
- to the output of amavisd-status add a simple bar graph display (with
an exponential-decay peak indicator) of the number of active processes;
- if a module Unix::Getrusage is available, the timing report log entry
(at log level 2) is enhanced: in addition to total elapsed time
(wall clock) spent in processing a message, it also shows a sum of
CPU user and system times spent by amavisd process and its spawned
processes.
OTHER
- the 'amavisd genrsa' command will now warn if the requested DKIM
signing key size is below 1024 bits, as required by RFC 6376;
- on amavisd startup a check on available private DKIM signing keys
(as declared by dkim_key) will now warn if a key size is below
1024 bits as required by RFC 6376, and log an information message
if a key size is below a configured $dkim_minimum_key_bits size
(defaults to 1024, currently 768 would still be a sensible value);
- for purposes of DKIM-based whitelisting (@author_to_policy_bank_maps)
and @signer_reputation_maps spam scores, valid signatures with
public keys shorter than $dkim_minimum_key_bits bits (default 1024,
equivalent to a lower limit as presently used by Google) are now
ignored, with an informational message logged at level 1.
- consider Unique local addresses (ULA) fc00::/7 non-public (RFC 4193),
dropped site-local addresses fec0::/10 (deprecated by RFC 3879),
adjusting the default setting of @mynetworks accordingly;
- consider the "Shared Address Space" 100.64.0.0/10 non-public (RFC 6598);
- adjust parsing the <zone_id> syntax of a scoped IPv6 address
as per RFC 6874;
- updated an AV entry for a Sophos Anti Virus: the scanning program
used to be named 'sweep', now it is 'savscan';
- updated a default value of @virus_name_to_spam_score_maps: updated
entry for Doppelstern and added entries for Bofhland and PORCUPINE;
- increase an arbitrary sanity limit on %smtp_reason_by_ccat strings
from 100 to 450 characters (RFC 5321 allows 512 character reply lines);
- relax testing file type of a configuration file, now a configuration
may also be passed to amavisd through a named pipe (fifo), possibly
facilitating testing or unusual deployments;
- relax a requirement that a $QUARANTINEDIR directory needs to be
writable: if $*_quarantine_method template settings include a
subdirectory (e.g.: $spam_quarantine_method='local:W%P/spam/%m.gz'),
such subdirectories must already exist and should be writable,
but the top-level $QUARANTINEDIR directory need not be writable;
- convert an IPv4-mapped IPv6 address into a plain IPv4 dot-quad form
when found in Received header fields, in socket local or peer address,
in ADDR field of an XFORWARD smtp extension command, or in an AM.PDP
attribute client_address. See draft-cmetz-v6ops-v4mapped-api-harmful
and draft-itojun-v6ops-v4mapped-harmful for potential caveats;
- drop a support for direct queries to p0f v2, as it never worked well
due to bugs in p0f v2. The p0f v3 changed the query protocol, but
a query does not include port numbers (see RFC 6302), so using the
p0f-analyzer.pl interface is still the only reliable approach;
- use sysread() instead of read() when reading from /dev/urandom
to avoid leaving entropy data in I/O buffers; also changed interface
name to sub read_random_bytes(), which now reads directly into a
scalar buffer, provided by an argument;
- fix uniform random distribution when generating a random PIN for an
attachment password (when releasing and $release_format is 'attach')
(not a security issue);
- added keepalive options to a call to Net::LDAP->new, recognized since
Net::LDAP 0.53;
- removed option inet6 from a default LDAP setup ( $ldap_sys_default ),
as Net::LDAP changed semantics in an incompatible way;
presumably the Net::LDAP now does the right thing by default;
- use a low-level 0MQ interface instead of ZeroMQ / ZMQ abstractions;
(i.e. ZeroMQ raw interface or ZMQ::LibZMQ3 or ZMQ::LibZMQ2);
- taking advantage of 0MQ multi-part messages the number of IP packets
transmitted is now radically decreased in favour of sending larger
but fewer packets;
- when generating 'Abuse Reporting Format (ARF) Reports' add a field
Source-IP and use UTC timestamps in the Arrival-Date field, in accordance
with RFC 6692;
- drop (opportunistic) loading of a module Devel::SawAmpersand and testing
the Devel::SawAmpersand::sawampersand(), variables $&, $` and $' are
no longer slow since Perl 5.17.7, the PL_sawampersand became a constant,
there is no longer any need of report it;
- documentation update: remove vestiges of a field 'spam_modifies_subj'
in README.sql, README.sql-mysql, this field was obsoleted in 2.7.0;
---------------------------------------------------------------------------
June 30, 2012
amavisd-new-2.8.0 release notes
COMPATIBILITY
- removed an old compatibility measure: default value of @banned_admin_maps
was changed from:
@banned_admin_maps = (\$banned_admin, \%virus_admin, \$virus_admin);
to a more consistent:
@banned_admin_maps = (\$banned_admin);
- quarantining to an mbox format file used to include a local time in an
mbox separator line, which differs from RFC 4155 and common practices
of using an UTC timestamp; a time zone of a timestamp in separator lines
is now changed to UTC;
BUG FIXES
- fixed initial evaluation of dynamic (i.e. per policy bank) values of
$enable_dkim_verification, $enable_dkim_signing and $bypass_decode_parts
across all declared policy banks; these policy bank entries may be scalars
of references to such;
- finely adjust a message size for de-stuffed dots according to a size
definition in RFC 1870; avoids occasional message size mismatch when
using an antispam interface module SpamdClient (implementing client-side
of a spamc/spamd protocol);
- updated LDAP.ldif to match LDAP.schema; provided by Quanah Gibson-Mount;
- updated AMAVIS-MIB.txt and amavisd-snmp-subagent: changed type of
SNMP variables *MsgsSize* in the group amavisStats 7 from Counter32
to Counter64 for consistency with other *MsgsSize* variables in groups
amavisStats 3 and amavisStats 9;
NEW FEATURES SUMMARY
- For monitoring and statistics gathering purposes a new set of utilities
and service processes is available based on a message passing paradigm,
using a 0MQ (a.k.a. ZMQ, ZeroMQ, or Crossroads I/O) library. This
replaces a functionally similar set of utilities based on a shared
BerkeleyDB database, with a benefit of avoiding lock contention
altogether. This can bring sigificant speedups, most pronounced on
a host with many busy amavisd child processes.
- Applied numerous fine-grained optimizations based on a NYTProf profiler
results. Optimizations include a reduction in a number of generated
Perl opcodes and similar micro-optimizations. This accounts for a large
amount of small changes in the code.
- Our current statistics (Q4 2011) shows that 80 % of messages are below
30.000 bytes, and 90 % of mail messages are below 100.000 bytes in
size. As an optimization, messages below 100 KiB in size are now kept
and processed in memory, including passing them more optimally to
SpamAssassin 3.4.0. Some file activity is still there, but is much
reduced. If $TEMPBASE also resides on an SSD disk (or a RAM disk),
observed speedup between 2.7.2 and 2.8.0 was 3 to 8 percent on a
busy host (with monitoring disabled, so as not to skew a measurement).
- Use a module IO::Socket::IP if available, instead of dealing directly
with low-level modules IO::Socket::INET and IO::Socket::INET6.
The IO::Socket::IP is a Perl core module since Perl version 5.19.8;
- choose more appropriate defaults if running on an IPv6-only host
(like connecting to ::1 instead of 127.0.0.1 which may not exist);
- amavisd-release now also supports connecting to amavisd over IPv6;
- as a debugging aid it is now possible that a late event triggers full
logging of earlier events that occurred during processing of a current
mail message;
- $enable_ldap setting is now dynamic, i.e. can be changed by a policy
bank, which makes it possible to selectively disable LDAP lookups
per policy bank;
- optionally avoid persistent connections to SQL and LDAP servers;
- it is now possible to disable calling an external file(1) utility
but still have MIME parts decoding enabled;
- added support in Amavis::SpamControl::ExtProg for an external spam scanner
Bogofilter;
- added locking options to @spam_scanners entries, to be used with external
scanners which need but do not implement locking of their resources
by themselves;
- added a global configuration setting $sa_userprefs_file, which is passed
on to SpamAssassin as a 'userprefs_filename' parameter at initialization;
- added a subroutine iso8601_weekday(), potentially useful with partitioning;
- added several new macros available to logging and notification templates;
NEW FEATURES - OTHER
- if a module IO::Socket::IP is available, amavisd will use this module
to create its client-side inet or inet6 sockets, instead of using the
low-level modules IO::Socket::INET and IO::Socket::INET6. This delegates
some of the dirty details handling to IO::Socket::IP, such as using the
getaddrinfo(3) system service to resolve host names, and dealing with
dual-stack multihomed host names. If IO::Socket::IP is not available,
the IO::Socket::INET or IO::Socket::INET6 are used directly instead,
to preserve compatibility. Please use a fairly recent version of
IO::Socket::IP, testing was done with versions 0.08 and 0.16.
- added a subroutine read_cidr() which can read a Postfix style CIDR file,
with a syntax interpreted according a Postfix cidr_table(5) man page.
The subroutine returns a ref to an array by default (but can also
produce a hash, and is able to add data to an existing array or hash).
- as a debugging aid it is now possible that a late event triggers full
logging of earlier events that occurred during processing of a current
mail message.
- the @decoders list is made a bit more flexible: the first entry in
each tuple (a short type name) may be a scalar string as before,
or may be a reference to a list of such names, in which case the
tuple applies to all listed short types.
- support an external decompressor lrzip for a .lrz format.
- $enable_ldap setting is now dynamic, i.e. can be changed by a policy
bank, which makes it possible to selectively disable LDAP lookups
per policy bank.
- optionally avoid persistent connections to SQL and LDAP servers - at
the expense of about 3 to 7 ms elapsed time for a reconnect.
- it is now possible to disable calling an external file(1) utility
but still have MIME parts decoding enabled: $file = undef;
- added Amavis::SpamControl::ExtProg support for an external spam scanner
Bogofilter.
- added Amavis::SpamControl::ExtProg support for auto-learning on external
spam scanners; experimental: works, but may change in future versions;
- added locking options to @spam_scanners entries, to be used with
external scanners which do not implement database locking by themselves.
- added a global configuration setting $sa_userprefs_file (undef by
default), which is passed on to SpamAssassin as a 'userprefs_filename'
parameter during its initialization.
- added a subroutine iso8601_weekday() which takes a Unix time as an
argument (seconds since 1970-01-01T00:00Z), and returns a weekday number
based on local time: a number from 1 through 7, beginning with Monday and
ending with Sunday, as specified in ISO 8601 (EN 28601).
- added a macro 'weekday', which expands to a weekday number
of the current message reception time, as provided by a call
to iso8601_weekday($msginfo->rx_time);
- added a macro 'secret_id', which expands to a secret counterpart to
mail_id, such that: b64_encode(md5(b64_decode(secret_id))) == mail_id.
It is encoded in base64url (RFC 4648), e.g. laL-rCJ6MBTm
(with a counterpart mail_id: XlZbJeFhn4OE).
- added a macro 'mail_id' as a synonym to a macro 'i', which is a
long-term unique mail_id on this system, possibly used in log and in
quarantine names, encoded in base64url (RFC 4648), e.g. XlZbJeFhn4OE
(with a counterpart secret_id: laL-rCJ6MBTm);
- added a macro 'log_id' as a synonym to a macro 'n', which is an
internal log id (also called task id, am_id) as shown in the log
and by amavisd-nanny, e.g. 58725-05-2;
- added a macro 'hexenc', which encodes its string arguments as
hex digits, high nybble first;
- added macros 'b64enc' and 'b64urlenc', which encode their arguments
as base64 strings, removing the final null padding '=' characters.
The 'b64enc' encodes into a character set [A-Za-z0-9+/], while the
'b64urlenc' encodes into a character set [A-Za-z0-9-_] in accordance
with RFC 4648;
- added a macro 'body_digest', which expands to a digest (a hash) of a
body of a mail message as computed by the algorithm chosen by a setting
$mail_digest_algorithm (defaults to 'MD5', can be 'SHA-1' or 'SHA-256').
- added a configuration setting $mail_digest_algorithm which chooses an
algorithm name for generating a mail header digest and a mail body digest.
OTHER
- quarantining to a mbox format file was using mboxo rule for protecting
a "From " line in a mail body, which made an original ">From " line
indistinguishable from a protected From; now a mboxrd format rule is
used, see http://en.wikipedia.org/wiki/Mbox
- make MIME::Parser use $TEMPBASE as a temporary directory for scratch
files instead of its default (which was /tmp, or failing over to a
current directory, disregarding a TMPDIR environment variable).
- distinguish an absence of an SMTP response from a negative SMTP response
in an SMTP/LMTP client code for improved logging/debugging purposes;
report delay time in case of a failure;
- a default value for $inet_socket_bind now reflects the availability
of socket protocol families INET (IPv4) and INET6 (IPv6).
- $forward_method, $notify_method and $requeue_method now default to an IPv6
address of a loopback interface ::1 instead of 127.0.0.1 when INET6 support
is available and INET is unavailable (IPv6-only host);
- remove an existing Authentication-Results header field only if we are
capable of generating our own: keep it if $enable_dkim_verification
is false or if $allowed_added_header_fields{'authentication-results'}
is false;
- add a field "Source-Port:" to "Abuse report format" (ARF) messages
as per draft-kucherawy-marf-source-ports;
- Avira SAVAPI av scanner: only log a warning instead of aborting
when a QUIT command at the end of a session fails;
- load all (both) applicable policy banks when %interface_policy contain
both a "SOCK" entry and a Unix socket path name; and similarly when it
contains both the "IPaddress:port" and a "port" entries.
- make use of a new SpamAssassin 3.4.0 option "skip_prng_reseeding".
- no longer pre-load a module Mail::SpamAssassin::Plugin::SpamCop
to avoid unnecessarily dragging-in modules Net::SMTP and Net::Cmd;
- a spamd client code in Amavis::SpamControl::SpamdClient now obeys an
option 'mail_body_size_limit' in a @spam_scanners entry and truncates
a message passed to spamd (like other spam scanner interfaces do),
instead of skipping a call to spamd.
- modules Convert::TNEF is now made optional, instead of being required;
do not load it if @decoders list is empty;
- avoid a warning issued when encountering an empty ehlo-keyword in a
response to an EHLO command (like on testing with a smtp-sink utility);
- some fine-grained reduction in a number of generated opcodes and
similar tiny optimizations; this accounts for numerous small changes
in the code;
- avoid some warnings issued by Test::Perl::Critic;
- just in case: make sure that our SMTP responses at the incoming session
are truly flushed to the socket and not stuck in a perlio I/O buffer;
- updated 2.7.0 release notes, documenting that a policy bank may also be
loaded based on a path name of a Unix socket receiving a connection;
- updated and clarified schema and instructions in README.sql-pg
- fixed spelling mistakes in comments;
- internal incompatible change: changed arguments and a result of a
subroutine write_header; also, now it rewinds a message file by itself;
---------------------------------------------------------------------------
August xx, 2012
amavisd-new-2.7.3 release notes
BUG FIXES
- fixed a bug in the SMTP client code, where the final SMTP status did
not reflect a failure status of a DATA command from a back-end MTA.
- if a configuration file path as given through a command line option -c
or as an argument to include_config_files() was not an absolute path,
and that file contained an error, the do() would search the @INC list
for alternative files of the same name, and report an unrelated error
(typically: No such file or directory) instead of reporting the true
reason for a failure;
- fixed a regular expression in amavisd.conf for an 'Avast!' AV entry
to properly extract a virus name;
OTHER
- updated an AV entry for a Sophos Anti Virus: the scanning program
used to be named 'sweep', now it is 'savscan';
- documentation update: remove vestiges of a field 'spam_modifies_subj'
in README.sql, README.sql-mysql, this field was obsoleted in 2.7.0;
---------------------------------------------------------------------------
June 30, 2012
amavisd-new-2.7.2 release notes
BUG FIXES
- a generated Received header field was missing the 'IPv6:' prefix
in the TCP-info component of a 'by' subfield (as required by RFC 5321,
section 4.1.3) when amavisd received a message over an IPv6 protocol;
- changed data type of an SNMP variable LogRetries from C32 to C64
for consistency with the MIB;
- updated AV entry 'AVG Anti-Virus' to consider status 403 continuation
lines when searching for a virus name;
OTHER
- reduce a log level to 5 on a log message:
Amavis::IO::RW: Error flushing on close: ...
to avoid an innocent but sinister-looking warning when a pipe
to a virus scanner is broken and needs to be re-established;
- updated an AV entry for 'F-Secure Linux Security' to version 9.14;
- fix a Unix socket compatibility issue with Net::Server versions 2.000,
2.001 and 2.002, where a method NS_unix_path no longer exists.
---------------------------------------------------------------------------
April 29, 2012
amavisd-new-2.7.1 release notes
BUG FIXES
- prevent rmdir() from failing with 'Invalid argument' on Solaris 10 when
deleting a temporary directory: current working directory must not be
within a directory which is about to be deleted;
- forwarding or quarantining through a 'pipe:' method failed with
"Insecure dependency in exec while running with -T switch" when a
sendmail command-line option -N was needed;
- when multiple sockets are specified (e.g. in $forward_method) as a
redundancy/failover mechanism, and SMTP session caching is enabled,
a failed forwarding session does not clear a cached session, so all
further attempts are stuck with the failed server, instead of picking
a different server from the list;
- on establishing a SMTP session when multiple sockets are specified
(e.g. in $forward_method) as a redundancy/failover mechanism, the
random choice never picked the last socket in a list;
- fix defanging by mimedefang, it was failing with perl 5.10 or later
due to an unhandled "Insecure dependency in sprintf" while logging the
result if the $log_level was 2 or higher, or when debugging was enabled;
- fix defanging by Anomy::Sanitizer, it was failing with an error message:
"mangling by anomy failed: replacement size 0, mail will pass unmodified";
- fix the 'xz' entry in a default @decoders list (in files amavisd.conf,
amavisd.conf-default and amavisd); the first two variants ('xzdec' and
'xz') were glued together, so the xz decoder was only available if found
under names 'unxz' or 'xzcat';
- provide a workaround for a bug [rt.cpan.org #64642] in a perl module
Encode, which gratuitously untaints a string when encoding or decoding it.
- avoid the use of Encode::is_utf8 due to a bug in a perl module Encode
as bundled with versions of Perl 5.8.0 to 5.8.8 (fixed in March 2007):
Perl bug tracking: #32687:
Encode::is_utf8 on tainted UTF8 string returns false
https://rt.perl.org/rt3/Public/Bug/Display.html?id=32687
also referenced by #37170:
https://rt.perl.org/rt3/Public/Bug/Display.html?id=37170
- fix a warning: _WARN: Invalid conversion in sprintf: "%a"
- write informational messages during a stop/start/restart to stdout,
instead of to stderr, avoiding unnecessary cron job messages;
- fix a syntactically incorrect 'Avira SAVAPI' av entry (missing
closing bracket) in a sample configuration file amavisd.conf;
- minor: get_body_digest incorrectly logged 8-bit body as 8-bit header;
- no longer insist on a minimal version 2.22 of a module Digest::MD5,
the 'clone' method is no longer needed since amavisd-new-2.7.0;
- do not call $parser->max_parts($MAXFILES) with some old versions
of MIME::Parser which did not yet provide this method;
- pre-load a module File::Glob even with perl 5.8.0, otherwise
autowhitelisting in SpamAssasssin may fail with "Insecure dependency";
- documentation: (files README.sql-mysql and README.sql-pg):
fixed a field name "policy.unchecked_lover", previously incorrectly
specified as "policy.unchecked_lovers_maps";
- documentation: fixed the two SELECT examples in files README.sql-pg and
README.sql-mysql, the field 'select' needs to be qualified with a table
name: 'msgrcpt.content' to avoid ambiguity;
- documentation bug in amavisd.conf-default: 'ESMTP' is not a valid
setting for $protocol, just use 'SMTP' instead;
COMPATIBILITY
- commented out the LHA entry in the default @decoders list and in
do_executable(). The program seems to be unmaintained, was seen crashing
and as such it may pose a security risk;
- due to popular demand, bring the 'spam-tag:' log line back to log level 2
(version 2.7.0 dropped it to log level 3) to retain compatibility with
some log analyzers. Caveat: 'spam-tag' string is now entirely in lowercase.
OTHER
- if a message is quarantined to more than one location using different
quarantine methods, the SQL field msgs.quar_type indicates only the
type of the last one.
- SMTP session caching now no longer re-uses old sessions which are
in use for more than a minute since their establishment;
- having the archive quarantine enabled should not be a sufficient reason
to store information to SQL when $sql_store_info_for_all_msgs is off;
- ClamAV-clamd and ClamAV-clamd-stream av scanners: changed socket name
in a sample configuration file amavisd.conf to /var/run/clamav/clamd.sock
(previously the socket name was /var/run/clamav/clamd);
- documentation updates;
ea5c665f5e