a) refer 'perl' in their Makefile, or
b) have a directory name of p5-*, or
c) have any dependency on any p5-* package
Like last time, where this caused no complaints.
to trigger/signal a rebuild for the transition 5.10.1 -> 5.12.1.
The list of packages is computed by finding all packages which end
up having either of PERL5_USE_PACKLIST, BUILDLINK_API_DEPENDS.perl,
or PERL5_PACKLIST defined in their make setup (tested via
"make show-vars VARNAMES=..."), minus the packages updated after
the perl package update.
sno@ was right after all, obache@ kindly asked and he@ led the
way. Thanks!
Pkgsrc changes:
- Removed most of the package options; using the options framework for
those choices was not quite correct in the first place. Some have now
fixed values (Perl warnings and taint checks always enabled), some
got converted to variables settable from the make command line, see
options.mk.
- pkglint complained about the variable PLIST_ADD so I renamed it to
DYNAMIC_PLIST.
- SpamAssassin does not come with rules anymore. As a starting point
the official archive of rules at the time of the SA release is
included. At installation time this set of rules gets installed
through the pkgsrc INSTALL file.
- Removed patch-ba and patch-bc, both were integrated upstream.
- Removed patch-be. The quick fix for the bad rule was replaced by
a permanent solution.
Summary of major changes since 3.3.0
====================================
bug 6335: add Spamhaus DBL as URIBL_DBL_SPAM rule
Bug 6370: update ImageInfo plugin to latest release
bug 6215, bug 6294: RCVD_IN_CSS rule was broken. the check_rbl_sub() syntax
was incorrect, resulting in missing hits
bug 6361: list 2tld and 3tld sub-domain hosters for URIBL/SURBL/DBL queries;
NOTE for SARE users: This file replaces the SARE file
http://www.rulesemporium.com/rules/90_2tld.cf, which will be deprecated as from
2010-05-01.
Bug 6369, 6356, 6373: WIN32 support for spamd improved
Bug 6267: Solaris 10 requires --syslog-socket=native
bug 6304 spamd is spawning and killing processes too often - Added spamd
adjustments to info level and more information for administrators + small fix
to Makefile.PL
Bug 6310: sa-learn --import gives Insecure dependency in open
Bug 6313: -Q or -q AND -x should not result in creation of a ~/.spamassassin
dir; plus: taint issues fixed
Bug 6342: make test failure on if_can under perl 5.6
Bug 6340: Impossible to find user home directory of VPOPMAIL alias
Bug 6072, 6343: POD warnings, documentation fixes
Bug 6304 (trivial), reduce sysadmin's stress level by lowercasing
the 'INTERRUPTED' in a logged message:
spamd: handled cleanup of child pid [...] due to SIGCHLD: INTERRUPTED
Bug 6329: POSIX::strftime in call under Win32 ActivePerl causes Perl to hang up;
formatting option %e is not in a POSIX standard, use %d instead and edit
Bug 6322: In DKIM ADSP eval test check_dkim_adsp() the '*' is handled incorrectly
Bug 6327: Fix calling argument in utility used to determine DCC's homedir
Bug 6316: DCC.pm, wrong options for dcc_proc, (plus: avoid a warning on undef
in logger when dccifd socket is not provided)
Bug 6287: improved DKIM plugin debugging
Bug 6321 - _TOKENSUMMARY_ not working in 3.3.0 (Plugin/Bayes.pm looks-up a tag
from wrong location)
Bug 6312 - uninitialized value $start_time in spamd
bug 5761: trivial doc fix: document SPAMD_LOCALHOST test-control env variable
Summary of major changes since 3.2.5
====================================
COMPATIBILITY WITH 3.2.5
- rules are no longer distributed with the package, but installed by
sa-update - either automatically fetched from the network (preferably)
or from a tar archive, which is available for downloading separately
(see below, section INSTALLING RULES);
- CPAN module requirements:
- minimum required version of ExtUtils::MakeMaker is 6.17;
- modules now required: Time::HiRes, NetAddr::IP (4.000 or later),
Archive::Tar (1.23 or later), IO::Zlib;
- minimal version of Mail::DKIM is 0.31 (preferred: 0.37 or later);
expect some tests in t/dkim2.t to fail with versions older than 0.36_5;
- no longer used: Mail::DomainKeys, Mail::SPF::Query;
- either Digest::SHA or the older Digest::SHA1 is required, though
note that the DKIM plugin requires Digest::SHA for sha256 hashes
and Razor agents still need Digest::SHA1;
- some IPv6 functionality requires IO::Socket::INET6;
- if keeping the AWL database in SQL, the field awl.ip must be extended to
40 characters. The change is necessary to allow AWL to keep track of IPv6
addresses which may appear in a mail header even on non-IPv6 -enabled host.
While at it, consider also adding a field 'signedby' to the SQL table 'awl'
(and adding 'auto_whitelist_distinguish_signed 1' to local.cf);
see sql/README.awl for details. The change need not be undone even if
downgrading back to 3.2.* for some reason;
- fixing a protocol implementation error regarding a PING command required
bumping up the SPAMC protocol version to 1.5. Spamd retains compatibility
with older spamc clients. Combining new spamc clients with pre-3.3 versions
of a spamd daemon is not supported (but happens to work, except for the
PING and SKIP commands);
- if using one of the plugins (FreeMail, PhishTag, Reuse) which were
previously not part of the official package, please retire your local copy
to avoid it conflicting with a new native plugin;
- as the plugin AWL is no longer loaded by default, to continue using it
the following line is needed in one of the .pre files (e.g. local.pre):
loadplugin Mail::SpamAssassin::Plugin::AWL
- it may be worth mentioning that a rule DKIM_VERIFIED has been renamed
to DKIM_VALID to match its semantics;
- the DKIM plugin is now enabled by default for new installs, if the perl
module Mail::DKIM is installed. However, installation of SpamAssassin
will not overwrite existing .pre configuration files, so to use DKIM when
upgrading from a previous release that did not use DKIM, a directive:
loadplugin Mail::SpamAssassin::Plugin::DKIM
will need to be uncommented in file "v312.pre", or added to some
other .pre file, such as local.pre;
- due to changes in some internal data structures (like Bug 6185, 6254),
some third-party plugins may need to be updated. One such example is
the ClamAVPlugin plugin - please find a fresh version, which can be used
with both SpamAssassin versions 3.2.5 and 3.3.0, on its wiki page at
http://wiki.apache.org/spamassassin/ClamAVPlugin
- versions of amavisd-new between 2.5.2 and 2.6.1 (inclusive) are incompatible
with SpamAssassin 3.3; please upgrade amavisd to 2.6.2 or later, or apply
a workaround https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6257
- support for versions of perl 5.6.* is being gradually revoked
(may still work, but no promises and no support);
- preferred versions of perl are 5.8.8, 5.8.9, and 5.10.1 or later;
- on FreeBSD, please avoid using multithreaded versions of perl older
than 5.10.0 due to small default main thread's stack size, which may
not suffice for some regular expression evaluations;
INSTALLING RULES
Rules are normally installed by running a sa-update command.
The version of sa-update program should match the version of SpamAssassin
modules, so invoking sa-update should be performed only after installing
or upgrading SpamAssassin code, not before.
Installing rules from network is done with a single command,
normally run as root:
sa-update
Installing rules from files:
obtain all the following files:
Mail-SpamAssassin-rules-xxx.tgz
Mail-SpamAssassin-rules-xxx.tgz.asc
Mail-SpamAssassin-rules-xxx.tgz.md5
Mail-SpamAssassin-rules-xxx.tgz.sha1
(where xxx may look something like '3.3.0.r893295')
install rules from a compressed tar archive:
sa-update --install Mail-SpamAssassin-rules-xxx.tgz
(sa-update will need corresponding .asc and .sha1 files with the
same base name in the same directory as the .tgz file)
MAIN NEW FEATURES
- IPv6 support was substantially improved (see below);
- many improvements to the DKIM plugin (understands author domain signatures,
supports multiple signatures, ADSP support with overrides) - (see below);
- added 'if can(Class::method)' conditional statement, allowing configuration
settings to be conditional on plugin capabilities without requiring
new version releases to do so;
- added a --verbose option to the sa-update utility to show updated channels;
- added a configuration option 'time_limit', defaulting to 300 seconds
or whatever the caller (like spamd) provides; attempting to gracefully
terminate the checking when a time limit is reached, reporting the score
and test hits that were collected so far, along with an added hit on
a rule TIME_LIMIT_EXCEEDED;
- more expensive code sections are now instrumented with timing measurements;
timing report is logged as a debug message by the end of processing,
and made available to a caller and to 'add_header' directives through
a TIMING tag;
- added a configuration option skip_uribl_checks to the URIDNSBL plugin,
cross-documented it with skip_rbl_checks;
- preserve order of declared 'add_header' header fields;
- configurable network mask length for the AWL plugin (see below);
- added support for DCC reputations (see below);
- improved error handling and robustness (see below);
- added timestamps when logging on stderr;
- allowed debug areas to be excluded from debugging,
e.g.: -D all,norules,noconfig,nodcc
BUILDING AND PACKAGING
- rules are no longer distributed with the package, but installed by
sa-update
- Makefile.PL has been simplified and a bug fixed in a DESTDIR support
by increasing the minimum required version of ExtUtils::MakeMaker to 6.17
- tools check_whitelist and check_spamd are now included in the distribution,
now called 'sa-awl' and 'sa-check_spamd'
WORKAROUNDS TO PERL BUGS AND LIMITATIONS
- modified the Check.pm plugin to produce smaller chunks of source code
from rules (60 kB) to avoid Perl compiler crashing on exceeding stack size;
- localized global variables $1, $2, etc at several places, avoiding taint
issue from propagating;
- avoided Perl I/O bug by replacing line-by-line reading with read() where
suitable, or played down the EBADF status in other places and only report
it as a dbg instead of a die - while also providing a little speedup
(10 .. 25 %) on reading a message;
- provided a new sub Message::split_into_array_of_short_lines to split
a text into array of paragraph chunks of sizes between 1 kB and 2 kB,
giving less opportunity to runaway regular expressions in rules;
fixes bugs: 5717, 5644, 5795, 5486, 5801, 5041;
MEMORY FOOTPRINT
- as a side-effect of compiling rules in smaller chunks (to avoid compiler
crashes), virtual memory footprint of SpamAssassin is reduced;
- saved some memory by not importing the Pod::Usage unless it is needed;
- saved 350k+ of memory in sa-compile by replacing DynaLoader with XSLoader;
- removed unneeded index from MySQL bayes_token table;
IPv6 SUPPORT
- added IPv6 support for trusted_networks, internal_networks, msa_networks,
whitelist_from_rcvd, and other stuff that uses NetSet and the Received
header field parser, using NetAddr::IP;
- allowed usage of a remote dccifd host through an INET or INET6 socket;
- added IPv6 support to AWL plugin and its utility modules; a network
mask length is now configurable and defaults to /48, which controls
what data is stored in an AWL database;
- sql/README.awl and sql/awl_*.sql: increased suggested awl.ip field width
to 40 characters to be able to hold IPv6 addresses;
- IP_PRIVATE now includes ipv6 variants of private address space,
as well as the ipv6-mapped ipv4 addresses.
- NetSet now understands that ::ffff:192.168.1.2 and 192.168.1.2 are
the same address;
- IPv6 addresses are now properly read from Received header fields;
- when reading Received header fields, the "IPv6:" prefix is stripped from
IPv6 addresses, and "::ffff:" is removed from IPv6-mapped IPv4 addresses
(so strings can match them as simply IPv4 addresses);
- ::1/128 is always included in the trusted_networks/internal_networks set
similar to 127.0.0.0/8;
- some of the IPv6 functionality in SpamAssassin requires that a perl module
IO::Socket::INET6 is available (like accessing a DNS resolver over inet6,
talking to a dccifd host over inet6 socket, SPAMC protocol);
SPAMC
- Mail::SpamAssasin::Client ping may erroneously result in broken pipe;
bump spamc protocol version to 1.5, updated spamd, spamc and Client.pm;
- added -n / --connect-timeout switch to spamc, allowing to separate
a connection timeout from communication timeout;
- added --filter-retries and --filter-retry-sleep;
- increased allowed line length in spamc.conf files to 8 KiB and report
an error when the limit is exceeded;
- fixed issue where spamc would not time out connections to a hung spamd;
- spamc client library leaked the zlib compression buffer if compression
is used;
- spamc long option '--dest' was broken;
SPAMD
- when spamd is started with the daemonize option do not exit the parent
until a child signals that it has logged the pid, to allow a wrapper
script to simply continue immediately after starting spamd;
- additional tempfile cleanup in kill_handler;
- added SPAMD_LOCALHOST option to "make test" to allow specifying
non-127.0.0.1 IP address for use in FreeBSD jail;
API
- adding one optional argument to Mail::SpamAssassin::parse allows caller
to pass additional out-of-band information to SpamAssassin (such as a
deadline time, DKIM verification results, information about a SMTP session,
or dynamic rule hits); this information is made available to plugins and
the rest of the code through a 'suppl_attrib' hash;
- added option 'master_deadline' to the suppl_attrib argument of a
Mail::SpamAssassin::parse method, allowing the caller to override a
time_limit configuration setting;
- Plugin::Check - pick up 'rule_hits' from caller via the new mechanism
and call got_hit() on them;
- simplified adding dynamic score hits and dynamic rules by plugins
(such as AWL, CRM114, FuzzyOcr, Check) by letting got_hit() accept
options tflags and description, and letting it store a supplied
dynamic score for proper reporting;
- let the timing breakdown information be accessible to a caller through
the existing get_tag mechanism (tag TIMING);
- let the generated header fields ('add_header' configuration options)
be accessible to a caller through the existing get_tag mechanism
(tags ADDEDHEADER, ADDEDHEADERHAM, ADDEDHEADERSPAM);
RULES
- rules are no longer distributed with the package;
- new scores were generated by a genetic algorithm (GA) and then manually
tweaked based on cleaned datasets supplied by a dozen volunteers;
- dropped redundant rules or rules causing too many false positives;
- added or updated many rules; incomplete list in no particular order:
vbounce, lotsa_money, muchmoney, image spam, fill_this_form, FreeMail,
European Parliament, HTML attachments, uri_obfu*, urinsrhsbl, urinsrhssub,
urifullnsrhsbl, URI_OBFU_X9_WS, rDNS=localhost, INVALID_DATE_TZ_ABSURD,
RCVD_IN_PSBL, FRT_VALIUM*, BOUNCE_MESSAGE, VBOUNCE_MESSAGE,
__BOUNCE_UNDELIVERABLE, HELO_STATIC_HOST, FILL_THIS_FORM_FRAUD_PHISH,
CHALLENGE_RESPONSE, DKIM_VALID, DKIM_VALID_AU, DKIM_ADSP_*,
NML_ADSP_CUSTOM_{LOW,MED,HIGH}, __VIA_ML, MIME_BASE64_TEXT, LOTTO_URI,
FORGED_MUA_THEBAT_BOUN, FORGED_MUA_THEBAT_CS, UNRESOLVED_TEMPLATE,
__THEBAT_MUA, __ANY_OUTLOOK_MUA, RP_MATCHES_RCVD, one-word X-Mailer,
SPAN rules, skype and misquoted-HTML rules, HTML obfuscation and
Google feedproxy URI rules, advance_fee updates including further
evolved advance fee second-order metarules, test rule for
postmaster+abuse missing, FROM_MISSPACED, fixed FROM_CONTAINS_TAB, a
Facebook redirector pattern, fixed FPs with TVD_SPACE_RATIO regarding
one-word emails and ISO-2022-JP, added exclusion for __ISO_2022_JP_DELIM
to OBFUSCATING_COMMENT, GAPPY_SUBJECT, PLING_QUERY and FM_FRM_RN_L_BRACK
rules, RATWARE_BOUNDARY plus variant, superseded all previous
RATWARE_OUTLOOK stuff, resolved FP in obfuscated URI rule, fixed breakage
in tbird image rule, fixed SUBJECT_FUZZY_MEDS FP on unobfuscated "meds",
added misspaced From header field rule, numeric+cctld URI rule,
updated FH_DATE_PAST_20XX, ...
- added PSBL blacklist - http://psbl.surriel.com/
- added support for http://www.spamhaus.org/css/
- replaces HABEAS, BSP and SSC with RP CERTIFIED;
- use ReturnPath's RNBL, replacing SSBL;
- added rule for plain text attachments with octet-stream MIME type;
- avoided false positives on ISO-2022-JP messages in several rules;
- removed massmailers from uridnsbl_skip_domain in 25_uribl.cf;
- updated various default whitelists, uridnsbl_skip_domain, adsp_override, ...
PLUGINS
- new plugins: FreeMail, PhishTag, Reuse;
- now enabled by default: DKIM;
- now disabled by default: AWL;
- retired plugin: DomainKeys;
AWL PLUGIN
- plugin AWL is now disabled by default;
- added new configuration options auto_whitelist_ipv4_mask_len and
auto_whitelist_ipv6_mask_len to allow more control on what part of
an IP address is stored into an AWL database;
- README.awl: increased a suggested awl.ip field width to 40 characters
to support IPv6 addresses;
- AutoWhitelist.pm: allowed storing a canonicalized IPv6 address, cropped
to a configurable network mask (previously causing SQL server errors:
'value too long');
- let AWL with SQL keep separate records for DKIM-signed and unsigned mail
(when auto_whitelist_distinguish_signed configuration option is true,
and a field awl.signedby exists);
- avoided a race condition in SQLBasedAddrList.pm when multiple processes
try to insert-or-update an awl SQL record: trying INSERT first, and if
that fails go for UPDATE;
- gracefully handle NaN from corrupted database or a broken emulator or
virtualizer;
DCC PLUGIN
- added support for DCC reputations, added setting dcc_rep_percent,
new test check_dcc_reputation_range(), new tag DCCREP
(DCC servers supply reputation data only to licensed clients);
- allowed usage of a remote dccifd host through an INET or INET6 socket;
DKIM PLUGIN
- the DKIM plugin is now enabled by default for new installs if the perl
module Mail::DKIM is installed. However, installing SpamAssassin will
not overwrite existing .pre configuration files, so to use DKIM when
upgrading from a previous release that did not use DKIM, the directive:
loadplugin Mail::SpamAssassin::Plugin::DKIM
will need to be uncommented in file "v312.pre", or added to some
other .pre file, such as local.pre;
- absolute minimal version of Mail::DKIM is 0.31;
support for ADSP requires Mail::DKIM 0.34;
a DNS test (and rule) for NXDOMAIN is operational since Mail::DKIM 0.36_5,
so effectively the recommended version is Mail::DKIM 0.37 or later;
- a perl module Digest::SHA is required if the DKIM plugin is enabled.
If a perl module Digest::SHA is available, the module Digest::SHA1
becomes optional as far as SpamAssassin is concerned, but is still
needed by Razor agents;
- added support for multiple signatures (useful for whitelisting);
- plugin now distinguishes author domain signatures from third party
signatures (useful for whitelisting);
- provides a tag DKIMIDENTITY (in addition to DKIMDOMAIN);
- DKIM now supports Author Domain Signing Practices - ADSP (RFC 5617);
- use the Mail::DKIM::AuthorDomainPolicy instead of Mail::DKIM::DkimPolicy,
when available (since Mail::DKIM 0.34);
- implements an 'adsp_override' configuration directive and adds
an eval:check_dkim_adsp check, which is used by new DKIM_ADSP_* rules;
- rules contain an initial set of 'adsp_override' directives, listing
some of the more popular target domains for phishing (applicable only to
domains which sign all their direct mail with a DKIM or DK signature);
- this plugin can now re-use Mail::DKIM verification results if made
available by a caller, which saves resources and makes it possible
for SpamAssassin to work on a truncated large mail without breaking
DKIM signatures;
- check_dkim_signed and check_dkim_adsp eval rules can now take an optional
list of domain names, which limits their action to listed domains only.
It facilitates building DKIM-based rules for specific domains, without
having to resort to meta rules;
- draft-ietf-dkim-ssp-10/RFC-5617 made Author Domain Signature based on 'd':
updated ADSP code accordingly; changed whitelisting code to be based on
SDID ('d') instead of AUID ('i');
- Plugin/DKIM.pm: terminology changes in comments and logging according
to RFC 5617 and draft-ietf-dkim-rfc4871-errata-07;
BUG FIXES
- fixed Rule2XSBody segfaults;
- no longer treat user data as perl booleans (a string "0" is a false);
- avoid data from the wild be interpreted as perl regular expressions;
- ArchiveIterator: prevent _scan_directory from passing directories
to _scan_file (on NFS it would fail with EISDIR on read(2);
- fixed inserting the SpamAssassin -generated header fields after a
multiline Return-Path header field;
- fixed vpopmail support;
- fixed incorrect mode bits when creating lock files for AWL;
- fixed some cases where :addr headers were parsed incorrectly;
- fixed leakage of 'whitelist_from_rcvd' entries between spamd users;
- fixing run_and_catch, which failed to catch a non-timed run;
- 127/8 isn't an illegal IP;
- reworked the M::S::Timeout module to deal with nested timers as one would
expect: an inner timer shouldn't be able to extend an outer timer's limit;
account for time elapsed in the submitted subroutine when restarting an
outer timer; reset() should have accounted for time already spent;
deal with nested timed runs where alarm(0) does not provide remaining time;
- the 'exists:' evaluator in HEADER rules now works as documented
and tests for existence of a header field, instead of testing for
a header field body being nonempty; internally, the pms->get can
also now distinguish between empty and nonexistent header fields;
- applied fixes to header fields parsing in several places: header field
names are case-insensitive, whitespace is not required after a colon,
obsolete rfc822 syntax allowed whitespace before a colon;
VBounce: match "Received:" only at the beginning of a line;
- fixed bugs 6237 and 6295: 1.0.0.0/8 and 2.0.0.0/8 are now valid allocated
address ranges, fixed a corresponding rule RCVD_ILLEGAL_IP;
- fixed bug 6205 comment 5 in URIDetail.pm;
- 'pyzor_options' in Plugin/Pyzor.pm was not untainted;
- made the URIDetail plugin taint safe;
- fixed parsing of multi-line Received header fields for
BOUNCE_MESSAGE/VBOUNCE_MESSAGE et al;
- Bug 6206, Bug 2536: spamd: untaint directory as obtained from a password
file or from vpopmail utilities, avoid implicit untainting; report error
if user preferences file exists but cannot be accessed;
- avoided using raw data from DNS as a regexp in Plugin/ASN.pm;
- ensured the dbg() and info() calls always return the same value (true)
regardless of log level;
- suppressed logging of $& when its value is not available (i.e. when
no regexp has been evaluated during rule evaluation);
- Exporter never really worked in SA, was not enclosed in BEGIN {};
- masses/runGA and masses/mk-baseline-results: prevent a shell 'source'
command from loading an unrelated file named 'config' which happens to be
in the current PATH - must use a ./ in an arg to a 'source' command;
ERROR HANDLING, ROBUSTNESS
- improved error detection and reporting: test status of all system calls
and I/O operations (or explicitly document where not), and report
unexpected failures;
- eval calls now check for eval result instead of testing the $@, which
is not always reliable;
- localized $@ and $! in DESTROY methods to prevent potential calls to eval
and calls to system routines in code executed from a DESTROY method
from clobbering global variables $@ and $!;
- Util::helper_app_pipe_open_unix: contain a failing exec with an eval
to prevent additional cases of process cloning. The exec could fail
this way when given tainted arguments;
- Util::helper_app_pipe_open_unix: flush stdout and stderr before forking,
otherwise an error reported by exec (such as 'insecure dependency')
was lost in a buffer;
- eval-protected an open($fh,'-|') to capture implied fork failures
due to lack of system resource;
- explicit untainting: combine "use re 'taint'" with untaint_var(),
avoiding implicit perl untainting, along with workarounds to prevent it;
- added 'use strict' where missing;
- avoided a bunch of warnings on "Use of uninitialized value";
- clearly report reasons for helper application process failures;
- t/SATest.pm: provide information about the process failure reason
if a system() call fails; improved its reporting of failures;
- improved error reporting in Plugin/DCC.pm on finding a DCC home directory
to facilitate troubleshooting;
OTHER CHANGES
- pseudoheader "ALL:raw" returns a pristine header section,
and pseudoheader "ALL" returns a cleaned header section
- total rewrite of URI detection in plain text body;
- many updates to the list of top level domains;
- added 'util_rb_3tld', allowing 3-level TLDs to be listed in URIBLs and
allowing new 3TLDs to be added from rule updates;
- avoided trusted_networks bog down due to O(n^2) loop with millions
of entries;
- applied fixes to Plugin/VBounce.pm, updated VBounce ruleset;
- added support for a 'Communigate Pro' Received header field;
- parse Communigate Pro "with HTTPU" auth token;
- let DependencyInfo.pm understand a concept of recommended module version,
besides a required version;
- provided a workaround for Net::DNS::Packet::new inconsistency;
- let SpamAssassin use either Digest::SHA or Digest::SHA1, whichever is
available (the Digest::SHA is now a base module since perl 5.10.0);
- improved parsing of eval-type rules: allow unquoted domain names as
arguments, disallow unmatched quotes;
- provided a new module Mail::SpamAssassin::BayesStore::BDB. It should be
treated as alpha-quality (needs more testing) and is not yet ready for
production use;
- exposed existing function 'received_within_months' as an eval function
in Plugin/HeaderEval.pm;
- moved rc script to /var/lock/subsys/spamd instead of
/var/lock/subsys/spamassassin so 'service spamd status' will work;
- added feature to re-download MIRRRORED.BY files at least once a week, or if
'sa-update --refreshmirrors' switch is used;
- input delimiter $/ can be corrupted by a plugin, localize $/ and $\ before
calling a plugin;
- bumped the retry counter to 180 seconds for starting spamd on slow machines;
- resolved Bug 5325: syslog severity level in spamc/libspamc.c for max
message size (changed LOG_ERR into LOG_NOTICE for the message:
"skipped message, greater than max message size");
- added checker to avoid taint warnings if hostname is returned as '(none)';
- altered sa-update to produce an error message if a channel doesn't exist;
- Bug 6150, Bug 6127, Bug 5981, Bug 5950, Bug 6191: let spamd log/report
a child process exit status or aborting condition in an informative way;
- added checker to detect accidental match-everything regexps in rules;
- updated garescorer for 3.3.0: use more epochs in GA runs for better scores;
clarify some mass-check warning output, ensure rule name always appears at
start of line; if a rule had no default/existing score in 50_scores.cf,
don't tell the GA that 1.0 is an appropriate default value, instead pick
the midway point of its score range. this produces better results;
remove some dead code from masses/score-ranges-from-freqs;
- set garescorer.c to report performance as iterations per second;
- added test to ensure that all config settings are correctly handled when
switching between users; added more config setting type metadata to enable
those tests to work; and fix URIDetail to store config on the {conf} object,
not on the plugin;
- moved 'release tests' to xt/ directory; mirror long-running, net-tests and
stress tests with xt/50_testname.t scripts to enforce their run before a
release;
- made numerous additional and updated self-tests;
- added a Test::Perl::Critic release-test;
- cleaned up some code based on suggestions by perl module Test::Perl::Critic,
among others:
. enable TestingAndDebugging::ProhibitNoStrict test but allow the
use of 'no strict "refs"';
. deal with BuiltinFunctions::RequireGlobFunction;
. deal with ControlStructures::ProhibitMutatingListFunctions
removing this exception from xt/60_perlcritic.t;
. deal with BayesStore/BDB.pm, Variables::ProhibitConditionalDeclarations
. now that the module Time::HiRes is a required module, we can afford
to replace a select() with Time::HiRes::sleep, and remove exception
BuiltinFunctions::ProhibitSleepViaSelect from xt/60_perlcritic.t;
- updated documentation, fixing numerous typos and mistakes in documentation
text and in log messages;
- extensively improved development process:
. automated testing through Hudson, a continuous integration tool;
. improved mass-check system and rules oversight;
Recognized_Att_Keys) were made private, which broke SpamAssassin's
(ab)use of those variables. For details see
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6131.
Perl 5.10.1 includes ExtUtils::MakeMaker 6.55_2 and consequently building
SpamAssassin 3.2.5 with DESTDIR support in pkgsrc does not work anymore.
The fix changes the decision whether the used EU::MM module has good
enough DESTDIR support to depend only on the version number
(ie. mm_has_good_destdir is true) instead of the availability of the
key 'DESTDIR' in the (now non-public) Recognized_Att_Keys hash.
Ok to commit during freeze by wiz@
- removed packages p5-IO-Compress-Base, p5-IO-Compress-Zlib,
p5-IO-Compress-Bzip2 and p5-Compress-Zlib because they are
merged into p5-IO-Compress
- Updated dependend packages to depend on p5-IO-Compress
and bump PKGREVISION
Upstream changes:
2.017 30 March 2009
* Merged IO-Compress-Base, IO-Compress-Bzip2, IO-Compress-Zlib &
Compress-Zlib into IO-Compress.
* The interface to Compress-Raw-Zlib now uses the new LimitOutput
feature. This will make all of the zlib-related IO-Compress modules
less greedy in their memory consumption.
* Removed MAN3PODS from Makefile.PL
* A few changes to get the test harness to work on VMS courtesy of
Craig. A. Berry.
* IO::Compress::Base & IO::Uncompress::Base
Downgraded some croaks in the constructors to just set $! (by letting
the code attempt to open a file and fail).
This makes the behavior more consistent to a standard open.
[RT #42657]
* IO::Uncompress::Base
Doing a seek with MultiStream could drop some of the uncompressed
data. Fixed.
* IO::Compress::Zip
- Fixed problem with the uncompressed & uncompressed fields when
zip64 is enabled. They were set to 0x0000FFFF instead of
0xFFFFFFFF. Also the ZIP64 extra field was 4 bytes short.
Problem spotted by Dino Chiesa.
* IO::Uncompress::Unzip
- use POSIX::mktime instead of Time::Local::timelocal to convert
the zip DOS time field into Unix time.
* Compress::Zlib
- Documented Compress::Zlib::zlib_version()
to trigger/signal a rebuild for the transition 5.8.8 -> 5.10.0.
The list of packages is computed by finding all packages which end
up having either of PERL5_USE_PACKLIST, BUILDLINK_API_DEPENDS.perl,
or PERL5_PACKLIST defined in their make setup (tested via
"make show-vars VARNAMES=...").
Pkgsrc changes:
- p5-DB_File is now required on all systems, even those where Perl
already detects the native db-functions (dbopen,...) and thus
provides DB_File.
This should prevent subtle errors like the one in PR pkg/37751 at
the price of installing an additional package.
- Added explanation to patch-ay.
- patch-bc was adapted to the changes for the path of compiled rulesets.
- patch-bd is no longer necessary, the public key is now cross-verified.
- shut up some warnings from pkglint regarding "set -e" and quoted
variables.
Changes since version 3.2.4:
============================
3.2.5 is a minor bug-fix release. Summary of changes:
- bug 5775: newer gpg versions require keys to be cross-certified (backsig).
Did a cross-verify on our sa-update public key and re-exported. (If you
are already seeing "GPG validation failed" errors from sa-update, see
http://wiki.apache.org/spamassassin/SaUpdateKeyNotCrossCertified .)
- bug 5899: add perl version string to the storage area for compiled
rulesets, to avoid crashes when perl is upgraded between major versions
(e.g perl 5.8.x to 5.10.0) and the ABI breaks
- bug 5496, bug 5910: clear some FORGED_MUA_OUTLOOK false positives,
particularly on the new-format Message-ID generated by the Outlook
Express version used in Windows XP service pack 3
- bug 5730: when using Postgres >= 8.1.0 with Bayes, this error occurs:
'WARNING: nonstandard use of \ in a string literal at character'. fix,
thanks to Tomasz Ostrowski
- bug 5769: fix 'sa-compile: eval failed: Can't find label NO' error,
caused in rare circumstances when sa-compile attempted to deal with
rules written using 'replace_rules' features
- bug 5858: fix circular reference memory leak caused by some messages
- bug 5815: update 2TLD list to include .rs CCTLD
- bug 4706: remove HG_HORMOME rules due to poor performance
- bug 5835: typo in POD docs for SPF plugin; thanks to Benny Pedersen for fix
- bug 5839: a missing or failed eval rule function could mistakenly count
as a rule hit, fixed
- trivial bugfix for the VBounce ruleset: __BOUNCE_FROM_DAEMON incorrectly
used + instead of *, so some From addresses were not being recognised as
bounce senders
Pkgsrc changes:
- Due to "user-destdir" mode not working yet switched to "destdir" mode
for the time being.
- Explicitly listed licence information.
- Listed submitted bug identifiers for patch-ba and patch-bc.
Changes since version 3.2.3:
============================
3.2.4 is a major bug-fix release, with a few minor new features. Summary of
changes:
- bug 5599: allow load distribution of SA nameserver queries across all
nameservers listed in resolv.conf, using 'dns_options rotate'. thanks
to Pawel Sasin <hannibal /at/ wp-sa.pl>
- bug 5673: 'ALL' header was including spurious extra spaces between header
names and values. fix
- bug 5594: several major sa-compile fixes. major increase in overall speed;
cache results between runs to further increase speed; and fix a danger of
massive memory usage
- bug 5556: fix a variety of sa-compile portability issues, and support for
5.6.x perls
- bug 5514: make 'score set for a non-existent rule' a debug message, instead
of a lint warning, since it's a very frequent FAQ
- bug 5493: sa-compile fails to correctly deal with escaped backslashes. fix
- bug 5672: remove DNS_FROM_SECURITYSAGE (DNSBL lookups against
securitysage.com) due to unreliability
- bug 5476: update Bonded Sender (now Sender Score Certified) rules, and add
a rule for their strictly-confirmed-opt-in-required zone
- bug 5538: remove FORGED_MUA_AOL_FROM and FORGED_AOL_TAGS entirely; they're
obsolete, given the current capabilities of AOL mail user agents
- bug 5632: remove all completewhois.com DNSBL lookups, site seems to have
disappeared without warning
- bug 5715: allow for more than one sa-update MIRRORED.BY file host in DNS,
for redundancy
- bug 5662: DKIM changes: recognize author signature and multiple signatures
for whitelisting (with Mail::DKIM 0.29); disable useless
"check_dkim_signsome"; new eval rules "check_dkim_valid_author_sig" and
"check_dkim_valid" (an alias for a "check_dkim_verified" misnomer); new
tags _DKIMIDENTITY_ and _DKIMDOMAIN_; updated terminology; verification
speedup with Mail::DKIM 0.30 (or its pre-releases)
- bug 5696: sa-compile: cut regexp base strings at Unicode high codepoints,
to avoid corruption of patterns containing UTF-8
- bug 5637: bayes_file_mode is handled incorrectly when creating bayes.mutex,
resulting in incorrect permissions on that file; fix by Mihaly Barasz
- bug 5612: DB_File version 4.2.x has a bug that loops infinitely if files
named '__db.{filename}' are present; work around. thanks to J. Nick
Koston for the report and fix
- bug 5606: too-early init_learner() call causes root's user prefs file to
be read when spamd is started; this is inappropriate. fix
- bug 4179: if allow_user_rules is 1, user rules are not unique to each
user; one user's user rules can appear in later scans for other users
that are run using the same spamd process. fix
- bug 5680: ALL_TRUSTED can fire if a trusted MSA or webmail system receives
the message from an untrusted X-Originating-IP: header. fix
- bug 5626: in the 'spamassassin' script, install a signal handler for SIGHUP,
SIGINT, SIGTERM and SIGPIPE to ensure that temporary files are removed
- bug 5557: some temporary files are left not cleaned up on Windows; fix
- bug 5661: speed up Bayes SQL queries by allowing the use of indexes when
expiring
- bug 5611: support 'spamd --nouser-config -u username', which setuids to
'username' but does not read user_prefs files from anywhere
- bug 5665: spamd may fail to notice that a child has completed exiting,
and keeps in the child list in state 'K', eventually filling up the
child list with 'ghost' children. fix
- bug 5735: spamc should allow retry_sleep 0
- bug 5728: spamd: require -u with --sql-config or --ldap-config
- bug 5682: remove FH_HOST_ALMOST_IP, FH_HOST_EQ_D_D_D_D, due to false
positives and redundancy with RDNS_DYNAMIC; remove FH_HOST_EQ_D_D_D_DB
due to no hits
- bug 5681: look up IP addresses found in 'X-Yahoo-Post-IP' and
'X-SenderIP' headers, too, thanks to Martin Blapp
- Bug 5589: Refined async events handling and DNS lookup completions
- bug 5586: RDNS_NONE has false positives if the MTA doesn't put the hostname
in the Received header, like Communigate Pro. add an exception for this
- bug 5748: fix locale problem with use of external sort in sa-compile
Pkgsrc changes:
- Added security/gnupg as a required package. Without it, sa-update does not
work.
- Added requirement for p5-INET6 which may be needed for IPv6 nameservers.
Until now, this only worked because p5-Net-DNS already pulled in this
package.
- New option "inet6" (enabled by default) for switching off the requirement
for p5-INET6.
- Got rid of some backslashes in options.mk
- patch-ba and patch-bc correct discrepancies between the man page
and the code. Additionally, they adapt hard-coded paths to the actual
installation. Submitted upstream.
Relevant changes since version 3.2.1:
=====================================
3.2.3 is a major bug-fix release. Summary of changes:
- bug 5574: fix new setuid code to work with perl 5.6.1, and to support DCC and
Pyzor in all releases of perl
- bug 5107: change default 'user_scores_ldap_username' to be the null string,
allowing anonymous binding; fix 'schema' syntax error in LDAP config support
- zeroing an 'eval' rule's score did not stop it running. fix, thanks to
Richard Birkett <richard+spamassassin at musicbox.net>
- bug 5571: allow for new message ID format we have seen from Vista or Windows
2003 Server MAPI to avoid false positives
- bug 5397: RDNS_DYNAMIC should never fire on a PTR with 'static' in it; thanks
to Martin Blapp <mbr at freebsd.org>. bug 5563: RDNS_DYNAMIC rules use
order-dependent fields where it is unsafe to depend on this, fix. bug 5564:
__RDNS_DYNAMIC_IPADDR does not hit all of its test patterns, fix.
- bug 5475: fix FORGED_MUA_AOL_FROM to allow <*@{aol,cs}.*> addresses instead
of just <*@{aol,cs}.com>
3.2.2 is a minor bug-fix release. Summary of changes:
- bug 5548: Certain mail input can take a long time to scan with 100% CPU
utilisation, due to backtracking in a rule's regexp. fix
- bugs 5510, 5518, 5529: fix 'make test' when running as root, needed for CPAN
- bug 5419: kill -HUP of pidof spamd causes the ps name to change from spamd
to perl. fixed
- bug 5535: 'make test' errors in Windows caused by nonportable use of
getpwuid
- bug 5462: multiple DNS records for a host name should allow use of spamd -H
for load balancing installs to work
- bugs 5509, 5511: fix network lookup timeouts, where lookups were being lost
once a timeout was hit; also fix code to match documentation on
rbl_timeout's scaling and minimum duration of 1 second; and attempt to
collect already-received DNS responses when the timeout is reached; improve
related debugging output. Thanks to Mark Martinec
- bugs 5412, 5478, 5522: Fix problems using the spamc -x option with certain
other options; 'spamc -x -R' always returned 0, instead of the exit code, on
error. Bug 5478: in addition, 'spamc -x -e /command' would still run the
command, even if errors meant that the filtered text would be unavailable,
which contradicted -x.
- bug 5445: body eval tests defined in user_rules cause ugly 'Subroutine
_eval_tests_type11_prineg400_set3 redefined' warnings
- bug 5355: add in new entries for RegistrarBoundaries
- bug 5515: libsslspamc.so & libsslspamc.so can not build without -fPIC, but
we were picking up the wrong CFLAGS to do this.
- bug 5501: zero score for FH_HAS_XID
- bug 5449: allow_user_rules causes sa-compile / Rule2XSBody plugin to emit
spurious warnings; fix. also, add a new 'user_conf_parsing_end' plugin
hook, which is called after the per-user configuration is parsed
- bug 5182: update the sa-learn doc to mention that -u is only usable w/ sql
- bug 5534: fix harmless-but-ugly C compiler warning in sa-compile
Pkgsrc changes:
- Added some comments to patch files.
- Adapting patch-aq to changes in the README file.
- Added forgotten direct requirement for p5-Compress-Zlib.
- Since 3.2.0 at least version 3.43 of p5-HTML-Parser is needed.
- Installation to DESTDIR is possible as root.
- Substituted correct paths in spamc.pod (source file for spamc.1).
- Added VARBASE to BUILD_DEFS to silence pkglint warnings.
Changes since version 3.2.0:
============================
3.2.1 is a major bug-fix release, including a potential local DoS. The
major highlights are:
- bug 5480: fix for CVE-2007-2873: a local user symlink-attack DoS
vulnerability. It only affects systems where spamd is run as root, is used
with vpopmail or virtual users via the "-v"/"--vpopmail" OR
"--virtual-config-dir" switch, AND with the "-x"/"--no-user-config AND
WITHOUT the "-u"/"--username" switch AND with the "-l"/"--allow-tell" switch.
This is not default on any distro package, and is not a common configuration.
More details of the vulnerability can be read at
<http://spamassassin.apache.org/advisories/cve-2007-2873.txt>.
- bug 5488: zero some rules causing false positives: FH_HOST_EQ_D_D_D_DB and
FH_HOST_EQ_D_D_D_D.
- bug 5257: re-raise autolearn ham threshold to 1.0; the lower value
used in 3.2.0 was creating problems.
- bug 5422: in spamd, deleting hash entries from the SIGCHLD signal handler is
unsafe, causes corruption of the data structure, and results in 'prefork:
ordered child N to accept, but they reported state '1', killing rogue'
errors. fix.
- bug 5102: tighten up regexp for FORGED_HOTMAIL_RCVD to avoid some FPs.
- bug 5457: spamc build and test should handle not having zlib available.
- bug 5379: spamd could crash at startup if its preloading temporary directory
already exists. fix.
- bug 4616: spamc config can cause command line options to be ignored. fix.
- bug 5485: zero score DK/DKIM_POLICY_SIGNSOME rules since they'll always fire
due to defaults (unless there's an explicit SIGNALL policy).
- bug 5492: VBounce rule was looking in header instead of body for whitelisted
relays. fix.
- bug 5487: prevent multiple "urirhssub"s using the same zone from overwriting
each other.
- bug 5432 - Change default in Win32 build to not build spamc.
- bug 5446: add --updatedir option to sa-compile and remove inaccurate re2c
required version info from pod.
- bug 5436: add omitted "ifplugin" statements to the configuration, which would
otherwise cause lint errors if the default plugins were disabled.
- bug 5477: prevent Rule2XSBody info message from appearing on stderr during
spamd startup.
Pkgsrc changes:
- Removed PKG_DESTDIR_SUPPORT until the issue with encoded ownership in
+INSTALL files is resolved.
- made pkglint shut up about some warnings (CONFIGURE_DIRS, BUILD_DIRS,
hidden commands with @)
- parse-rules-for-masses has moved in the source archive.
- The directories "masses" and "tools" are no longer distributed in the
archive so I simplified the post-install target.
- Since "tools" is gone, the post-extract: target is obsolete.
- MESSAGE now points at sa-compile.
- Spamc depends on zlib now, so we needed the appropriate buildlink3 file.
Summary of changes since version 3.1.8:
=======================================
* new behavior for trusted_networks/internal_networks: the 127.* network
is now always considered trusted and internal, regardless of configuration.
* bug 3109: short-circuiting of 'definite ham' or 'definite spam' messages
based on individual short-circuit rules using the 'shortcircuit' setting,
by Dallas Engelken <dallase /at/ uribl.com>.
* bug 5305: implement 'msa_networks', for ISPs to specify their Mail
Submission Agents, and extend network trust accordingly.
* bug 4636: Add support for charset normalization, so rules can be written
in UTF-8 to match text in other charsets.
* sa-compile: compilation of SpamAssassin rules into a fast parallel-matching
DFA, implemented in native code.
* "tflags multiple": allow writing of rules that count multiple hits in a
single message.
* bug 4363: if a message uses CRLF for line endings, we should use it as
well, otherwise stay with LF as usual; important for Windows users.
* bug 4515: content preview was omitting first paragraph when no Subject:
header was present.
* The third-party modules used by sa-update are now required by the
SpamAssassin package, instead of being optional.
* Bug 5165: 'sa-update --checkonly' added to check for updates without
applying them; thanks to <anomie /at/ users.sourceforge.net>
* Bugs 4606, 4609: Adjust MIME parsing limits for nested multipart/* and
message/rfc822 MIME parts.
* bug 5295: add 'whitelist_auth', to whitelist addresses that send mail
using sender-authorization systems like SPF, Domain Keys, and DKIM
* Removed dependency on Text::Wrap CPAN module.
* Received header parsing updates/fixes/additions.
Spamc / spamd:
* bug 4603: Mail::SpamAssassin::Spamd::Apache2 -- mod_perl2 module,
implementing spamd as a mod_perl module, contributed as a Google Summer of
Code project by Radoslaw Zielinski.
* bug 3991: spamd can now listen on UNIX domain, TCP, and SSL sockets
simultaneously. Command-line semantics extended slightly, although fully
backwards compatibly; add the --ssl-port switch to allow TCP and SSL
listening at the same time.
* bug 3466: do Bayes expiration, if required, after results have been
passed back to the client from spamd; this helps avoid client timeouts.
* more complete IPv6 support.
* spamc: Add '-K' switch, to ping spamd.
* spamc: add '-z' switch, which compresses mails to be scanned using
zlib compression; very useful for long-distance use of spamc over the
internet.
* bug 5296: spamc '--headers' switch, which scans messages and transmits
back just rewritten headers. This is more bandwidth-efficient than the
normal mode of scanning, but only works for 'report_safe 0'.
* Bump spamd's protocol version to 1.4, to reflect new HEADERS verb used
for '--headers'.
Mail::SpamAssassin modules and API:
* bug 4589: allow M::SA::Message to use IO::File objects to read in
message (same as GLOB).
* bug 4517: rule instrumentation plugin hooks, to measure performance,
from John Gardiner Myers <jgmyers /at/ proofpoint.com>.
* add two features to core rule-parsing code; 1. optional behaviour to
recurse through subdirs looking for .cf/.pre's, to support rules compilers
working on rulesrc dir. 2. call back into invoking code on lint failure,
so rule compiler can detect which rules exactly fail the lint check.
* bug 5206: detect duplicate rules, and silently merge them internally
for greater efficiency.
* bug 5243: add Plugin::register_method_priority() API, allowing plugins
to control the relative ordering of plugin callbacks relative to other
plugins' implementations.
* Reduced memory footprint.
Plugins:
* bug 5236: Support Mail::SPF replacement for Mail::SPF::Query.
* bug 5127: allow mimeheader :raw rules to match newlines and folded-header
whitespace in MIME header strings.
* bug 4770: add ASN.pm plugin, contributed by Matthias Leisi <matthias at
leisi.net>
* bug 5271: move ImageInfo ruleset into 3.2.0 core rules, thanks to
Dallas Engelken <dallase /at/ uribl.com>.
* VBounce ruleset and plugin: detect spurious bounce messages sent by
broken mail systems in response to spam or viruses. (Based on Tim
Jackson's "bogus-virus-warnings.cf" ruleset.)
* DomainKeys/DKIM: Mail::DKIM is now preferred over Mail::DomainKeys,
since the latter module is no longer actively maintained, and Mail::DKIM
can handle both DomainKeys and DKIM signatures.
* DKIM: separate signature verification from fetching a policy: can save
a DNS lookup for each unverified message by setting score to 0 for all
policy-related rules (DKIM_POLICY_SIGNALL, DKIM_POLICY_SIGNSOME, and
DKIM_POLICY_TESTING). (thanks to Mark Martinec)
* DKIM: support testing flags in the public key, as well as in the policy
record. (thanks to Mark Martinec)
* DKIM: skip fetching a policy (SSP) if a signature does verify, according
to draft-allman-dkim-ssp-02 (thanks to Mark Martinec)
* Move rule functionality and checking into separate Check plugin, allowing
third parties to implement alternative scanner core algorithms.
* core EvalTests code moved into various plugins.
* Plus lots of miscellaneous bug fixes.
Pkgsrc changes:
- some cleanup of the Makefile to shut up a few pkglint notes and warnings
(USE_TOOLS, SUBST_CLASSES).
Changes since version 3.1.7:
============================
3.1.8 is a major bug-fix release, including a potential DoS. The major
highlights are:
- bug 5318: fix for CVE-2007-0451: possible DoS due to incredibly
long URIs found in the message content.
- bug 5240: disable perl module usage in update channels unless
--allowplugins is specified
- bug 5288: files with names starting/ending in whitespace weren't usable
- bug 5056: remove Text::Wrap related code due to upstream issues
- bug 5145: update spamassassin and sa-learn to better deal with STDIN
- bug 5140 and 5179: improvements and bug fixes related to DomainKeys
and DKIM support
- several updates for Received header parsing
- several documentation updates and random taint-variable related issues
Pkgsrc changes:
- none
Changes since version 3.1.6:
============================
3.1.7 is a "quick-fix" release; it contains only a fix for one bug,
introduced accidentally in 3.1.6:
- bug 5119: if admins had set rule scores in the site configuration in
/etc, sa-update would fail. Back out this change
bug 5044: include local site config in sa-update lint checks
bug 5048: --lint should not use network rules or AWL
bug 5081: sometimes, SIGHUPing spamd would leave one child process still alive due to a race in the SIGHUP handler and the preforking code. fixed
bug 5040: if in no-net scoreset, don't warn about net rules being zero-score dependencies of meta rules
bug 5105: M::SA::Client doesn't always catch failed connection to spamd, fixed
bug 5094: check for unit'd value used in AntiVirus.pm
bug 5089: enable adding headers with single digit zero value
bug 5077: fix false SPF_SOFTFAIL's when SPF queries timeout
bug 5080: fix bug in update RCVD_ILLEGAL_IP evaltest to properly deal with 127/8 fix
bug 5111: fix FORGED_JUNO_RCVD FP on webmail from untd.com's own systems
bug 4940: _get_date_header_time() should try dates one at a time not all dates joined together
bug 5098: add support for ecelerity Received headers, thanks to Joe Schaefer <joe+gmane at sunstarsys.com>
bug 4975: (trivial) avoid use of unit'd value in prefork warning message
bug 4418: remove no longer useful HTML_WEB_BUGS and HTML_LINK_IMAGE_BUG rules
bug 5101: fix bug in mbx code introduced in a backport of 3.2 AICache code
try to work around horribleness when checking in the entire built tree for the website update procedure
bug 5076: unescape hash characters in the config
add my nagios plugin for monitoring spamd to contrib/
bug 5018: update RegistrarBoundaries with new list of 2TLDs
bug 5020: drop __OUTLOOK_DOLLARS_MSGID, replace with the superset
rule __OE_MSGID_2
update replace_license_blocks, have it deal with C-style comments
better, put a sample commandline in place
bug 4492: bayes_ignore_header was case sensitive
add logging, enable zmi and doc mass-checks
bug 5049: allow commented lines in gpgkeyfile and channelfile files
bug 4952: set a default value for DEF_RULES_DIR_, LOCAL_RULES_DIR,
and LOCAL_STATE_DIR. this fixes an issue where third-party code
which hasn't been updated to deal with LOCAL_STATE_DIR can still
use updates.
bug 4848: Pyzor, DCC, and SpamCop plugins weren't looking at the
right place to see the 'dont_report_to_...' options
bug 4843: skip text/calendar parts when generating the body text
arrays
bug 5013: deal octal obfuscation of IP addrs in URLs
bug 5049: handle comments and whitespace in sa-update config files
and fix an error in channel name validation
bug 4981: remove broken urirhssub support for regexp subrule and
make perldoc match the code
bug 5060: backport ArchiveIterator changes for rule-QA system,
namely: the AICache caching subsystem for faster mass-checks, support
for combined --head/--tail semantics, more verbose 'showdots' to
avoid buildbot timeouts, and a new intra-child IPC me chanism for
multi-child mass-checks
bug 5034: fix endless loop possible from bad input or network error
bug 5065: implement whitelist_from_dk
bug 4823: clarify M::SA::Plugin documentation
Pkgsrc changes:
- Incorporated new version requirements for p5-Archive-Tar and p5-IO-Zlib.
Relevant changes since version 3.1.3:
=====================================
- bug 4941: if the first sa-update run failed and wasn't re-run to successful
completion, the local state directory would exist, and therefore SA sees no
rules. now, wait as long as possible to create the directory, and try to
remove it on failure.
- bug 4997: increase module version requirements for Archive::Tar to 1.23 and
IO::Zlib to 1.04
- bug 4966: fix major BSMTP bug, which rendered SA unusable with exim4 when
BSMTP is used.
- bug 4899: Windows had issues with single quotes around filenames so certain
things like pyzor, etc, wouldn't function.
- bug 4958: sa-update should work on Windows
- bug 4908: gtube.t test failed in non-english locales
- bug 4488: deal with potential memory leak due to Bayes and BayesStore
circular references
- bug 4862: update macro values in update channels (ie: @@CONTACT_ADDRESS@@)
Pkgsrc changes:
- patch-bb for no longer necessary (integrated upstream).
Changes since version 3.1.2:
============================
- bug 4926: given a certain set of parameters to spamd and a specially
formatted input message, users could cause spamd to execute arbitrary
commands as the spamd user
- bug 4932: the userstate dir and userprefs file would not be created
under certain conditions.
Pkgsrc changes:
- The updates for rule files go into $VARBASE/spamassassin/.
- This above directory and the directory sa-update-keys for the GPG keys
are now handled automatically by OWN_DIRS.
- The growing number of *.pre files are managed in a loop in the Makefile.
They are no longer contained in the static PLIST.
- Removed some unnecessary trailing slashes.
- Patching init.pre in order to disable the SPF plugin broke the spf.t
test. This is now fixed, although in a rather ugly way :-/.
- patch-ab no longer needs to use BSD_INSTALL_DATA_DIR because we create
the directories through INSTALLATION_DIRS.
- patch-ad and patch-az were removed (changes integrated upstream).
- patch-bb fixes a small documentation error.
- Fixed some warnings by pkglint about the SUBST framework in Makefile
and options.mk.
Relevant changes since version 3.1.1:
=====================================
- bug 4802: implement DKIM plugin, including whitelist_from_dkim support
- bug 3838: work around Perl bug causing captured RE variables to become
tainted -- thanks to Mark Martinec for pointing out the bug with
Perl itself
- bug 4850: re-enable the Razor2 plugin by default due to a service
policy change
- bug 4826: Razor2 plugin needs to load Mail::SpamAssassin::Timeout module
- bug 4827: M::SA::first_existing_path() would return the last array
entry passed in if none of the paths were found. Now return undef
instead and handle the error when it happens.
- bug 4813: generally open RE causes sendmail received header get read
in as qmail in error
- bug 4839: Logger.pm converts control chars including tab into
underscores which confuses a bunch of users when checking debug output.
Convert tab into space instead, etc.
- bug 4884: if a null message is passed in, there are several variables
which end up undefined causing warnings. fake an empty message if no
input is given.
- bug 4793: when replacing tags in a message (_TAG_), leave the tags
that don't exist alone instead of just removing them
- bug 4861, 4760: handle dccifd and dccproc failover properly, backport
relays_internal and relays_external code, backport bug 4760 fix so
that it's not possible to be in internal_networks without being in
trusted_networks as well
- bug 4901: deal more properly with failures in bgsend(). also, use
the proper variable to show when errors occur.
- bug 4867: fetchmail changed header formats at some point making Received
parsing fail in certain conditions
- bug 4699: use M::SA::Timeout for spamd copy_config call and allow for
empty $@ values
- bug 3754: if there's a problem opening a file via sa-learn or
spamassassin, return an error exit value.
Pkgsrc changes:
- Generic option "online-tests" replaces "spamassassin-test-net".
- Removed underscore from package-internal variables (pkglint
complained).
- patch-ay disables the SPF plugin to avoid confusing warnings in the log
files.
- patch-az fixes http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4826.
Relevant changes since version 3.1.0:
=====================================
- better validate a number of different configuration options
- support new Mail::DomainKeys API, which changed incompatibly between
0.18 and 0.80 without warning
- more properly handle new Received header formats
- bug 4788: backport sa-update from 3.2 along with the local_state_dir
code, etc.
- bug 4760: strictly validate trusted/internal network configurations
- bug 4696: consolidated fixes for timeout bugs
- bug 3710: add timeout to connect so spamc -t works
- bug 4363: if a message uses CRLF for line endings, use it for header
rewrites as well
- bug 4748: add ExpressionEngine and Google redirector patterns
- bug 3815: add _RELAYCOUNTRY_ tag so that the RelayCountry plugin can
put in the list of countries relayed through
- bug 4090: x86_64 platforms (linux specifically) have an issue compiling
libspamc.so causing RPM build failures
- bug 4791: fix issue where perl would throw a UTF-8 warning for certain
messages
- bugs 4606, 4609: Adjust MIME parsing limits
- bug 4780: fix IP_ADDRESS & LOCALHOST regexes to correctly parse IPv6
addresses
- bug 4728: DUL rules should only use the last external IP, not all but
the first of the external IPs
- bug 4700: certain privileged configuration settings can inject code,
due to a bad fix for bug 3846. Back that out
