pkgsrc/mail/spamdyke/Makefile

50 lines
1.3 KiB
Makefile
Raw Normal View History

# $NetBSD: Makefile,v 1.37 2018/07/29 08:41:18 schmonz Exp $
#
Update to 5.0.1. From the changelog: Fixed a typo in the README file for the OpenSSL "dhparam" command. Thanks to Eric Shubert for reporting this one. Removed unused variables from the dns* commands in the utils folder to fix compiler warnings. Fixed a bug in read_file() that returned uninitialized pointers if a file contained blank lines or comments at the top, causing segfaults when they were free()d. Thanks to Jeffrey Gordon and Quinn Comendant for reporting this one. Changed the directory naming scheme in the "generator" program to include the flowchart step numbers in the name. The old pattern was just too hard to follow visually and far too difficult to search for a specific test. Added more steps to the recipient validation flowchart and spamdyke-qrv's recipient validation filter to correctly handle addresses that are forwarded to an external address. Thanks to Stephen Marley for reporting this one. Changed search_file() in spamdyke-qrv to return a "not found" result when the file does not exist, instead of an error. Added a delay loop to exec_command_argv() in spamdyke and spamdyke-qrv to work around a race condition -- sometimes the child process will close its pipes in preparation for exiting and the parent's waitpid() will fire before the child has fully exited. This leads to erroneous returns showing the child has not exited when it really only needed another timeslice or two. This is fixed by looping with nanosleep() to wait a few tenths of a second after seeing this return code. Added a way to stop a test script run by creating a file named "stop". This allows it to be stopped without killing the process and potentially leaving the test platform in a partially (mis)configured state. Fixed the accessor function for the header-blacklist-entry and header-blacklist-file options to find their data in the filter_settings object instead of the option_set object. This is because the data is moved from the option_set immediately after it is set so the blacklist effect is cumulative when set from configuration directories. Reading from the wrong location meant the config-test feature was never testing those options at all. Thanks to Stefan for reporting this one. Fixed a pair of bugs in process_config_file(): one that would add empty values to the end of a list of blacklist/whitelist files if a directive was followed by a blank line and a commented-out directive (causing errors when the values are used), the other that would throw errors if a line in a configuration file contained only one space. Thanks to Les Fenison for reporting these. Fixed a bug in middleman() that would return an improper greeting when injecting both AUTH and STARTTLS banners into the EHLO response. Clients seeing this improper greeting would hang forever and eventually timeout. Thanks to Elliot Denk for reporting this one and sending a patch! Fixed a major thinko in smtp_filter that was carrying over the rejection data between recipients, even if a recipient had a configuration directory file that altered the overall configuration. This was leading to some recipients being incorrectly rejected under very specific (and likely very rare) conditions, which just happened to be met on my own server. Fixed a bug in copy_base_options that was not copying the "reason" data from the last rejection. Fixed an infinite loop in dnsdummy when priorities over 0 are used. Fixed a typo in dnsdummy that was truncating data when the verbose flags were used (weird, yes). Changed dnsdummy to fork a child process to return each query. This was the easiest solution to implement to allow new queries to be processed while waiting n seconds to send answers to previous queries. This is a fragile and wasteful solution -- if dnsdummy were intended for production use, a queue would be a much better solution. Changed all of the "verbose"-level error messages to include the name of the function, file and line that generated it. Every other message prefixed with "ERROR" already did this, so this makes things more consistent. Renamed all of the "FILTER" messages and added a new logging macro to print them named SPAMDYKE_LOG_FILTER(). This way they can continue to be output without function, file and line information. Renamed the SPAMDYKE_LOG_CONFIG_TEST() macro to SPAMDYKE_LOG_CONFIG_TEST_ERROR() and changed it to use LOG_LEVEL_ERROR instead of having a special LOG_LEVEL_CONFIG_TEST setting. This way the config-test messages can be changed to emit file, function and lines if needed (or not). Added SPAMDYKE_LOG_CONFIG_TEST_INFO() and SPAMDYKE_LOG_CONFIG_TEST_VERBOSE() as analogs to SPAMDYKE_LOG_CONFIG_TEST_ERROR(). Changed dnsdummy to encode multiple answers in the same response, if its config file contains multiple matches for the same query. Fixed a bug in dnsdummy that was adding extra bytes to the end of each answer. This turned out to be covering a matching (compensating) bug in spamdyke's DNS parsing code. I really hate it when that happens! Fixed a serious bug in nihdns_expand() that was causing spamdyke to incorrectly parse DNS responses with multiple answers; it would use the first answer, then skip the wrong number of bytes, causing it to conclude any subsequent answers were corrupted. Changed nihdns_expand() to return separate values for the number of bytes in the decoded string and the number of bytes the string occupies in the DNS packet. Due to packet compression, the numbers can be very different. Changed generator to add records to the named configuration so domains will resolve correctly during testing. Since using port numbers in resolv.conf is not allowed, there is no easy way to use dnsdummy for these tests. Discovered qmail-send does not check the percenthack or virtualdomains files when resolving forward addresses, only locals and assign. Updated spamdyke-qrv to behave the same way. Refined the success/failure detection in generator after learning more about qmail's behavior. If only it had some kind of accurate documentation... Extended the tests created by generator to also test conditions where spamdyke-qrv calls vpopmail to look up addresses. This increased the number of spamdyke-qrv tests more than tenfold! Added a "diagnostic output" flag to spamdyke-qrv to print the decision path it used to evaluate the address. Also added a test to the test generator to compare the diagnostic output with the expected decision path, to catch tests that may be producing the desired effect for the wrong reason. Fixed a bug in set_config_value() to make it possible to set CONFIG_TYPE_NAME_MULTIPLE options to "none" or unset specific values. Thanks to Konstantin for reporting this one. Added flags to smtpdummy to advertise STARTTLS support in response to EHLO. It doesn't actually do TLS, it just advertises it. Fixed smtp_filter() to block a client's STARTTLS command if tls-level is "none". Thanks to Les Fenison for reporting this one. Added a flag to the configure script for both spamdyke and spamdyke-qrv to compile with the address sanitizer library to catch memory access errors. Adjusted the version string to show when the sanitizer is in use. Changed the test scripts to always compile spamdyke with the address sanitizer (if available) when testing. The tests run a lot slower, but the sanitizer is too awesome to not use. Fixed a buffer underrun in examine_entry that was causing segfaults when searching files where wildcards are allowed at the beginning of the lines. Thanks to Dirk Kannapinn for reporting this one. Discovered a horrible problem with snprintf()'s %n format -- it returns the number of bytes it _would_have_ written *if* there were infinite space, not the number of bytes *actually* written as the man page states. So using %n at the end of the format string as a substitute for immediately calling strlen() is not safe. Good thing I don't ever do that, right? ...wait, I use that feature EVERYWHERE! (grrrrr) Thanks to the Google Address Sanitizer team for finding this one. Whoever implemented the %n feature in glibc can report to me any time for a free punch in the throat. I want my weekend back. Reverted the (apparently) useless change from 4.3.0 to use %n in snprintf() instead of the return value and replaced snprintf() with a macro named SNPRINTF() that explicitly compares the return value with the size of the buffer and returns the number of bytes ACTUALLY written. Fixed a harmless buffer overrun in sub_examine_tcprules_entry() that could have overwritten one byte of another variable on the stack with a null byte. Since the address was valid and that other variable is set just after the overwrite anyway, it wasn't actually a problem. But fixing it makes the address sanitizer happy, so it's fixed. Fixed a pair of huge buffer overruns in config_test_file_read() and config_test_file_read_write() that could load 63K of file contents past the end of the buffer (on the stack). Fortunately, these functions are only used by the config-test feature, never during normal operation. Fixed a buffer overflow in find_address() that would overwrite a single byte in the caller's stack with a null byte when parsing BATV addresses. From what I can tell, the effect of this bug would be to either truncate the parsed address or cause a segfault. Added undo_softlimit() to try to increase the "soft" limits on address space, stack size and memory size to maximum if they are less than infinite (and squawk if they cannot be reset to maximum). This will (hopefully) prevent problems caused by DJB's "softlimit" program, which is a useless piece of trash many qmail install guides *still* recommend using. Fixed a bug in the logging code of tls_read() that was using an "error" message to log at "verbose" level. The error message had more printf() format specifiers than the verbose logger was providing, which was leading to segfaults when the message was printed. Many thanks to Konstanin for a lot of help tracking this one down.
2015-05-03 16:22:08 +02:00
DISTNAME= spamdyke-5.0.1
PKGREVISION= 2
2012-01-17 01:15:24 +01:00
CATEGORIES= mail
MASTER_SITES= ${HOMEPAGE}releases/
EXTRACT_SUFX= .tgz
2012-01-17 01:15:24 +01:00
MAINTAINER= schmonz@NetBSD.org
HOMEPAGE= http://www.spamdyke.org/
COMMENT= Filters incoming SMTP connections to qmail
2012-01-17 01:15:24 +01:00
LICENSE= gnu-gpl-v2
2012-01-17 01:15:24 +01:00
WRKSRC= ${WRKDIR}/${PKGNAME_NOREV}/${PKGBASE}
Update to 3.1.6. Note that some command-line options and the format of some log entries have changed since 2.6.3; see UPGRADING.txt. Also, pkgsrc no longer installs the random extra utilities that are explicitly marked as unnecessary for spamdyke operation. From the changelog: VERSION 3.1.6 -- 2/11/2008 Fixed a serious bug in middleman() -- when the remote server sent its message data and QUIT command in a burst and disconnected before spamdyke read() all of the data, the last data returned from read() was printed twice. This could cause message corruption, especially in the case of attachments. Fixed a serious bug in middleman() -- when the remote server sent its data in bursts of 4096 bytes AND there were two lines of text in the data AND the 4096th character was not a newline AND there was a delay between the data bursts, memmove()ing the buffered data was causing corruption because the moved data was not being properly re-terminated. While processing the remaining buffered data (and waiting for another burst from the remote server), strchr() would seek past the end of the data to an old newline character and middleman() would erroneously conclude the next line of data was complete, ready for processing. Many thanks to Andreas Galatis and Dragomir Denev for reporting and helping me reproduce this one. Added a -W flag to sendrecv to introduce a delay between message data bursts. Added a -o flag to smtpdummy to save the message data to a file. VERSION 3.1.5 -- 1/22/2008 Fixed sendrecv to correctly process corrupted TLS negotiations instead of covering up bugs in spamdyke. Fixed spamdyke to not add garbage output at the beginning of TLS passthrough negotiations. This was causing SSL handshakes to fail. Thanks to Ronnie Tartar for reporting this one. VERSION 3.1.4 -- 1/21/2008 Fixed all of the Makefiles to remove a symbols directory Leopard's gcc seems to create when compiling in debug mode. Fixed middleman() to log the timeout message only once. Fixed middleman() to not expect input from the child process when the child process' input is being ignored or after the child process has exited. Fixed middleman() to correctly handle a rare situation -- when the child process was too slow responding that spamdyke's idle timeout was passed AND spamdyke was processing TLS data AND there was still data in the SSL buffer, spamdyke would loop infinitely, consuming 100% CPU. This was a very tricky bug to find and fix. Thanks to Pablo Gonzalez and Paolo for reporting this one and helping me debug it. Fixed middleman() to send message data to the child process line-by-line, even when the buffer is full. Added a new test program: smtpdummy. This one simulates an SMTP server and can add delays after specific commands. Changed sendrecv to use a 64K buffer for input and output data. Changed sendrecv to kill the its child process after its timeout expires. Changed sendrecv to optionally continue sending data in bursts after the end for the message data. Some mail servers do this. Changed sendrecv to deliberately send corrupt data while TLS is active. Changed test regression_009 to build its message payload at runtime instead of including a 0.75M file. This file was unnecesarily increasing the size of the spamdyke tarball. Fixed compiling on Solaris. Again. Thanks to Davide Bozzelli for reporting this. Again. Sigh. VERSION 3.1.3 -- 1/3/2008 Fixed the format string LOG_INFO_DNS_TXT to assign the parameters correctly and prevent bus errors when the DNS response text is long. Thanks to Stephan Rosenke for reporting this one. VERSION 3.1.2 -- 12/11/2007 Fixed smtp_filter() to set a flag after some SMTP commands to force middleman() to wait for input from the child process before proceeding. Some (nonspammer) mail servers send their data in bursts without waiting for responses. This was causing spamdyke to skip logging (but not filtering) if the DATA command was sent in a burst with RCPT TO. Thanks to Sebastien Guilbaud and Bucky Carr for reporting this one. Added a "-b" flag to sendrecv to simulate servers that send their message data (but not their SMTP commands) in bursts. VERSION 3.1.1 -- 11/12/2007 Added excessive logging to search_domain_directory() to log the directory search pattern. Changed all calls to spamdyke_log() to use the macros SPAMDYKE_LOG_NONE(), SPAMDYKE_LOG_ERROR(), SPAMDYKE_LOG_INFO(), SPAMDYKE_LOG_DEBUG() and SPAMDYKE_LOG_EXCESSIVE() instead. The macro tests the current log level without forcing a function call and also paves the way toward eliminating some logging code at compile-time. Fixed process_access() to correctly search for the RELAYCLIENT variable in spamdyke's environment. Thanks to Steve Cole for reporting this one. VERSION 3.1.0 -- 11/5/2007 Changed the "graylist-dir" and "no-graylist-dir" options to take multiple directories for servers that are hosting so many domains that they can't create enough domain folders in one place (wow). Added minimum and maximum values to all integer options and changed set_config_value() to generate error messages when values are out of range. Change usage() to print minimum and maximum integer values. Alphabetized the option list by long option name and changed process_config_file() to use a binary search algorithm when identifying directives, a theoretical improvement from O(n/2) to O(log n). Changed prepare_settings() to create an array of options indexed by the short option code. This introduces some constant-time work (O(1)) and greater memory usage. Changed process_command_line() to use the indexed array of options, theoretically reducing command line parsing work from O(n/2) to O(1). This is a win if the command line has many parameters or if it has parameters that are near the end of the unindexed option array. Testing confirms a small performance gain. Added command line options "config-test-smtpauth-username" and "config-test-smtpauth-password". Changed config_test_smtpauth() to run the authentication command(s) if a username and password are provided. This incorporates the functionality of checkpassword into spamdyke. Added the command line option "config-test-user" to change user and group IDs before running the configuration tests. This makes it easier to simulate running as the mail server. Changed process_config_file() and process_command_line() to print errors and stop when they encounter an option that is not legal in that location. At the moment, "help", "version", "config-test", "config-test-smtpauth-username", "config-test-smtpauth-password" and "config-test-user" are not valid in files; all options are valid on the command line. Changed config_test_dir_read() and config_test_graylist() to never examine the "." or ".." folders, even if readdir() and/or stat() report they are not folders. Thanks to Paulo Henrique for reporting this one. Changed set_config_value() to remove trailing slashes from directory paths. Added test_spamdyke_binary() to check if the spamdyke binary is setuid root (it should not be). Renamed test_settings() to config_test(). Moved all of the configuration test functions to config_test.[ch] -- they were cluttering up configuration.c. Made a few small updates to the help message text. Added additional vchkpw exit codes to exec_checkpassword() to explain why vchkpw exited, since it doesn't follow DJB's published checkpassword API. Moved md5.[ch] from the "utils" folder to the "spamdyke" folder and updated Makefile to compile them into spamdyke. Removed passwordcheck from the "utils" folder since spamdyke now contains its functionality. Added a README file to the "utils" folder to answer the biggest FAQ about those utilities. Fixed exec_command() to connect the output pipe to the child process's stdin instead of file descriptor 3. The bug was due to copying exec_checkpassword() and forgetting to change the value. Renamed exec_checkpassword() to exec_checkpassword_argv() and changed its arguments to expect a filename and an argument array. Added exec_checkpassword() to parse a command string into an argument array and call exec_checkpassword_argv(). Renamed exec_command() to exec_command_argv() and changed its arguments to expect a filename and an argument array. Added exec_command() to parse a command string into an argument array and call exec_command_argv(). Fixed numerous bugs in exec_command_argv() that were preventing it from actually gathering any input from the child process. Changed exec_command_argv() and exec_checkpassword_argv() to always log their child process errors to syslog, regardless of the user's preferences. Otherwise, the errors will be lost. Added the function find_path() to search the PATH for the given command without executing it. Changed exec_command_argv() and exec_checkpassword_argv() to use find_path() to locate the executable before fork()ing to catch typos. The child processes then use execve() to execute the command instead of exec_path(). Otherwise, the parent has a hard time determining that the child process quit because the command path was invalid. Changed exec_command_argv() and exec_checkpassword_argv() not to wait indefinitely for the child to exit after the timeout expires. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to limit the total number of queries they will recursively perform. This is to prevent a DoS situation where some domain has an unreasonable number of chained (non-circular) CNAME records. The limit is (arbitrarily) set at 16. Added the function config_test_child_capabilities() to test the qmail binary for SMTP AUTH and TLS patches. Depending on what is found, recommendations for spamdyke flags are made. Changed check_rdns_keywords() to allow top-level domains (like .com) to be used as keywords. This allows a way to reject connections from remote servers with rDNS names that contain the IP address and a two-letter country code. Unlike check_country_code(), specific country codes can now be chosen. Fixed do_spamdyke() not to wait indefinitely for all child processes to exit. This behavior was causing problems with DJB's recordio because recordio fork()s and uses its parent process to exec() spamdyke. This is very unusual. Changing wait(NULL) to waitpid() fixes the problem. Thanks to Bob Hutchinson for reporting this one. Added dns_initialize() and dns_get() to perform DNS queries by sending UDP packets instead of using the resolver library to do it. The resolver functions are just too slow and they try to do too much unnecessary work. dns_get() performs multiple requests for records (one for each kind of desired record) and, if no responses are received, sends requests to the secondary nameservers as well. Timeouts and retransmission times can now be controlled. This has resulted in a significant speedup in DNS resolutions; testing shows as much as a 10x performance increase in some situations. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to search all of the answers for the desired answer type before recursively querying CNAME answers. Some nameservers always put the CNAME answers first, even if other answer types are also given. This should allow spamdyke to find answers faster when domain admins have used a lot of CNAMEs. Added dns_a() to perform A record queries and changed all uses of gethostbyname() to use dns_a() instead. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to only perform their specific queries, not ask for CNAME records as well. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to send their own UDP packets instead of using the resolver library. Added dnscname to the "utils" folder to perform CNAME queries. Added dnsany to the "utils" folder to perform ANY queries and perform recursive CNAME lookups. Added "log-target" option to allow logging to stderr instead of syslog. Some people apparently like using the qmail-style "multilog" instead of syslog. I can't understand why but I'm here to serve. Thanks to John Hallam for suggesting this one. Changed all of the error messages about unexpected file types to specify what file type was found -- "non-regular file" was too vague to be useful. Changed the header in the files created by full logging to include the spamdyke version. Changed tls_end_inner() to use SSL_get_shutdown() to see if a shutdown signal has already been received. If SSL_shutdown() is used on a closed file descriptor, spamdyke will crash with SIGPIPE. Changed all instances of read(), write(), SSL_read() and SSL_write() to read or write as many bytes as possible in each call. This should provide a significant performance increase. The single-byte read()s and write()s were only used because I had badly misunderstood the relationship between select() and read()/write() -- blocking only occurs when select() indicates a file descriptor is not ready. If it is ready, read() and write() will handle as many bytes as they can without blocking. Thanks to Trog for setting me straight on this one. Rewrote most of sendrecv in the "tests" folder to use a multi-byte read(). Also took the opportunity to make sendrecv much faster and more polite, so it doesn't consume 100% CPU while waiting for qmail output. Fixed compiling errors on 64 bit Linux systems (Debian Etch x86_64 and Gentoo AMD64). Thanks to Juha-Pekka Jarvenpaa and FireBall for reporting this. Added config_test_file_type() to use stat() to find a file's type if readdir() either doesn't report it (Solaris) or reports "unknown" for all files (XFS). Thanks to Paulo Henrique for reporting this one. Fixed compiling errors on Solaris. Thanks to Limperis Antonis for reporting this. Changed the logging severity of the "unable to write X bytes to file descriptor" to debug instead of error. 99% of the time, the error occurs because the remote client disconnected unexpectedly and there's nothing the administrator can do about it anyway. Changed do_spamdyke() to ignore SIGPIPE signals. Changed do_spamdyke(), exec_command_argv() and exec_command_checkpassword() to change the SIGPIPE signal handler back to default for child processes after fork()ing but before exec()ing. Added a new logging level: excessive (4). It's to be used for printing very detailed debugging statements. Changed process_access() to permit access when no matching lines are found in the access file. Although DJB's tcprules documentation doesn't explicitly say so, no matching lines should allow access. Thanks to Steve Cole for reporting this one. VERSION 3.0.1 -- 9/12/2007 Fixed "configure" to remove the "_beta1" tag from the version number. That should never have been published. Changed usage() to show that optional values to long commands must be separated by an equals sign. getopt_long() is really becoming a hassle. Thanks to Richard Kreider for reporting this one. Fixed find_address() to accept addresses that aren't correctly delimited with <> characters and/or have multiple (illegal) spaces after the colon. Thanks to Davide Bozzelli for reporting this one. Fixed prepare_settings() to set the idle timeout seconds to the correct variable instead of setting the connection timeout variable. Thanks to Carlo Blohm for reporting this one. Fixed smtp_filter() to print the rejection message to HELO and EHLO, even if those commands appear in an improper place in the protocol. Fixed smtp_filter() to print the rejection message with an error code in response to STARTTLS if the command is given in an improper place in the protocol. Added some regression tests to find these bugs in the future. Fixed the usage statement in sendrecv to show the -w flag. VERSION 3.0.0 -- 9/11/2007 Added command line options never-graylist-rdns-dir, always-graylist-rdns-dir and rdns-whitelist-dir to search domain directory structures just like rdns-blacklist-dir. Added the command line option rdns-blacklist-file to search a file just like rdns-whitelist-file. Moved the command line option labels into configuration.c so they can be shared with the config file parser. Changed process_command_line() to build the list of short options from the list of long options instead of hardcoding them. Less maintenance this way. Modified check_rdns_keywords(), search_file() and search_tcprules_file() to correctly track line numbers and return the matching line number instead of just 1. Changed logging to allow the amount of information to be turned up or down. This should make spamdyke less chatty in the syslog for small errors. Modified smtp_filter() and run_tests() to report the matching filename and line number from check_rdns_keywords(), search_file() and search_tcprules_file() in syslog if the logging level is high enough. Fixed find_address() to locate the real email address and ignore BATV tags, relay paths and bang paths. Thanks to Walter Russo for reporting this one (again). Changed middleman() to obey minimums and maximums for the amount of time to select() for traffic. If spamdyke waits too long, the qmail process might not get wait()ed for a while, leaving a lot of defunct/zombie processes around. On a busy server, this could be a problem. Thanks to Jason M for reporting this one. Added process_config_file() to process configuration files instead of requiring all configuration to be done on the command line. At the moment, the file just uses the same (long option) directives as the command line. Added test_settings() to run tests on every configuration option and (hopefully) identify misconfigurations before someone makes them on a live server. Added the command line option "config-test" to run test_settings(). Renamed log_writeln() and log_write_rejection() to output_writeln() and output_write_rejection(), respectively, to make it clearer what they're doing. Changed smtp_filter() to allow multiple authentication attempts. Some clients retry authentication several times, presumably to deal with servers that can't use the authentication method they prefer. Changed middleman() to collect (and send) whole lines of input instead of single characters. Single character write()s were causing problems with Nagios and Windows clients. Changed output_write_rejection() to create a single output line and send it to output_writeln() all at once instead of sending a piece at a time. This keeps packets together for stupid Windows clients that just can't handle reassembling TCP packets correctly. Changed main() to always run spamdyke (as opposed to starting qmail without spamdyke listening) even if a whitelist is matched. This way, spamdyke can report all traffic to syslog, not just traffic that _may_ be filtered. Changed smtp_filter() and middleman() to catch the return codes from qmail when the remote client gives the recipient address. Now, if spamdyke doesn't block the recipient command but qmail does (e.g. for relaying), spamdyke will log the correct message. Incorporated GNU autoconf to create a "configure" script for spamdyke and the "utils" folder. The days of "make no_tls" and "make bsd" are thankfully over. Renamed all of the test folders to group them by function so it's easier to see what tests exist. Sequential numbers just weren't working. Changed dns_mx() to lookup the MX record before returning success. This means the sender MX filter now requires a mail exchanger record _and_ at least one mail exchanger must have an IP address. Before, the MX record was enough, even if there was no corresponding A record. Changed usage() to read the options and help text from get_spamdyke_options() in configuration.c so the help message won't ever be out of sync with the available options again. Added the command line option "tls-privatekey-password-file" to allow the SSL private key password to be read from a file instead of the command line. This way, the password isn't visible to everyone who can view a process list. Changed search_file(), search_tcprules_file() and check_rdns_keywords() so they no longer build their fscanf() patterns into a stack variable but instead use a literal search pattern assembled at compile time with STRINGIFY(). Added the command line options "hostname-file" and "hostname-command" to support reading the local hostname from a file or from a command (e.g. "hostname -f") instead of forcing it to be specified on the command line. Changed middleman() and smtp_filter() to always monitor and trust authentication carried out by qmail, even if "smtp-auth-command" was not given. This means spamdyke will always disable its filters for authenticated users even if it can't check the authentication itself. I'm not sure why I didn't design spamdyke this way in the first place. Added command line options recipient-whitelist-file and sender-whitelist-file so specific sender and recipient addresses can bypass the filters. Sender addresses are very easy to fake and recipient addresses are, of course, known to spammers, so both of these options are ill-advised. I've only added them due to popular demand. Added command line option check-rhsbl to check righthand-side blacklists. Both the server's rDNS domain name and the sender's email domain name are checked. Added command line options check-dns-whitelist and check-rhs-whitelist to allow DNS RBLs and RHSBLs to act as whitelists instead of blacklists. Anyone using DNS-based blacklists _and_ whitelists had better have some seriously fast DNS servers. Changed dns_txt(), dns_mx() and dns_ptr_lookup() to pass a stack of previous queries whenever they recursively lookup CNAME records, to prevent a cylical CNAME structure from leading to infinite recursion. NOT BACKWARDS COMPATIBLE: Changed the syslog entry format: renamed "origin" to "origin_ip", added "origin_rdns:" before the rDNS name, added "auth:" before the authenticated username and added "reason:" before the rejection reason when a timeout occurs. Changed process_command_line() to assume the remote IP address is 0.0.0.0 if the environment variable TCPREMOTEIP is not set. Added a ton more test scripts for all of the new options and for testing config files. Added dnsa, dnsns and dnssoa to the "utils" folder for performing DNS queries of A, NS and SOA records, respectively. Wouldn't it be AMAZING if the libc maintainers added standard functions to do these queries?! NOT BACKWARDS COMPATIBLE: Changed the "flag" options to take optional arguments instead of simply assuming "true" when the option was given. Unfortunately, getopt_long() is too stupid to handle them properly, which means clustered options (e.g. -rRc) can no longer be used. They must be separated (e.g. -r -R -c). Also, arguments given with the short version must not be separated by a space (e.g. -l3). NOT BACKWARDS COMPATIBLE: Renamed the long command line option "use-syslog" to "log-level". Fixed middleman() to completely bypass all processing when TLS passthrough is active. The additional processing was buffering TLS traffic until the data contained a newline character (purely by coincidence). This buffering was preventing the passthrough from functioning properly. Thanks to Dominik Dausch for reporting this one.
2008-02-25 08:38:00 +01:00
2012-01-17 01:15:24 +01:00
GNU_CONFIGURE= yes
BUILD_TARGET= ${PKGBASE}
MAKE_FLAGS+= CFLAGS=${CFLAGS:Q}
MAKE_FLAGS+= LDFLAGS=${LDFLAGS:Q}
SUBST_CLASSES+= paths
Update to 3.1.6. Note that some command-line options and the format of some log entries have changed since 2.6.3; see UPGRADING.txt. Also, pkgsrc no longer installs the random extra utilities that are explicitly marked as unnecessary for spamdyke operation. From the changelog: VERSION 3.1.6 -- 2/11/2008 Fixed a serious bug in middleman() -- when the remote server sent its message data and QUIT command in a burst and disconnected before spamdyke read() all of the data, the last data returned from read() was printed twice. This could cause message corruption, especially in the case of attachments. Fixed a serious bug in middleman() -- when the remote server sent its data in bursts of 4096 bytes AND there were two lines of text in the data AND the 4096th character was not a newline AND there was a delay between the data bursts, memmove()ing the buffered data was causing corruption because the moved data was not being properly re-terminated. While processing the remaining buffered data (and waiting for another burst from the remote server), strchr() would seek past the end of the data to an old newline character and middleman() would erroneously conclude the next line of data was complete, ready for processing. Many thanks to Andreas Galatis and Dragomir Denev for reporting and helping me reproduce this one. Added a -W flag to sendrecv to introduce a delay between message data bursts. Added a -o flag to smtpdummy to save the message data to a file. VERSION 3.1.5 -- 1/22/2008 Fixed sendrecv to correctly process corrupted TLS negotiations instead of covering up bugs in spamdyke. Fixed spamdyke to not add garbage output at the beginning of TLS passthrough negotiations. This was causing SSL handshakes to fail. Thanks to Ronnie Tartar for reporting this one. VERSION 3.1.4 -- 1/21/2008 Fixed all of the Makefiles to remove a symbols directory Leopard's gcc seems to create when compiling in debug mode. Fixed middleman() to log the timeout message only once. Fixed middleman() to not expect input from the child process when the child process' input is being ignored or after the child process has exited. Fixed middleman() to correctly handle a rare situation -- when the child process was too slow responding that spamdyke's idle timeout was passed AND spamdyke was processing TLS data AND there was still data in the SSL buffer, spamdyke would loop infinitely, consuming 100% CPU. This was a very tricky bug to find and fix. Thanks to Pablo Gonzalez and Paolo for reporting this one and helping me debug it. Fixed middleman() to send message data to the child process line-by-line, even when the buffer is full. Added a new test program: smtpdummy. This one simulates an SMTP server and can add delays after specific commands. Changed sendrecv to use a 64K buffer for input and output data. Changed sendrecv to kill the its child process after its timeout expires. Changed sendrecv to optionally continue sending data in bursts after the end for the message data. Some mail servers do this. Changed sendrecv to deliberately send corrupt data while TLS is active. Changed test regression_009 to build its message payload at runtime instead of including a 0.75M file. This file was unnecesarily increasing the size of the spamdyke tarball. Fixed compiling on Solaris. Again. Thanks to Davide Bozzelli for reporting this. Again. Sigh. VERSION 3.1.3 -- 1/3/2008 Fixed the format string LOG_INFO_DNS_TXT to assign the parameters correctly and prevent bus errors when the DNS response text is long. Thanks to Stephan Rosenke for reporting this one. VERSION 3.1.2 -- 12/11/2007 Fixed smtp_filter() to set a flag after some SMTP commands to force middleman() to wait for input from the child process before proceeding. Some (nonspammer) mail servers send their data in bursts without waiting for responses. This was causing spamdyke to skip logging (but not filtering) if the DATA command was sent in a burst with RCPT TO. Thanks to Sebastien Guilbaud and Bucky Carr for reporting this one. Added a "-b" flag to sendrecv to simulate servers that send their message data (but not their SMTP commands) in bursts. VERSION 3.1.1 -- 11/12/2007 Added excessive logging to search_domain_directory() to log the directory search pattern. Changed all calls to spamdyke_log() to use the macros SPAMDYKE_LOG_NONE(), SPAMDYKE_LOG_ERROR(), SPAMDYKE_LOG_INFO(), SPAMDYKE_LOG_DEBUG() and SPAMDYKE_LOG_EXCESSIVE() instead. The macro tests the current log level without forcing a function call and also paves the way toward eliminating some logging code at compile-time. Fixed process_access() to correctly search for the RELAYCLIENT variable in spamdyke's environment. Thanks to Steve Cole for reporting this one. VERSION 3.1.0 -- 11/5/2007 Changed the "graylist-dir" and "no-graylist-dir" options to take multiple directories for servers that are hosting so many domains that they can't create enough domain folders in one place (wow). Added minimum and maximum values to all integer options and changed set_config_value() to generate error messages when values are out of range. Change usage() to print minimum and maximum integer values. Alphabetized the option list by long option name and changed process_config_file() to use a binary search algorithm when identifying directives, a theoretical improvement from O(n/2) to O(log n). Changed prepare_settings() to create an array of options indexed by the short option code. This introduces some constant-time work (O(1)) and greater memory usage. Changed process_command_line() to use the indexed array of options, theoretically reducing command line parsing work from O(n/2) to O(1). This is a win if the command line has many parameters or if it has parameters that are near the end of the unindexed option array. Testing confirms a small performance gain. Added command line options "config-test-smtpauth-username" and "config-test-smtpauth-password". Changed config_test_smtpauth() to run the authentication command(s) if a username and password are provided. This incorporates the functionality of checkpassword into spamdyke. Added the command line option "config-test-user" to change user and group IDs before running the configuration tests. This makes it easier to simulate running as the mail server. Changed process_config_file() and process_command_line() to print errors and stop when they encounter an option that is not legal in that location. At the moment, "help", "version", "config-test", "config-test-smtpauth-username", "config-test-smtpauth-password" and "config-test-user" are not valid in files; all options are valid on the command line. Changed config_test_dir_read() and config_test_graylist() to never examine the "." or ".." folders, even if readdir() and/or stat() report they are not folders. Thanks to Paulo Henrique for reporting this one. Changed set_config_value() to remove trailing slashes from directory paths. Added test_spamdyke_binary() to check if the spamdyke binary is setuid root (it should not be). Renamed test_settings() to config_test(). Moved all of the configuration test functions to config_test.[ch] -- they were cluttering up configuration.c. Made a few small updates to the help message text. Added additional vchkpw exit codes to exec_checkpassword() to explain why vchkpw exited, since it doesn't follow DJB's published checkpassword API. Moved md5.[ch] from the "utils" folder to the "spamdyke" folder and updated Makefile to compile them into spamdyke. Removed passwordcheck from the "utils" folder since spamdyke now contains its functionality. Added a README file to the "utils" folder to answer the biggest FAQ about those utilities. Fixed exec_command() to connect the output pipe to the child process's stdin instead of file descriptor 3. The bug was due to copying exec_checkpassword() and forgetting to change the value. Renamed exec_checkpassword() to exec_checkpassword_argv() and changed its arguments to expect a filename and an argument array. Added exec_checkpassword() to parse a command string into an argument array and call exec_checkpassword_argv(). Renamed exec_command() to exec_command_argv() and changed its arguments to expect a filename and an argument array. Added exec_command() to parse a command string into an argument array and call exec_command_argv(). Fixed numerous bugs in exec_command_argv() that were preventing it from actually gathering any input from the child process. Changed exec_command_argv() and exec_checkpassword_argv() to always log their child process errors to syslog, regardless of the user's preferences. Otherwise, the errors will be lost. Added the function find_path() to search the PATH for the given command without executing it. Changed exec_command_argv() and exec_checkpassword_argv() to use find_path() to locate the executable before fork()ing to catch typos. The child processes then use execve() to execute the command instead of exec_path(). Otherwise, the parent has a hard time determining that the child process quit because the command path was invalid. Changed exec_command_argv() and exec_checkpassword_argv() not to wait indefinitely for the child to exit after the timeout expires. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to limit the total number of queries they will recursively perform. This is to prevent a DoS situation where some domain has an unreasonable number of chained (non-circular) CNAME records. The limit is (arbitrarily) set at 16. Added the function config_test_child_capabilities() to test the qmail binary for SMTP AUTH and TLS patches. Depending on what is found, recommendations for spamdyke flags are made. Changed check_rdns_keywords() to allow top-level domains (like .com) to be used as keywords. This allows a way to reject connections from remote servers with rDNS names that contain the IP address and a two-letter country code. Unlike check_country_code(), specific country codes can now be chosen. Fixed do_spamdyke() not to wait indefinitely for all child processes to exit. This behavior was causing problems with DJB's recordio because recordio fork()s and uses its parent process to exec() spamdyke. This is very unusual. Changing wait(NULL) to waitpid() fixes the problem. Thanks to Bob Hutchinson for reporting this one. Added dns_initialize() and dns_get() to perform DNS queries by sending UDP packets instead of using the resolver library to do it. The resolver functions are just too slow and they try to do too much unnecessary work. dns_get() performs multiple requests for records (one for each kind of desired record) and, if no responses are received, sends requests to the secondary nameservers as well. Timeouts and retransmission times can now be controlled. This has resulted in a significant speedup in DNS resolutions; testing shows as much as a 10x performance increase in some situations. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to search all of the answers for the desired answer type before recursively querying CNAME answers. Some nameservers always put the CNAME answers first, even if other answer types are also given. This should allow spamdyke to find answers faster when domain admins have used a lot of CNAMEs. Added dns_a() to perform A record queries and changed all uses of gethostbyname() to use dns_a() instead. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to only perform their specific queries, not ask for CNAME records as well. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to send their own UDP packets instead of using the resolver library. Added dnscname to the "utils" folder to perform CNAME queries. Added dnsany to the "utils" folder to perform ANY queries and perform recursive CNAME lookups. Added "log-target" option to allow logging to stderr instead of syslog. Some people apparently like using the qmail-style "multilog" instead of syslog. I can't understand why but I'm here to serve. Thanks to John Hallam for suggesting this one. Changed all of the error messages about unexpected file types to specify what file type was found -- "non-regular file" was too vague to be useful. Changed the header in the files created by full logging to include the spamdyke version. Changed tls_end_inner() to use SSL_get_shutdown() to see if a shutdown signal has already been received. If SSL_shutdown() is used on a closed file descriptor, spamdyke will crash with SIGPIPE. Changed all instances of read(), write(), SSL_read() and SSL_write() to read or write as many bytes as possible in each call. This should provide a significant performance increase. The single-byte read()s and write()s were only used because I had badly misunderstood the relationship between select() and read()/write() -- blocking only occurs when select() indicates a file descriptor is not ready. If it is ready, read() and write() will handle as many bytes as they can without blocking. Thanks to Trog for setting me straight on this one. Rewrote most of sendrecv in the "tests" folder to use a multi-byte read(). Also took the opportunity to make sendrecv much faster and more polite, so it doesn't consume 100% CPU while waiting for qmail output. Fixed compiling errors on 64 bit Linux systems (Debian Etch x86_64 and Gentoo AMD64). Thanks to Juha-Pekka Jarvenpaa and FireBall for reporting this. Added config_test_file_type() to use stat() to find a file's type if readdir() either doesn't report it (Solaris) or reports "unknown" for all files (XFS). Thanks to Paulo Henrique for reporting this one. Fixed compiling errors on Solaris. Thanks to Limperis Antonis for reporting this. Changed the logging severity of the "unable to write X bytes to file descriptor" to debug instead of error. 99% of the time, the error occurs because the remote client disconnected unexpectedly and there's nothing the administrator can do about it anyway. Changed do_spamdyke() to ignore SIGPIPE signals. Changed do_spamdyke(), exec_command_argv() and exec_command_checkpassword() to change the SIGPIPE signal handler back to default for child processes after fork()ing but before exec()ing. Added a new logging level: excessive (4). It's to be used for printing very detailed debugging statements. Changed process_access() to permit access when no matching lines are found in the access file. Although DJB's tcprules documentation doesn't explicitly say so, no matching lines should allow access. Thanks to Steve Cole for reporting this one. VERSION 3.0.1 -- 9/12/2007 Fixed "configure" to remove the "_beta1" tag from the version number. That should never have been published. Changed usage() to show that optional values to long commands must be separated by an equals sign. getopt_long() is really becoming a hassle. Thanks to Richard Kreider for reporting this one. Fixed find_address() to accept addresses that aren't correctly delimited with <> characters and/or have multiple (illegal) spaces after the colon. Thanks to Davide Bozzelli for reporting this one. Fixed prepare_settings() to set the idle timeout seconds to the correct variable instead of setting the connection timeout variable. Thanks to Carlo Blohm for reporting this one. Fixed smtp_filter() to print the rejection message to HELO and EHLO, even if those commands appear in an improper place in the protocol. Fixed smtp_filter() to print the rejection message with an error code in response to STARTTLS if the command is given in an improper place in the protocol. Added some regression tests to find these bugs in the future. Fixed the usage statement in sendrecv to show the -w flag. VERSION 3.0.0 -- 9/11/2007 Added command line options never-graylist-rdns-dir, always-graylist-rdns-dir and rdns-whitelist-dir to search domain directory structures just like rdns-blacklist-dir. Added the command line option rdns-blacklist-file to search a file just like rdns-whitelist-file. Moved the command line option labels into configuration.c so they can be shared with the config file parser. Changed process_command_line() to build the list of short options from the list of long options instead of hardcoding them. Less maintenance this way. Modified check_rdns_keywords(), search_file() and search_tcprules_file() to correctly track line numbers and return the matching line number instead of just 1. Changed logging to allow the amount of information to be turned up or down. This should make spamdyke less chatty in the syslog for small errors. Modified smtp_filter() and run_tests() to report the matching filename and line number from check_rdns_keywords(), search_file() and search_tcprules_file() in syslog if the logging level is high enough. Fixed find_address() to locate the real email address and ignore BATV tags, relay paths and bang paths. Thanks to Walter Russo for reporting this one (again). Changed middleman() to obey minimums and maximums for the amount of time to select() for traffic. If spamdyke waits too long, the qmail process might not get wait()ed for a while, leaving a lot of defunct/zombie processes around. On a busy server, this could be a problem. Thanks to Jason M for reporting this one. Added process_config_file() to process configuration files instead of requiring all configuration to be done on the command line. At the moment, the file just uses the same (long option) directives as the command line. Added test_settings() to run tests on every configuration option and (hopefully) identify misconfigurations before someone makes them on a live server. Added the command line option "config-test" to run test_settings(). Renamed log_writeln() and log_write_rejection() to output_writeln() and output_write_rejection(), respectively, to make it clearer what they're doing. Changed smtp_filter() to allow multiple authentication attempts. Some clients retry authentication several times, presumably to deal with servers that can't use the authentication method they prefer. Changed middleman() to collect (and send) whole lines of input instead of single characters. Single character write()s were causing problems with Nagios and Windows clients. Changed output_write_rejection() to create a single output line and send it to output_writeln() all at once instead of sending a piece at a time. This keeps packets together for stupid Windows clients that just can't handle reassembling TCP packets correctly. Changed main() to always run spamdyke (as opposed to starting qmail without spamdyke listening) even if a whitelist is matched. This way, spamdyke can report all traffic to syslog, not just traffic that _may_ be filtered. Changed smtp_filter() and middleman() to catch the return codes from qmail when the remote client gives the recipient address. Now, if spamdyke doesn't block the recipient command but qmail does (e.g. for relaying), spamdyke will log the correct message. Incorporated GNU autoconf to create a "configure" script for spamdyke and the "utils" folder. The days of "make no_tls" and "make bsd" are thankfully over. Renamed all of the test folders to group them by function so it's easier to see what tests exist. Sequential numbers just weren't working. Changed dns_mx() to lookup the MX record before returning success. This means the sender MX filter now requires a mail exchanger record _and_ at least one mail exchanger must have an IP address. Before, the MX record was enough, even if there was no corresponding A record. Changed usage() to read the options and help text from get_spamdyke_options() in configuration.c so the help message won't ever be out of sync with the available options again. Added the command line option "tls-privatekey-password-file" to allow the SSL private key password to be read from a file instead of the command line. This way, the password isn't visible to everyone who can view a process list. Changed search_file(), search_tcprules_file() and check_rdns_keywords() so they no longer build their fscanf() patterns into a stack variable but instead use a literal search pattern assembled at compile time with STRINGIFY(). Added the command line options "hostname-file" and "hostname-command" to support reading the local hostname from a file or from a command (e.g. "hostname -f") instead of forcing it to be specified on the command line. Changed middleman() and smtp_filter() to always monitor and trust authentication carried out by qmail, even if "smtp-auth-command" was not given. This means spamdyke will always disable its filters for authenticated users even if it can't check the authentication itself. I'm not sure why I didn't design spamdyke this way in the first place. Added command line options recipient-whitelist-file and sender-whitelist-file so specific sender and recipient addresses can bypass the filters. Sender addresses are very easy to fake and recipient addresses are, of course, known to spammers, so both of these options are ill-advised. I've only added them due to popular demand. Added command line option check-rhsbl to check righthand-side blacklists. Both the server's rDNS domain name and the sender's email domain name are checked. Added command line options check-dns-whitelist and check-rhs-whitelist to allow DNS RBLs and RHSBLs to act as whitelists instead of blacklists. Anyone using DNS-based blacklists _and_ whitelists had better have some seriously fast DNS servers. Changed dns_txt(), dns_mx() and dns_ptr_lookup() to pass a stack of previous queries whenever they recursively lookup CNAME records, to prevent a cylical CNAME structure from leading to infinite recursion. NOT BACKWARDS COMPATIBLE: Changed the syslog entry format: renamed "origin" to "origin_ip", added "origin_rdns:" before the rDNS name, added "auth:" before the authenticated username and added "reason:" before the rejection reason when a timeout occurs. Changed process_command_line() to assume the remote IP address is 0.0.0.0 if the environment variable TCPREMOTEIP is not set. Added a ton more test scripts for all of the new options and for testing config files. Added dnsa, dnsns and dnssoa to the "utils" folder for performing DNS queries of A, NS and SOA records, respectively. Wouldn't it be AMAZING if the libc maintainers added standard functions to do these queries?! NOT BACKWARDS COMPATIBLE: Changed the "flag" options to take optional arguments instead of simply assuming "true" when the option was given. Unfortunately, getopt_long() is too stupid to handle them properly, which means clustered options (e.g. -rRc) can no longer be used. They must be separated (e.g. -r -R -c). Also, arguments given with the short version must not be separated by a space (e.g. -l3). NOT BACKWARDS COMPATIBLE: Renamed the long command line option "use-syslog" to "log-level". Fixed middleman() to completely bypass all processing when TLS passthrough is active. The additional processing was buffering TLS traffic until the data contained a newline character (purely by coincidence). This buffering was preventing the passthrough from functioning properly. Thanks to Dominik Dausch for reporting this one.
2008-02-25 08:38:00 +01:00
SUBST_FILES.paths= spamdyke.h
SUBST_SED.paths= -e 's,@PREFIX@,${PREFIX:Q},g'
SUBST_SED.paths+= -e 's,@QMAILDIR@,${QMAILDIR:Q},g'
SUBST_STAGE.paths= do-configure
BUILD_DEFS+= QMAILDIR
Update to 4.0.0. The changelog is unwieldy; highlights: * Several options have changed names, behaviors, defaults, and/or possible settings, all of which are explained in the upgrade instructions. * Many new options have been added: config-dir config-dir-search dns-blacklist-file dns-level dns-max-retries-total dns-max-retries-primary dns-server-ip dns-server-ip-primary dns-timeout-secs dns-whitelist-file filter-level graylist-exception-ip-entry graylist-exception-rdns-entry graylist-level ip-blacklist-entry ip-in-rdns-keyword-blacklist-entry ip-in-rdns-keyword-whitelist-entry ip-in-rdns-keyword-whitelist-file ip-whitelist-entry local-domain-entry rdns-blacklist-entry rdns-whitelist-entry recipient-blacklist-entry recipient-whitelist-entry rejection-text-access-denied rejection-text-dns-blacklist rejection-text-earlytalker rejection-text-empty-rdns rejection-text-graylist rejection-text-ip-blacklist rejection-text-ip-in-cc-rdns rejection-text-ip-in-rdns-keyword-blacklist rejection-text-local-recipient rejection-text-max-recipients rejection-text-missing-sender-mx rejection-text-rdns-blacklist rejection-text-recipient-blacklist rejection-text-reject-all rejection-text-relaying-denied rejection-text-rhs-blacklist rejection-text-sender-blacklist rejection-text-smtp-auth-required rejection-text-timeout rejection-text-unresolvable-rdns relay-level rhs-blacklist-file rhs-whitelist-file run-as-user sender-blacklist-entry sender-whitelist-entry smtp-auth-level tls-level * An example spamdyke.conf has been provided.
2008-07-16 09:03:21 +02:00
EGDIR= ${PREFIX}/share/examples/${PKGBASE}
CONF_FILES+= ${EGDIR}/spamdyke.conf.example ${PKG_SYSCONFDIR}/spamdyke.conf
INSTALLATION_DIRS= bin share/doc/${PKGBASE} share/examples/${PKGBASE}
Update to 2.6.0. From the changelog: Added support for STARTTLS, similar to the way SMTP AUTH is implemented -- if a server certificate is available, spamdyke takes care of the TLS. If not but qmail supports TLS, spamdyke passes it through. Changed the read() and write() calls to the network to use macros named NETWORK_READ() and NETWORK_WRITE() that are replaced by TLS routines when TLS support has been compiled in. Changed the smtp_filter() return codes to use a mask/flag system because the possible permutations of PASS/INTERCEPT/QUIT with ADD/REMOVE/CAPTURE AUTH and ADD/REMOVE/CAPTURE TLS and CHILD QUIT/CONTINUE were getting too complex. Fixed search_file() to match a file entry where the search text matches the entry completely but the entry has wildcard markers at the start and/or end. Added TLS support to tests/sendrecv so TLS can be tested from scripts. Fixed numerous small bugs in tests/sendrecv that were causing inaccurate test results (false positives and false negatives). Updated all of the test scripts to make renumbering them easier. Added a new test script to exercise a small whitelist wildcard bug I found. Added 10 new test scripts to exercise the new TLS features. Changed process_command_line() and usage() to print a brief usage message if no parameters are given. Changed process_command_line() and usage() to print a brief error message if a bad parameter is given. Changed process_command_line() and usage() to print the full usage message if -h or --help is given. Changed process_command_line() and usage() to print the version header if -v or --version is given. Renamed test_smtpauth_crammd5, test_smtpauth_login and test_smtpauth_plain to smtpauth_crammd5, smtpauth_login and smtpauth_plain, respectively. Moved smtpauth_crammd5, smtpauth_login and smtpauth_plain from the utils folder to tests/smtpauth, since they're only used by the test scripts anyway. Added alternate command line options for people who spell "gray" with an "e". They do the same thing. Updated the documentation.
2007-05-30 07:58:10 +02:00
.include "options.mk"
do-install:
Update to 3.1.6. Note that some command-line options and the format of some log entries have changed since 2.6.3; see UPGRADING.txt. Also, pkgsrc no longer installs the random extra utilities that are explicitly marked as unnecessary for spamdyke operation. From the changelog: VERSION 3.1.6 -- 2/11/2008 Fixed a serious bug in middleman() -- when the remote server sent its message data and QUIT command in a burst and disconnected before spamdyke read() all of the data, the last data returned from read() was printed twice. This could cause message corruption, especially in the case of attachments. Fixed a serious bug in middleman() -- when the remote server sent its data in bursts of 4096 bytes AND there were two lines of text in the data AND the 4096th character was not a newline AND there was a delay between the data bursts, memmove()ing the buffered data was causing corruption because the moved data was not being properly re-terminated. While processing the remaining buffered data (and waiting for another burst from the remote server), strchr() would seek past the end of the data to an old newline character and middleman() would erroneously conclude the next line of data was complete, ready for processing. Many thanks to Andreas Galatis and Dragomir Denev for reporting and helping me reproduce this one. Added a -W flag to sendrecv to introduce a delay between message data bursts. Added a -o flag to smtpdummy to save the message data to a file. VERSION 3.1.5 -- 1/22/2008 Fixed sendrecv to correctly process corrupted TLS negotiations instead of covering up bugs in spamdyke. Fixed spamdyke to not add garbage output at the beginning of TLS passthrough negotiations. This was causing SSL handshakes to fail. Thanks to Ronnie Tartar for reporting this one. VERSION 3.1.4 -- 1/21/2008 Fixed all of the Makefiles to remove a symbols directory Leopard's gcc seems to create when compiling in debug mode. Fixed middleman() to log the timeout message only once. Fixed middleman() to not expect input from the child process when the child process' input is being ignored or after the child process has exited. Fixed middleman() to correctly handle a rare situation -- when the child process was too slow responding that spamdyke's idle timeout was passed AND spamdyke was processing TLS data AND there was still data in the SSL buffer, spamdyke would loop infinitely, consuming 100% CPU. This was a very tricky bug to find and fix. Thanks to Pablo Gonzalez and Paolo for reporting this one and helping me debug it. Fixed middleman() to send message data to the child process line-by-line, even when the buffer is full. Added a new test program: smtpdummy. This one simulates an SMTP server and can add delays after specific commands. Changed sendrecv to use a 64K buffer for input and output data. Changed sendrecv to kill the its child process after its timeout expires. Changed sendrecv to optionally continue sending data in bursts after the end for the message data. Some mail servers do this. Changed sendrecv to deliberately send corrupt data while TLS is active. Changed test regression_009 to build its message payload at runtime instead of including a 0.75M file. This file was unnecesarily increasing the size of the spamdyke tarball. Fixed compiling on Solaris. Again. Thanks to Davide Bozzelli for reporting this. Again. Sigh. VERSION 3.1.3 -- 1/3/2008 Fixed the format string LOG_INFO_DNS_TXT to assign the parameters correctly and prevent bus errors when the DNS response text is long. Thanks to Stephan Rosenke for reporting this one. VERSION 3.1.2 -- 12/11/2007 Fixed smtp_filter() to set a flag after some SMTP commands to force middleman() to wait for input from the child process before proceeding. Some (nonspammer) mail servers send their data in bursts without waiting for responses. This was causing spamdyke to skip logging (but not filtering) if the DATA command was sent in a burst with RCPT TO. Thanks to Sebastien Guilbaud and Bucky Carr for reporting this one. Added a "-b" flag to sendrecv to simulate servers that send their message data (but not their SMTP commands) in bursts. VERSION 3.1.1 -- 11/12/2007 Added excessive logging to search_domain_directory() to log the directory search pattern. Changed all calls to spamdyke_log() to use the macros SPAMDYKE_LOG_NONE(), SPAMDYKE_LOG_ERROR(), SPAMDYKE_LOG_INFO(), SPAMDYKE_LOG_DEBUG() and SPAMDYKE_LOG_EXCESSIVE() instead. The macro tests the current log level without forcing a function call and also paves the way toward eliminating some logging code at compile-time. Fixed process_access() to correctly search for the RELAYCLIENT variable in spamdyke's environment. Thanks to Steve Cole for reporting this one. VERSION 3.1.0 -- 11/5/2007 Changed the "graylist-dir" and "no-graylist-dir" options to take multiple directories for servers that are hosting so many domains that they can't create enough domain folders in one place (wow). Added minimum and maximum values to all integer options and changed set_config_value() to generate error messages when values are out of range. Change usage() to print minimum and maximum integer values. Alphabetized the option list by long option name and changed process_config_file() to use a binary search algorithm when identifying directives, a theoretical improvement from O(n/2) to O(log n). Changed prepare_settings() to create an array of options indexed by the short option code. This introduces some constant-time work (O(1)) and greater memory usage. Changed process_command_line() to use the indexed array of options, theoretically reducing command line parsing work from O(n/2) to O(1). This is a win if the command line has many parameters or if it has parameters that are near the end of the unindexed option array. Testing confirms a small performance gain. Added command line options "config-test-smtpauth-username" and "config-test-smtpauth-password". Changed config_test_smtpauth() to run the authentication command(s) if a username and password are provided. This incorporates the functionality of checkpassword into spamdyke. Added the command line option "config-test-user" to change user and group IDs before running the configuration tests. This makes it easier to simulate running as the mail server. Changed process_config_file() and process_command_line() to print errors and stop when they encounter an option that is not legal in that location. At the moment, "help", "version", "config-test", "config-test-smtpauth-username", "config-test-smtpauth-password" and "config-test-user" are not valid in files; all options are valid on the command line. Changed config_test_dir_read() and config_test_graylist() to never examine the "." or ".." folders, even if readdir() and/or stat() report they are not folders. Thanks to Paulo Henrique for reporting this one. Changed set_config_value() to remove trailing slashes from directory paths. Added test_spamdyke_binary() to check if the spamdyke binary is setuid root (it should not be). Renamed test_settings() to config_test(). Moved all of the configuration test functions to config_test.[ch] -- they were cluttering up configuration.c. Made a few small updates to the help message text. Added additional vchkpw exit codes to exec_checkpassword() to explain why vchkpw exited, since it doesn't follow DJB's published checkpassword API. Moved md5.[ch] from the "utils" folder to the "spamdyke" folder and updated Makefile to compile them into spamdyke. Removed passwordcheck from the "utils" folder since spamdyke now contains its functionality. Added a README file to the "utils" folder to answer the biggest FAQ about those utilities. Fixed exec_command() to connect the output pipe to the child process's stdin instead of file descriptor 3. The bug was due to copying exec_checkpassword() and forgetting to change the value. Renamed exec_checkpassword() to exec_checkpassword_argv() and changed its arguments to expect a filename and an argument array. Added exec_checkpassword() to parse a command string into an argument array and call exec_checkpassword_argv(). Renamed exec_command() to exec_command_argv() and changed its arguments to expect a filename and an argument array. Added exec_command() to parse a command string into an argument array and call exec_command_argv(). Fixed numerous bugs in exec_command_argv() that were preventing it from actually gathering any input from the child process. Changed exec_command_argv() and exec_checkpassword_argv() to always log their child process errors to syslog, regardless of the user's preferences. Otherwise, the errors will be lost. Added the function find_path() to search the PATH for the given command without executing it. Changed exec_command_argv() and exec_checkpassword_argv() to use find_path() to locate the executable before fork()ing to catch typos. The child processes then use execve() to execute the command instead of exec_path(). Otherwise, the parent has a hard time determining that the child process quit because the command path was invalid. Changed exec_command_argv() and exec_checkpassword_argv() not to wait indefinitely for the child to exit after the timeout expires. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to limit the total number of queries they will recursively perform. This is to prevent a DoS situation where some domain has an unreasonable number of chained (non-circular) CNAME records. The limit is (arbitrarily) set at 16. Added the function config_test_child_capabilities() to test the qmail binary for SMTP AUTH and TLS patches. Depending on what is found, recommendations for spamdyke flags are made. Changed check_rdns_keywords() to allow top-level domains (like .com) to be used as keywords. This allows a way to reject connections from remote servers with rDNS names that contain the IP address and a two-letter country code. Unlike check_country_code(), specific country codes can now be chosen. Fixed do_spamdyke() not to wait indefinitely for all child processes to exit. This behavior was causing problems with DJB's recordio because recordio fork()s and uses its parent process to exec() spamdyke. This is very unusual. Changing wait(NULL) to waitpid() fixes the problem. Thanks to Bob Hutchinson for reporting this one. Added dns_initialize() and dns_get() to perform DNS queries by sending UDP packets instead of using the resolver library to do it. The resolver functions are just too slow and they try to do too much unnecessary work. dns_get() performs multiple requests for records (one for each kind of desired record) and, if no responses are received, sends requests to the secondary nameservers as well. Timeouts and retransmission times can now be controlled. This has resulted in a significant speedup in DNS resolutions; testing shows as much as a 10x performance increase in some situations. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to search all of the answers for the desired answer type before recursively querying CNAME answers. Some nameservers always put the CNAME answers first, even if other answer types are also given. This should allow spamdyke to find answers faster when domain admins have used a lot of CNAMEs. Added dns_a() to perform A record queries and changed all uses of gethostbyname() to use dns_a() instead. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to only perform their specific queries, not ask for CNAME records as well. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to send their own UDP packets instead of using the resolver library. Added dnscname to the "utils" folder to perform CNAME queries. Added dnsany to the "utils" folder to perform ANY queries and perform recursive CNAME lookups. Added "log-target" option to allow logging to stderr instead of syslog. Some people apparently like using the qmail-style "multilog" instead of syslog. I can't understand why but I'm here to serve. Thanks to John Hallam for suggesting this one. Changed all of the error messages about unexpected file types to specify what file type was found -- "non-regular file" was too vague to be useful. Changed the header in the files created by full logging to include the spamdyke version. Changed tls_end_inner() to use SSL_get_shutdown() to see if a shutdown signal has already been received. If SSL_shutdown() is used on a closed file descriptor, spamdyke will crash with SIGPIPE. Changed all instances of read(), write(), SSL_read() and SSL_write() to read or write as many bytes as possible in each call. This should provide a significant performance increase. The single-byte read()s and write()s were only used because I had badly misunderstood the relationship between select() and read()/write() -- blocking only occurs when select() indicates a file descriptor is not ready. If it is ready, read() and write() will handle as many bytes as they can without blocking. Thanks to Trog for setting me straight on this one. Rewrote most of sendrecv in the "tests" folder to use a multi-byte read(). Also took the opportunity to make sendrecv much faster and more polite, so it doesn't consume 100% CPU while waiting for qmail output. Fixed compiling errors on 64 bit Linux systems (Debian Etch x86_64 and Gentoo AMD64). Thanks to Juha-Pekka Jarvenpaa and FireBall for reporting this. Added config_test_file_type() to use stat() to find a file's type if readdir() either doesn't report it (Solaris) or reports "unknown" for all files (XFS). Thanks to Paulo Henrique for reporting this one. Fixed compiling errors on Solaris. Thanks to Limperis Antonis for reporting this. Changed the logging severity of the "unable to write X bytes to file descriptor" to debug instead of error. 99% of the time, the error occurs because the remote client disconnected unexpectedly and there's nothing the administrator can do about it anyway. Changed do_spamdyke() to ignore SIGPIPE signals. Changed do_spamdyke(), exec_command_argv() and exec_command_checkpassword() to change the SIGPIPE signal handler back to default for child processes after fork()ing but before exec()ing. Added a new logging level: excessive (4). It's to be used for printing very detailed debugging statements. Changed process_access() to permit access when no matching lines are found in the access file. Although DJB's tcprules documentation doesn't explicitly say so, no matching lines should allow access. Thanks to Steve Cole for reporting this one. VERSION 3.0.1 -- 9/12/2007 Fixed "configure" to remove the "_beta1" tag from the version number. That should never have been published. Changed usage() to show that optional values to long commands must be separated by an equals sign. getopt_long() is really becoming a hassle. Thanks to Richard Kreider for reporting this one. Fixed find_address() to accept addresses that aren't correctly delimited with <> characters and/or have multiple (illegal) spaces after the colon. Thanks to Davide Bozzelli for reporting this one. Fixed prepare_settings() to set the idle timeout seconds to the correct variable instead of setting the connection timeout variable. Thanks to Carlo Blohm for reporting this one. Fixed smtp_filter() to print the rejection message to HELO and EHLO, even if those commands appear in an improper place in the protocol. Fixed smtp_filter() to print the rejection message with an error code in response to STARTTLS if the command is given in an improper place in the protocol. Added some regression tests to find these bugs in the future. Fixed the usage statement in sendrecv to show the -w flag. VERSION 3.0.0 -- 9/11/2007 Added command line options never-graylist-rdns-dir, always-graylist-rdns-dir and rdns-whitelist-dir to search domain directory structures just like rdns-blacklist-dir. Added the command line option rdns-blacklist-file to search a file just like rdns-whitelist-file. Moved the command line option labels into configuration.c so they can be shared with the config file parser. Changed process_command_line() to build the list of short options from the list of long options instead of hardcoding them. Less maintenance this way. Modified check_rdns_keywords(), search_file() and search_tcprules_file() to correctly track line numbers and return the matching line number instead of just 1. Changed logging to allow the amount of information to be turned up or down. This should make spamdyke less chatty in the syslog for small errors. Modified smtp_filter() and run_tests() to report the matching filename and line number from check_rdns_keywords(), search_file() and search_tcprules_file() in syslog if the logging level is high enough. Fixed find_address() to locate the real email address and ignore BATV tags, relay paths and bang paths. Thanks to Walter Russo for reporting this one (again). Changed middleman() to obey minimums and maximums for the amount of time to select() for traffic. If spamdyke waits too long, the qmail process might not get wait()ed for a while, leaving a lot of defunct/zombie processes around. On a busy server, this could be a problem. Thanks to Jason M for reporting this one. Added process_config_file() to process configuration files instead of requiring all configuration to be done on the command line. At the moment, the file just uses the same (long option) directives as the command line. Added test_settings() to run tests on every configuration option and (hopefully) identify misconfigurations before someone makes them on a live server. Added the command line option "config-test" to run test_settings(). Renamed log_writeln() and log_write_rejection() to output_writeln() and output_write_rejection(), respectively, to make it clearer what they're doing. Changed smtp_filter() to allow multiple authentication attempts. Some clients retry authentication several times, presumably to deal with servers that can't use the authentication method they prefer. Changed middleman() to collect (and send) whole lines of input instead of single characters. Single character write()s were causing problems with Nagios and Windows clients. Changed output_write_rejection() to create a single output line and send it to output_writeln() all at once instead of sending a piece at a time. This keeps packets together for stupid Windows clients that just can't handle reassembling TCP packets correctly. Changed main() to always run spamdyke (as opposed to starting qmail without spamdyke listening) even if a whitelist is matched. This way, spamdyke can report all traffic to syslog, not just traffic that _may_ be filtered. Changed smtp_filter() and middleman() to catch the return codes from qmail when the remote client gives the recipient address. Now, if spamdyke doesn't block the recipient command but qmail does (e.g. for relaying), spamdyke will log the correct message. Incorporated GNU autoconf to create a "configure" script for spamdyke and the "utils" folder. The days of "make no_tls" and "make bsd" are thankfully over. Renamed all of the test folders to group them by function so it's easier to see what tests exist. Sequential numbers just weren't working. Changed dns_mx() to lookup the MX record before returning success. This means the sender MX filter now requires a mail exchanger record _and_ at least one mail exchanger must have an IP address. Before, the MX record was enough, even if there was no corresponding A record. Changed usage() to read the options and help text from get_spamdyke_options() in configuration.c so the help message won't ever be out of sync with the available options again. Added the command line option "tls-privatekey-password-file" to allow the SSL private key password to be read from a file instead of the command line. This way, the password isn't visible to everyone who can view a process list. Changed search_file(), search_tcprules_file() and check_rdns_keywords() so they no longer build their fscanf() patterns into a stack variable but instead use a literal search pattern assembled at compile time with STRINGIFY(). Added the command line options "hostname-file" and "hostname-command" to support reading the local hostname from a file or from a command (e.g. "hostname -f") instead of forcing it to be specified on the command line. Changed middleman() and smtp_filter() to always monitor and trust authentication carried out by qmail, even if "smtp-auth-command" was not given. This means spamdyke will always disable its filters for authenticated users even if it can't check the authentication itself. I'm not sure why I didn't design spamdyke this way in the first place. Added command line options recipient-whitelist-file and sender-whitelist-file so specific sender and recipient addresses can bypass the filters. Sender addresses are very easy to fake and recipient addresses are, of course, known to spammers, so both of these options are ill-advised. I've only added them due to popular demand. Added command line option check-rhsbl to check righthand-side blacklists. Both the server's rDNS domain name and the sender's email domain name are checked. Added command line options check-dns-whitelist and check-rhs-whitelist to allow DNS RBLs and RHSBLs to act as whitelists instead of blacklists. Anyone using DNS-based blacklists _and_ whitelists had better have some seriously fast DNS servers. Changed dns_txt(), dns_mx() and dns_ptr_lookup() to pass a stack of previous queries whenever they recursively lookup CNAME records, to prevent a cylical CNAME structure from leading to infinite recursion. NOT BACKWARDS COMPATIBLE: Changed the syslog entry format: renamed "origin" to "origin_ip", added "origin_rdns:" before the rDNS name, added "auth:" before the authenticated username and added "reason:" before the rejection reason when a timeout occurs. Changed process_command_line() to assume the remote IP address is 0.0.0.0 if the environment variable TCPREMOTEIP is not set. Added a ton more test scripts for all of the new options and for testing config files. Added dnsa, dnsns and dnssoa to the "utils" folder for performing DNS queries of A, NS and SOA records, respectively. Wouldn't it be AMAZING if the libc maintainers added standard functions to do these queries?! NOT BACKWARDS COMPATIBLE: Changed the "flag" options to take optional arguments instead of simply assuming "true" when the option was given. Unfortunately, getopt_long() is too stupid to handle them properly, which means clustered options (e.g. -rRc) can no longer be used. They must be separated (e.g. -r -R -c). Also, arguments given with the short version must not be separated by a space (e.g. -l3). NOT BACKWARDS COMPATIBLE: Renamed the long command line option "use-syslog" to "log-level". Fixed middleman() to completely bypass all processing when TLS passthrough is active. The additional processing was buffering TLS traffic until the data contained a newline character (purely by coincidence). This buffering was preventing the passthrough from functioning properly. Thanks to Dominik Dausch for reporting this one.
2008-02-25 08:38:00 +01:00
cd ${WRKSRC}; \
Update to 2.4.0. From the changelog: Added search_ip_file() to search files for IP addresses and allow wildcards, network sizes (numbers of bits, e.g. 11.22.33.44/16) and netmasks (e.g. 11.22.33.0/255.255.255.0). Added new options to allow graylisting exclusions by IP address and rDNS name. Added new options to activate graylisting for only certain IP addresses and rDNS names. Updated the documentation. Added some explicit casting to printf("%.*s") arguments to make them ints instead of longs. gcc 3.3 on OpenBSD was complaining. Changed a datatype in log_write() from long to time_t. gcc 3.3 on OpenBSD was complaining. Changed all uses of sprintf() to snprintf(), even though they were already safe from buffer overruns. gcc 3.3 on OpenBSD was complaining. Added a new target to the Makefile in the spamdyke and utils folders named "openbsd" that compiles without the "-lresolv" flag. gcc 3.3 on OpenBSD includes the resolver library automatically and throws an error when it is explicitly specified. Added some additional #include directives in dnsmx.c, dnsptr.c and dnstxt.c because they're not included by resolv.h on OpenBSD. Gained a broader understanding of the resolver library and DNS packet structure, then rewrote most of dns_txt() and dns_ptr_lookup() to (more) correctly process DNS data. Added dns_mx() to perform MX record lookups. Changed process_command_line() to use getopt_long() and added long option equivalents to the existing command line flags. Added the option "reject-missing-sender-mx" to reject email from senders whose domains aren't local and don't have MX records and/or A records. AOL does this. Updated the usage statement to show the new (long) command line options. Updated the README.txt to show the new (long) command line options. Moved domain2path and domainsplit into a new folder named "utils". Created dnsmx, dnsptr and dnstxt in the utils folder by copying the dns_mx(), dns_ptr() and dns_txt() functions from spamdyke. They are simple command line utilities that no one will ever use except as examples of how to make MX, PTR and TXT DNS queries using libc. As far as Google knows, there are no such examples anywhere else on the internet. Changed the DNS RBL code to check for A records in addition to TXT records. Some RBLs are using A records, I don't know why. Fixed the DNS RBL code not to check 127.0.0.1 for RBL entries.
2007-04-11 16:50:37 +02:00
for f in spamdyke; do \
${INSTALL_PROGRAM} $${f} ${DESTDIR}${PREFIX}/bin; \
Update to 2.4.0. From the changelog: Added search_ip_file() to search files for IP addresses and allow wildcards, network sizes (numbers of bits, e.g. 11.22.33.44/16) and netmasks (e.g. 11.22.33.0/255.255.255.0). Added new options to allow graylisting exclusions by IP address and rDNS name. Added new options to activate graylisting for only certain IP addresses and rDNS names. Updated the documentation. Added some explicit casting to printf("%.*s") arguments to make them ints instead of longs. gcc 3.3 on OpenBSD was complaining. Changed a datatype in log_write() from long to time_t. gcc 3.3 on OpenBSD was complaining. Changed all uses of sprintf() to snprintf(), even though they were already safe from buffer overruns. gcc 3.3 on OpenBSD was complaining. Added a new target to the Makefile in the spamdyke and utils folders named "openbsd" that compiles without the "-lresolv" flag. gcc 3.3 on OpenBSD includes the resolver library automatically and throws an error when it is explicitly specified. Added some additional #include directives in dnsmx.c, dnsptr.c and dnstxt.c because they're not included by resolv.h on OpenBSD. Gained a broader understanding of the resolver library and DNS packet structure, then rewrote most of dns_txt() and dns_ptr_lookup() to (more) correctly process DNS data. Added dns_mx() to perform MX record lookups. Changed process_command_line() to use getopt_long() and added long option equivalents to the existing command line flags. Added the option "reject-missing-sender-mx" to reject email from senders whose domains aren't local and don't have MX records and/or A records. AOL does this. Updated the usage statement to show the new (long) command line options. Updated the README.txt to show the new (long) command line options. Moved domain2path and domainsplit into a new folder named "utils". Created dnsmx, dnsptr and dnstxt in the utils folder by copying the dns_mx(), dns_ptr() and dns_txt() functions from spamdyke. They are simple command line utilities that no one will ever use except as examples of how to make MX, PTR and TXT DNS queries using libc. As far as Google knows, there are no such examples anywhere else on the internet. Changed the DNS RBL code to check for A records in addition to TXT records. Some RBLs are using A records, I don't know why. Fixed the DNS RBL code not to check 127.0.0.1 for RBL entries.
2007-04-11 16:50:37 +02:00
done; \
Update to 3.1.6. Note that some command-line options and the format of some log entries have changed since 2.6.3; see UPGRADING.txt. Also, pkgsrc no longer installs the random extra utilities that are explicitly marked as unnecessary for spamdyke operation. From the changelog: VERSION 3.1.6 -- 2/11/2008 Fixed a serious bug in middleman() -- when the remote server sent its message data and QUIT command in a burst and disconnected before spamdyke read() all of the data, the last data returned from read() was printed twice. This could cause message corruption, especially in the case of attachments. Fixed a serious bug in middleman() -- when the remote server sent its data in bursts of 4096 bytes AND there were two lines of text in the data AND the 4096th character was not a newline AND there was a delay between the data bursts, memmove()ing the buffered data was causing corruption because the moved data was not being properly re-terminated. While processing the remaining buffered data (and waiting for another burst from the remote server), strchr() would seek past the end of the data to an old newline character and middleman() would erroneously conclude the next line of data was complete, ready for processing. Many thanks to Andreas Galatis and Dragomir Denev for reporting and helping me reproduce this one. Added a -W flag to sendrecv to introduce a delay between message data bursts. Added a -o flag to smtpdummy to save the message data to a file. VERSION 3.1.5 -- 1/22/2008 Fixed sendrecv to correctly process corrupted TLS negotiations instead of covering up bugs in spamdyke. Fixed spamdyke to not add garbage output at the beginning of TLS passthrough negotiations. This was causing SSL handshakes to fail. Thanks to Ronnie Tartar for reporting this one. VERSION 3.1.4 -- 1/21/2008 Fixed all of the Makefiles to remove a symbols directory Leopard's gcc seems to create when compiling in debug mode. Fixed middleman() to log the timeout message only once. Fixed middleman() to not expect input from the child process when the child process' input is being ignored or after the child process has exited. Fixed middleman() to correctly handle a rare situation -- when the child process was too slow responding that spamdyke's idle timeout was passed AND spamdyke was processing TLS data AND there was still data in the SSL buffer, spamdyke would loop infinitely, consuming 100% CPU. This was a very tricky bug to find and fix. Thanks to Pablo Gonzalez and Paolo for reporting this one and helping me debug it. Fixed middleman() to send message data to the child process line-by-line, even when the buffer is full. Added a new test program: smtpdummy. This one simulates an SMTP server and can add delays after specific commands. Changed sendrecv to use a 64K buffer for input and output data. Changed sendrecv to kill the its child process after its timeout expires. Changed sendrecv to optionally continue sending data in bursts after the end for the message data. Some mail servers do this. Changed sendrecv to deliberately send corrupt data while TLS is active. Changed test regression_009 to build its message payload at runtime instead of including a 0.75M file. This file was unnecesarily increasing the size of the spamdyke tarball. Fixed compiling on Solaris. Again. Thanks to Davide Bozzelli for reporting this. Again. Sigh. VERSION 3.1.3 -- 1/3/2008 Fixed the format string LOG_INFO_DNS_TXT to assign the parameters correctly and prevent bus errors when the DNS response text is long. Thanks to Stephan Rosenke for reporting this one. VERSION 3.1.2 -- 12/11/2007 Fixed smtp_filter() to set a flag after some SMTP commands to force middleman() to wait for input from the child process before proceeding. Some (nonspammer) mail servers send their data in bursts without waiting for responses. This was causing spamdyke to skip logging (but not filtering) if the DATA command was sent in a burst with RCPT TO. Thanks to Sebastien Guilbaud and Bucky Carr for reporting this one. Added a "-b" flag to sendrecv to simulate servers that send their message data (but not their SMTP commands) in bursts. VERSION 3.1.1 -- 11/12/2007 Added excessive logging to search_domain_directory() to log the directory search pattern. Changed all calls to spamdyke_log() to use the macros SPAMDYKE_LOG_NONE(), SPAMDYKE_LOG_ERROR(), SPAMDYKE_LOG_INFO(), SPAMDYKE_LOG_DEBUG() and SPAMDYKE_LOG_EXCESSIVE() instead. The macro tests the current log level without forcing a function call and also paves the way toward eliminating some logging code at compile-time. Fixed process_access() to correctly search for the RELAYCLIENT variable in spamdyke's environment. Thanks to Steve Cole for reporting this one. VERSION 3.1.0 -- 11/5/2007 Changed the "graylist-dir" and "no-graylist-dir" options to take multiple directories for servers that are hosting so many domains that they can't create enough domain folders in one place (wow). Added minimum and maximum values to all integer options and changed set_config_value() to generate error messages when values are out of range. Change usage() to print minimum and maximum integer values. Alphabetized the option list by long option name and changed process_config_file() to use a binary search algorithm when identifying directives, a theoretical improvement from O(n/2) to O(log n). Changed prepare_settings() to create an array of options indexed by the short option code. This introduces some constant-time work (O(1)) and greater memory usage. Changed process_command_line() to use the indexed array of options, theoretically reducing command line parsing work from O(n/2) to O(1). This is a win if the command line has many parameters or if it has parameters that are near the end of the unindexed option array. Testing confirms a small performance gain. Added command line options "config-test-smtpauth-username" and "config-test-smtpauth-password". Changed config_test_smtpauth() to run the authentication command(s) if a username and password are provided. This incorporates the functionality of checkpassword into spamdyke. Added the command line option "config-test-user" to change user and group IDs before running the configuration tests. This makes it easier to simulate running as the mail server. Changed process_config_file() and process_command_line() to print errors and stop when they encounter an option that is not legal in that location. At the moment, "help", "version", "config-test", "config-test-smtpauth-username", "config-test-smtpauth-password" and "config-test-user" are not valid in files; all options are valid on the command line. Changed config_test_dir_read() and config_test_graylist() to never examine the "." or ".." folders, even if readdir() and/or stat() report they are not folders. Thanks to Paulo Henrique for reporting this one. Changed set_config_value() to remove trailing slashes from directory paths. Added test_spamdyke_binary() to check if the spamdyke binary is setuid root (it should not be). Renamed test_settings() to config_test(). Moved all of the configuration test functions to config_test.[ch] -- they were cluttering up configuration.c. Made a few small updates to the help message text. Added additional vchkpw exit codes to exec_checkpassword() to explain why vchkpw exited, since it doesn't follow DJB's published checkpassword API. Moved md5.[ch] from the "utils" folder to the "spamdyke" folder and updated Makefile to compile them into spamdyke. Removed passwordcheck from the "utils" folder since spamdyke now contains its functionality. Added a README file to the "utils" folder to answer the biggest FAQ about those utilities. Fixed exec_command() to connect the output pipe to the child process's stdin instead of file descriptor 3. The bug was due to copying exec_checkpassword() and forgetting to change the value. Renamed exec_checkpassword() to exec_checkpassword_argv() and changed its arguments to expect a filename and an argument array. Added exec_checkpassword() to parse a command string into an argument array and call exec_checkpassword_argv(). Renamed exec_command() to exec_command_argv() and changed its arguments to expect a filename and an argument array. Added exec_command() to parse a command string into an argument array and call exec_command_argv(). Fixed numerous bugs in exec_command_argv() that were preventing it from actually gathering any input from the child process. Changed exec_command_argv() and exec_checkpassword_argv() to always log their child process errors to syslog, regardless of the user's preferences. Otherwise, the errors will be lost. Added the function find_path() to search the PATH for the given command without executing it. Changed exec_command_argv() and exec_checkpassword_argv() to use find_path() to locate the executable before fork()ing to catch typos. The child processes then use execve() to execute the command instead of exec_path(). Otherwise, the parent has a hard time determining that the child process quit because the command path was invalid. Changed exec_command_argv() and exec_checkpassword_argv() not to wait indefinitely for the child to exit after the timeout expires. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to limit the total number of queries they will recursively perform. This is to prevent a DoS situation where some domain has an unreasonable number of chained (non-circular) CNAME records. The limit is (arbitrarily) set at 16. Added the function config_test_child_capabilities() to test the qmail binary for SMTP AUTH and TLS patches. Depending on what is found, recommendations for spamdyke flags are made. Changed check_rdns_keywords() to allow top-level domains (like .com) to be used as keywords. This allows a way to reject connections from remote servers with rDNS names that contain the IP address and a two-letter country code. Unlike check_country_code(), specific country codes can now be chosen. Fixed do_spamdyke() not to wait indefinitely for all child processes to exit. This behavior was causing problems with DJB's recordio because recordio fork()s and uses its parent process to exec() spamdyke. This is very unusual. Changing wait(NULL) to waitpid() fixes the problem. Thanks to Bob Hutchinson for reporting this one. Added dns_initialize() and dns_get() to perform DNS queries by sending UDP packets instead of using the resolver library to do it. The resolver functions are just too slow and they try to do too much unnecessary work. dns_get() performs multiple requests for records (one for each kind of desired record) and, if no responses are received, sends requests to the secondary nameservers as well. Timeouts and retransmission times can now be controlled. This has resulted in a significant speedup in DNS resolutions; testing shows as much as a 10x performance increase in some situations. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to search all of the answers for the desired answer type before recursively querying CNAME answers. Some nameservers always put the CNAME answers first, even if other answer types are also given. This should allow spamdyke to find answers faster when domain admins have used a lot of CNAMEs. Added dns_a() to perform A record queries and changed all uses of gethostbyname() to use dns_a() instead. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to only perform their specific queries, not ask for CNAME records as well. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to send their own UDP packets instead of using the resolver library. Added dnscname to the "utils" folder to perform CNAME queries. Added dnsany to the "utils" folder to perform ANY queries and perform recursive CNAME lookups. Added "log-target" option to allow logging to stderr instead of syslog. Some people apparently like using the qmail-style "multilog" instead of syslog. I can't understand why but I'm here to serve. Thanks to John Hallam for suggesting this one. Changed all of the error messages about unexpected file types to specify what file type was found -- "non-regular file" was too vague to be useful. Changed the header in the files created by full logging to include the spamdyke version. Changed tls_end_inner() to use SSL_get_shutdown() to see if a shutdown signal has already been received. If SSL_shutdown() is used on a closed file descriptor, spamdyke will crash with SIGPIPE. Changed all instances of read(), write(), SSL_read() and SSL_write() to read or write as many bytes as possible in each call. This should provide a significant performance increase. The single-byte read()s and write()s were only used because I had badly misunderstood the relationship between select() and read()/write() -- blocking only occurs when select() indicates a file descriptor is not ready. If it is ready, read() and write() will handle as many bytes as they can without blocking. Thanks to Trog for setting me straight on this one. Rewrote most of sendrecv in the "tests" folder to use a multi-byte read(). Also took the opportunity to make sendrecv much faster and more polite, so it doesn't consume 100% CPU while waiting for qmail output. Fixed compiling errors on 64 bit Linux systems (Debian Etch x86_64 and Gentoo AMD64). Thanks to Juha-Pekka Jarvenpaa and FireBall for reporting this. Added config_test_file_type() to use stat() to find a file's type if readdir() either doesn't report it (Solaris) or reports "unknown" for all files (XFS). Thanks to Paulo Henrique for reporting this one. Fixed compiling errors on Solaris. Thanks to Limperis Antonis for reporting this. Changed the logging severity of the "unable to write X bytes to file descriptor" to debug instead of error. 99% of the time, the error occurs because the remote client disconnected unexpectedly and there's nothing the administrator can do about it anyway. Changed do_spamdyke() to ignore SIGPIPE signals. Changed do_spamdyke(), exec_command_argv() and exec_command_checkpassword() to change the SIGPIPE signal handler back to default for child processes after fork()ing but before exec()ing. Added a new logging level: excessive (4). It's to be used for printing very detailed debugging statements. Changed process_access() to permit access when no matching lines are found in the access file. Although DJB's tcprules documentation doesn't explicitly say so, no matching lines should allow access. Thanks to Steve Cole for reporting this one. VERSION 3.0.1 -- 9/12/2007 Fixed "configure" to remove the "_beta1" tag from the version number. That should never have been published. Changed usage() to show that optional values to long commands must be separated by an equals sign. getopt_long() is really becoming a hassle. Thanks to Richard Kreider for reporting this one. Fixed find_address() to accept addresses that aren't correctly delimited with <> characters and/or have multiple (illegal) spaces after the colon. Thanks to Davide Bozzelli for reporting this one. Fixed prepare_settings() to set the idle timeout seconds to the correct variable instead of setting the connection timeout variable. Thanks to Carlo Blohm for reporting this one. Fixed smtp_filter() to print the rejection message to HELO and EHLO, even if those commands appear in an improper place in the protocol. Fixed smtp_filter() to print the rejection message with an error code in response to STARTTLS if the command is given in an improper place in the protocol. Added some regression tests to find these bugs in the future. Fixed the usage statement in sendrecv to show the -w flag. VERSION 3.0.0 -- 9/11/2007 Added command line options never-graylist-rdns-dir, always-graylist-rdns-dir and rdns-whitelist-dir to search domain directory structures just like rdns-blacklist-dir. Added the command line option rdns-blacklist-file to search a file just like rdns-whitelist-file. Moved the command line option labels into configuration.c so they can be shared with the config file parser. Changed process_command_line() to build the list of short options from the list of long options instead of hardcoding them. Less maintenance this way. Modified check_rdns_keywords(), search_file() and search_tcprules_file() to correctly track line numbers and return the matching line number instead of just 1. Changed logging to allow the amount of information to be turned up or down. This should make spamdyke less chatty in the syslog for small errors. Modified smtp_filter() and run_tests() to report the matching filename and line number from check_rdns_keywords(), search_file() and search_tcprules_file() in syslog if the logging level is high enough. Fixed find_address() to locate the real email address and ignore BATV tags, relay paths and bang paths. Thanks to Walter Russo for reporting this one (again). Changed middleman() to obey minimums and maximums for the amount of time to select() for traffic. If spamdyke waits too long, the qmail process might not get wait()ed for a while, leaving a lot of defunct/zombie processes around. On a busy server, this could be a problem. Thanks to Jason M for reporting this one. Added process_config_file() to process configuration files instead of requiring all configuration to be done on the command line. At the moment, the file just uses the same (long option) directives as the command line. Added test_settings() to run tests on every configuration option and (hopefully) identify misconfigurations before someone makes them on a live server. Added the command line option "config-test" to run test_settings(). Renamed log_writeln() and log_write_rejection() to output_writeln() and output_write_rejection(), respectively, to make it clearer what they're doing. Changed smtp_filter() to allow multiple authentication attempts. Some clients retry authentication several times, presumably to deal with servers that can't use the authentication method they prefer. Changed middleman() to collect (and send) whole lines of input instead of single characters. Single character write()s were causing problems with Nagios and Windows clients. Changed output_write_rejection() to create a single output line and send it to output_writeln() all at once instead of sending a piece at a time. This keeps packets together for stupid Windows clients that just can't handle reassembling TCP packets correctly. Changed main() to always run spamdyke (as opposed to starting qmail without spamdyke listening) even if a whitelist is matched. This way, spamdyke can report all traffic to syslog, not just traffic that _may_ be filtered. Changed smtp_filter() and middleman() to catch the return codes from qmail when the remote client gives the recipient address. Now, if spamdyke doesn't block the recipient command but qmail does (e.g. for relaying), spamdyke will log the correct message. Incorporated GNU autoconf to create a "configure" script for spamdyke and the "utils" folder. The days of "make no_tls" and "make bsd" are thankfully over. Renamed all of the test folders to group them by function so it's easier to see what tests exist. Sequential numbers just weren't working. Changed dns_mx() to lookup the MX record before returning success. This means the sender MX filter now requires a mail exchanger record _and_ at least one mail exchanger must have an IP address. Before, the MX record was enough, even if there was no corresponding A record. Changed usage() to read the options and help text from get_spamdyke_options() in configuration.c so the help message won't ever be out of sync with the available options again. Added the command line option "tls-privatekey-password-file" to allow the SSL private key password to be read from a file instead of the command line. This way, the password isn't visible to everyone who can view a process list. Changed search_file(), search_tcprules_file() and check_rdns_keywords() so they no longer build their fscanf() patterns into a stack variable but instead use a literal search pattern assembled at compile time with STRINGIFY(). Added the command line options "hostname-file" and "hostname-command" to support reading the local hostname from a file or from a command (e.g. "hostname -f") instead of forcing it to be specified on the command line. Changed middleman() and smtp_filter() to always monitor and trust authentication carried out by qmail, even if "smtp-auth-command" was not given. This means spamdyke will always disable its filters for authenticated users even if it can't check the authentication itself. I'm not sure why I didn't design spamdyke this way in the first place. Added command line options recipient-whitelist-file and sender-whitelist-file so specific sender and recipient addresses can bypass the filters. Sender addresses are very easy to fake and recipient addresses are, of course, known to spammers, so both of these options are ill-advised. I've only added them due to popular demand. Added command line option check-rhsbl to check righthand-side blacklists. Both the server's rDNS domain name and the sender's email domain name are checked. Added command line options check-dns-whitelist and check-rhs-whitelist to allow DNS RBLs and RHSBLs to act as whitelists instead of blacklists. Anyone using DNS-based blacklists _and_ whitelists had better have some seriously fast DNS servers. Changed dns_txt(), dns_mx() and dns_ptr_lookup() to pass a stack of previous queries whenever they recursively lookup CNAME records, to prevent a cylical CNAME structure from leading to infinite recursion. NOT BACKWARDS COMPATIBLE: Changed the syslog entry format: renamed "origin" to "origin_ip", added "origin_rdns:" before the rDNS name, added "auth:" before the authenticated username and added "reason:" before the rejection reason when a timeout occurs. Changed process_command_line() to assume the remote IP address is 0.0.0.0 if the environment variable TCPREMOTEIP is not set. Added a ton more test scripts for all of the new options and for testing config files. Added dnsa, dnsns and dnssoa to the "utils" folder for performing DNS queries of A, NS and SOA records, respectively. Wouldn't it be AMAZING if the libc maintainers added standard functions to do these queries?! NOT BACKWARDS COMPATIBLE: Changed the "flag" options to take optional arguments instead of simply assuming "true" when the option was given. Unfortunately, getopt_long() is too stupid to handle them properly, which means clustered options (e.g. -rRc) can no longer be used. They must be separated (e.g. -r -R -c). Also, arguments given with the short version must not be separated by a space (e.g. -l3). NOT BACKWARDS COMPATIBLE: Renamed the long command line option "use-syslog" to "log-level". Fixed middleman() to completely bypass all processing when TLS passthrough is active. The additional processing was buffering TLS traffic until the data contained a newline character (purely by coincidence). This buffering was preventing the passthrough from functioning properly. Thanks to Dominik Dausch for reporting this one.
2008-02-25 08:38:00 +01:00
cd ../documentation; \
Update to 4.0.0. The changelog is unwieldy; highlights: * Several options have changed names, behaviors, defaults, and/or possible settings, all of which are explained in the upgrade instructions. * Many new options have been added: config-dir config-dir-search dns-blacklist-file dns-level dns-max-retries-total dns-max-retries-primary dns-server-ip dns-server-ip-primary dns-timeout-secs dns-whitelist-file filter-level graylist-exception-ip-entry graylist-exception-rdns-entry graylist-level ip-blacklist-entry ip-in-rdns-keyword-blacklist-entry ip-in-rdns-keyword-whitelist-entry ip-in-rdns-keyword-whitelist-file ip-whitelist-entry local-domain-entry rdns-blacklist-entry rdns-whitelist-entry recipient-blacklist-entry recipient-whitelist-entry rejection-text-access-denied rejection-text-dns-blacklist rejection-text-earlytalker rejection-text-empty-rdns rejection-text-graylist rejection-text-ip-blacklist rejection-text-ip-in-cc-rdns rejection-text-ip-in-rdns-keyword-blacklist rejection-text-local-recipient rejection-text-max-recipients rejection-text-missing-sender-mx rejection-text-rdns-blacklist rejection-text-recipient-blacklist rejection-text-reject-all rejection-text-relaying-denied rejection-text-rhs-blacklist rejection-text-sender-blacklist rejection-text-smtp-auth-required rejection-text-timeout rejection-text-unresolvable-rdns relay-level rhs-blacklist-file rhs-whitelist-file run-as-user sender-blacklist-entry sender-whitelist-entry smtp-auth-level tls-level * An example spamdyke.conf has been provided.
2008-07-16 09:03:21 +02:00
${INSTALL_DATA} spamdyke.conf.example ${DESTDIR}${EGDIR}; \
Update to 3.1.6. Note that some command-line options and the format of some log entries have changed since 2.6.3; see UPGRADING.txt. Also, pkgsrc no longer installs the random extra utilities that are explicitly marked as unnecessary for spamdyke operation. From the changelog: VERSION 3.1.6 -- 2/11/2008 Fixed a serious bug in middleman() -- when the remote server sent its message data and QUIT command in a burst and disconnected before spamdyke read() all of the data, the last data returned from read() was printed twice. This could cause message corruption, especially in the case of attachments. Fixed a serious bug in middleman() -- when the remote server sent its data in bursts of 4096 bytes AND there were two lines of text in the data AND the 4096th character was not a newline AND there was a delay between the data bursts, memmove()ing the buffered data was causing corruption because the moved data was not being properly re-terminated. While processing the remaining buffered data (and waiting for another burst from the remote server), strchr() would seek past the end of the data to an old newline character and middleman() would erroneously conclude the next line of data was complete, ready for processing. Many thanks to Andreas Galatis and Dragomir Denev for reporting and helping me reproduce this one. Added a -W flag to sendrecv to introduce a delay between message data bursts. Added a -o flag to smtpdummy to save the message data to a file. VERSION 3.1.5 -- 1/22/2008 Fixed sendrecv to correctly process corrupted TLS negotiations instead of covering up bugs in spamdyke. Fixed spamdyke to not add garbage output at the beginning of TLS passthrough negotiations. This was causing SSL handshakes to fail. Thanks to Ronnie Tartar for reporting this one. VERSION 3.1.4 -- 1/21/2008 Fixed all of the Makefiles to remove a symbols directory Leopard's gcc seems to create when compiling in debug mode. Fixed middleman() to log the timeout message only once. Fixed middleman() to not expect input from the child process when the child process' input is being ignored or after the child process has exited. Fixed middleman() to correctly handle a rare situation -- when the child process was too slow responding that spamdyke's idle timeout was passed AND spamdyke was processing TLS data AND there was still data in the SSL buffer, spamdyke would loop infinitely, consuming 100% CPU. This was a very tricky bug to find and fix. Thanks to Pablo Gonzalez and Paolo for reporting this one and helping me debug it. Fixed middleman() to send message data to the child process line-by-line, even when the buffer is full. Added a new test program: smtpdummy. This one simulates an SMTP server and can add delays after specific commands. Changed sendrecv to use a 64K buffer for input and output data. Changed sendrecv to kill the its child process after its timeout expires. Changed sendrecv to optionally continue sending data in bursts after the end for the message data. Some mail servers do this. Changed sendrecv to deliberately send corrupt data while TLS is active. Changed test regression_009 to build its message payload at runtime instead of including a 0.75M file. This file was unnecesarily increasing the size of the spamdyke tarball. Fixed compiling on Solaris. Again. Thanks to Davide Bozzelli for reporting this. Again. Sigh. VERSION 3.1.3 -- 1/3/2008 Fixed the format string LOG_INFO_DNS_TXT to assign the parameters correctly and prevent bus errors when the DNS response text is long. Thanks to Stephan Rosenke for reporting this one. VERSION 3.1.2 -- 12/11/2007 Fixed smtp_filter() to set a flag after some SMTP commands to force middleman() to wait for input from the child process before proceeding. Some (nonspammer) mail servers send their data in bursts without waiting for responses. This was causing spamdyke to skip logging (but not filtering) if the DATA command was sent in a burst with RCPT TO. Thanks to Sebastien Guilbaud and Bucky Carr for reporting this one. Added a "-b" flag to sendrecv to simulate servers that send their message data (but not their SMTP commands) in bursts. VERSION 3.1.1 -- 11/12/2007 Added excessive logging to search_domain_directory() to log the directory search pattern. Changed all calls to spamdyke_log() to use the macros SPAMDYKE_LOG_NONE(), SPAMDYKE_LOG_ERROR(), SPAMDYKE_LOG_INFO(), SPAMDYKE_LOG_DEBUG() and SPAMDYKE_LOG_EXCESSIVE() instead. The macro tests the current log level without forcing a function call and also paves the way toward eliminating some logging code at compile-time. Fixed process_access() to correctly search for the RELAYCLIENT variable in spamdyke's environment. Thanks to Steve Cole for reporting this one. VERSION 3.1.0 -- 11/5/2007 Changed the "graylist-dir" and "no-graylist-dir" options to take multiple directories for servers that are hosting so many domains that they can't create enough domain folders in one place (wow). Added minimum and maximum values to all integer options and changed set_config_value() to generate error messages when values are out of range. Change usage() to print minimum and maximum integer values. Alphabetized the option list by long option name and changed process_config_file() to use a binary search algorithm when identifying directives, a theoretical improvement from O(n/2) to O(log n). Changed prepare_settings() to create an array of options indexed by the short option code. This introduces some constant-time work (O(1)) and greater memory usage. Changed process_command_line() to use the indexed array of options, theoretically reducing command line parsing work from O(n/2) to O(1). This is a win if the command line has many parameters or if it has parameters that are near the end of the unindexed option array. Testing confirms a small performance gain. Added command line options "config-test-smtpauth-username" and "config-test-smtpauth-password". Changed config_test_smtpauth() to run the authentication command(s) if a username and password are provided. This incorporates the functionality of checkpassword into spamdyke. Added the command line option "config-test-user" to change user and group IDs before running the configuration tests. This makes it easier to simulate running as the mail server. Changed process_config_file() and process_command_line() to print errors and stop when they encounter an option that is not legal in that location. At the moment, "help", "version", "config-test", "config-test-smtpauth-username", "config-test-smtpauth-password" and "config-test-user" are not valid in files; all options are valid on the command line. Changed config_test_dir_read() and config_test_graylist() to never examine the "." or ".." folders, even if readdir() and/or stat() report they are not folders. Thanks to Paulo Henrique for reporting this one. Changed set_config_value() to remove trailing slashes from directory paths. Added test_spamdyke_binary() to check if the spamdyke binary is setuid root (it should not be). Renamed test_settings() to config_test(). Moved all of the configuration test functions to config_test.[ch] -- they were cluttering up configuration.c. Made a few small updates to the help message text. Added additional vchkpw exit codes to exec_checkpassword() to explain why vchkpw exited, since it doesn't follow DJB's published checkpassword API. Moved md5.[ch] from the "utils" folder to the "spamdyke" folder and updated Makefile to compile them into spamdyke. Removed passwordcheck from the "utils" folder since spamdyke now contains its functionality. Added a README file to the "utils" folder to answer the biggest FAQ about those utilities. Fixed exec_command() to connect the output pipe to the child process's stdin instead of file descriptor 3. The bug was due to copying exec_checkpassword() and forgetting to change the value. Renamed exec_checkpassword() to exec_checkpassword_argv() and changed its arguments to expect a filename and an argument array. Added exec_checkpassword() to parse a command string into an argument array and call exec_checkpassword_argv(). Renamed exec_command() to exec_command_argv() and changed its arguments to expect a filename and an argument array. Added exec_command() to parse a command string into an argument array and call exec_command_argv(). Fixed numerous bugs in exec_command_argv() that were preventing it from actually gathering any input from the child process. Changed exec_command_argv() and exec_checkpassword_argv() to always log their child process errors to syslog, regardless of the user's preferences. Otherwise, the errors will be lost. Added the function find_path() to search the PATH for the given command without executing it. Changed exec_command_argv() and exec_checkpassword_argv() to use find_path() to locate the executable before fork()ing to catch typos. The child processes then use execve() to execute the command instead of exec_path(). Otherwise, the parent has a hard time determining that the child process quit because the command path was invalid. Changed exec_command_argv() and exec_checkpassword_argv() not to wait indefinitely for the child to exit after the timeout expires. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to limit the total number of queries they will recursively perform. This is to prevent a DoS situation where some domain has an unreasonable number of chained (non-circular) CNAME records. The limit is (arbitrarily) set at 16. Added the function config_test_child_capabilities() to test the qmail binary for SMTP AUTH and TLS patches. Depending on what is found, recommendations for spamdyke flags are made. Changed check_rdns_keywords() to allow top-level domains (like .com) to be used as keywords. This allows a way to reject connections from remote servers with rDNS names that contain the IP address and a two-letter country code. Unlike check_country_code(), specific country codes can now be chosen. Fixed do_spamdyke() not to wait indefinitely for all child processes to exit. This behavior was causing problems with DJB's recordio because recordio fork()s and uses its parent process to exec() spamdyke. This is very unusual. Changing wait(NULL) to waitpid() fixes the problem. Thanks to Bob Hutchinson for reporting this one. Added dns_initialize() and dns_get() to perform DNS queries by sending UDP packets instead of using the resolver library to do it. The resolver functions are just too slow and they try to do too much unnecessary work. dns_get() performs multiple requests for records (one for each kind of desired record) and, if no responses are received, sends requests to the secondary nameservers as well. Timeouts and retransmission times can now be controlled. This has resulted in a significant speedup in DNS resolutions; testing shows as much as a 10x performance increase in some situations. Changed dns_txt(), dns_ptr_lookup() and dns_mx() to search all of the answers for the desired answer type before recursively querying CNAME answers. Some nameservers always put the CNAME answers first, even if other answer types are also given. This should allow spamdyke to find answers faster when domain admins have used a lot of CNAMEs. Added dns_a() to perform A record queries and changed all uses of gethostbyname() to use dns_a() instead. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to only perform their specific queries, not ask for CNAME records as well. Changed dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt in the "utils" folder to send their own UDP packets instead of using the resolver library. Added dnscname to the "utils" folder to perform CNAME queries. Added dnsany to the "utils" folder to perform ANY queries and perform recursive CNAME lookups. Added "log-target" option to allow logging to stderr instead of syslog. Some people apparently like using the qmail-style "multilog" instead of syslog. I can't understand why but I'm here to serve. Thanks to John Hallam for suggesting this one. Changed all of the error messages about unexpected file types to specify what file type was found -- "non-regular file" was too vague to be useful. Changed the header in the files created by full logging to include the spamdyke version. Changed tls_end_inner() to use SSL_get_shutdown() to see if a shutdown signal has already been received. If SSL_shutdown() is used on a closed file descriptor, spamdyke will crash with SIGPIPE. Changed all instances of read(), write(), SSL_read() and SSL_write() to read or write as many bytes as possible in each call. This should provide a significant performance increase. The single-byte read()s and write()s were only used because I had badly misunderstood the relationship between select() and read()/write() -- blocking only occurs when select() indicates a file descriptor is not ready. If it is ready, read() and write() will handle as many bytes as they can without blocking. Thanks to Trog for setting me straight on this one. Rewrote most of sendrecv in the "tests" folder to use a multi-byte read(). Also took the opportunity to make sendrecv much faster and more polite, so it doesn't consume 100% CPU while waiting for qmail output. Fixed compiling errors on 64 bit Linux systems (Debian Etch x86_64 and Gentoo AMD64). Thanks to Juha-Pekka Jarvenpaa and FireBall for reporting this. Added config_test_file_type() to use stat() to find a file's type if readdir() either doesn't report it (Solaris) or reports "unknown" for all files (XFS). Thanks to Paulo Henrique for reporting this one. Fixed compiling errors on Solaris. Thanks to Limperis Antonis for reporting this. Changed the logging severity of the "unable to write X bytes to file descriptor" to debug instead of error. 99% of the time, the error occurs because the remote client disconnected unexpectedly and there's nothing the administrator can do about it anyway. Changed do_spamdyke() to ignore SIGPIPE signals. Changed do_spamdyke(), exec_command_argv() and exec_command_checkpassword() to change the SIGPIPE signal handler back to default for child processes after fork()ing but before exec()ing. Added a new logging level: excessive (4). It's to be used for printing very detailed debugging statements. Changed process_access() to permit access when no matching lines are found in the access file. Although DJB's tcprules documentation doesn't explicitly say so, no matching lines should allow access. Thanks to Steve Cole for reporting this one. VERSION 3.0.1 -- 9/12/2007 Fixed "configure" to remove the "_beta1" tag from the version number. That should never have been published. Changed usage() to show that optional values to long commands must be separated by an equals sign. getopt_long() is really becoming a hassle. Thanks to Richard Kreider for reporting this one. Fixed find_address() to accept addresses that aren't correctly delimited with <> characters and/or have multiple (illegal) spaces after the colon. Thanks to Davide Bozzelli for reporting this one. Fixed prepare_settings() to set the idle timeout seconds to the correct variable instead of setting the connection timeout variable. Thanks to Carlo Blohm for reporting this one. Fixed smtp_filter() to print the rejection message to HELO and EHLO, even if those commands appear in an improper place in the protocol. Fixed smtp_filter() to print the rejection message with an error code in response to STARTTLS if the command is given in an improper place in the protocol. Added some regression tests to find these bugs in the future. Fixed the usage statement in sendrecv to show the -w flag. VERSION 3.0.0 -- 9/11/2007 Added command line options never-graylist-rdns-dir, always-graylist-rdns-dir and rdns-whitelist-dir to search domain directory structures just like rdns-blacklist-dir. Added the command line option rdns-blacklist-file to search a file just like rdns-whitelist-file. Moved the command line option labels into configuration.c so they can be shared with the config file parser. Changed process_command_line() to build the list of short options from the list of long options instead of hardcoding them. Less maintenance this way. Modified check_rdns_keywords(), search_file() and search_tcprules_file() to correctly track line numbers and return the matching line number instead of just 1. Changed logging to allow the amount of information to be turned up or down. This should make spamdyke less chatty in the syslog for small errors. Modified smtp_filter() and run_tests() to report the matching filename and line number from check_rdns_keywords(), search_file() and search_tcprules_file() in syslog if the logging level is high enough. Fixed find_address() to locate the real email address and ignore BATV tags, relay paths and bang paths. Thanks to Walter Russo for reporting this one (again). Changed middleman() to obey minimums and maximums for the amount of time to select() for traffic. If spamdyke waits too long, the qmail process might not get wait()ed for a while, leaving a lot of defunct/zombie processes around. On a busy server, this could be a problem. Thanks to Jason M for reporting this one. Added process_config_file() to process configuration files instead of requiring all configuration to be done on the command line. At the moment, the file just uses the same (long option) directives as the command line. Added test_settings() to run tests on every configuration option and (hopefully) identify misconfigurations before someone makes them on a live server. Added the command line option "config-test" to run test_settings(). Renamed log_writeln() and log_write_rejection() to output_writeln() and output_write_rejection(), respectively, to make it clearer what they're doing. Changed smtp_filter() to allow multiple authentication attempts. Some clients retry authentication several times, presumably to deal with servers that can't use the authentication method they prefer. Changed middleman() to collect (and send) whole lines of input instead of single characters. Single character write()s were causing problems with Nagios and Windows clients. Changed output_write_rejection() to create a single output line and send it to output_writeln() all at once instead of sending a piece at a time. This keeps packets together for stupid Windows clients that just can't handle reassembling TCP packets correctly. Changed main() to always run spamdyke (as opposed to starting qmail without spamdyke listening) even if a whitelist is matched. This way, spamdyke can report all traffic to syslog, not just traffic that _may_ be filtered. Changed smtp_filter() and middleman() to catch the return codes from qmail when the remote client gives the recipient address. Now, if spamdyke doesn't block the recipient command but qmail does (e.g. for relaying), spamdyke will log the correct message. Incorporated GNU autoconf to create a "configure" script for spamdyke and the "utils" folder. The days of "make no_tls" and "make bsd" are thankfully over. Renamed all of the test folders to group them by function so it's easier to see what tests exist. Sequential numbers just weren't working. Changed dns_mx() to lookup the MX record before returning success. This means the sender MX filter now requires a mail exchanger record _and_ at least one mail exchanger must have an IP address. Before, the MX record was enough, even if there was no corresponding A record. Changed usage() to read the options and help text from get_spamdyke_options() in configuration.c so the help message won't ever be out of sync with the available options again. Added the command line option "tls-privatekey-password-file" to allow the SSL private key password to be read from a file instead of the command line. This way, the password isn't visible to everyone who can view a process list. Changed search_file(), search_tcprules_file() and check_rdns_keywords() so they no longer build their fscanf() patterns into a stack variable but instead use a literal search pattern assembled at compile time with STRINGIFY(). Added the command line options "hostname-file" and "hostname-command" to support reading the local hostname from a file or from a command (e.g. "hostname -f") instead of forcing it to be specified on the command line. Changed middleman() and smtp_filter() to always monitor and trust authentication carried out by qmail, even if "smtp-auth-command" was not given. This means spamdyke will always disable its filters for authenticated users even if it can't check the authentication itself. I'm not sure why I didn't design spamdyke this way in the first place. Added command line options recipient-whitelist-file and sender-whitelist-file so specific sender and recipient addresses can bypass the filters. Sender addresses are very easy to fake and recipient addresses are, of course, known to spammers, so both of these options are ill-advised. I've only added them due to popular demand. Added command line option check-rhsbl to check righthand-side blacklists. Both the server's rDNS domain name and the sender's email domain name are checked. Added command line options check-dns-whitelist and check-rhs-whitelist to allow DNS RBLs and RHSBLs to act as whitelists instead of blacklists. Anyone using DNS-based blacklists _and_ whitelists had better have some seriously fast DNS servers. Changed dns_txt(), dns_mx() and dns_ptr_lookup() to pass a stack of previous queries whenever they recursively lookup CNAME records, to prevent a cylical CNAME structure from leading to infinite recursion. NOT BACKWARDS COMPATIBLE: Changed the syslog entry format: renamed "origin" to "origin_ip", added "origin_rdns:" before the rDNS name, added "auth:" before the authenticated username and added "reason:" before the rejection reason when a timeout occurs. Changed process_command_line() to assume the remote IP address is 0.0.0.0 if the environment variable TCPREMOTEIP is not set. Added a ton more test scripts for all of the new options and for testing config files. Added dnsa, dnsns and dnssoa to the "utils" folder for performing DNS queries of A, NS and SOA records, respectively. Wouldn't it be AMAZING if the libc maintainers added standard functions to do these queries?! NOT BACKWARDS COMPATIBLE: Changed the "flag" options to take optional arguments instead of simply assuming "true" when the option was given. Unfortunately, getopt_long() is too stupid to handle them properly, which means clustered options (e.g. -rRc) can no longer be used. They must be separated (e.g. -r -R -c). Also, arguments given with the short version must not be separated by a space (e.g. -l3). NOT BACKWARDS COMPATIBLE: Renamed the long command line option "use-syslog" to "log-level". Fixed middleman() to completely bypass all processing when TLS passthrough is active. The additional processing was buffering TLS traffic until the data contained a newline character (purely by coincidence). This buffering was preventing the passthrough from functioning properly. Thanks to Dominik Dausch for reporting this one.
2008-02-25 08:38:00 +01:00
for f in *.txt *.html; do \
${INSTALL_DATA} $${f} ${DESTDIR}${PREFIX}/share/doc/${PKGBASE};\
done
.include "../../mk/resolv.buildlink3.mk"
.include "../../mk/bsd.pkg.mk"