Features
- Password protected ssl keys
- Add OpenBSD module to system() source
- Add Ubuntu Trusty support to Docker build
Bugfixes
- Fix increased memory usage during saving disk-buffer
- Fix maximum record length limitations of disk-buffer
- Fix a memory leak in cfg-lexer
- Fix some issues found by pylint in python module
- Fix a crash due to a race condition in kv-parser()
- Fix a crash due to a race condition in file() destination
- Fix deprecated API usage in python module tests
- Fix a race condition in internal() source
- Fix a locale issue in merge-grammar python tool
- Fix compile problems with autotools when '--disable-all-modules'
used
- Fix a file descriptor leak in persist-state
- Fix a file descriptor leak in pseudofile()
- Fix memory/fd leaks in loggen tool
- Fix compile problems on Fedora, RHEL6, CentOS6 and SUSE based
platforms
- Fix a crash when large variety of keys added to messages
- Fix compile problems when PATH_MAX not defined
- Fix integer overflow problems in grammar
- Fix a memory leak in filter()
- Fix memory leak of persist-name() option
- Fix message corruption caused by a bug in the subst() rewrite rule
- Fix silently dropped messages in elasticsearch2() when sending in
bulk mode
- Fix broken disk-buffer() support in elasticsearch2()
- Fix Hy support in python module
- Fix an event scheduler related crash during reloading syslog-ng
- Fix a crash with SIGBUS when persist file cannot grow
Other changes
- Improve error reporting in "block" definitions in config
- Add warning message when disk-buffer() directory is changed in
configuration
- Syslog-ng debun improvements
- Refactor in rewrite() module init
- Missing child program (exit status 127) handling is changed in
program() destination:
- stopping destination instead of polling for the child program
- Refactor in filter() module
- Improve thread synchronization in mainloop and refactor
- Adapted json-c v0.13 API changes to json-parser
- Add filters as selectors in contextual data
- Remove separate sysutils/syslog-ng-json package, it is now a core
module/dependency.
3.13.2
Fixes
- Missing manpages from release tarball
- Package syslog-ng-mod-json is removed from
- Drop syslog-ng-abi virtual packages
3.13.1
Features
- Add app-parser() framework (automatic parsing of log messages)
- Support microseconds in Riemann destination
- Add osquery destination as an SCL plugin
- Add network load balancer destination
- Add possibility to only signal re-open of file handles (SIGUSR1)
- It is possible from now to limit the number of registered dynamic
counters
- Add $(binary) template function
- Add experimental transport for transferring messages in whole
between syslog-ng instances (EWMM)
- Docker based build and debian package generation
- Add auto-parse(yes/no) to app-paser(), system() and
default-network-drivers()
- Add Graylog2 destination and $(format-gelf) template function
Bugfixes
- Exit when a read fails on an included config file instead of
starting up with an empty configuration.
- Fix double free
- Add missing discarded counter to groupingby
- Fix a reference leak in Python destination
- Fix timezone issue in snmptrapd parser
- Fix potential crash in stdin driver
- Fix a crash when initializing new config fails for socket with
keep_alive off
- Fix filter evaluation in case of contexts with multiple elements
- Various grouping-by fixes
- Fix potential use after free around dns-cache during shutdown
- Fix access to indirect values within Java destination
- Fix a crash in affile
- Fix a memory leak
- Fix a crash when getent is used empty group
- Fix jvm-options()
- Fix a crash in Python language binding
- Fix a crash in afmongodb
- Fix a memory leak in afmongodb
- Fix name-to-GID calculation in the $(getent) template function
- Fix a crash when redis is configured without the command() option
- Fix a race condition in kv-parser()
Other changes
- Cleanup diskq related warning messages
- Provide tls block for tls options in amqp(), http(), riemann()
destination drivers
- It it possible from now to register blocks and generators as plugins
- Drop compatiblity with configurations below 3.0
- Do not change permissions of a file by default
- Allow source files to specify permissions locally
- Minor performance improvement
- The current config version can be queried with "--version"
- Increase the performance of kv-parser()
# Features
- HDFS: support macro in filename
- HDFS: add append support
- Java: allow to use sequence numbers in templates
- TLS improvements
- Add PKCS 12 support with the new `pkcs12-file()` TLS option
- startup time `ssl-options()` and `peer-verify()` check
- startup time `key_file`, `cert_file`, `ca_dir`, `crl_dir` and
`cipher_suite` check
- ECDH cipher support (OpenSSL 1.0.1, 1.0.2, 1.1.0) with the
`ecdh-curve-list()` option (only available >= 1.0.2)
- for < 1.0.2, a hard-coded curve is used
- for >= 1.0.2, automatic curve selection is used (the
`ecdh-curve-list()` option can restrict this list)
- DH cipher support with the `dhparam-file()` option
- if the option is not specified, fallback RFC 3526 parameters
are used
- minor fixes
- `stdin()` source driver
- Implement `read_old_records` option for systemd-journal source
- Add tags-parser: a new module to parse $TAGS values
- Add a Windows eventlog parser scl module
- Add XML parser module
# Bugfixes
- Fix cannot parse ipv6 into hostname
- Speedup add-contextual-data by making ordering optional
- Fix `monitor-method()` option not working for `wildcard-file()` source
- Sanitize SDATA keys in syslog-protocol messages to avoid generating
non-valid messages
- Fix memory leaks reported using Valgrind
- Fix memory leak related to cloning pipes and reload
- Fix getent protocol number returns incorrect value
- Fix elasticsearch2 destination flush mechanism
- Fix file destination related memory leak
- Fix a possible memory leak around affile destination
# Other changes
- Improve syslog-ng debun functionality
- Java: allow to set JVM options form global syslog-ng options
- Do steps towards Python 3 support:
- Fix string compatibility for Python 3
- Improve Python version auto detection
- HTTP destination: display verbose logs on debug level
- Improvements for Solaris packing
3.11.1
Features
- Add geoip2 parser and template function.
- Add SSL support to AMQP.
- Add template option to apache-accesslog-parser.
- Add configurable event time to Riemann destination.
- Add drop-unmatched() option to dbparser.
- Add Ubuntu Xenial to the bundled docker images.
- Support multi-instance support for Solaris 10 and 11.
- Support multi-instance for systemd.
- Add configurable timeout to HTTP destination.
- Add prefix() option to cisco-parser.
Bugfixes
- Fix a memory usage counter underflow for threaded destination drivers
and writers.
- Fix a potential crash in AMQP.
- Fix a potential crash during reload.
- Fix a reload/shutdown issue.
- Fix a potential crash in afsocket destination during reload.
- Fix a counter registration bug.
- Fix a build issue on FreeBSD.
- Fix a memory leak in diskq plugin.
- Fix systemd-journal error codes validation.
- Fix a potential crash in diskq when it is used with file
destination and the file is reaped.
- Fix a memory leak in HTTP destination
- Fix ENABLE_DEBUG in dbparser.
- Fix a unit tests that caused build issue on 32 bit platforms.
Other changes
- The eventlog library is part of syslog-ng from now.
- Improve error messages when the config cannot be initialized.
- Improve source suspended/resumed debug messages.
- Rename syslog-debun to syslog-ng-debun.
- Update manpages to v3.11
- Remove tgz2build directory.
3.10.1
Features
- Support https in http (curl) module
- Docker support : from now Dockerfile for CentOS7, Ubuntu Zesty and for
Debian Jessie is part of our upstream
- Add --database parameter for geoip template function
- Metric improvements
- Add snmp-parser (v1, v2)
- Add snmp-soure
- Add osquery source
- Add cisco-parser
- Add wildcard filesource
- Add startdate template function
- Add $(basename) and $(dirname) template functions
- Add Kerberos support for HDFS destination
- Add AUTH support for redis destination
- Add map-value-pairs() parser
- Extend Python language binding by Python parser
- Add support for extract-stray-words() option in kv-parser()
- Add $(context-values) template function
- Add $(context-lookup) function
- Add list related template functions
- Add add query commands to syslog-ng-ctl
- Support multiple servers in elasticsearch2-http destination
- Implements elastic-v2 https in http mode
- Add getent module (ported from incubator)
- Add support for IP_FREEBIND
Bugfixes
- Fix a libnet detection check error that caused problem configuring
enable-spoof-source.
- Avoid warnings about _DEFAULT_SOURCE on recent glibc versions
- Fix invalid database warning for geoip parser
- Fix prefix() default in systemd-journal for new config versions
- Fix a potential message loss in Riemann destination
- Fix a potential crash in the Riemann destination when the client is
not connected to the Riemann server.
- Fix a possible add-contextual-data() related data loss in case of
multiple
reference to the same add-contextual-data parser in several logpaths.
- Fix dbparser deadlock
- Fix Python destination
- Fix processed stats counter for afsocket
- Fix stats source for pipes
- Fix csv-parser multithreaded support
- Fix a message loss in case of filesource when syslog-ng was restarted
and the log_msg_size > file size.
- Fix a potential crash in cryptofuncs
- Fix a potential crash in syslog-ng-ctl when no command line parameters
was set.
- Fix token duplication in the output of '--preprocess-into'
- Fix UTF-8 support in syslog-ng-ctl
- Fix a potential crash during X.509 certificate validation.
- Fix a segfault in Python module startup
- Fix a possible endless reading loop issue in case of multi-line
filesource.
- Fix soname for the http module from "curl" to "http"
- Avoid openssl 1.1.0 deprecated APIs
Other changes
- Increase processed counter by queued counter after reload or restart
when diskqueue is used otherwise the newly added written counter would
underflow.
- Set the default time-zone to UTC for elasticsearch2
- Add retries support for python destination
- Prefer server side cipher suite order
- Always include librabbitmq in the dist tarball
- Always include ivykis in the dist tarball
- Marking parse error locations with >@<.
- Default log_msg_size is increased to 64Kbyte from 8Kb
- Tons of syslog-debun improvements
- Exit with 0 return code when --help is specified for syslog-ng-ctl
- syslog-ng: make '--preprocess-into' foreground only
- Add debug messages on log_msg_set_value()
- Add more detail to filter evaluation related debug messages
Features
- Improve parsing performance in case of keep-timestamp(no)
- TLS based transports will publish the peer's certificate in a set of
name-value pairs.
- Improve performance of the tcp() source, due to a bug, syslog-ng
attempted to apply position tracking to messages coming over a TCP
transport, which is used for file position tracking and causing
performance degradation.
- Make it possible to configure the listen-backlog() for any stream based
transports (unix-stream and tcp).
- Add a groupunset() rewrite rule that pairs up with groupset() but instead
of setting values it unsets them.
- Add support for Elastic Shield and SearchGuard
- kv-parser() is now able to cope with unquoted values with an embedded
space in them, it also trims whitespace from keys/values and is in
general more reliable in extracting key-value pairs from arbitrary log
messages.
- Improve performance for java based destinations.
- Add prefix() option to add-contextual-data()
Bugfixes
- Fix a potential crash in the file destination, in case it is a template
based filename and time-reap() is elapsed.
- Fix a potential ACK problem within syslog-ng that can cause input windows
to overflow queue sizes over time, effectively causing message drops that
shouldn't occur.
- Fix a heap corruption bug in the DNS cache, in case the maximum number of
DNS cache entries is reached.
- Fix timestamp for suppression messages.
- Fix add-contextual-data() to support CRLF line endings in its CSV input
files.
- Fixed key() option parsing in riemann() destinations.
- Find libsystemd-journal related functions in both libsystemd-journal.so
and libsystemd.so, as recent systemd versions bundled all systemd
related libs into the same library.
- Fixed the build-time detection of system-wide installed librabbitmq,
libmongoc and libcap.
- Fix the file source to repeatedly check for unexisting files, as a bug
caused syslog-ng to stop after two attempts previously.
- The performance testing tool "loggen" crashed if it was used to generate
messages on multiple threads over TLS. This was now fixed.
- Fix an issue in the syslog-parser() parser, so that timestamps parsed
earlier in the log path are properly overwritten.
- Due to a compilation issue, tcp-keepalive-time(), tcp-keepalive-intvl() and
tcp-keepalive-probes() were not working, now they are again.
- The --disable-shm-counters option is now passed to mongo-c-driver to work
around a minor security issue.
- Fix compilation issues on FreeBSD.
- Add support to month names in all caps in syslog timestamps. At least one
device seems to generate these.
- The options() option to java destination can now accept numbers and not
just strings.
- Fix a memory leak in the java destination driver, that may affect java
based destinations like ElasticSearch, Kafka & HDFS.
Other changes
- HDFS was updated to 2.7.3
- Elasticsearch was updated to 2.4.0
- Support was added for OpenSSL 1.1.x
3.8.1
Library updates
- Kafka-client updated to version to 0.9.0.0
- Minimal required version of hiredis is set to 0.11.0 to avoid
possible deadlocks
- Minimal version of libdbi is set to 0.9.0
Improvements and features
- Added the long-waited disk-buffer.
- date-parser ported from incubator to upstream
- New template functions: min, max, sum, average
- Added Apache-accesslog-parser
- Added loggly destination
- Added logmatic destination
- Added template function for supporting CEF.
- cURL-based HTTP destination driver added (implemented in C
programming language)
- SELinux policy installer script now has support for Red Hat
Enterprise Linux/CentOS/ Oracle Linux 5, 6 and 7.
- Implemented add-contextual-data: With add-context-data syslog-ng
can use an external database file to append custom name-value
pairs on incoming logs (to enrich messages).
Program destination/source drivers
- Added inherit-environment configuration option to program source
and destination.
- Added keep-alive option to program destination (afprog).
Java drivers
- HTTP destination: Added the ability to use templates in both url
and message.
- ElasticSearch Destination driver: Support 2.2.x series of
ElasticSearch (transport and node mode).
MongoDB destination driver
- Replaced submodule limongo-client with mongo-c-driver.
- Additional support for previous syntax used by libmongo-client
before we started using mongo-c-driver and its URI syntax
exclusively.
Riemann destination driver
- Use cert-file() and key-file() options to match afsocket
keywords as the same way as afsocket drivers use these options.
Rewrite rules
- Introduced template options in rewrite rules.
- Added unset operation to make it possible to unset a specific
name-value pair for a logmessage.
Parsers
- kvformat: make it possible to specify name-value separator
- linux-audit-scanner: recognize a0-a9* as fields to be decoded
- csv-parser has been refactored, extended with new dialect and
prefix options.
PatternDB
- added groupingby() parser that can perform simple correlation on
log messages
- added create-context action
- Added NLSTRING parser that captures a string until the following
end-of-line
Miscellaneous features
- syslog-debun (debug bundle script for syslog-ng) has been
improved
Bugfixes
- geoip-parser: When default database if not specified, syslog-ng
crashed.
- Added support for multiple drivers with the same name in
syslog-ng config.
- Fixed aack counting logic for junctions that have branches that
modify the LogMessage.
- Fixed a potential crash for code that uses log_msg_clear() in
production (e.g. syslog-parser()).
- Fixed potential crash in reload logic
- system(): use string comparison instead of numeric in PID
rewrite
- Support encoding on glib compiled with libiconv
- pdbtool: Fix the ordering of the debug-info list in PatternDB
- afprog: Don't kill our own process group
- Handle option names with hyphen (-) characters in java scls
- dnscache performance improved
- Fixed IPv6 parser in patterndb.
- Fixed journald program name flapping
- Fixed create-dirs() inheritance in file destinations
- Fixed pass-unix-credentials() global inheritance in afunix
- Fixed create-dirs() global inheritance in afunix
- Fixed byteorder handling on bigendian systems in netmask6 filter
- Fixed flow-control issue when overflow queue is full (suspending
source by setting the window size to 0).
- Log HTTP response error codes in HTTPDestination (Java).
- Fixed potential leaks related $(sanitize) argument parsing in
basicfuncs.
- Fixed a memory leak in python debugger
- Fixed a use-after-free bug in templates.
- Fixed a memory leak around reload in netmask6 filter.
- Fixed a memory leak in LogProtoBufferedServer in case the
encoding() option is used.
- configure: don't override $enable_python while executing
pkg-config
- Fixed BSD timestamp parsing in syslog-format.
- Fixed a SIGPIPE bug in program destination.
- Error handling has been improved in AMQP destination.
- value-pairs performance improvements, memleak fixes
- Various issues around UTF-8 support fixed.
- Fixed integer overflow in numerical operations template function
- Fixed an integer underflow in afsocket.
- Fixed numerical comperisons issues around filters.
- Fixed kernel log message time drift on Linux.
- Take CRLF sequences equivalent to an LF in patterndb.
- When syslog-ng failed to insert data into Redis, it has crashed.
- When device file is set as a file destination then syslog-ng
will not try to change the permission of the device file.
- Various fixes around config file parsing:
3.7.3
Improvements
- Updated Python package requirements.
- Can now compile without MongoDB.
- Added eventlog to the list of required pkg-config packages.
- Basic FreeBSD and HP-UX support of syslog debug bundle generator
by improving POSIX shell compatibility.
- Keep the program destination open between configuration reloads.
- system-source now uses keep-timestamp(no) for Linux kernel log.
The time source used by /dev/kmsg is not updated after system
SUSPEND/RESUME.
Fixes
- Fix a SIGSEGV when a Redis command returns an error.
- Resolve deadlock in logwriter triggered by suppress()
- Mitigate possible deadlock in patterndb
- Fixed global inheritance of pass-unix-credentials() and
create-dirs().
- Certain compilers complained about an undefined symbol when
setting keep-alive(yes).
- For certain use cases, afsocket would not handle procfs read
errors due to an integer underflow.
- Enhanced Java version check and the handling of
SyslogNgInternalLogger (used by Kafka), the FATAL loglevel and
getLocationInformation().
- When a big amount of kernel log was produced in a very short
time, the syslog-ng process sometimes entered into a spin and
stop processing messages.
Rework and clean up the package, split off various bindings
into separate packages. Add SMF support.
Major features and improvements introduced in major releases since 3.2.
3.7
- OpenSSL is now a required dependency for syslog-ng.
- Java-destination driver ported from syslog-ng-incubator.
- Python language support is ported from syslog-ng incubator.
- New Java destination drivers
- New Parsers
3.6
- PCRE is now a required dependency of syslog-ng.
- Threaded mode is now enabled by default.
3.5
- Multi-line support
- STOMP destination
- Redis destination
- Template type hinting
- Template options honored everywhere
- Support for unit suffixes in the configuration
- The Incubator project
3.4
- New plugins: AMQP & SMTP destinations, JSON parser.
- New parsers for patterndb: HOSTNAME, EMAIL, PCRE and LLADDR.
- It is now possible to control what db-parser() sees as its input
via it's new template() option.
- value-pairs() gained support for programmatically
rewriting key names in bulk, via the rekey() method.
- The network() driver is introduced, unifying and extending
tcp(), udp(), syslog(), unix-dgram() and
unix-stream(). The old drivers are still available, but
- Support for junctions & channels were added, which improve
the flexibility of the syslog-ng configuration language.
3.3
- multi-core/CPU scaling: the new multi-threaded architecture allows
syslog-ng to scale into the 800k msg/sec region.
- MongoDB support: using MongoDB instead of SQL is faster and
allows better representation of log data.
- JSON support: using the $(format-json) template function it is
now possible to construct JSON (JavaScript Object Notation)
output for log messages.
- A number of enhancements all over the place: SQL, patterndb.
- The default ports have changed. syslog-ng is using the standard
* Using libtool.
* fixes configure option for pidfile.
* tell sysconfigdir to configure.
* syslog2ng is using awk, add runtime dependency on awk and fix shebang.
* and let not to patch hard-coded uname path for NetBSD specific.
* VARBASE is used for various directory, set to BUILDE_DEFS.
* remove distractions from PLIST, libtoolized shlib files and an empty line.
PR pkg/45419
* fixes config file handling with CONF_FILES.
* require dbdir specified by --localstatedir.
Bump PKGREVISION.
Some key changes:
* configure.in: changed "source" to "." as the source command is a
bashism, changed a couple of double equal signs to single ones,
moved libol to the statically linked libs as AIX links to .a files
dynamically unless static linking is explicitly requested
* src/filters.c (do_filter_netmask): fixed negation for the
netmask() filter
* src/macros.c: added LEVEL_NUM and FACILITY_NUM macros
