Beats version 6.8.8
Add support for Cisco syslog format used by their switch. 10760
Beats version 6.8.7
Fixed bug with elasticsearch/cluster_stats metricset not recording license expiration date correctly. 14541 14591
Make kibana module more resilient to Kibana unavailability. 15258 15270
Beats version 6.8.6
Fix recording of SSL cert metadata for Expired/Unvalidated x509 certs. 13687
Fix marshaling of ms-since-epoch values in elasticsearch/cluster_stats metricset. 14378
Fixed bug with elasticsearch/cluster_stats metricset not recording license ID in the correct field. 14592
Beats version 6.8.5
Convert indexed ms-since-epoch timestamp fields in elasticsearch/ml_job metricset to ints from float64s. 14220 14222
Beats version 6.8.4
Fix delay in enforcing close_renamed and close_removed options. 13488 13907
Fix merging of fields specified in global scope with fields specified under an input’s scope. 3628 13909
Fixed early expiration of templates (Netflow v9 and IPFIX). 13821
Fixed bad handling of sequence numbers when multiple observation domains were exported by a single device (Netflow V9 and IPFIX). 13821
Fixed increased memory usage with large files when multiline pattern does not match. 14068
Mark Kibana usage stats as collected only if API call succeeds. 13881
Beats version 6.8.3
Iterate over journal correctly, so no duplicate entries are sent. 12716
Fix panic in Redis Key metricset when collecting information from a removed key. 13426
Remove _nodes field from under cluster_stats as it’s not being used. 13010
Collect license expiry date fields as well. 11652
Beats version 6.8.2
Process dataset: Do not show non-root warning on Windows. 12740
Host dataset: Export Host fields to gob encoder. 12940
Skipping unparsable log entries from docker json reader 12268
Limit memory usage of Redis replication sessions. https://github.com/elastic/beats/issues/12657[12657
Beats version 6.8.1
Fixed a memory leak when using the add_process_metadata processor under Windows. 12100
Package dataset: Log error when Homebrew is not installed. 11667
Process dataset: Fixed a memory leak under Windows. 12100
Login dataset: Fix re-read of utmp files. 12028
Package dataset: Fixed a crash inside librpm after Auditbeat has been running for a while. 12147 12168
Fix direction of incoming IPv6 sockets. 12248
Package dataset: Auto-detect package directories. 12289
System module: Start system module without host ID. 12373
Host dataset: Fix reboot detection logic. 12591
Fix goroutine leak happening when harvesters are dynamically stopped. 11263
Fix initialization of the TCP input logger. 11605
Fix goroutine leak caused on initialization failures of log input. 12125
Fix memory leak in Filebeat pipeline acker. 12063
Fix goroutine leak on non-explicit finalization of log input. 12164
When TLS is configured for the TCP input and a certificate_authorities is configured we now default to required for the client_authentication. 12584
Avoid generating hints-based configuration with empty hosts when no exposed port is suitable for the hosts hint. 8264 12086
Fix direction of incoming IPv6 sockets. 12248
Validate that kibana/status metricset cannot be used when xpack is enabled. 12264
In the kibana/stats metricset, only log error (don’t also index it) if xpack is enabled. 12353
The elasticsearch/index_summary metricset gracefully handles an empty Elasticsearch cluster when xpack.enabled: true is set. 12489 12487
When TLS is configured for the http metricset and a certificate_authorities is configured we now default to required for the client_authentication. 12584
Fixed a memory leak when using process monitoring under Windows. 12100
Improved debug logging efficiency in PGQSL module. 12150
Add support to the system package dataset for the SUSE OS family. 11634
Add validation for elasticsearch and kibana modules' metricsets when xpack.enabled is set to true. 12386
Beats version 6.8.0
Updates to support changes to licensing of security features.
Some Elastic Stack security features, such as encrypted communications, file and native authentication, and role-based access control, are now available in more subscription levels. For details, see https://www.elastic.co/subscriptions.
Beats version 6.7.2
Relax validation of the X-Pack license UID value. 11640
Fix a parsing error with the X-Pack license check on 32-bit system. 11650
Fix OS family classification in add_host_metadata for Amazon Linux, Raspbian, and RedHat Linux. 9134 11494
Fix false positives reported in the host.containerized field added by add_host_metadata. 11494
Fix the add_host_metadata’s host.id field on older Linux versions. 11494
Package dataset: dlopen versioned librpm shared objects. 11565
Package dataset: Nullify Librpm’s rpmsqEnable. 11628
Don’t apply multiline rules in Logstash json logs. 11346
Prevent the docker/memory metricset from processing invalid events before container start 11676
Add support to the system package dataset for the SUSE OS family. 11634
Beats version 6.7.1
Initialize the Paths before the keystore and save the keystore into data/{beatname}.keystore. 10706
Remove IP fields from default_field in Elasticsearch template. 11399
Beats version 6.7.0
Port settings have been deprecated in redis/logstash output and will be removed in 7.0. 9915
Update the code of Central Management to align with the new returned format. 10019
Allow Central Management to send events back to kibana. 9382
Fix panic if fields settting is used to configure hosts.x fields. 10824 10935
Introduce query.default_field as part of the template. 11205
Beats Xpack now checks for Basic license on connect. 11296
Filesets with multiple ingest pipelines added in 8914 only work with Elasticsearch >= 6.5.0 10001
Add grok pattern to support redis 5.0.3 log timestamp. 9819 10033
Ingesting Elasticsearch audit logs is only supported with Elasticsearch 6.5.0 and above 8852
Remove ecs option from user_agent processors when loading pipelines with Filebeat 6.7.x into Elasticsearch < 6.7.0. 10655 11362
Remove monitor generator script that was rarely used. 9648
Fix TLS certificate DoS vulnerability. 10303
Fix panic and file unlock in spool on atomic operation (arm, x86-32). File lock was not released when panic occurs, leading to the beat deadlocking on startup. 10289
Adding logging traces at debug level when the pipeline client receives the following events: onFilteredOut, onDroppedOnPublish. 9016
Do not panic when no tokenizer string is configured for a dissect processor. 8895
Fix a issue when remote and local configuration didn’t match when fetching configuration from Central Management. 10587
Add ECS-like selectors and dedotting to docker autodiscover. 10757 10862
Fix encoding of timestamps when using disk spool. 10099
Include ip and boolean type when generating index pattern. 10995
Using an environment variable for the password when enrolling a beat will now raise an error if the variable doesn’t exist. 10936
Cancelling enrollment of a beat will not enroll the beat. 10150
Remove IP fields from default_field in Elasticsearch template. 11399
Package: Disable librpm signal handlers. 10694
Login: Handle different bad login UTMP types. 10865
Fix hostname references in System module dashbords. 11064
User dataset: Numerous fixes to error handling. 10942
Support IPv6 addresses with zone id in IIS ingest pipeline. 9836 error log: 9869 access log: 10029
Fix bad bytes count in docker input when filtering by stream. 10211
Fixed data types for roles and indices fields in elasticsearch/audit fileset 10307
Cover empty request data, url and version in Apache2 modulehttps://github.com/elastic/beats/pull/10846[10846]
Fix a bug with the convert_timezone option using the incorrect timezone field. 11055 11164
Change URLPATH grok pattern to support brackets. 11135 11252
Add support for iis log with different address format. 11255 11256
Add fix to parse syslog message with priority value 0. 11010
Host header can now be overridden for HTTP requests sent by Heartbeat monitors. 9516
Fix checks for TCP send/receive data 10777
Do not stop collecting events when journal entries change. 9994
Fix MongoDB dashboard that had some incorrect field names from status Metricset 9795 9715
Fix issue that would prevent collection of processes without command line on Windows. 10196
Fixed data type for tags field in docker/container metricset 10307
Fixed data type for tags field in docker/image metricset 10307
Fixed data type for isr field in kafka/partition metricset 10307
Fixed data types for various hosts fields in mongodb/replstatus metricset 10307
Added function to close sql database connection. 10355
Fix parsing error using GET in Jolokia module. 11075 11071
Fix Winlogbeat escaping CR, LF and TAB characters. 11328 11357
Correctly extract Kinesis Data field from the Kinesis Record. 11141
Add the required permissions to the role when deployment SQS functions. 9152
Affecting all Beats
Add ip fields to default_field in Elasticsearch template. 11035
Add cleanup_timeout option to docker autodiscover, to wait some time before removing configurations after a container is stopped. 10374 10905
System module process dataset: Add user information to processes. 9963
Add system package dataset. 10225
Add system module login dataset. 9327
Add entity_id fields. 10500
Add seven dashboards for the system module. 10511
Add field log.source.address and log.file.path to replace source. 9435
Support mysql 5.7.22 slowlog starting with time information. 7892 9647
Add support for ssl_request_log in apache2 module. 8088 9833
Add support for iis 7.5 log format. 9753 9967
Add support for MariaDB in the slowlog fileset of mysql module. 9731
Add convert_timezone to nginx module. 9839 10148
Add support for Percona in the slowlog fileset of mysql module. 6665 10227
Added support for ingesting structured Elasticsearch audit logs 8852
New iptables module that receives iptables/ip6tables logs over syslog or file. Supports Ubiquiti Firewall extensions. 8781 10176
Populate more ECS fields in the Suricata module. 10006
Made monitors.d configuration part of the default config. 9004
Autodiscover metadata is now included in events by default. So, if you are using the docker provider for instance, you’ll see the correct fields under the docker key. 10258
Add field event.dataset which is {module}.{metricset}.
Add more TCP statuses to socket_summary metricset. 9430
Remove experimental tag from ceph metricsets. 9708
Add key metricset to the Redis module. 9582 9657
Add DeDot for kubernetes labels and annotations. 9860 9939
Add docker event metricset. 9856
Release Ceph module as GA. 10202
Release windows Metricbeat module as GA. 10163
Release traefik Metricbeat module as GA. 10166
List filesystems on Windows that have an access path but not an assigned letter 8916 10196
Release uswgi Metricbeat module GA. 10164
Release php_fpm module as GA. 10198
Release Memcached module as GA. 10199
Release etcd module as GA. 10200
Release kubernetes apiserver and event metricsets as GA 10212
Release Couchbase module as GA. 10201
Release aerospike module as GA. 10203
Release envoyproxy module GA. 10223
Release mongodb.metrics and mongodb.replstatus as GA. 10242
Release mysql.galera_status as Beta. 10242
Release postgresql.statement as GA. 10242
Release RabbitMQ Metricbeat module GA. 10165
Release Dropwizard module as GA. 10240
Release Graphite module as GA. 10240
Release http.server metricset as GA. 10240
Add support for MySQL 8.0 and tests also for Percona and MariaDB. 10261
Release use of xpack.enabled: true flag in Elasticsearch and Kibana modules as GA. 10222
Release Elastic stack modules (Elasticsearch, Logstash, and Kibana) as GA. 10094
Add remaining memory metrics of pods in Kubernetes metricbeat module 10157
Added 'server' Metricset to Zookeeper Metricbeat module 8938 10341
Add overview dashboard to Zookeeper Metricbeat module 10379
Mark Functionbeat as GA. 10564
Functionbeat can now deploy a function for Kinesis. 10116
Allow functionbeat to use the keystore. 9009
Deprecate field source. Will be replaced by log.source.address and log.file.path in 7.0. 9435
Deprecate field metricset.rtt. Replaced by event.duration which is in nano instead of micro seconds.
Support new TLS version negotiation introduced in TLS 1.3. 8647.
Journalbeat requires at least systemd v233 in order to follow entries after journal changes (rotation, vacuum).
Pkgsrc changes:
* Add patches to work around runtime.support_avx and runtime.support_avx2
being undefined when building with go 1.11.
* Add patch to cludge around build issue by commenting out BindToInterface
invocations, which is only implemented for Linux and Darwin.
Upstream changes:
6.6.2:
======
Auditbeat
* System module: Fix and unify bucket closing logic. 10897
Filebeat
* Fix a bug when converting NetFlow fields to snake_case. 10950
Metricbeat
* Fix issue in kubernetes module preventing usage percentages to
be properly calculated. 10946
Packetbeat
* Avoid reporting unknown MongoDB opcodes more than once. 10878
Winlogbeat
* Prevent Winlogbeat from dropping events with invalid XML. 11006
6.6.1:
======
Breaking changes
Affecting all Beats
* Fix stopping of modules started by kubernetes autodiscover. 10476
Auditbeat
* Enable System module config on Windows. 10237
Filebeat
* Fix bad bytes count in docker input when filtering by stream. 10211
* Add convert_timezone option to Logstash module to convert dates
to UTC. 9756 9797
* Add convert_timezone option to Elasticsearch module to convert
dates to UTC. 9756 9761
* Make elasticsearch/audit fileset be more lenient in parsing node
name. 10035 10135
Journalbeat
* Fix fields.yml indentation of audit group which had the effect
of creating an incomplete Elasticsearch index template. 10556
Metricbeat
* Fix issue with elasticsearch/node_stats metricset (x-pack) not
indexing source_node field. 10639
Packetbeat
* Fixed a crash when using af_packet capture 10477
Functionbeat
* Ensure that functionbeat is logging at info level not debug. 10262
Added
Filebeat
* Teach elasticsearch/audit fileset to parse out some more fields.
10134 10137
Journalbeat
* Migrate registry from previously incorrect path. 10486
6.6.0:
======
Breaking changes
Affecting all Beats
* Dissect syntax change, use * instead of ? when working with field
reference. 8054
Filebeat
* Allow beats to blacklist certain part of the configuration while
using Central Management. 9099
Metricbeat
* Allow beats to blacklist certain part of the configuration while
using Central Management. 9099
Functionbeat
* The CLI will now log CloudFormation Stack events. 8912
* Correctly normalize Cloudformation resource name. 10087
Bugfixes
Affecting all Beats
* Fix autodiscover configurations stopping when metadata is missing. 8851
* Refresh host metadata in add_host_metadata. 9359
* When collecting swap metrics for beats telemetry or system
metricbeat module handle cases of free swap being bigger than
total swap by assuming no swap is being used. 6271 9383
* Ignore non index fields in default_field for Elasticsearch. 9549
* Update Golang to 1.10.6. 9563
* Update Kibana index pattern attributes for objects that are disabled. 9644
* Enforce validation for the Central Management access token. 9621
* Fix registry handle leak on Windows
(https://github.com/elastic/go-sysinfo/pull/33). 9920
* Gracefully handle TLS options when enrolling a Beat. 9129
* Allow to unenroll a Beat from the UI. 9452
* The backing off now implements jitter to better distribute the load. 10172
* Fix config appender registration. 9873
* Fix TLS certificate DoS vulnerability. 10304
Filebeat
* Fix improperly set config for CRI Flag in Docker Input 8899
* Just enabling the elasticsearch fileset and starting Filebeat
no longer causes an error. 8891
* Support IPv6 addresses with zone id in IIS ingest pipeline. 9836
error log: 9869 access log: 10030
* Support haproxy log lines without captured headers. 9463 9958
Heartbeat
* Heartbeat now always downloads the entire body of HTTP endpoints,
even if no checks against the body content are declared. This
fixes an issue where timing metrics would be incorrect in
scenarios where the body wasn't used since the connection would
be closed soon after the headers were sent, but before the entire
body was. 8894
Metricbeat
* Add missing namespace field in http server metricset 7890
* Fix issue with not collecting Elasticsearch cross-cluster
replication stats correctly. 9179
* The node.name field in the elasticsearch/node metricset now
correctly reports the Elasticsarch node name. Previously this
field was incorrectly reporting the node ID instead. 9209
* Fix panics in vsphere module when certain values where not
returned by the API. 9784
* Fix pod UID metadata enrichment in Kubernetes module. 10081
Packetbeat
* Fix issue with process monitor associating traffic to the wrong
process. 9151 9443
* Fix DHCPv4 dashboard that wouldn't load in Kibana. 9850
Added
Affecting all Beats
* Unify dashboard exporter tools. 9097
* Dissect will now flag event on parsing error. 8751
* Added the redirect_stderr option that allows panics to be logged
to log files. 8430
* Add cache.ttl to add_host_metadata. 9359
* Add support for index lifecycle management (beta). 7963
* Always include Pod UID as part of Pod metadata. {pull]9517[9517]
* Release Jolokia autodiscover as GA. 9706
Auditbeat
* Add system module. 9546
Filebeat
* Added detect_null_bytes selector to detect null bytes from a io.reader. 9210
* Added syslog_host variable to HAProxy module to allow syslog
listener to bind to configured host. 9366
* Allow to force CRI format parsing for better performance 8424
* Add event.dataset to module events. 9457
* Add field log.source.address and log.file.path to replace source. 9435
* Add support for multi-core thread_id in postgresql module 9156 9482
* Added netflow input type that supports NetFlow v1, v5, v6, v7,
v8, v9 and IPFIX. 9399
Journalbeat
* Add the ability to check against JSON HTTP bodies with conditions. 8667
* Add cursor_seek_fallback option. 9234
Metricbeat
* Collect custom cluster display_name in elasticsearch/cluster_stats
metricset. 8445
* Test etcd module with etcd 3.3. 9068
* All elasticsearch metricsets now have module-level cluster.id
and cluster.name fields. 8770 8771 9164 9165 9166 9168
* All elasticsearch node-level metricsets now have node.id and
node.name fields. 9168 9209
* Add settings to disable docker and cgroup cpu metrics per core.
9187 9194 9589
* The elasticsearch/node metricset now reports the Elasticsearch
cluster UUID. 8771
* Support GET requests in Jolokia module. 8566 9226
* Add freebsd support for the uptime metricset. 9413
* Add host.os.name field to add_host_metadata processor. 8948 9405
* Add field event.dataset which is {module}.{metricset). 9393
Deprecated
Filebeat
* Deprecate field source. Will be replaced by log.source.address
and log.file.path in 7.0. 9435
Metricbeat
* Deprecate field metricset.rtt. Replaced by event.duration which
is in nano instead of micro seconds. 9393
Packetbeat
* Support new TLS version negotiation introduced in TLS 1.3. 8647.
6.5.4:
======
Bugfixes
Affecting all Beats
* Update Golang to 1.10.6. This fixes an issue in remote certificate
validation CVE-2018-16875. 9563
Filebeat
* Fix saved objects in filebeat haproxy dashboard. 9417
* Fixed a memory leak when harvesters are closed. 7820
Added
Filebeat
* Added support on Traefik for Common Log Format and Combined Log
Format mixed which is the default Traefik format 8015 6111 8768.
6.5.3:
======
Bugfixes
Affecting all Beats
* Log events at the debug level when dropped by encoding problems. 9251
Filebeat
* Correctly parse December or Dec in the Syslog input. 9349
* Don't generate incomplete configurations when logs collection
is disabled by hints. 9305
* Stop runners disabled by hints after previously being started. 9305
* Fix installation of haproxy dashboard. 9307 9313
6.5.2:
======
Bugfixes
Affecting all Beats
* Propagate Sync error when running SafeFileRotate. 9069
Metricbeat
* Fix panic on docker healthcheck collection on dockers without
healthchecks. 9171
* Fix issue preventing diskio metrics collection for idle disks.
9124 9125
6.5.1:
======
Bugfixes
Affecting all Beats
* Fix windows binaries not having an enroll command. 9096 8836
Journalbeat
* Fix journalbeat sometimes hanging if output is unavailable. 9106
Metricbeat
* Fix race condition when enriching events with kubernetes metadata. 9055 9067
Added
Journalbeat
* Add minimal kibana dashboard. 9106
6.5.0:
======
Bugfixes
Affecting all Beats
* Fixed add_host_metadata not initializing correctly on Windows.
7715
* Fixed missing file unlock in spool file on Windows, so file can
be reopened and locked. 7859
* Fix spool file opening/creation failing due to file locking on
Windows. 7859
* Fix size of maximum mmaped read area in spool file on Windows.
7859
* Fix potential data loss on OS X in spool file by using fcntl
with F_FULLFSYNC. 7859
* Improve fsync on linux, by assuming the kernel resets error
flags of failed writes. 7859
* Remove unix-like permission checks on Windows, so files can be
opened. 7849
* Replace index patterns in TSVB visualizations. 7929
* Deregister pipeline loader callback when inputsRunner is stopped.
7893[7893]
* Add backoff support to x-pack monitoring outputs. 7966
* Removed execute permissions systemd unit file. 7873
* Fix a race condition with the add_host_metadata and the event
serialization. 8223 8653
* Enforce that data used by k8s or docker doesn't use any reference. 8240
* Switch to different UUID lib due to to non-random generated UUIDs. 8485
* Fix race condition when publishing monitoring data. 8646
* Fix bug in loading dashboards from zip file. 8051
* Fix in-cluster kubernetes configuration on IPv6. 8754
* The export config subcommand should not display real value for
field reference. 8769
* The setup command will not fail if no dashboard is available to
import. 8977
* Fix central management configurations reload when a configuration
is removed in Kibana. 9010
Auditbeat
* Fixed a crash in the file_integrity module under Linux. 7753
* Fixed the RPM by designating the config file as configuration
data in the RPM spec. 8075
* Fixed a concurrent map write panic in the auditd module. 8158
* Fixed a data race in the file_integrity module. 8009
* Fixed a deadlock in the file_integrity module. 8027
Filebeat
* Fix date format in Mongodb Ingest pipeline. 7974
* Fixed a docker input error due to the offset update bug in
partial log join.8177
* Update CRI format to support partial/full tags. 8265
* Fix some errors happening when stopping syslog input. 8347
* Fix RFC3339 timezone and nanoseconds parsing with the syslog input. 8346
* Mark the TCP and UDP input as GA. 8125
* Support multiline logs in logstash/log fileset of Filebeat. 8562
Heartbeat
* Fixed bug where HTTP responses with larger bodies would
incorrectly report connection errors. 8660
Metricbeat
* Fix golang.heap.gc.cpu_fraction type from long to float in Golang
module. 7789
* Fixed the RPM by designating the modules.d config files as
configuration data in the RPM spec. 8075
* Fixed the location of the modules.d dir in Deb and RPM packages. 8104
* Add docker diskio stats on Windows. 6815 8126
* Fix incorrect type conversion of average response time in Haproxy
dashboards 8404
* Added io disk read and write times to system module 8473 8508
* Avoid mapping issues in kubernetes module. 8487
* Recover metrics for old apache versions removed by mistake on #6450. 7871
* Fix dropwizard module parsing of metric names. 8365 8385
* Fix issue that would prevent kafka module to find a proper broker
when port is not set 8613
* Fix range colors in multiple visualizations. 8633 8634
* Fix incorrect header parsing on http metricbeat module 8564 8585
* Fixed a panic when the kvm module cannot establish a connection
to libvirtd. 7792.
* Add missing namespace field in http server metricset 7890
Packetbeat
* Fixed a seccomp related error where the fcntl64 syscall was
not permitted on 32-bit Linux and the sniffer failed to start.
7839
* Added missing cmdline and client_cmdline fields to index template. 8258
Added
Affecting all Beats
* Added time-based log rotation. 8349
* Add backoff on error support to redis output. 7781
* Allow for cloud-id to specify a custom port. This makes cloud-id
work in ECE contexts. 7887
* Add support to grow or shrink an existing spool file between
restarts. 7859
* Make kubernetes autodiscover ignore events with empty container
IDs 7971
* Implement CheckConfig in RunnerFactory to make autodiscover
check configs 7961
* Add DNS processor with support for performing reverse lookups
on IP addresses. 7770
* Support for Kafka 2.0.0 in kafka output 8399
* Add setting setup.kibana.space.id to support Kibana Spaces 7942
* Better tracking of number of open file descriptors. 7986
* Report number of open file handles on Windows. 8329
* Added the add_process_metadata processor to enrich events with
process information. 6789
* Add Beats Central Management 8559
* Report configured queue type. 8091
* Enable host and cloud metadata processors by default. 8596
Filebeat
* Add tag "truncated" to "log.flags" if incoming line is longer
than configured limit. 7991
* Add haproxy module. 8014
* Add tag "multiline" to "log.flags" if event consists of multiple
lines. 7997
* Release docker input as GA. 8328
* Keep unparsed user agent information in user_agent.original. 7832
* Added default and TCP parsing formats to HAproxy module 8311 8637
* Add Suricata IDS/IDP/NSM module. 8153 8693
* Support for Kafka 2.0.0 8853
Heartbeat
* Heartbeat is marked as GA.
* Add automatic config file reloading. 8023
* Added autodiscovery support 8415
* Added support for extra TLS/x509 metadata. 7944
* Added stats and state metrics for number of monitors and endpoints
started. 8621
Journalbeat
* Add journalbeat. 8703
Metricbeat
* Add replstatus metricset to MongoDB module 7604
* Add experimental socket summary metricset to system module 6782
* Move common kafka fields (broker, topic and partition.id) to
the module level to facilitate events correlation 7767
* Add fields for memory fragmentation, memory allocator stats,
copy on write, master-slave status, and active defragmentation
to info metricset of Redis module. 7695
* Increase ignore_above for system.process.cmdline to 2048. 8100
* Add support to renamed fields planned for redis 5.0. 8167
* Allow TCP helper to support delimiters and graphite module to
accept multiple metrics in a single payload. 8278
* Added died PID state to process_system metricset on system module 8275
* Add metrics metricset to MongoDB module. 7611
* Added ccr metricset to Elasticsearch module. 8335
* Support for Kafka 2.0.0 8399
* Added support for query params in configuration 8286 8292
* Add container image for docker metricsets. 8214 8438
* Precalculate composed id fields for kafka dashboards. 8504
* Add support for full status page output for php-fpm module as
a separate metricset called process. 8394
* Add Kafka dashboard. 8457
* Release Kafka module as GA. 8854
Packetbeat
* Added DHCP protocol support. 7647
Functionbeat
* Initial version of Functionbeat. 8678
Deprecated
Heartbeat
* watch.poll_file is now deprecated and superceded by automatic
config file reloading.
Metricbeat
* Redis info replication.master_offset has been deprecated in
favor of replication.master.offset.7695
* Redis info clients fields longest_output_list and biggest_input_buf
have been renamed to max_output_buffer and max_input_buffer
based on the names they will have in Redis 5.0, both fields will
coexist during a time with the same value 8167.
* Move common kafka fields (broker, topic and partition.id) to
the module level 7767.
6.4.3:
======
Bugfixes
Affecting all Beats
* Fix a race condition with the add_host_metadata and the event
serialization. 8223 8653
* Fix race condition when publishing monitoring data. 8646
* Fix bug in loading dashboards from zip file. 8051
* The export config subcommand should not display real value for
field reference. 8769
Filebeat
* Fix typo in Filebeat IIS Kibana visualization. 8604
Metricbeat
* Recover metrics for old Apache versions removed by mistake on
#6450. 7871
* Avoid mapping issues in Kubernetes module. 8487
* Fixed a panic when the KVM module cannot establish a connection
to libvirtd. 7792
6.4.2:
======
Bugfixes
Filebeat
* Fix some errors happening when stopping syslog input. 8347
* Fix RFC3339 timezone and nanoseconds parsing with the syslog input. 8346
Metricbeat
* Fix incorrect type conversion of average response time in Haproxy
dashboards 8404
* Fix dropwizard module parsing of metric names. 8365 8385
6.4.1:
======
Bugfixes
Affecting all Beats
* Add backoff support to x-pack monitoring outputs. 7966
* Removed execute permissions systemd unit file. 7873
* Fix a race condition with the add_host_metadata and the event
serialization. 8223
* Enforce that data used by k8s or docker doesn't use any reference.
8240
* Implement CheckConfig in RunnerFactory to make autodiscover
check configs 7961
* Make kubernetes autodiscover ignore events with empty container
IDs 7971
Auditbeat
* Fixed a concurrent map write panic in the auditd module. 8158
* Fixed the RPM by designating the config file as configuration
data in the RPM spec. 8075
Filebeat
* Fixed a docker input error due to the offset update bug in
partial log join.8177
* Update CRI format to support partial/full tags. 8265
Metricbeat
* Fixed the location of the modules.d dir in Deb and RPM packages.
8104
* Fixed the RPM by designating the modules.d config files as
configuration data in the RPM spec. 8075
* Fix golang.heap.gc.cpu_fraction type from long to float in Golang
module. 7789
Packetbeat
* Added missing cmdline and client_cmdline fields to index template. 8258
6.4.0:
======
Known issue
Due to a packaging mistake, the modules.d configuration directory
is installed in the wrong path in the Metricbeat DEB and RPM
packages. This issue results in an empty list when you run metricbeat
modules list and failures when you try to enable or disable modules.
To work around this issue, run the following command:
sudo cp -r /usr/share/metricbeat/modules.d /etc/metricbeat/
This issue affects all new installations on DEB and RPM. Upgrades
will run, but use old configurations defined in the modules.d
directory from the previous installation.
The issue will be fixed in the 6.4.1 release.
Breaking changes
Affecting all Beats
* Set default kafka version to 1.0.0 in kafka output. Older versions
are still supported by configuring the version setting. Minimally
supported version is 0.11 (older versions might work, but are
untested). 7025
Heartbeat
* Rename http.response.status to http.response.status_code to
align with ECS. 7274
* Remove type field as not needed. 7307
Metricbeat
* Fixed typo in values for state_container status.phase, from
terminate to terminated. 6916
* RabbitMQ management plugin path is now configured at the module
level instead of having to do it in each of the metricsets. New
management_path_prefix option should be used now 7074
* RabbitMQ node metricset only collects metrics of the instance
it connects to, node.collect: cluster can be used to collect
all nodes as before. 6556 6971
* Change http/server metricset to put events by default under
http.server and prefix config options with server.. 7100
* Disable dedotting in docker module configuration. This will
change the out-of-the-box behaviour, but not the one of already
configured instances. 7485
* Fix typo in etcd/self metricset fields from *.bandwithrate to
*.bandwidthrate. 7456
* Changed the definition of the system.cpu.total.pct and
system.cpu.total.norm.cou fields to exclude the IOWait time.
7691
Bugfixes
Affecting all Beats
* Error out on invalid Autodiscover template conditions settings. 7200
* Allow to override the ignore_above option when defining new
field with the type keyword. 7238
* Fix a panic on the Dissect processor when we have data remaining
after the last delimiter. 7449
* When we fail to build a Kubernetes' indexer or matcher we produce
a warning but we don't add them to the execution. 7466
* Fix default value for logging.files.keepfiles. It was being set
to 0 and now it's set to the documented value of 7. 7494
* Retain compatibility with older Docker server versions. 7542
* Fix errors unpacking configs modified via CLI by ignoring -E
key=value pairs with missing value. 7599
Auditbeat
* Allow auditbeat setup to run without requiring elevated privileges
for the audit client. 7111
* Fix goroutine leak that occurred when the auditd module was stopped. 7163
Filebeat
* Fix a data race between stopping and starting of the harvesters. 6879
* Fix an issue when parsing ISO8601 dates with timezone definition 7367
* Fix Grok pattern of MongoDB module. 7568
* Fix registry duplicates and log resending on upgrade. 7634
Metricbeat
* Fix Windows service metricset when using a 32-bit binary on a
64-bit OS. 7294
* Do not report Metricbeat container host as hostname in Kubernetes
deployment. 7199
* Ensure metadata updates don't replace existing pod metrics. 7573
* Fix kubernetes pct fields reporting. 7677
* Add support for new kube_node_status_condition in Kubernetes
state_node. 7699
Added
Affecting all Beats
* Add dissect processor. 6925
* Add IP-addresses and MAC-addresses to add_host_metadata. 6878
* Added a seccomp (secure computing) filter on Linux that whitelists
the necessary system calls used by each Beat. 5213
* Ship fields.yml as part of the binary 4834
* Added options to dev-tools/cmd/dashboards/export_dashboard.go:
-indexPattern to include index-pattern in output, -quiet to be
quiet. 7101
* Add Indexer indexing by pod uid. Enable pod uid metadata gathering
in add_kubernetes_metadata. Extended Matcher log_path matching
to support volume mounts 7072
* Add default_fields to Elasticsearch template when connecting to
Elasticsearch >= 7.0. 7015
* Add support for loading a template.json file directly instead
of using fields.yml. 7039
* Add support for keyword multifields in field.yml. 7131
* Add experimental Jolokia Discovery autodiscover provider. 7141
* Add owner object info to Kubernetes metadata. 7231
* Add Beat export dashboard command. 7239
* Add support for docker autodiscover to monitor containers on
host network 6708
* Add ability to define input configuration as stringified JSON
for autodiscover. 7372
* Add processor definition support for hints builder 7386
* Add support to disable html escaping in outputs. 7445
* Refactor error handing in schema.Apply(). 7335
* Add additional types to Kubernetes metadata 7457
* Add module state reporting for Beats Monitoring. 7075
* Release the rename processor as GA. 7656
* Add support for Openstack Nova in add_cloud_metadata processor.
7663
Auditbeat
* Added XXH64 hash option for file integrity checks. 7311
* Added the show auditd-rules and show auditd-status commands to
show kernel rules and status. 7114
* Add Kubernetes specs for auditbeat file integrity monitoring 7642
Filebeat
* Add Kibana module with log fileset. 7052
* Support MySQL 5.7.19 by mysql/slowlog 6969
* Correctly join partial log lines when using docker input. 6967
* Add support for TLS with client authentication to the TCP input 7056
* Converted part of pipeline from treafik/access metricSet to
dissect to improve efficiency. 7209
* Add GC fileset to the Elasticsearch module. 7305
* Add Audit log fileset to the Elasticsearch module. 7365
* Add Slow log fileset to the Elasticsearch module. 7473
* Add deprecation fileset to the Elasticsearch module. 7474
* Add convert_timezone option to Kafka module to convert dates to
UTC. 7546 7578
* Add patterns for kafka 1.1 logs. 7608
* Move debug messages in tcp input source 7712
Metricbeat
* Add experimental Elasticsearch index metricset. 6881
* Add dashboards and visualizations for haproxy metrics. 6934
* Add Jolokia agent in proxy mode. 6475
* Add message rates to the RabbitMQ queue metricset 6442 6606
* Add exchanges metricset to the RabbitMQ module 6442 6607
* Add Elasticsearch index_summary metricset. 6918
* Add shard metricset to Elasticsearch module. 7006
* Add apiserver metricset to Kubernetes module. 7059
* Add maxmemory to redis info metricset. 7127
* Set guest as default user in RabbitMQ module. 7107
* Add postgresql statement metricset. 7048 7060
* Update state_container metricset to support latest kube-state-metrics
version. 7216
* Add TLS support to MongoDB module. 7401
* Added Traefik module with health metricset. 7413
* Add Elasticsearch ml_job metricsets. 7196
* Add support for bearer token files to HTTP helper. 7527
* Add Elasticsearch index recovery metricset. 7225
* Add locks, global_locks, oplatencies and process fields to status
metricset of MongoDB module. 7613
* Run Kafka integration tests on version 1.1.0 7616
* Release raid and socket metricset from system module as GA. 7658
* Release elasticsearch module and all its metricsets as beta. 7662
* Release munin and traefik module as beta. 7660
* Add envoyproxy module. 7569
* Release prometheus collector metricset as GA. 7660
* Add Elasticsearch cluster_stats metricset. 7638
* Added basepath setting for HTTP-based metricsets 7700
Packetbeat
* The process monitor now reports the command-line for all processes,
under Linux and Windows. 7135
* Updated the TLS protocol parser with new cipher suites added to
TLS 1.3. 7455
* Flows are enriched with process information using the process
monitor. 7507
* Added UDP support to process monitor. 7571
Deprecated
Metricbeat
* Kubernetes state_container cpu.limit.nanocores and cpu.request.nanocores
have been deprecated in favor of cpu.*.cores. 6916
6.3.2:
======
Bugfixes
Affecting all Beats
* Fix registry duplicates and log resending on upgrade. 7634
* Fix default value for logging.files.keepfiles. It was being set
to 0 and now it's set to the documented value of 7. 7494
* Retain compatibility with older Docker server versions. 7542
Metricbeat
* Fix missing hosts config option in Ceph module. 7596
* Ensure metadata updates don't replace existing pod metrics. 7573
Added
Metricbeat
* Add support for bearer token files to HTTP helper. 7527
Packetbeat
* Updated the TLS protocol parser with new cipher suites added
to TLS 1.3. 7455
6.3.1:
======
Bugfixes
Affecting all Beats
* Allow index-pattern only setup when setup.dashboards.only_index=true.
7285
* Preserve the event when source matching fails in add_docker_metadata.
7133
* Negotiate Docker API version from our client instead of using
a hardcoded one. 7165
* Fix duplicating dynamic_fields in template when overwriting the
template. 7352
Auditbeat
* Fixed parsing of AppArmor audit messages. 6978
Filebeat
* Comply with PostgreSQL database name format 7198
* Optimize PostgreSQL ingest pipeline to use anchored regexp and
merge multiple regexp into a single expression. 7269
* Keep different registry entry per container stream to avoid
wrong offsets. 7281
* Fix offset field pointing at end of a line. 6514
* Commit registry writes to stable storage to avoid corrupt registry
files. 6792
Metricbeat
* Fix field mapping for the system process CPU ticks fields. 7230
* Ensure canonical naming for JMX beans is disabled in Jolokia module. 7047
* Fix Jolokia attribute mapping when using wildcards and MBean
names with multiple properties. 7321
Packetbeat
* Fix an out of bounds access in HTTP parser caused by malformed
request. 6997
* Fix missing type for http.response.body field. 7169
Added
Auditbeat
* Added caching of UID and GID values to auditd module. 6978
* Updated syscall tables for Linux 4.16. 6978
* Added better error messages for when the auditd module fails
due to the Linux kernel not supporting auditing (CONFIG_AUDIT=n).
7012
Metricbeat
* Collect accumulated docker network metrics and mark old ones
as deprecated. 7253
6.3.0:
======
Breaking changes
Affecting all Beats
* De dot keys of labels and annotations in kubernetes meta processors
to prevent collisions. 6203
* Rename beat.cpu.*.time metrics to beat.cpu.*.time.ms. 6449
* Add host.name field to all events, to avoid mapping conflicts.
This could be breaking Logstash configs if you rely on the host
field being a string. 7051
Filebeat
* Add validation for Stdin, when Filebeat is configured with Stdin
and any other inputs, Filebeat will now refuse to start. 6463
* Mark system.syslog.message and system.auth.message as text
instead of keyword. 6589
Metricbeat
* De dot keys in kubernetes/event metricset to prevent collisions.
6203
* Add config option for windows/perfmon metricset to ignore non
existent counters. 6432
* Refactor docker CPU calculations to be more consistent with
docker stats. 6608
* Update logstash.node_stats metricset to write data under
logstash.node.stats.*. 6714
Bugfixes
Affecting all Beats
* Fix panic when Events containing a float32 value are normalized. 6129
* Fix setup.dashboards.always_kibana when using Kibana 5.6. 6090
* Fix for Kafka logger. 6430
* Remove double slashes in Windows service script. 6491
* Ensure Kubernetes labels/annotations don't break mapping 6490
* Ensure that the dashboard zip files can't contain files outside
of the kibana directory. 6921
* Fix map overwrite panics by cloning shared structs before doing
the update. 6947
* Fix delays on autodiscovery events handling caused by blocking
runner stops. 7170
* Do not emit Kubernetes autodiscover events for Pods without IP
address. 7235
* Fix self metrics when containerized 6641
Auditbeat
* Add hex decoding for the name field in audit path records. 6687
* Fixed a deadlock in the file_integrity module under Windows. 6864
Filebeat
* Fix panic when log prospector configuration fails to load. 6800
* Fix memory leak in log prospector when files cannot be read. 6797
* Add raw JSON to message field when JSON parsing fails. 6516
Commit registry writes to stable storage to avoid corrupt registry
files. 6877
* Fix a parsing issue in the syslog input for RFC3339 timestamp
and time with nanoseconds. 7046
* Fix an issue with an overflowing wait group when using the TCP
input. 7202
* Fix an issue when parsing ISO8601 dates with timezone definition 7367
Heartbeat
* Fix race due to updates of shared a map, that was not supposed
to be shared between multiple go-routines. 6616
Metricbeat
* Fix the default configuration for Logstash to include the default
port. 6279
* Fix dealing with new process status codes in Linux kernel 4.14+. 6306
* Add filtering option by exact device names in system.diskio.
diskio.include_devices. 6085
* Add connections metricset to RabbitMQ module 6548
* Fix panic in http dependent modules when invalid config was used. 6205
* Fix system.filesystem.used.pct value to match what df reports. 5494
* Fix namespace disambiguation in Kubernetes state_* metricsets. 6281
* Fix Windows perfmon metricset so that it sends metrics when an
error occurs. 6542
* Exclude bind mounts in fsstat and filesystem metricsets. 6819
* Don't stop Metricbeat if aerospike server is down. 6874
* Disk reads and write count metrics in RabbitMQ queue metricset
made optional. 6876
* Add mapping for docker metrics per cpu. 6843
Winlogbeat
* Fixed a crash under Windows 2003 and XP when an event had less
insert strings than required by its format string. 6247
Added
Affecting all Beats
* Update Golang 1.9.4 6326
* Add the ability to log to the Windows Event Log. 5913
* The node name can be discovered automatically by machine-id
matching when beat deployed outside Kubernetes cluster. 6146
* Panics will be written to the logger before exiting. 6199
* Add builder support for autodiscover and annotations builder 6408
* Add plugin support for autodiscover builders, providers 6457
* Preserve runtime from container statuses in Kubernetes autodiscover 6456
* Experimental feature setup.template.append_fields added. 6024
* Add appender support to autodiscover 6469
* Add add_host_metadata processor 5968
* Retry configuration to load dashboards if Kibana is not reachable
when the beat starts. 6560
* Add has_fields conditional to filter events based on the existence
of all the given fields. 6285 6653
* Add support for spooling to disk to the beats event publishing pipeline. 6581
* Added logging of system info at Beat startup. 5946
* Do not log errors if X-Pack Monitoring is enabled but Elastisearch
X-Pack is not. 6627
* Add rename processor. 6292
* Allow override of dynamic template match_mapping_type for fields
with object_type. 6691
Filebeat
* Add IIS module to parse access log and error log. 6127
* Renaming of the prospector type to the input type and all
prospectors are now moved to the input folder, to maintain
backward compatibility type aliasing was used to map the old
type to the new one. This change also affect YAML configuration.
6078
* Addition of the TCP input 6700
* Add option to convert the timestamps to UTC in the system module. 5647
* Add Logstash module support for main log and the slow log,
support the plain text or structured JSON format 5481
* Add stream filtering when using docker prospector. 6057
* Add support for CRI logs format. 5630
* Add json.ignore_decoding_error config to not log json decoding
erors. 6547
* Make registry file permission configurable. 6455
* Add MongoDB module. 6238
* Add Ingest pipeline loading to setup. 6814
* Add support of log_format combined to NGINX access logs. 6858
* Release config reloading feature as GA.
* Add support human friendly size for the UDP input. 6886
* Add Syslog input to ingest RFC3164 Events via TCP and UDP 6842
* Remove the undefined username option from the Redis input and
clarify the documentation. 6662
Heartbeat
* Made the URL field of Heartbeat aggregateable. 6263
* Use match.Matcher for checking Heartbeat response bodies with
regular expressions. 6539
Metricbeat
* Support apache status pages for versions older than 2.4.16. 6450
* Add support for huge pages on Linux. 6436
* Support to optionally de dot keys in http/json metricset to
prevent collisions. 5970
* Add graphite protocol metricbeat module. 4734
* Add http server metricset to support push metrics via http. 4770
* Make config object public for graphite and http server 4820
* Add system uptime metricset. 4848
* Add experimental queue metricset to RabbitMQ module. 4788
* Add additional php-fpm pool status kpis for Metricbeat module 5287
* Add etcd module. 4970
* Add ip address of docker containers to event. 5379
* Add ceph osd tree information to metricbeat 5498
* Add ceph osd_df to metricbeat 5606
* Add basic Logstash module. 5540
* Add dashboard for Windows service metricset. 5603
* Add pct calculated fields for Pod and container CPU and memory usages. 6158
* Add statefulset support to Kubernetes module. 6236
* Refactor prometheus endpoint parsing to look similar to upstream
prometheus 6332
* Making the http/json metricset GA. 6471
* Add support for array in http/json metricset. 6480
* Making the jolokia/jmx module GA. 6143
* Making the MongoDB module GA. 6554
* Allow to disable labels dedot in Docker module, in favor of a
safe way to keep dots. 6490
* Add experimental module to collect metrics from munin nodes. 6517
* Add support for wildcards and explicit metrics grouping in
jolokia/jmx. 6462
* Set collector as default metricset in Prometheus module. 6636 6747
* Set mntr as default metricset in Zookeeper module. 6674
* Set default metricsets in vSphere module. 6676
* Set status as default metricset in Apache module. 6673
* Set namespace as default metricset in Aerospike module. 6669
* Set service as default metricset in Windows module. 6675
* Set all metricsets as default metricsets in uwsgi module. 6688
* Allow autodiscover to monitor unexposed ports 6727
* Mark kubernetes.event metricset as beta. 6715
* Set all metricsets as default metricsets in couchbase module. 6683
* Mark uwsgi module and metricset as beta. 6717
* Mark Golang module and metricsets as beta. 6711
* Mark system.raid metricset as beta. 6710
* Mark http.server metricset as beta. 6712
* Mark metricbeat logstash module and metricsets as beta. 6713
* Set all metricsets as default metricsets in Ceph module. 6676
* Set container, cpu, diskio, healthcheck, info, memory and network
in docker module as default. 6718
* Set cpu, load, memory, network, process and process_summary as
default metricsets in system module. 6689
* Set collector as default metricset in Dropwizard module. 6669
* Set info and keyspace as default metricsets in redis module. 6742
* Set connection as default metricset in rabbitmq module. 6743
* Set all metricsets as default metricsets in Elasticsearch module. 6755
* Set all metricsets as default metricsets in Etcd module. 6756
* Set server metricsets as default in Graphite module. 6757
* Set all metricsets as default metricsets in HAProxy module. 6758
* Set all metricsets as default metricsets in Kafka module. 6759
* Set all metricsets as default metricsets in postgresql module. 6761
* Set status metricsets as default in Kibana module. 6762
* Set all metricsets as default metricsets in Logstash module. 6763
* Set container, node, pod, system, volume as default in Kubernetes
module. https://github.com/elastic/beats/pull/ 6764[6764]
* Set stats as default in memcached module. 6765
* Set all metricsets as default metricsets in Mongodb module. 6766
* Set pool as default metricset for php_fpm module. 6768
* Set status as default metricset for mysql module.
https://github.com/elastic/beats/pull/ 6769[6769]
* Set stubstatus as default metricset for nginx module. 6770
* Added support for haproxy 1.7 and 1.8. 6793
* Add accumulated I/O stats to diskio in the line of docker stats. 6701
* Ignore virtual filesystem types by default in system module. 6819
* Release config reloading feature as GA. 6891
* Kubernetes deployment: Add ServiceAccount config to system metricbeat. 6824
* Kubernetes deployment: Add DNS Policy to system metricbeat. 6656
Packetbeat
* Add support for condition on bool type 5659 5954
* Fix high memory usage on HTTP body if body is not published. 6680
* Allow to capture the HTTP request or response bodies independently. 6784
* HTTP publishes an Error event for unmatched requests or responses. 6794
Winlogbeat
* Use bookmarks to persist the last published event. 6150
Users who have previously installed this package by building as root should
very carefully check their Go installations. Running "pkg_admin check" will
almost certainly fail.
Auditbeat
- Add hex decoding for the name field in audit path records.
Filebeat
- Fix panic when log prospector configuration fails to load.
Packetbeat
- HTTP parses successfully on empty status phrase.
