==== Breaking changes
Affecting all Beats
- The log format may differ due to logging library changes.
- The default value for pipelining is reduced to 2 to avoid high
memory in the Logstash beats input.
Auditbeat
- Split the audit.kernel and audit.file metricsets into their own
modules
named auditd and file_integrity, respectively. This change requires
existing users to update their config.
- Renamed file_integrity module fields.
- Renamed auditd module fields.
Metricbeat
- Rename `golang.heap.system.optained` field to
`golang.heap.system.obtained`.
- De dot keys in jolokia/jmx metricset to prevent collisions.
==== Bugfixes
Auditbeat
- Fixed an issue where the proctitle value was being truncated.
- Fixed an issue where values were incorrectly interpretted as hex
data.
- Fixed parsing of the `key` value when multiple keys are present.
- Fix possible resource leak if file_integrity module is used with
config
reloading on Windows or Linux.
Filebeat
- Fix variable name for `convert_timezone` in the system module.
Metricbeat
- Fix error `datastore '*' not found` in Vsphere module.
- Fix error `NotAuthenticated` in Vsphere module.
- Fix mongodb session consistency mode to allow command execution on
secondary nodes.
- Fix kubernetes `state_pod` `status.phase` so that the active phase
is returned instead of `unknown`.
- Fix error collecting network_names in Vsphere module.
- Fix process cgroup memory metrics for memsw, kmem, and kmem_tcp.
- Fix kafka OffsetFetch request missing topic and partition
parameters.
Packetbeat
- Fix mysql SQL parser to trim `\r` from Windows Server
`SELECT\r\n\t1`.
==== Added
Affecting all Beats
- Adding a local keystore to allow user to obfuscate password
- Add autodiscover for kubernetes.
- Add Beats metrics reporting to Xpack.
- Update the command line library cobra and add support for zsh
completion
- Update to Golang 1.9.2
- Moved `ip_port` indexer for `add_kubernetes_metadata` to all beats.
- `ip_port` indexer now index both IP and IP:port pairs.
- Add the ability to write structured logs.
- Use structured logging for the metrics that are periodically logged
via the
`logging.metrics` feature.
- Improve Elasticsearch output metrics to count number of dropped and
duplicate (if event ID is given) events.
- Add the abilility for the add_docker_metadata process to enrich
based on process ID.
- The `add_docker_metadata` and `add_kubernetes_metadata` processors
are now GA, instead of Beta.
- Update go-ucfg library to support top level key reference and cyclic
key reference for the
keystore
Auditbeat
- Auditbeat is marked as GA, no longer Beta.
- Add support for BLAKE2b hash algorithms to the file integrity
module.
- Add support for recursive file watches.
Filebeat
- Add Osquery module.
- Add stream filtering when using `docker` prospector.
Metricbeat
- Add ceph osd_df to metricbeat
- Add field network_names of hosts and virtual machines.
- Add experimental system/raid metricset.
- Add a dashboard for the Nginx module.
- Add experimental mongodb/collstats metricset.
- Update the MySQL dashboard to use the Time Series Visual Builder.
- Add experimental uwsgi module.
- Docker and Kubernetes modules are now GA, instead of Beta.
- Support haproxy stats gathering using http (additionaly to tcp
socket).
- Support to optionally 'de dot' keys in http/json metricset to
prevent collisions.
Packetbeat
- Configure good defaults for `add_kubernetes_metadata`.
6.2.0
=== Breaking changes
Aggregations::
- Add a new cluster setting to limit the total number of buckets
returned by a request
Core::
- Forbid granting the all permission in production
Highlighting::
- Limit the analyzed text for highlighting
Search::
- Introduce limit to the number of terms in Terms Query
=== Breaking Java changes
Java API::
- Remove `operationThreaded` from Java API
Java High Level REST Client::
- REST high-level client: remove index suffix from indices client
method names
=== Deprecations
Analysis::
- Backport delimited payload filter renaming
Suggesters::
- deprecating `jarowinkler` in favor of `jaro_winkler`
- Deprecating `levenstein` in favor of `levensHtein`
=== New features
Plugin Ingest GeoIp::
- Enable ASN support for Ingest GeoIP plugin.
Plugin Lang Painless::
- Painless: Add spi jar that will be published for extending
whitelists
- Painless: Add a simple cache for whitelist methods and fields.
Plugins::
- Add the ability to bundle multiple plugins into a meta plugin
Rank Evaluation::
- Backport of ranking evaluation API (#27478)
Recovery::
- Backport for using lastSyncedGlobalCheckpoint in deletion policy
Reindex API::
- Add scroll parameter to _reindex API
=== Enhancements
Allocation::
- Fix cluster.routing.allocation.enable and
cluster.routing.rebalance.enable case
- Add node id to shard failure message
Analysis::
- Limit the analyzed text for highlighting (#27934)
- Allow TrimFilter to be used in custom normalizers
Circuit Breakers::
- Add accounting circuit breaker and track segment memory usage
Cluster::
- Adds wait_for_no_initializing_shards to cluster health API
Core::
- Introduce elasticsearch-core jar
- Rename core module to server
- Introduce elasticsearch-core jar
- Add Writeable.Reader support to TransportResponseHandler
- Simplify rejected execution exception
- Add node name to thread pool executor name
Discovery::
- Add information when master node left to DiscoveryNodes'
shortSummary()
Engine::
- Move uid lock into LiveVersionMap
- Optimize version map for append-only indexing
Geo::
- [GEO] Add WKT Support to GeoBoundingBoxQueryBuilder
- [Geo] Add Well Known Text (WKT) Parsing Support to ShapeBuilders
Highlighting::
- Include all sentences smaller than fragment_size in the unified
highlighter
Ingest::
- Enable convert processor to support Long and Double
Internal::
- Make KeyedLock reentrant
- Make AbstractQueryBuilder.declareStandardFields to be protected
- Tighten the CountedBitSet class
- Avoid doing redundant work when checking for self references.
Java API::
- Add missing delegate methods to NodeIndicesStats
- Java api clean-up : consistency for `shards_acknowledged` getters
Java High Level REST Client::
- add toString implementation for UpdateRequest.
- Add Close Index API to the high level REST client
- Add Open Index API to the high level REST client
- Added Create Index support to high-level REST client
- Add multi get api to the high level rest client
- Add msearch api to high level client
Mapping::
- Allow `_doc` as a type.
Network::
- Add NioGroup for use in different transports
- Add read timeouts to http module
- Implement byte array reusage in `NioTransport`
- Introduce resizable inbound byte buffer
- Decouple nio constructs from the tcp transport
Packaging::
- Extend JVM options to support multiple versions
- Add explicit coreutils dependency
- Detect mktemp from coreutils
- Enable GC logs by default
- Use private directory for temporary files
Percolator::
- also extract match_all queries when indexing percolator queries
Plugin Lang Painless::
- Painless: Add whitelist extensions
- Painless: Modify Loader to Load Classes Directly from Definition
- Clean Up Painless Cast Object
- Painless: Only allow Painless type names to be the same as the
equivalent Java class.
Plugins::
- Add client actions to action plugin
- Plugins: Add validation to plugin descriptor parsing
- Plugins: Add plugin extension capabilities
- Add support for filtering mappings fields
Rank Evaluation::
- Simplify RankEvalResponse output
Recovery::
- Truncate tlog cli should assign global checkpoint
- Replica starts peer recovery with safe commit
- Primary send safe commit in file-based recovery
- Fail resync-failed shards in subsequent writes
- Introduce promoting index shard state
- Non-peer recovery should set the global checkpoint
- Persist global checkpoint when finalizing a peer recovery
- Rollback a primary before recovering from translog
Rollover::
- Fail rollover if duplicated alias found in templates
Search::
- Use typeName() to check field type in GeoShapeQueryBuilder
- Optimize search_after when sorting in index sort order
Sequence IDs::
- Do not keep 5.x commits when having 6.x commits
- Use lastSyncedGlobalCheckpoint in deletion policy
- Use CountedBitSet in LocalCheckpointTracker
- Only fsync global checkpoint if needed
- Keep commits and translog up to the global checkpoint
- Adjust CombinedDeletionPolicy for multiple commits
- Keeps index commits up to the current global checkpoint
- Dedup translog operations by reading in reverse
Settings::
- Add validation of keystore setting names
Snapshot/Restore::
- Use AmazonS3.doesObjectExist() method in S3BlobContainer
- Remove XContentType auto detection in BlobStoreRepository
- Include include_global_state in Snapshot status API (#22423)
Task Manager::
- Add ability to associate an ID with tasks
Translog::
- Simplify MultiSnapshot#SeqNoset
- Enclose CombinedDeletionPolicy in SnapshotDeletionPolicy
=== Bug fixes
Aggregations::
- Adds metadata to rewritten aggregations
- Fix NPE on composite aggregation with sub-aggregations that need
scores
- StringTerms.Bucket.getKeyAsNumber detection type
- Fix incorrect results for aggregations nested under a nested
aggregation
- Fix global aggregation that requires breadth first and scores
- Fix composite aggregation when after term is missing in the shard
- Fix preserving FiltersAggregationBuilder#keyed field on rewrite
- Using DocValueFormat::parseBytesRef for parsing missing value
parameter
- Fix illegal cast of the "low cardinality" optimization of the
`terms` aggregation.
- Always include the _index and _id for nested search hits.
Allocation::
- Do not open indices with broken settings
Core::
- Fix lock accounting in releasable lock
- Fixes ByteSizeValue to serialise correctly
- Do not set data paths on no local storage required
- Ensure threadcontext is preserved when refresh listeners are invoked
- Ensure logging is configured for CLI commands
Engine::
- Replica recovery could go into an endless flushing loop
- Use `_refresh` to shrink the version map on inactivity
- Allow resize version map under lock even if there are pending
operations
- Reset LiveVersionMap on sync commit
Geo::
- Correct two equality checks on incomparable types
- Handle case where the hole vertex is south of the containing
polygon(s)
Highlighting::
- Fix highlighting on a keyword field that defines a normalizer
Inner Hits::
- Add version support for inner hits in field collapsing (#27822)
Internal::
- Never return null from Strings.tokenizeToStringArray
- Fallback to TransportMasterNodeAction for cluster health retries
- Retain originalIndex info when rewriting FieldCapabilities requests
Java REST Client::
- Do not use system properties when building the HttpAsyncClient
Mapping::
- Ignore null value for range field (#27845)
- Pass `java.locale.providers=COMPAT` to Java 9 onwards
- Allow update of `eager_global_ordinals` on `_parent`.
- Fix merging of _meta field
Network::
- Only bind loopback addresses when binding to local
- Remove potential nio selector leak
- Fix issue where the incorrect buffers are written
- Throw UOE from compressible bytes stream reset
- Bubble exceptions when closing compressible streams
Packaging::
- Allow custom service names when installing on windows
Percolator::
- Avoid TooManyClauses exception if number of terms / ranges is
exactly equal to 1024
Plugin Analysis ICU::
- Catch InvalidPathException in IcuCollationTokenFilterFactory
Plugin Analysis Phonetic::
- Fix daitch_mokotoff phonetic filter to use the dedicated Lucene
filter
Plugin Lang Painless::
- Painless: Fix variable scoping issue in lambdas
- Painless: Fix errors allowing void to be assigned to def.
Plugin Repository HDFS::
- Fix SecurityException when HDFS Repository used against HA Namenodes
Plugins::
- Make sure that we don't detect files as maven coordinate when
installing a plugin
- Fix upgrading indices which use a custom similarity plugin.
Recovery::
- Open engine should keep only starting commit
- Allow shrinking of indices from a previous major
- Set global checkpoint before open engine from store
- Check and repair index under the store metadata lock
- Flush old indices on primary promotion and relocation
Rollover::
- Make index rollover action atomic
Scripting::
- Ensure we protect Collections obtained from scripts from
self-referencing
Scroll::
- Reject scroll query if size is 0 (#22552)
- Fix scroll query with a sort that is a prefix of the index sort
Search::
- Fix simple_query_string on invalid input
- Use the underlying connection version for CCS connections
- Fix synonym phrase query expansion for cross_fields parsing
- Carry forward weights, etc on rescore rewrite
- Fix routing with leading or trailing whitespace
Sequence IDs::
- Recovery from snapshot may leave seq# gaps
- No longer unidle shard during recovery
- Obey translog durability in global checkpoint sync
Settings::
- Settings: Introduce settings updater for a list of settings
- Fix setting notification for complex setting (affixMap settings)
that could cause transient settings to be ignored
- Fix environment variable substitutions in list setting
- Allow index settings to be reset by wildcards
Snapshot/Restore::
- Consistent updates of IndexShardSnapshotStatus
- Avoid concurrent snapshot finalizations when deleting an INIT
snapshot
- Do not start snapshots that are deleted during initialization
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic()
- Consistent update of stage and failure message in
IndexShardSnapshotStatus
- Fail restore when the shard allocations max retries count is reached
- Delete shard store files before restoring a snapshot
Stats::
- Fixes DocStats to properly deal with shards that report -1 index
size
- Include internal refreshes in refresh stats
Term Vectors::
- Fix term vectors generator with keyword and normalizer
=== Upgrades
Core::
- Dependencies: Update joda time to 2.9.9
- upgrade to lucene 7.2.1
- Upgrade jna from 4.4.0-1 to 4.5.1
Ingest::
- update ingest-attachment to use Tika 1.17 and newer deps
6.1.3
Engine::
- Replica recovery could go into an endless flushing loop
Internal::
- Never return null from Strings.tokenizeToStringArray
- Fallback to TransportMasterNodeAction for cluster health retries
Mapping::
- Allow update of `eager_global_ordinals` on `_parent`.
Scripting::
- Ensure we protect Collections obtained from scripts from
self-referencing
Settings::
- Fix setting notification for complex setting (affixMap settings)
that could cause transient settings to be ignored
- Fix environment variable substitutions in list setting
Snapshot/Restore::
- Avoid concurrent snapshot finalizations when deleting an INIT
snapshot
- Do not start snapshots that are deleted during initialization
- agent: Fixed several areas where reading from catalog, health, or
agent HTTP endpoints could make unintended mofidications to Consul's
state in a way that would cause unnecessary anti-entropy syncs back to
the Consul servers
This was added 8 years ago intending to ensure pkgsrc libtool was used, except
it actually did the opposite. By default we will try to replace any script
named "libtool" up to the override depth. Setting LIBTOOL_OVERRIDE changes
the behaviour to instead override the list of filenames specified by the
LIBTOOL_OVERRIDE variable. Thus we were trying to replace any file named
"YES" instead of "libtool".
Noticed by SmartOS/clang builds of php-intl which were using the supplied
broken libtool script instead of the correct pkgsrc one.
3.0.13
- Fix bug where simple field aliases were being ignored.
- More strict about column type inference for postgres + pwiz.
3.0.12
- Fix queries of the form INSERT ... VALUES (SELECT...) so that
sub-select is wrapped in parentheses.
- Improve model-graph resolution when selecting from multiple tables
that are joined by foreign-keys, and an intermediate table is
omitted from selection.
- Docs update to reflect deletion of post_init signal.
3.0.11
- Add note to changelog about cursor() method.
- Add hash method to postgres indexedfield subclasses.
- Add TableFunction to sqlite_ext module namespace.
- Fix bug regarding NOT IN queries where the right-hand-side is an
empty set.
- Fallback implementations of bm25f and lucene search ranking
algorithms.
- Fixed DecimalField issue.
- Fixed issue with BlobField when database is a Proxy object.
Added support for the BoarSSL / Twrch test framework.
Header files now include the ritual mantras that make them compatible with C++.
Better Makefile behaviour with Clang (FreeBSD compatibility).
Worked around a bug of GCC 4.8 and 4.9 in 32-bit x86 mode.
Incoming application data after initiating closure is now properly ignored.
Some instances of (critical) Certificate Policies extensions are now ignored (when it is safe to do so).
Fixed some behavioural bugs with regards to renegotiation (all were failing safe).
Added encoded OID for hash functions in the public API, to help with using RSA signatures in non-SSL contexts.
Fixed bug in AES/CBC decryption on x86 with AES-NI opcode (this was breaking decryption with AES/CBC cipher suites and TLS 1.0 only).
Added an explicit stack buffer initialisation (some provably harmless accesses to uninitialised data were performed, Valgrind was not happy).
Fixed bug in the search tree for the cache of SSL sessions (occasional lock-ups).
Fixed bug in modular reduction in the special field for P-256 (this was infrequently breaking ECDSA signatures).
Added support for exporting keying material (RFC 5705).
Added new general-purpose API for AEAD implementations (in non-SSL contexts) and an AES/GCM implementation that follows that API.
Added a function to forget saved session parameter in the LRU cache.
Added a new ChaCha20 implementation that uses SSE2 on x86 architectures (both 32-bit and 64-bit).
3.4.11:
Resolved issues
* GH-121. the record list was still not correct due to PEP3147
and __pycache__ directories.
3.4.10:
Resolved issues
* When creating ElGamal keys, the generator wasn't a square residue:
ElGamal encryption done with those keys cannot be secure under
the DDH assumption.
3.4.9:
New features
* More meaningful error messages while importing an ECC key.
Resolved issues
* GH-123 and #125. The SSE2 command line switch was not always passed on 32-bit x86 platforms.
* GH-121. The record list (--record) was not always correctly filled for the
pycryptodomex package.
-march=native yields an error in some configurations.
doxygen is not a dependency, so if the user has it and it's detected,
we have plist mismatches.
PKGREVISION++
