What's new in psycopg 2.7
-------------------------
New features:
- Added `~psycopg2.sql` module to generate SQL dynamically (🎫`#308`).
- Added :ref:`replication-support` (🎫`#322`). Main authors are
Oleksandr Shulgin and Craig Ringer, who deserve a huge thank you.
- Added `~psycopg2.extensions.parse_dsn()` and
`~psycopg2.extensions.make_dsn()` functions (🎟️`#321, #363`).
`~psycopg2.connect()` now can take both *dsn* and keyword arguments, merging
them together.
- Added `~psycopg2.__libpq_version__` and
`~psycopg2.extensions.libpq_version()` to inspect the version of the
``libpq`` library the module was compiled/loaded with
(🎟️`#35, #323`).
- The attributes `~connection.notices` and `~connection.notifies` can be
customized replacing them with any object exposing an `!append()` method
(🎫`#326`).
- Adapt network types to `ipaddress` objects when available. When not
enabled, convert arrays of network types to lists by default. The old `!Inet`
adapter is deprecated (🎟️`#317, #343, #387`).
- Added `~psycopg2.extensions.quote_ident()` function (🎫`#359`).
- Added `~connection.get_dsn_parameters()` connection method (🎫`#364`).
- `~cursor.callproc()` now accepts a dictionary of parameters (🎫`#381`).
- Give precedence to `!__conform__()` over superclasses to choose an object
adapter (🎫`#456`).
- Using Python C API decoding functions and codecs caching for faster
unicode encoding/decoding (🎫`#473`).
- `~cursor.executemany()` slowness addressed by
`~psycopg2.extras.execute_batch()` and `~psycopg2.extras.execute_values()`
(🎫`#491`).
- Added ``async_`` as an alias for ``async`` to support Python 3.7 where
``async`` will become a keyword (🎫`#495`).
- Unless in autocommit, do not use :sql:`default_transaction_*` settings to
control the session characteristics as it may create problems with external
connection pools such as pgbouncer; use :sql:`BEGIN` options instead
(🎫`#503`).
- `~connection.isolation_level` is now writable and entirely separated from
`~connection.autocommit`; added `~connection.readonly`,
`~connection.deferrable` writable attributes.
Bug fixes:
- Fixed error caused by missing decoding `~psycopg2.extras.LoggingConnection`
(🎫`#483`).
- Fixed integer overflow in :sql:`interval` seconds (🎫`#512`).
Other changes:
- Dropped support for Python 2.5 and 3.1.
- Dropped support for client library older than PostgreSQL 9.1 (but older
server versions are still supported).
- `~connection.isolation_level` doesn't read from the database but will return
`~psycopg2.extensions.ISOLATION_LEVEL_DEFAULT` if no value was set on the
connection.
- Empty arrays no more converted into lists if they don't have a type attached
(🎫`#506`)
5.2.2
- The request circuit breaker, which track the size of in-flight
requests, was not decrementing its counter when the connection was
closed by the client before the response could be returned. This could
result in no further requests being accepted until the node has been
restarted. All users should upgrade to take advantage of this bug fix.
- The cgroup functionality (for reading OS stats for containers) had a
bug in the regex for parsing /proc/self/cgroup which prevented nodes
with multiple hierarchies from starting up.
- Requests which have been delayed by a shard lock might be executed
later on a different thread which lacks the request context, resulting
in the requests being incorrectly rejected as unauthorised.
- Removal of an undocumented syntax for include/exclude parameters in
the terms aggregation broke certain aggregations in Kibana.
5.2.1
- When stored fields are configured with best_compression, we rely on
JVM garbage collection to reclaim Deflater/Inflater instances. However
these classes use little JVM memory but may use significant native
memory, so if may happen that the OS runs out of native memory before
the JVM collects these unreachable Deflater/Inflater instances.
- Certain queries can hold a reference to the IndexReader that was used
to build them. When these queries are cached, this reference causes
Lucene to hold onto segments long after they should have been deleted.
Features
- Quote index names in metadata CQL generation (PYTHON-616)
- On column deserialization failure, keep error message consistent
between python and cython (PYTHON-631)
- TokenAwarePolicy always sends requests to the same replica for a given
key (PYTHON-643)
- Added cql types to result set (PYTHON-648)
- Add __len__ to BatchStatement (PYTHON-650)
- Duration Type for Cassandra (PYTHON-655)
- Send flags with PREPARE message in v5 (PYTHON-684)
Bug Fixes
- Potential Timing issue if application exits prior to session pool
initialization (PYTHON-636)
- "Host X.X.X.X has been marked down" without any exceptions
(PYTHON-640)
- NoHostAvailable or OperationTimedOut when using execute_concurrent
with a generator that inserts into more than one table (PYTHON-642)
- ResponseFuture creates Timers and don't cancel them even when result
is received which leads to memory leaks (PYTHON-644)
- Driver cannot connect to Cassandra version > 3 (PYTHON-646)
- Unable to import model using UserType without setuping connection
since 3.7 (PYTHON-649)
- Don't prepare queries on ignored hosts on_up (PYTHON-669)
- Sockets associated with sessions not getting cleaned up on
session.shutdown() (PYTHON-673)
- Make client timestamps strictly monotonic (PYTHON-676)
- cassandra.cqlengine.connection.register_connection broken when
hosts=None (PYTHON-692)
Other
- Create a cqlengine doc section explaining None semantics (PYTHON-623)
- Resolve warnings in documentation generation (PYTHON-645)
- Cython dependency (PYTHON-686)
- Drop Support for Python 2.6 (PYTHON-690)
5.2.0 (2017-02-12)
- The client now automatically sends Content-Type http header set
to application/json. If you are explicitly passing in other
encoding than json you need to set the header manually.
5.1.0 (2017-01-11)
- Fixed sniffing
=== Breaking changes
Core::
- Add system call filter bootstrap check
=== Breaking Java changes
Allocation::
- Cluster Explain API uses the allocation process to explain shard
allocation decisions
Cluster::
- Remove PROTO-based custom cluster state components
Ingest::
- Change type of ingest doc meta-data field 'TIMESTAMP' to `Date`
Internal::
- Consolidate the last easy parser construction
- Introduce XContentParser#namedObject
Plugins::
- Plugins: Replace Rest filters with RestHandler wrapper
Query DSL::
- Resolve index names in indices_boost
Search::
- Cluster search shards improvements: expose ShardId, adjust visibility
of some members
=== Deprecations
Java API::
- Deprecate and remove "minimumNumberShouldMatch" in BoolQueryBuilder
Shadow Replicas::
- Add a deprecation notice to shadow replicas
=== New features
Analysis::
- Synonym Graph Support (LUCENE-6664)
Ingest::
- introduce KV Processor in Ingest Node
Mapping::
- Add the ability to set a normalizer on keyword fields.
- Add RangeFieldMapper for numeric and date range types
=== Enhancements
Aggregations::
- `value_type` is useful regardless of scripting.
- Support for partitioning set of terms
Allocation::
- Adds setting level to allocation decider explanations
- Improves allocation decider decision explanation messages
- Prepares allocator decision objects for use with the allocation
explain API
Analysis::
- Expose FlattenGraphTokenFilter
CRUD::
- Use correct block levels for TRA subclasses
Cluster::
- Keep NodeConnectionsService in sync with current nodes in the cluster
state
- Add a generic way of checking version before serializing custom
cluster object
- Add validation for supported index version on node join, restore,
upgrade & open index
- Let ClusterStateObserver only hold onto state that's needed for change
detection
- Remove mutable status field from cluster state
Core::
- Rename bootstrap.seccomp to bootstrap.system_call_filter
- Cleanup random stats serialization code
- Avoid corruption when deserializing booleans
- Reduce memory pressure when sending large terms queries.
Discovery::
- Simplify Unicast Zen Ping
- Prefer joining node with conflicting transport address when becoming
master
Ingest::
- add `ignore_missing` flag to ingest plugins
- Added ability to remove pipelines via wildcards (#22149)
- Enables the ability to inject serialized json fields into root of
document
- compile ScriptProcessor inline scripts when creating ingest pipelines
Internal::
- Remove some more usages of ParseFieldMatcher
- Remove some more usages of ParseFieldMatcher
- Remove some ParseFieldMatcher usages
- Add infrastructure to manage network connections outside of
Transport/TransportService
- Replace strict parsing mode with response headers assertions
- Start using `ObjectParser` for aggs.
- Don't output null source node in RecoveryFailedException
- ClusterService should expose "applied" cluster states (i.e., remove
ClusterStateStatus)
Java High Level REST Client::
- Add fromxcontent methods to index response
- Add fromXContent() methods for ReplicationResponse
- Add parsing method for ElasticsearchException
- Add fromXContent method to GetResponse
Java REST Client::
- Warn log deprecation warnings received from server
Logging::
- Log deleting indices at info level
- Log failure to connect to node at info instead of debug
Mapping::
- Atomic mapping updates across types
- Only update DocumentMapper if field type changes
- Better error message when `_parent` isn't an object
Network::
- Disable the Netty recycler
- Tell Netty not to be unsafe in transport client
- Introduce a low level protocol handshake
- Detach handshake from connect to node
- Reduce number of connections per node depending on the nodes role
- Add a connect timeout to the ConnectionProfile to allow per node
connect timeouts
- Grant Netty permission to read system somaxconn
- Remove connectToNodeLight and replace it with a connection profile
Packaging::
- Improve the out-of-the-box experience
Plugin Discovery EC2::
- Add support for ca-central-1 region to EC2 and S3 plugins
- Support for eu-west-2 (London) cloud-aws plugin
- Add us-east-2 AWS region
- Add setting to set read timeout for EC2 discovery and S3 repository
plugins
Plugin Ingest GeoIp::
- Cache results of geoip lookups
Plugin Lang Painless::
- Improve painless's ScriptException generation
Plugins::
- Pass ThreadContext to transport interceptors to allow header
modification
- Provide helpful error message if a plugin exists
- Add shutdown hook for closing CLI commands
- Allow plugins to install bootstrap checks
Query DSL::
- Un-deprecate fuzzy query
- support numeric bounds with decimal parts for long/integer/short/byte
datatypes
Reindex API::
- Reindex: Better error message for pipeline in wrong place
- Timeout improvements for rest client and reindex
Scripting::
- Wrap VerifyError in ScriptException
- Log ScriptException's xcontent if file script compilation fails
Search::
- Move SearchTransportService and SearchPhaseController creation outside
of TransportSearchAction constructor
- Don't carry ShardRouting around when not needed in
AbstractSearchAsyncAction
- ShardSearchRequest to take ShardId constructor argument rather than
the whole ShardRouting
- Use index uuid as key in the alias filter map rather than the index
name
Settings::
- Speed up filter and prefix settings operations
Snapshot/Restore::
- Fixes retrieval of the latest snapshot index blob
- Synchronize snapshot deletions on the cluster state
Stats::
- Include unindexed field in FieldStats response
Task Manager::
- Improve the error message if task and node isn't found
- Add descriptions to create snapshot and restore snapshot tasks.
- Add proper descriptions to reindex, update-by-query and
delete-by-query tasks.
=== Bug fixes
Aggregations::
- Fix scaled_float numeric type in aggregations
- Allow terms aggregations on pure boolean scripts.
- Fix numeric terms aggregations with includes/excludes and
minDocCount=0
- Fix `missing` on aggs on `boolean` fields.
- IP range masks exclude the maximum address of the range.
- Fix `other_bucket` on the `filters` agg to be enabled if a key is set.
Allocation::
- Cannot force allocate primary to a node where the shard already exists
- Promote shadow replica to primary when initializing primary fails
- Trim in-sync allocations set only when it grows
Analysis::
- AsciiFoldingFilter's multi-term component should never preserve the
original token.
- Pre-built analysis factories do not implement MultiTermAware
correctly.
CRUD::
- Reject external versioning and explicit version numbers on create
Cluster::
- IndicesService handles all exceptions during index deletion
Core::
- Streamline foreign stored context restore and allow to perserve
response headers
- Support negative numbers in readVLong
Discovery::
- Close InputStream when receiving cluster state in
PublishClusterStateAction
- Do not reply to pings from another cluster
Engine::
- Close and flush refresh listeners on shard close
Exceptions::
- Fixing shard recovery error message to report the number of docs
correctly for each node
Highlighting::
- Fix FiltersFunctionScoreQuery highlighting
Index APIs::
- Ensure shrunk indices carry over version information from its source
Ingest::
- fix index out of bounds error in KV Processor
- Fixes GrokProcessor's ignorance of named-captures with same name.
Inner Hits::
- Inner hits and ignore unmapped
Internal::
- Index creation and setting update may not return deprecation logging
Java API::
- Don't output empty ext object in SearchSourceBuilder#toXContent
Java REST Client::
- Don't use null charset in RequestLogger
Mapping::
- The `_all` default mapper is not completely configured.
- Fix MapperService.allEnabled().
- Dynamic `date` fields should use the `format` that was used to detect
it is a date.
- Sub-fields should not accept `include_in_all` parameter
- Mappings: Fix get mapping when no indexes exist to not fail in
response generation
Nested Docs::
- Fix bug in query builder rewrite that ignores the ignore_unmapped
option
Network::
- Pass `forceExecution` flag to transport interceptor
- Ensure new connections won't be opened if transport is closed or
closing
- Prevent open channel leaks if handshake times out or is interrupted
- Execute low level handshake in #openConnection
- Properly configure Netty 3 ClientBootstrap when using custom
connection profile
- Handle connection close / reset events gracefully during handshake
- Do not lose host information when pinging
- DiscoveryNode and TransportAddress should preserve host information
Packaging::
- Another fix for handling of paths on Windows
- Fix handling of spaces in Windows paths
- Add option to skip kernel parameters on install
Percolator::
- Fix NPE in percolator's 'now' range check for percolator queries with
range queries
Plugin Analysis Stempel::
- Fix thread safety of Stempel's token filter factory
Plugin Lang Painless::
- Whitelist some ScriptDocValues in painless
- Update Painless Loop Counter to be Higher
- Fix some issues with painless's strings
- Test fix for def equals in Painless
- Fix a VerifyError bug in Painless
Plugin Mapper Attachment::
- NPE is raised when defining a non existing type within attachments
type
Plugin Repository Azure::
- Fixes default chunk size for Azure repositories
- readonly on azure repository must be taken into account
Query DSL::
- Fix script score function that combines `_score` and weight
REST::
- Adds percent-encoding for Location headers
Recovery::
- Fix primary relocation for shadow replicas
- Don't close store under CancellableThreads
- Use a fresh recovery id when retrying recoveries
- Allow flush/force_merge/upgrade on shard marked as relocated
Reindex API::
- Fix reindex from remote clearing scroll
- Fix source filtering in reindex-from-remote
- Remove content type detection from reindex-from-remote
- Don't close rest client from its callback
- Keep context during reindex's retries
Scripting::
- Expose `ip` fields as strings in scripts.
Search::
- Improve concurrency of ShardCoreKeyMap.
- Make `-0` compare less than `+0` consistently.
- Fix boost_mode propagation when the function score query builder is
rewritten
- FiltersAggregationBuilder: rewriting filter queries, the same way as
in FilterAggregationBuilder
- Fix cross_fields type on multi_match query with synonyms
Search Templates::
- SearchTemplateRequest to implement CompositeIndicesRequest
Settings::
- Handle spaces in `action.auto_create_index` gracefully
- Fix settings diff generation for affix and group settings
Snapshot/Restore::
- Gracefully handles pre 2.x compressed snapshots
- URLRepository should throw NoSuchFileException to correctly adhere to
readBlob contract
- Fixes shard level snapshot metadata loading when index-N file is
missing
Stats::
- Implement stats for geo_point and geo_shape field
- Use reader for doc stats
- Avoid NPE in NodeService#stats if HTTP is disabled
- Add support for "include_segment_file_sizes" in indices stats REST
handler
Term Vectors::
- Fix `_termvectors` with preference to not hit NPE
- Return correct term statistics when a field is not found in a shard
=== Regressions
Core::
- Source filtering: only accept array items if the previous include
pattern matches
Logging::
- Restores the original default format of search slow log
Search::
- Handle specialized term queries in MappedFieldType.extractTerm(Query)
=== Upgrades
Core::
- Upgrade to Lucene 6.4.0
- Update Jackson to 2.8.6
- Upgrade to lucene-6.4.0-snapshot-084f7a0.
- Upgrade to lucene-6.4.0-snapshot-ec38570
================================================================================
Redis 3.2.8 Released Sun Feb 12 16:11:18 CET 2017
================================================================================
Two important bug fixes, the first of one is critical:
1. Apparently Jemalloc 4.4.0 may contain a deadlock under particular
conditions. See https://github.com/antirez/redis/issues/3799.
We reverted back to the previously used Jemalloc versions and plan
to upgrade Jemalloc again after having more info about the
cause of the bug.
2. MIGRATE could crash the server after a socket error. See for reference:
https://github.com/antirez/redis/issues/3796.
================================================================================
Redis 3.2.7 Released Tue Jan 31 16:21:41 CET 2017
================================================================================
Main bugs fixes and improvements in this release:
1. MIGRATE could incorrectly move keys between Redis Cluster nodes by turning
keys with an expire set into persisting keys. This bug was introduced with
the multiple-keys migration recently. It is now fixed. Only applies to
Redis Cluster users that use the resharding features of Redis Cluster.
2. As Redis 4.0 beta and the unstable branch already did (for some months at
this point), Redis 3.2.7 also aliases the Host: and POST commands to QUIT
avoiding to process the remaining pipeline if there are pending commands.
This is a security protection against a "Cross Scripting" attack, that
usually involves trying to feed Redis with HTTP in order to execute commands.
Example: a developer is running a local copy of Redis for development
purposes. She also runs a web browser in the same computer. The web browser
could send an HTTP request to http://127.0.0.1:6379 in order to access the
Redis instance, since a specially crafted HTTP requesta may also be partially
valid Redis protocol. However if POST and Host: break the connection, this
problem should be avoided. IMPORTANT: It is important to realize that it
is not impossible that another way will be found to talk with a localhost
Redis using a Cross Protocol attack not involving sending POST or Host: so
this is only a layer of protection but not a definitive fix for this class
of issues.
3. A ziplist bug that could cause data corruption, could crash the server and
MAY ALSO HAVE SECURITY IMPLICATIONS was fixed. The bug looks complex to
exploit, but attacks always get worse, never better (cit). The bug is very
very hard to catch in practice, it required manual analysis of the ziplist
code in order to be found. However it is also possible that rarely it
happened in the wild. Upgrading is required if you use LINSERT and other
in-the-middle list manipulation commands.
4. We upgraded to Jemalloc 4.4.0 since the version we used to ship with Redis
was an early 4.0 release of Jemalloc. This version may have several
improvements including the ability to better reclaim/use the memory of
system.
* Approximately 25% better performance from the R-Tree extension.
* Uses compiler built-ins (ex: __builtin_bswap32() or _byteswap_ulong()) for byteswapping when available.
* Uses the sqlite3_blob key/value access object instead of SQL for pulling content out of R-Tree nodes
* Other miscellaneous enhancements such as loop unrolling.
* Add the SQLITE_DEFAULT_LOOKASIDE compile-time option.
* Increase the default lookaside size from 512,125 to 1200,100 as this provides better performance while only adding 56KB of extra memory per connection. Memory-sensitive applications can restore the old default at compile-time, start-time, or run-time.
* Use compiler built-ins __builtin_sub_overflow(), __builtin_add_overflow(), and __builtin_mul_overflow() when available. (All compiler built-ins can be omitted with the SQLITE_DISABLE_INTRINSIC compile-time option.)
* Added the SQLITE_ENABLE_NULL_TRIM compile-time option, which can result in significantly smaller database files for some applications, at the risk of being incompatible with older versions of SQLite.
* Change SQLITE_DEFAULT_PCACHE_INITSZ from 100 to 20, for improved performance.
* Added the SQLITE_UINT64_TYPE compile-time option as an analog to SQLITE_INT64_TYPE.
* Perform some UPDATE operations in a single pass instead of in two passes.
* Enhance the session extension to support WITHOUT ROWID tables.
* Fixed performance problems and potential stack overflows when creating views from multi-row VALUES clauses with hundreds of thousands of rows.
* Added the sha1.c extension.
* In the command-line shell, enhance the ".mode" command so that it restores the default column and row separators for modes "line", "list", "column", and "tcl".
* Enhance the SQLITE_DIRECT_OVERFLOW_READ option so that it works in WAL mode as long as the pages being read are not in the WAL file.
* Enhance the LEMON parser generator so that it can store the parser object as a stack variable rather than allocating space from the heap and make use of that enhancement in the amalgamation.
* Other performance improvements. Uses about 6.5% fewer CPU cycles.
Bug Fixes:
* Throw an error if the ON clause of a LEFT JOIN references tables to the right of the ON clause. This is the same behavior as PostgreSQL. Formerly, SQLite silently converted the LEFT JOIN into an INNER JOIN.
* Use the correct affinity for columns of automatic indexes.
* Ensure that the sqlite3_blob_reopen() interface can correctly handle short rows.
6.2.4:
* More complete fix for pkgsrc.
6.2.3:
* Update copyright notices.
* Solve a conflict between different installations of Berkeley DB
on some pkgsrc configurations.
6.2.2:
* Correctly detect Berkeley DB installations in SmartOS native zones.
* "Probably" (not tested) correctly detect Berkeley DB in pkgsrc systems.
New Features/Improvements
-------------------------
Added support for global message prefixes.
Added Replication Manager write forwarding, which enables a client to perform simple put and delete operations by forwarding them to the master.
Improved the way we mark a password as consumed.
Added support for IPv6.
Added support for Java 8.
Added support for ARMv7 and above, and for ARM64. ARMv6 and below are no longer supported.
Add a new SMP-optimized data model that is especially valuable for highly multithreaded applications. A sliced Berkeley DB instance contains a collection of related Berkeley DB environments, known as slices, each containing a disjoint subset of records. Transactional consistency is individually maintained within each slice, rather than across multiple slices.
DB now supports the client-server architecture by providing a stand-alone server program and Java client driver APIs. The server program offers remote access to DB features. The client driver APIs provide building blocks for applications that communicate with a database server. Multiple client applications can communicate with a single server simultaneously.
Enabled support of the SQLite user authentication extension.
Added Visual Studio 2012 projects.
Encryption and checksum can now be enabled with external files (formerly known as BLOBs). Note that only the data stored in the database will be encrypted and checksum protected. The data in the external files will not be encrypted or checksum protected.
The BLOB files feature has been renamed external files to reduce confusion with the SQL BLOB datatype. The BLOB related functions have been deprecated, and replaced with new function names; for example, ext_file_threshold has replaced blob_threshold, and blob_threshold is deprecated. See the Upgrading section of the documentation for more details.
MASTER_SITES= site1 \
site2
style continuation lines to be simple repeated
MASTER_SITES+= site1
MASTER_SITES+= site2
lines. As previewed on tech-pkg. With thanks to rillig for fixing pkglint
accordingly.
Aggregations::
- Allow terms aggregations on pure boolean scripts.
- IP range masks exclude the maximum address of the range.
- Fix `other_bucket` on the `filters` agg to be enabled if a key is set.
Allocation::
- Promote shadow replica to primary when initializing primary fails
- Trim in-sync allocations set only when it grows
Analysis::
- Pre-built analysis factories do not implement MultiTermAware
correctly.
CRUD::
- Reject external versioning and explicit version numbers on create
Core::
- Support negative numbers in readVLong
Exceptions::
- Fixing shard recovery error message to report the number of docs
correctly for each node
Index APIs::
- Ensure shrunk indices carry over version information from its source
Internal::
- Don't output null source node in RecoveryFailedException
Java REST Client::
- Don't use null charset in RequestLogger
Logging::
- Restores the original default format of search slow log
Mapping::
- Only update DocumentMapper if field type changes
- The `_all` default mapper is not completely configured.
- Fix MapperService.allEnabled().
Network::
- Disable the Netty recycler
- Tell Netty not to be unsafe in transport client
Nested Docs::
- Fix bug in query builder rewrite that ignores the ignore_unmapped
option
Packaging::
- Another fix for handling of paths on Windows
Percolator::
- Fix NPE in percolator's 'now' range check for percolator queries with
range queries
Plugins::
- Provide helpful error message if a plugin exists
- Add shutdown hook for closing CLI commands
Plugin Lang Painless::
- Update Painless Loop Counter to be Higher
Plugin Repository Azure::
- readonly on azure repository must be taken into account
Recovery::
- Fix primary relocation for shadow replicas
- Don't close store under CancellableThreads
- Use a fresh recovery id when retrying recoveries
- Allow flush/force_merge/upgrade on shard marked as relocated
Reindex API::
- Don't close rest client from its callback
Search::
- Improve concurrency of ShardCoreKeyMap.
- Make `-0` compare less than `+0` consistently.
- FiltersAggregationBuilder: rewriting filter queries, the same way as
in FilterAggregationBuilder
Stats::
- Implement stats for geo_point and geo_shape field
- Use reader for doc stats
- Avoid NPE in NodeService#stats if HTTP is disabled
- Add support for "include_segment_file_sizes" in indices stats REST
handler
Term Vectors::
- Fix _termvectors with preference to not hit NPE
- Return correct term statistics when a field is not found in a shard
sqlrelay 1.0.0
- updated md5 pwdenc to use rudiments md5 class
- solved a long-standing hang that could occur if the database password
expired while sqlrelay was running, and the sqlr-scaler attempted to
start new connections
- added convertcase, convertcasedoublequotes, and removedoublequotes
parameters to normalize translation module
- added ColumnCaseName parameter to ODBC driver
- fixed a subtle bug in the ODBC driver that caused the first row to be
skipped if columns were not bound
- added -backtrace option for debugging
- fixed mysql connection module so it can use the statement API on
Windows
- fixed semaphore reference-before-init bug in sqlrservercontroller
- added support for db, debug, columnnamecase, dontgetcolumninfo,
nullsasnulls, and lazyconnect connect-string options to all drivers
(though in some they are camel-case and others lower-case)
- in DB-abstraction-layer drivers, the debug, dontgetcolumninfo,
nullsasnulls, lazyconnect, krb, and tls connect-string options now
support any yes/no equivalent such as yes, Yes, y, Y, true, True, 1,
(and similar for no)
- fixed a regression that could cause crashes in the C#/ADO driver when
binding null or date values
- added all parameters to ODBC config screen
- removed calls to mysql_stmt_store_result/mysql_stmt_num_rows from
mysql connection to improve performance. mysql connection doesn't know
the total number of rows prior to full fetch now (which is consistent
with most other databases).
- updated postgresql connection to use an unnamed cursor instead of
naming it and having to deallocate it over an over
- tweaked skipWhitespaceAndComments and related methods to use
charstring::isNullOrEmpty and character::isWhitespace
- replaced regular expression comparisons in sap/freetds connections
with simpler charstring comparisons
- updated db2 connection to reuse the statement instead of recreating it
with each prepare
- made column-lookup by name case sensitive to improve performance
- fixed nagle-disable and sizing of socket read/write buffers
- changed default maxcursors to 5
- fixed an uninitialized variable in the firebird connection
- python api's are installed in dist-packages subdirectory on systems
that have that instead of site-packages
- added route-entire-session option to router modules
sqlrelay 0.67.0
- general documenation clean-up
- added missing all: target to man/Makefile
- mysql_info in drop-in library for mysql returns NULL instead of "" now
- mysql_errno in the drop-in library now returns the error number from
the DB, instead of CR_UNKNOWN_ERROR, if no error map is provided, and
the backend database is known to be mysql
- mysql_refresh in the drop-in library works with options other than
REFRESH_GRANT now
- mysql_shutdown/kill in the drop-in library return 1 on failure now
rather than 2000
- date/time and timestamp types are recognized as binary types now
- year/timestamp types are recognized as unsigned types now
- TINY/MEDIUM/LONG_TEXT types are recognized as blobs now
- mapping date/time to MYSQL_TYPE_DATE/TIME, rather than
MYSQL_TYPE_DATETIME is the default in the drop-in library now and
SQLR_MAP_DATETIME_TO_DATE has been removed
- mysql drop-in library sets column flags correctly now
- mysql drop-in library runs appropriate kill/shutdown queries now
- mysql connection uses non-stmt api for "show *" queries now
- mysql drop-in library runs "show processlist" for mysql_list_processes
- mysql drop-in library supports date/time/datetime input binds now
instead of just treating them all as datetimes
- connection modules differentiate between date/time/datetime binds now
instead of just treating them all as datetimes
- added result set row translation module framework
- updated firebird connection module to the fieldstruct.textbuffer
instead of fieldbuffer for non-text to text translations
- added support for negative times in mysql connection module and
drop-in library
- added translation-failure event
- general refactoring of server module API
- consolidated sqlr-scaler code
- fixed a 1-byte empty-blob memory leak in the client
- /etc/alternatives/java_sdk is preferred, if found, now
=== 4.42.1 (2017-01-12)
* Make dataset_module inherited to subclasses when using the single_table_inheritance plugin (jeremyevans) (#1284)
=== 4.42.0 (2017-01-01)
* Handle eager load callbacks correctly for one_to_one associations with orders or offsets when window functions are not supported (jeremyevans)
* Raise Sequel::Error if using an :eager_limit dataset option when eager loading a singular association (jeremyevans)
* Replace internal uses of Dataset#select_more with #select_append to save a method call (jeremyevans)
* Make Dataset#order_append the primary method, and #order_more the alias, for similarity to #select_append and #select_more (jeremyevans)
* Replace internal uses of Dataset#filter with #where to save a method call (jeremyevans)
* Do not set :auto_increment in the schema information for integer columns that are part of a composite primary key on SQLite (jeremyevans)
* Use autoincrement setting on integer primary key columns when emulating table modification methods on SQLite (thenrio, jeremyevans) (#1277, #1278)
* Make the pagination extension work on frozen datasets (jeremyevans)
* Make Dataset#server work for frozen model datasets using the sharding plugin (jeremyevans)
* Make Dataset#nullify in the null_dataset extension work on frozen datasets (jeremyevans)
* Make Model#set_server work when using a frozen model dataset (jeremyevans)
* Make Dataset#ungraphed work on a frozen model dataset (jeremyevans)
* Add Dataset#with_{autoid,fetch,numrows} to the mock adapter, returning cloned datasets with the setting changed (jeremyevans)
* Make looser_typecasting extension handle the strict BigDecimal parsing introduced in ruby 2.4rc1 (jeremyevans)
* Make Database#{db,opts}= in the sequel_3_dataset_methods extension raise for frozen datasets (jeremyevans)
* Speed up repeated calls to Dataset#{interval,range} for frozen datasets using a cached placeholder literalizer (jeremyevans)
* Speed up repeated calls to Dataset#get with a single argument for frozen datasets using a cached placeholder literalizer (jeremyevans)
* Speed up repeated calls to Dataset#{first,last} with arguments/blocks for frozen datasets using a cached placeholder literalizer (jeremyevans)
* Speed up repeated calls to Dataset#{avg,min,max,sum} for frozen datasets using a cached placeholder literalizer (jeremyevans)
* Cache dataset returned by Dataset#skip_locked for frozen datasets (jeremyevans)
* Cache dataset returned by Dataset#for_update for frozen datasets (jeremyevans)
* Cache dataset returned by Dataset#un{filtered,grouped,limited,ordered} for frozen datasets (jeremyevans)
* Cache dataset returned by Dataset#reverse (no args) for frozen datasets (jeremyevans)
* Cache dataset returned by Dataset#invert for frozen datasets (jeremyevans)
* Speed up repeated calls to Dataset#count with an argument or block for frozen datasets using a cached placeholder literalizer (jeremyevans)
* Using :on_duplicate_columns=>:warn Database option with duplicate_columns_handler now prepends file/line to the warning message (jeremyevans)
* Move identifier mangling code to identifier_mangling extension, load by default unless using :identifier_mangling=>false Database option (jeremyevans)
* Allow Dataset#with_extend to accept a block and create a module with that block that the object is extended with (jeremyevans)
* Speed up repeated calls to with_pk on the same frozen model dataset using a cached placeholder literalizer (jeremyevans)
* Add dataset_module methods such as select and order that define dataset methods which support caching for frozen datasets (jeremyevans)
* Cache subset datasets if they don't use blocks or procs for frozen model datasets (jeremyevans)
* Cache intermediate dataset used in Dataset#{last,paged_each} for frozen model datasets without an order (jeremyevans)
* Cache dataset returned by Dataset#naked for frozen datasets (jeremyevans)
* Cache intermediate dataset used in Dataset#last (no args) for frozen datasets (jeremyevans)
* Cache intermediate dataset used in Dataset#first (no args) and #single_record for frozen datasets (jeremyevans)
* Cache intermediate dataset used in Dataset#empty? for frozen datasets (jeremyevans)
* Cache intermediate dataset used in Dataset#count (no args) for frozen datasets (jeremyevans)
* Warn if :conditions option may be unexpectedly ignored during eager_graph/association_join (jeremyevans) (#1272)
* Cache SELECT and DELETE SQL for most frozen datasets (jeremyevans)
* Freeze most SQL::Expression objects and internal state by default (jeremyevans)
* Freeze Dataset::PlaceholderLiteralizer and Dataset::PlaceholderLiteralizer::Argument instances (jeremyevans)
* Freeze most dataset opts values to avoid unintentional modification (jeremyevans)
* Add Dataset#with_convert_smallint_to_bool on DB2, returning a clone with convert_smallint_to_bool set (jeremyevans)
* Make Dataset#freeze actually freeze the dataset on ruby 2.4+ (jeremyevans)
* Avoid using instance variables other than @opts for dataset data storage (jeremyevans)
* Add freeze_datasets extension, making all datasets for a given Database frozen (jeremyevans)
* Refactor prepared statement internals, using opts instead of instance variables (jeremyevans)
* Model.set_dataset now operates on a clone of the dataset given instead of modifying it, so it works with frozen datasets (jeremyevans)
=== 4.41.0 (2016-12-01)
* Add Dataset#with_mssql_unicode_strings on Microsoft SQL Server, returning a clone with mssql_unicode_strings set (jeremyevans)
* Add Dataset#with_identifier_output_method, returning a clone with identifier_output_method set (jeremyevans)
* Add Dataset#with_identifier_input_method, returning a clone with identifier_input_method set (jeremyevans)
* Add Dataset#with_quote_identifiers, returning a clone with quote_identifiers set (jeremyevans)
* Add Dataset#with_extend, returning a clone extended with given modules (jeremyevans)
* Add Dataset#with_row_proc, returning a clone with row_proc set (jeremyevans)
* Support use of SQL::AliasedExpressions as Model#to_json :include option keys in the json_serializer plugin (sensadrome) (#1269)
* Major improvements to type conversion in the ado adapter (vais, jeremyevans) (#1265)
* Avoid memory leak in ado adapter by closing result sets after yielding them (vais, jeremyevans) (#1259)
* Fix hook_class_methods plugin handling of commit hooks (jeremyevans)
* Make association dataset method correctly handle cases where key fields are nil (jeremyevans)
* Handle pure java exceptions that don't support message= when reraising the exception in the jdbc adapter (jeremyevans)
* Add support for :offset_strategy Database option on DB2, with :limit_offset and :offset_fetch values, to disable OFFSET emulation (#1254) (jeremyevans)
* Remove deprecated support for using Bignum class as a generic type (jeremyevans)
Changes since 1.5.0:
- allow mixed $ and non-$ query ops.
- Missing exports of mongoc_gridfs_file_set_*() functions.
- update define constants for "find" opts to be unique.
- Windows CA stores should be opened with read-only flag.
- Fix SEGFAULT with performance counters on NUMA (thanks to Jonathan Wang).
- Prevent rare assertion error in mongoc_cluster_stream_for_server.
- Improve error messages from auth failure.
- Escape quotes when appending CFLAGS to handshake metadata.
- Fix OpenSSL header lookups in non-default paths.
- Fix build failure with LibreSSL.
- Add connect options MYSQL_OPT_MAX_ALLOWED_PACKET, MYSQL_OPT_NET_BUFFER_LENGTH,
MYSQL_OPT_TLS_VERSION and MYSQL_OPT_SSL_MODE
- Fix Visual Studio 2015 build.
- Fix segmentation fault when inserting a large string
- Fix Compilation Failure with PRE-5.7 C API
2.1.16
- Avoid infinitely looping result set when paging SELECT queries with
an IN clause with duplicate keys by treating the IN values as a set
instead of a list
- Add system property to set the max number of native transport requests
in queue
- Include column family parameter when -st and -et are provided
- Fix queries with empty ByteBuffer values in clustering column
restrictions
- Disable passing control to post-flush after flush failure to prevent
data loss
- Allow STCS-in-L0 compactions to reduce scope with LCS
- cannot use cql since upgrading python to 2.7.11+
- Fix filtering on clustering columns when 2i is used
- Improve digest calculation in the presence of overlapping tombstones
2.1.15
- Account for partition deletions in tombstone histogram
- Avoid stalling paxos when the paxos state expires
- Remove finished incoming streaming connections from MessagingService
- Don't try to get sstables for non-repairing column families
- Prevent select statements with clustering key > 64k
- Avoid marking too many sstables as repaired
- Fix clock skew corrupting other nodes with paxos
- Remove distinction between non-existing static columns and existing
but null in LWTs
- Support mlockall on IBM POWER arch
- Cache local ranges when calculating repair neighbors
- Allow LWT operation on static column with only partition keys
- Create interval tree over canonical sstables to avoid missing sstables
during streaming
- cqlsh COPY FROM: shutdown parent cluster after forking, to avoid
corrupting SSL connections
- Updated cqlsh Python driver to fix DESCRIBE problem for legacy tables
- cqlsh: apply current keyspace to source command
- Backport CASSANDRA-11578
- Clear out parent repair session if repair coordinator dies
- Set default streaming_socket_timeout_in_ms to 24 hours
- Do not consider local node a valid source during replace
- Avoid holding SSTableReaders for duration of incremental repair
- Add message dropped tasks to nodetool netstats
- Don't compute expensive MaxPurgeableTimestamp until we've verified
there's an expired tombstone
- Fix paging on DISTINCT queries repeats result when first row in
partition changes
- Add option to disable use of severity in DynamicEndpointSnitch
- cqlsh COPY FROM fails for null values with non-prepared statements
- Make cython optional in pylib/setup.py
- Change order of directory searching for cassandra.in.sh to favor local
one
- cqlsh COPY FROM fails with []{} chars in UDT/tuple fields/values
- clqsh: COPY FROM throws TypeError with Cython extensions enabled
- cqlsh: COPY FROM ignores NULL values in conversion
- Validate levels when building LeveledScanner to avoid overlaps with
orphaned sstables
2.1.14
- Start L0 STCS-compactions even if there is a L0 -> L1 compaction going
- (cqlsh) Fix potential COPY deadlock when parent process is terminating
child processes
- Replace sstables on DataTracker before marking them as non-compacting
during anti-compaction
- Checking if an unlogged batch is local is inefficient
- Fix paging for COMPACT tables without clustering columns
- Fix out-of-space error treatment in memtable flushing
- Backport CASSANDRA-10859
- COPY FROM fails when importing blob
- Backport CASSANDRA-10679
- Don't do defragmentation if reading from repaired sstables
- Fix streaming_socket_timeout_in_ms not enforced
- Avoid dropping message too quickly due to missing unit conversion
- COPY FROM on large datasets: fix progress report and debug performance
- InvalidateKeys should have a weak ref to key cache
- Don't remove FailureDetector history on removeEndpoint
- Only notify if repair status changed
- Add partition key to TombstoneOverwhelmingException error message
- Use logback setting for 'cassandra -v' command
- Fix sstableloader to unthrottle streaming by default
- Fix incorrect warning in 'nodetool status'
- Properly release sstable ref when doing offline scrub
- Improve nodetool status performance for large cluster
- Make it clear what DTCS timestamp_resolution is used for
- Gossiper#isEnabled is not thread safe
- Avoid major compaction mixing repaired and unrepaired sstables in DTCS
- test_bulk_round_trip_blogposts is failing occasionally
- Add a -j parameter to scrub/cleanup/upgradesstables to state how
many threads to use
2.1.13
- Fix isJoined return true only after becoming cluster member
(CASANDRA-11007)
- Fix bad gossip generation seen in long-running clusters
- Avoid NPE when incremental repair fails
- Unmark sstables compacting once they are done in
cleanup/scrub/upgradesstables
- Revert CASSANDRA-10012 and add more logging
- Allow simultaneous bootstrapping with strict consistency when no
vnodes are used
- Log a message when major compaction does not result in a single file
- (cqlsh) fix cqlsh_copy_tests when vnodes are disabled
- (cqlsh) fix formatting bytearray values
- (cqlsh) Add request timeout option to cqlsh
- Avoid AssertionError while submitting hint with LWT
- If CompactionMetadata is not in stats file, use index summary instead
- Retry sending gossip syn multiple times during shadow round
- Fix pending range calculation during moves
- Sane default (200Mbps) for inter-DC streaming througput
- Match cassandra-loader options in COPY FROM
- Fix binding to any address in CqlBulkRecordWriter
- Fix the way we replace sstables after anticompaction
- cqlsh fails to decode utf-8 characters for text typed columns
- Log error when stream session fails
- Fix bugs in commit log archiving startup behavior
- (cqlsh) further optimise COPY FROM
- Allow CREATE TABLE WITH ID
- Make Stress compiles within eclipse
- Cassandra Daemon should print JVM arguments
- Allow cancellation of index summary redistribution
- sstableloader will fail if there are collections in the schema tables
- Disable reloading of GossipingPropertyFileSnitch
- Fix Stress profile parsing on Windows
2.1.12
- Fix incremental repair hang when replica is down
- Avoid writing range tombstones after END_OF_ROW marker
- Optimize the way we check if a token is repaired in anticompaction
- Add proper error handling to stream receiver
- Warn or fail when changing cluster topology live
- Status command in debian/ubuntu init script doesn't work
- Some DROP ... IF EXISTS incorrectly result in exceptions on
non-existing KS
- DeletionTime.compareTo wrong in rare cases
- Force encoding when computing statement ids
- Properly reject counters as map keys
- Fix the sstable-needs-cleanup check
- (cqlsh) Print column names before COPY operation
- Add Native-Transport-Requests back to tpstats
- Make paging logic consistent between searcher impls
- Fix CompressedInputStream for proper cleanup
- (cqlsh) Support counters in COPY commands
- Try next replica if not possible to connect to primary replica on
ColumnFamilyRecordReader
- Limit window size in DTCS
- sstableloader does not use MAX_HEAP_SIZE env parameter
- (cqlsh) Improve COPY TO performance and error handling
- Don't remove level info when running upgradesstables
- Create compression chunk for sending file only
- Make buffered read size configurable
- Forbid compact clustering column type changes in ALTER TABLE
- Reject incremental repair with subrange repair
- Add a nodetool command to refresh size_estimates
- Shutdown compaction in drain to prevent leak
- Invalidate cache after stream receive task is completed
- Reject counter writes in CQLSSTableWriter
- Remove superfluous COUNTER_MUTATION stage mapping
- Improve json2sstable error reporting on nonexistent columns
- (cqlsh) fix COPY using wrong variable name for time_format
- Do not run SizeEstimatesRecorder if a node is not a member of the ring
- Improve handling of dead nodes in gossip
- Fix logback-tools.xml incorrectly configured for outputing to
System.err
- Fix streaming to catch exception so retry not fail
- Add validation method to PerRowSecondaryIndex
- Support encrypted and plain traffic on the same port
- Do STCS in DTCS windows
- Don't try to get ancestors from half-renamed sstables
- Avoid repetition of JVM_OPTS in debian package
- Fix potential NPE from handling result of SIM.highestSelectivityIndex
- Fix paging issues with partitions containing only static columns data
- Fix conditions on static columns
- AssertionError: attempted to delete non-existing file CommitLog
- Merge range tombstones during compaction
- (cqlsh) Distinguish negative and positive infinity in output
- (cqlsh) allow custom time_format for COPY TO
- Don't allow startup if the node's rack has changed
- Fix sorting for queries with an IN condition on partition key columns
Fix the REPLACE statement for WITHOUT ROWID tables that lack secondary indexes so that it works correctly with triggers and foreign keys. This was a new bug caused by performance optimizations added in version 3.16.0. Ticket 30027b613b4
Fix the sqlite3_value_text() interface so that it correctly translates content generated by zeroblob() into a string of all 0x00 characters. This is a long-standing issue discovered after the 3.16.1 release by OSS-Fuzz
Fix the bytecode generator to deal with a subquery in the FROM clause that is itself a UNION ALL where one side of the UNION ALL is a view that contains an ORDER BY. This is a long-standing issue that was discovered after the release of 3.16.1. See ticket 190c2507.
Adjust the sqlite3_column_count() API so it more often returns the same values for PRAGMA statements as it did in prior releases, to minimize disruption to applications that might be using that interface in unexpected ways.
Fix a bug concerning the use of row values within triggers (see ticket 8c9458e7) that was in version 3.15.0 but was not reported until moments after the 3.16.0 release was published.
== 5.1.1 Release Notes
=== Breaking changes
Aliases::
- Validate alias names the same as index names
REST::
- Remove lenient stats parsing 5.x
=== Breaking Java changes
Discovery::
- Remove pluggability of ElectMasterService
Exceptions::
- Remove `IndexTemplateAlreadyExistsException` and
`IndexShardAlreadyExistsException`
- Replace IndexAlreadyExistsException with
ResourceAlreadyExistsException
- Backport: Replace IndexAlreadyExistsException with
ResourceAlreadyExistsException
Internal::
- Pass executor name to request interceptor to support async intercept
calls
Network::
- Unguice Transport and friends
Plugins::
- Plugins: Remove support for onModule
=== Deprecations
Analysis::
- Deprecating request parameters of _analyze API in 5.x
CRUD::
- Deprecate VersionType.FORCE
Core::
- Add deprecation logging for users that explicitly opt in for the
`default` fs type.
Mapping::
- Deprecate `timestamp` and `ttl` on index requests.
Query DSL::
- Add deprecation logging for lenient boolean queries
- Add deprecation logging message for 'fuzzy' query
Search::
- Deprecate ignored type parameter in search_shards api
Settings::
- Add deprecation logging for the case that store throttling is used.
=== New features
Analysis::
- Expose Lucenes Ukrainian analyzer
CAT API::
- Provides a cat api endpoint for templates.
Query DSL::
- Add "all fields" execution mode to simple_query_string query
- Add support for `quote_field_suffix` to `simple_query_string`.
- Add "all field" execution mode to query_string query
Reindex API::
- Add automatic parallelization support to reindex and friends
=== Enhancements
Aggregations::
- Rescorer should be applied in the TopHits aggregation
Allocation::
- Balance step in BalancedShardsAllocator for a single shard
- Process more expensive allocation deciders last
- Separates decision making from decision application in
BalancedShardsAllocator
- Split allocator decision making from decision application
Analysis::
- Remove AnalysisService and reduce it to a simple name to analyzer
mapping
CAT API::
- Adding built-in sorting capability to _cat apis.
- Add health status parameter to cat indices API
Cache::
- Do not cache term queries.
- Parse alias filters on the coordinating node
Cluster::
- Cache successful shard deletion checks
Core::
- Reduce memory pressure when sending large terms queries.
- Install a security manager on startup
- Log node ID on startup
- Ensure source filtering automatons are only compiled once
- Improve scheduling fairness when batching cluster state changes with
equal priority
- Do not log full bootstrap checks exception
Exceptions::
- Add BWC layer for Exceptions
Geo::
- Optimize geo-distance sorting.
Ingest::
- add `ignore_missing` option to SplitProcessor
Internal::
- Rename ClusterState#lookupPrototypeSafe to `lookupPrototype` and
remove "unsafe" unused variant
- ShardActiveResponseHandler shouldn't hold to an entire cluster state
- Remove unused ClusterService dependency from SearchPhaseController
- Remove special case in case no action filters are registered
- Use TimveValue instead of long for CacheBuilder methods
- Remove SearchContext#current and all it's threadlocals
Java REST Client::
- Provide error message when rest request path is null
Logging::
- Log failure to connect to node at info instead of debug
- Truncate log messages from the end
- Logging shutdown hack
- Disable console logging
Mapping::
- Create the QueryShardContext lazily in DocumentMapperParser.
Network::
- Grant Netty permission to read system somaxconn
- Lazy resolve unicast hosts
- Fix handler name on message not fully read
- Handle rejected pings on shutdown gracefully
Packaging::
- Add empty plugins dir for archive distributions
- Make explicit missing settings for Windows service
- Change permissions on config files
Plugin Lang Painless::
- Add Debug.explain to painless
- Implement the ?: operator in painless
- In painless suggest a long constant if int won't do
- Support decimal constants with trailing [dD] in painless
- Implement reading from null safe dereferences
- Painless negative offsets
Plugin Repository S3::
- Make the default S3 buffer size depend on the available memory.
Plugins::
- Clarify that plugins can be closed
- Plugins: Convert custom discovery to pull based plugin
- Removing plugin that isn't installed shouldn't trigger usage
information
- Remove pluggability of ZenPing
- Make UnicastHostsProvider extension pull based
Query DSL::
- Using ObjectParser in MatchAllQueryBuilder and IdsQueryBuilder
- Expose splitOnWhitespace in `Query String Query`
- Throw error if query element doesn't end with END_OBJECT
- Remove `lowercase_expanded_terms` and `locale` from query-parser
options.
Reindex API::
- Make reindex-from-remote ignore unknown fields
Scripting::
- Wrap VerifyError in ScriptException
- Support binary field type in script values
- Mustache: Add
- Expose `ctx._now` in update scripts
Search::
- Add indices and filter information to search shards api output
- remove pointless catch exception in TransportSearchAction
- Optimize query with types filter in the URL (t/t/_search)
- Makes search action cancelable by task management API
Search Templates::
- Add profile and explain parameters to template API
Snapshot/Restore::
- Abort snapshots on a node that leaves the cluster
Stats::
- Remove load average leniency
- Strengthen handling of unavailable cgroup stats
- Add basic cgroup CPU metrics
Task Manager::
- Add search task descriptions
Tribe Node::
- Add support for merging custom meta data in tribe node
=== Bug fixes
Aggregations::
- Rewrite Queries/Filter in FilterAggregationBuilder and ensure client
usage marks query as non-cachable
- Percentiles bucket fails for 100th percentile
- Thread safety for scripted significance heuristics
Allocation::
- Allow master to assign primary shard to node that has shard store
locked during shard state fetching
Analysis::
- Can load non-PreBuiltTokenFilter in Analyze API
CAT API::
- Consume `full_id` request parameter early
Cache::
- Fix the request cache keys to not hold references to the
SearchContext.
Circuit Breakers::
- ClusterState publishing shouldn't trigger circuit breakers
Cluster::
- Remove cluster update task when task times out
Core::
- Add a StreamInput#readArraySize method that ensures sane array sizes
- Use a buffer to do character to byte conversion in
StreamOutput#writeString
- Fix ShardInfo#toString
- Protect BytesStreamOutput against overflows of the current number of
written bytes.
- Return target index name even if _rollover conditions are not met
- .es_temp_file remains after system crash, causing it not to start
again
Dates::
- Fix time zone rounding edge case for DST overlaps
Engine::
- Die with dignity on the Lucene layer
- Fix `InternalEngine#isThrottled` to not always return `false`.
- Retrying replication requests on replica doesn't call `onRetry`
Highlighting::
- Fix FiltersFunctionScoreQuery highlighting
- Fix highlighting on a stored keyword field
Index APIs::
- Validate the `_rollover` target index name early to also fail if
dry_run=true
Index Templates::
- Fix integer overflows when dealing with templates.
Ingest::
- fix trace_match behavior for when there is only one grok pattern
- Stored scripts and ingest node configurations should be included into
a snapshot
Inner Hits::
- Skip adding a parent field to nested documents.
Internal::
- Rethrow ExecutionException from the loader to concurrent callers of
Cache#computeIfAbsent
- Restore thread's original context before returning to the ThreadPool
- Fix NPE in SearchContext.toString()
- Source filtering should treat dots in field names as sub objects.
Java API::
- Transport client: Fix remove address to actually work
- Add a HostFailureListener to notify client code if a node got
disconnected
- Fix InternalSearchHit#hasSource to return the proper boolean value
- Null checked for source when calling sourceRef
- ClusterAdminClient.prepareDeletePipeline method should accept pipeline
id to delete
Java REST Client::
- Rest client: don't reuse the same HttpAsyncResponseConsumer across
multiple retries
Logging::
- Do not prematurely shutdown Log4j
- Assert status logger does not warn on Log4j usage
- Fix logger names for Netty
Mapping::
- Fail to index fields with dots in field names when one of the
intermediate objects is nested.
- Uncommitted mapping updates should not efect existing indices
Network::
- DiscoveryNode and TransportAddress should preserve host information
- Die with dignity on the network layer
- Prevent double release in TcpTransport if send listener throws an
exception
Packaging::
- Set vm.max_map_count on systemd package install
- Export ES_JVM_OPTIONS for SysV init
- Debian: configure start-stop-daemon to not go into background
- Generate POM files with non-wildcard excludes
Plugin Lang Painless::
- Test fix for def equals in Painless
- Fix a VerifyError bug in Painless
- Fix Lambdas in Painless to be Able to Use Top-Level Variables Such as
params and doc
Plugin Mapper Attachment::
- NPE is raised when defining a non existing type within attachments
type
Query DSL::
- Fixes date range query using epoch with timezone
- Allow overriding all-field leniency when `lenient` option is specified
- Max score should be updated when a rescorer is used
REST::
- Strict level parsing for indices stats
- The routing query string param is supported by mget but was missing
from the rest spec
- fix thread_pool_patterns path variable definition
- ensure the XContentBuilder is always closed in RestBuilderListener
- XContentBuilder: Avoid building self-referencing objects
Reindex API::
- Ignore IllegalArgumentException with assertVersionSerializable
- Bump reindex-from-remote's buffer to 200mb
- Fix reindex-from-remote for parent/child from <2.0
Search::
- Fix match_phrase_prefix on boosted fields
- Respect default search timeout
- Remove LateParsingQuery to prevent timestamp access after context is
frozen
Search Templates::
- SearchTemplateRequest to implement CompositeIndicesRequest
Settings::
- Handle spaces in `action.auto_create_index` gracefully
- Fix settings diff generation for affix and group settings
- Don't reset non-dynamic settings unless explicitly requested
Snapshot/Restore::
- Fixes shard level snapshot metadata loading when index-N file is
missing
- Ensures cleanup of temporary index-* generational blobs during
snapshotting
- Fixes get snapshot duplicates when asking for _all
- Keep snapshot restore state and routing table in sync (5.x backport)
Task Manager::
- Task cancellation command should wait for all child nodes to receive
cancellation request before returning
Tribe Node::
- Add socket permissions for tribe nodes
=== Upgrades
Core::
- Upgrade to lucene-6.3.0.
Dates::
- Update Joda Time to version 2.9.5
Logging::
- Upgrade Log4j 2 to version 2.7
Network::
- Upgrade to Netty 4.1.6
Plugin Ingest Attachment::
- Update to Tika 1.14
== 5.0.2 Release Notes
=== Enhancements
Core::
- Install a security manager on startup
Exceptions::
- Add BWC layer for Exceptions
Logging::
- Truncate log messages from the end
Scripting::
- Wrap VerifyError in ScriptException
Snapshot/Restore::
- Abort snapshots on a node that leaves the cluster
=== Bug fixes
Allocation::
- Allow master to assign primary shard to node that has shard store
locked during shard state fetching
Cluster::
- Remove cluster update task when task times out
Core::
- Add a StreamInput#readArraySize method that ensures sane array sizes
- Use a buffer to do character to byte conversion in
StreamOutput#writeString
Engine::
- Die with dignity on the Lucene layer
- Fix `InternalEngine#isThrottled` to not always return `false`.
Index Templates::
- Fix integer overflows when dealing with templates.
Ingest::
- fix trace_match behavior for when there is only one grok pattern
Internal::
- Rethrow ExecutionException from the loader to concurrent callers of
Cache#computeIfAbsent
- Fixes potential NullPointerException on shard closing
Java API::
- Transport client: Fix remove address to actually work
- Add a HostFailureListener to notify client code if a node got
disconnected
Logging::
- Do not prematurely shutdown Log4j
Network::
- Die with dignity on the network layer
Plugin Lang Painless::
- Fix a VerifyError bug in Painless
- Fix Lambdas in Painless to be Able to Use Top-Level Variables Such as
params and doc
Search::
- Respect default search timeout
Settings::
- Don't reset non-dynamic settings unless explicitly requested
Tribe Node::
- Add socket permissions for tribe nodes
