- Continuous aggregates with compression
- time_bucket_ng support for N months and timezones on cont. aggregates
- Query planning performance is improved for hypertables with a large
number of chunks.
3.14.10
* Add shortcut for conditional insert using sub-select
* Add convenience `left_outer_join()` method to query.
* Add `selected_columns` property to Select queries.
* Add `name` property to Alias instances.
* Fix regression in tests introduced by change to DataSet in 3.14.9.
- 0.50 | 2021-12-04
- lob details bifurcated for "modern" Guile
Guile 2.2 (and later, presumably) have ‘scm_c_make_port’ and in
fact a whole rewrite of the ports extension mechanism. Thus,
all previous Guile-PG releases failed to build w/ Guile 2.2.
Specifically, all the large object (lob) low-level details
clashed irreconcilably.
This release of Guile-PG now handles things by configure-time
detection and selection of a suitably matching implementation.
The file test/OK now includes an entry w/ a subheading of "other
Guile" to show the range of supported Guile versions. More to
the point, if you build w/ Guile 2.2 and encounter problems,
then please report it.
NB: We haven't dipped into the Guile 3.0 waters yet. Experience
reports from those more adventurous are welcome, in any case.
SQLite Release 3.38.0 On 2022-02-22
Added the -> and ->> operators for easier processing of JSON. The new operators are compatible with MySQL and PostgreSQL.
The JSON functions are now built-ins. It is no longer necessary to use the -DSQLITE_ENABLE_JSON1 compile-time option to enable JSON support. JSON is on by default. Disable the JSON interface using the new -DSQLITE_OMIT_JSON compile-time option.
Enhancements to date and time functions:
Added the unixepoch() function.
Added the auto modifier and the julianday modifier.
Rename the printf() SQL function to format() for better compatibility. The original printf() name is retained as an alias for backwards compatibility.
Added the sqlite3_error_offset() interface, which can sometimes help to localize an SQL error to a specific character in the input SQL text, so that applications can provide better error messages.
Enhanced the interface to virtual tables as follows:
Added the sqlite3_vtab_distinct() interface.
Added the sqlite3_vtab_rhs_value() interface.
Added new operator types SQLITE_INDEX_CONSTRAINT_LIMIT and SQLITE_INDEX_CONSTRAINT_OFFSET.
Added the sqlite3_vtab_in() interface (and related) to enable a virtual table to process IN operator constraints all at once, rather than processing each value of the right-hand side of the IN operator separately.
CLI enhancements:
Columnar output modes are enhanced to correctly handle tabs and newlines embedded in text.
Added options like "--wrap N", "--wordwrap on", and "--quote" to the columnar output modes.
Added the .mode qbox alias.
The .import command automatically disambiguates column names.
Use the new sqlite3_error_offset() interface to provide better error messages.
Query planner enhancements:
Use a Bloom filter to speed up large analytic queries.
Use a balanced merge tree to evaluate UNION or UNION ALL compound SELECT statements that have an ORDER BY clause.
The ALTER TABLE statement is changed to silently ignores entries in the sqlite_schema table that do not parse when PRAGMA writable_schema=ON.
MariaDB 10.5.15 Release Notes
Notable Items
InnoDB
* Set innodb_change_buffering=none by default (MDEV-27734)
Security
* Fixes for the following security vulnerabilities:
* CVE-2021-46665
* CVE-2021-46664
* CVE-2021-46661
* CVE-2021-46668
* CVE-2021-46663
MariaDB 10.5.14 Release Notes
Notable Items
InnoDB
* --skip-symbolic-links does not disallow .isl file creation
(MDEV-26870)
* Indexed CHAR columns are broken with NO_PAD collations (MDEV-25440)
* insert-intention lock conflicts with waiting ORDINARY lock
(MDEV-27025)
* Crash recovery improvements (MDEV-26784, MDEV-27022, MDEV-27183,
MDEV-27610)
Galera
* Galera updated to 26.4.11
* Galera SST scripts should use ssl_capath (not ssl_ca) for CA directory
(MDEV-27181)
* Alter Sequence do not replicate to another nodes with in Galera
Cluster (MDEV-19353)
* Galera crash - Assertion. Possible parallel writeset problem
(MDEV-26803)
* CREATE TABLE with FOREIGN KEY constraint fails to apply in parallel
(MDEV-27276)
* Galera cluster node consider old server_id value even after
modification of server_id [wsrep_gtid_mode=ON] (MDEV-26223)
Replication
* Seconds behind master corrected from artificial spikes at relay-log
rotation (MDEV-16091)
* Statement rollback in binlog when transaction creates or drop
temporary table is set right (MDEV-26833)
* CREATE-or-REPLACE SEQUENCE is made to binlog with the DDL flag to
stabilize its parallel execution on slave (MDEV-27365)
Security
* Fixes for the following security vulnerabilities:
* CVE-2022-24052
* CVE-2022-24051
* CVE-2022-24050
* CVE-2022-24048
* CVE-2021-46659
MariaDB 10.6.7 Release Notes
InnoDB
* Set innodb_change_buffering=none by default (MDEV-27734)
Security
* Fixes for the following security vulnerabilities:
* CVE-2021-46665
* CVE-2021-46664
* CVE-2021-46661
* CVE-2021-46668
* CVE-2021-46663
MariaDB 10.6.6 Release Notes
Notable Items
InnoDB
* --skip-symbolic-links does not disallow .isl file creation
(MDEV-26870)
* Indexed CHAR columns are broken with NO_PAD collations (MDEV-25440)
* insert-intention lock conflicts with waiting ORDINARY lock
(MDEV-27025)
* Crash recovery improvements (MDEV-26784, MDEV-27022, MDEV-27183,
MDEV-27610)
* mariabackup skips valid .ibd file (MDEV-26326)
* Allow seamless upgrade despite ROW_FORMAT=COMPRESSED (MDEV-27736)
Galera
* Galera updated to 26.4.11
* Galera SST scripts should use ssl_capath (not ssl_ca) for CA directory
(MDEV-27181)
* Alter Sequence do not replicate to another nodes with in Galera
Cluster (MDEV-19353)
* Galera crash - Assertion. Possible parallel writeset problem
(MDEV-26803)
* CREATE TABLE with FOREIGN KEY constraint fails to apply in parallel
(MDEV-27276)
* Galera cluster node consider old server_id value even after
modification of server_id [wsrep_gtid_mode=ON] (MDEV-26223)
Replication
* Seconds behind master corrected from artificial spikes at relay-log
rotation (MDEV-16091)
* Statement rollback in binlog when transaction creates or drop
temporary table is set right (MDEV-26833)
* CREATE-or-REPLACE SEQUENCE is made to binlog with the DDL flag to
stabilize its parallel execution on slave (MDEV-27365)
Security
* Fixes for the following security vulnerabilities:
* CVE-2022-24052
* CVE-2022-24051
* CVE-2022-24050
* CVE-2022-24048
* CVE-2021-46659
This library is a mid-level Haskell binding to the SQLite database.
Sqlite-simple provides a convenient API to sqlite that does some level of
automatic data conversion between the database and Haskell types. The API
has been modeled directly after postgresql-simple which in turn borrows
from mysql-simple.
This package is not very different from the other SQLite3 bindings out
there, but it fixes a few deficiencies I was finding. As compared to
bindings-sqlite3, it is slightly higher-level, in that it supports
marshalling of data values to and from the database. In particular, it
supports strings encoded as UTF8, and BLOBs represented as ByteStrings.
3.14.9
Allow calling table_exists() with a model-class, refs
Improve is_connection_usable() method of MySQLDatabase class.
Better support for VIEWs with playhouse.dataset.DataSet and sqlite-web.
Support INSERT / ON CONFLICT in playhosue.kv for newer Sqlite.
Add ArrayField.contained_by() method, a corollary to contains() and the contains_any() methods.
Support cyclical foreign-key relationships in reflection/introspection, and also for sqlite-web.
Add magic methods for FTS5 field to optimize, rebuild and integrity check the full-text index.
Add fallbacks in setup.py in the event distutils is not available.
repmgr 5.3.1 is a minor release.
A.1.1. Bug fixes
Fix upgrade path from repmgr 4.2 and 4.3 to repmgr 5.3.
repmgrd: ensure potentially open connections are closed.
In some cases, when recovering from degraded state in local node monitoring, new connection was opened to the local node without closing the old one, which will result in memory leakage.
Version 1.23, 2022-02-04
* Bucket cache switched from balanced tree to hash table
Change suggested by Terence Kelly.
* Speed up flushing the changed buckets on disk
* New option codes for gdbm_setopt
** GDBM_GETDBFORMAT
Return the database format.
** GDBM_GETDIRDEPTH
Return the directory depth, i.e. the number of initial (most significant)
bits in hash value that are interpreted as index to the directory.
** GDBM_GETBUCKETSIZE
Return maximum number of keys per bucket.
** GDBM_GETCACHEAUTO
Return the status of the automatic cache adjustment.
** GDBM_SETCACHEAUTO
Enable or disable automatic cache adjustment.
7.0.2 (2022-02-08)
* Fix PG.connect keyword arguments deprecation warning on ruby 2.7.
Nikita Vasilevsky
* Fix the ability to exclude encryption params from being autofiltered.
Mark Gangl
* Dump the precision for datetime columns following the new defaults.
Rafael Mendonça França
* Make sure encrypted attributes are not being filtered twice.
Nikita Vasilevsky
* Dump the database schema containing the current Rails version.
Since #42297, Rails now generate datetime columns with a default precision
of 6. This means that users upgrading to Rails 7.0 from 6.1, when loading
the database schema, would get the new precision value, which would not
match the production schema.
To avoid this the schema dumper will generate the new format which will
include the Rails version and will look like this:
ActiveRecord::Schema[7.0].define
When upgrading from Rails 6.1 to Rails 7.0, you can run the rails
app:update task that will set the current schema version to 6.1.
Rafael Mendonça França
* Fix parsing expression for PostgreSQL generated column.
fatkodima
* Fix Mysql2::Error: Commands out of sync; you can't run this command now
when bulk-inserting fixtures that exceed max_allowed_packet configuration.
Nikita Vasilevsky
* Fix error when saving an association with a relation named record.
Dorian Marié
* Fix MySQL::SchemaDumper behavior about datetime precision value.
y0t4
* Improve associated with no reflection error.
Nikolai
* Fix PG.connect keyword arguments deprecation warning on ruby 2.7.
Fixes#44307.
Nikita Vasilevsky
* Fix passing options to check_constraint from change_table.
Frederick Cheung
7.0.2.1 (2022-02-11)
* No changes.
7.0.2.2 (2022-02-11)
* No changes.
This update contains security fix for CVE-2022-23633 in ruby-actionpack61.
Active Support 6.1.4.6 (2022-02-11)
* Fix Reloader method signature to work with the new Executor signature.
Action Pack 6.1.4.5 (2022-02-11)
* Under certain circumstances, the middleware isn't informed that the
response body has been fully closed which result in request state
not being fully reset before the next request.
[CVE-2022-23633]
Other packages have no change.
This update contains security fix for CVE-2022-23633 in ruby-actionpack60.
Active Support 6.0.4.6 (2022-02-11)
* Fix Reloader method signature to work with the new Executor signature.
Action Pack 6.0.4.6
6.0.4.5 (2022-02-11)
* Under certain circumstances, the middleware isn't informed that the
response body has been fully closed which result in request state
not being fully reset before the next request.
[CVE-2022-23633]
Other packages have no change.
This update contains security fix for CVE-2022-23633 in
Active Support 5.2.6.2 (2022-02-11)
* Fix Reloader method signature to work with the new Executor signature.
Action Pack 5.2.6.2 (2022-02-11)
* Under certain circumstances, the middleware isn't informed that the
response body has been fully closed which result in request state
not being fully reset before the next request.
[CVE-2022-23633]
5.52.0 (2022-01-01)
* Use Class#subclasses if available in the subclasses plugin, instead of a
custom Model.subclasses accessor (jeremyevans)
* Add Model.descendants and .freeze_descendants to subclasses plugin
(jeremyevans)
* Avoid use of deprecated Refinement#include on Ruby 3.1+ (jeremyevans)
* Add date_parse_input_handler extension for custom handling of input to
date parsing methods (jeremyevans)
* Make postgres adapter respect Database#default_string_column_size
(jeremyevans)
* Make pg_interval extension work with ActiveSupport 7.0 (jeremyevans)
* Make :ruby_default schema entry for type: :datetime respect
Sequel.datetime_class (jeremyevans)
* Make alter_table drop_constraint have an effect on MySQL 8.0.19+
(jeremyevans)
* Make mysql adapter support ruby-mysql 3 API (jeremyevans) (#1795)
* Make mysql adapter no longer use connection's server_version, since it
isn't accurate when using the ruby-mysql driver (jeremyevans)
* Add sql_comments plugin for automatically including comments on queries
generated by model class, instance, and dataset methods (jeremyevans)
* Make sql_comments Database extension support Database#with_comments, for
automatically including comments for queries executed inside the block
(jeremyevans)
* Fix sql_comments extension to not modify cached SQL for a dataset
(jeremyevans)
5.53.0 (2022-02-01)
* Make Dataset#_sql_comment private when using the Database sql_comments
extension (jeremyevans)
* Fix prepared statements in the mysql2 adapter to reuse native prepared
statements (jeremyevans) (#1832)
* Support H2 version 2+ in the jdbc/h2 adapter (jeremyevans) (#1817)
* Work around active_support breaking subclasses plugin on Ruby <3.1
(jeremyevans) (#1816)
* Fix error handling if trying to setup column_encryption plugin without
keys (jeremyevans) (#1815)
PostgreSQL 14.2, 13.6, 12.10, 11.15, and 10.20 Released!
This update fixes over 55 bugs that were reported in the last several months. The issues listed below affect PostgreSQL 14. Some of these issues may also affect other supported versions of PostgreSQL.
Included in this release:
Fix for a low probability scenario of index corruption when a HOT (heap-only tuple) chain changes state during VACUUM. Encountering this issue is unlikely, but if you are concerned, please consider reindexing.
Fix for using REINDEX CONCURRENTLY on TOAST table indexes to prevent corruption. You can fix any TOAST indexes by reindexing them again.
The psql \password command now defaults to setting the password for the role defined by CURRENT_USER. Additionally, the role name is now included in the password prompt.
Build extended statistics for partitioned tables. If you previously added extended statistics to a partitioned table, you should run ANALYZE on those tables. As autovacuum currently does not process partitioned tables, you must periodically run ANALYZE on any partitioned tables to update their statistics.
Fix crash with ALTER STATISTICS when the statistics object is dropped concurrently.
Fix crash with multiranges when extracting variable-length data types.
Several fixes to the query planner that lead to incorrect query results.
Several fixes for query plan memoization.
Fix startup of a physical replica to tolerate transaction ID wraparound.
When using logical replication, avoid duplicate transmission of a partitioned table's data when the publication includes both the child and parent tables.
Disallow altering data type of a partitioned table's columns when the partitioned table's row type is used as a composite type elsewhere.
Disallow ALTER TABLE ... DROP NOT NULL for a column that is part of a replica identity index.
Several fixes for caching that correct logical replication behavior and improve performance.
Fix memory leak when updating expression indexes.
Avoid leaking memory during REASSIGN OWNED BY operations that reassign ownership of many objects.
Fix display of whole-row variables appearing in INSERT ... VALUES rules.
Fix race condition that could lead to failure to localize error messages that are reported early in multi-threaded use of libpq or ecpglib.
Fix psql \d command for identifying parent triggers.
Fix failures on Windows when using the terminal as data source or destination. This affected the psql \copy command and using pg_recvlogical with -f -.
Fix the pg_dump --inserts and --column-inserts modes to handle tables that contain both generated and dropped columns.
Fix edge cases in how postgres_fdw handles asynchronous queries. These errors could lead to crashes or incorrect results when attempting to run parallel scans of foreign tables.
For the full list of changes available, please review the release notes.
4.1.3
Bug Fixes
Fix flushdb and flushall
Maintenance
Add redis5 and redis4 dockers
Change json.clear test multi to be up to date with redisjson
Fixing volume for unstable_cluster docker
Update changes file with changes since 4.0.0-beta2
4.9.9 (2022-01-22)
- issue #17305 Fix syntax error for PHP 5
- issue #17307 Fix hide_connection_errors being undefined when a controluser is set
4.9.8 (2022-01-20)
- issue #14321 Display a correct error page when "$cfg['Servers'][$i]['SignonURL']" is empty for auth_type=signon
- issue #14321 [security] Remove leaked HTML on signon page redirect before login for auth_type=signon
- issue [security] Add configuration directive $cfg['Servers'][$i]['hide_connection_errors'] to allow hiding host names and other error details when login fails
- issue [security] Add configuration directive $cfg['URLQueryEncryption'] to allow encrypting senstive information in the URL
- issue [security] Fix a scenario where an authenticated user can disable two factor authentication
6.0.2 (2022-01-24)
==================
Bugfixes
--------
- Revert :issue:`644`, restore type annotations to as-of 5.2.0 version. (:issue:`688`)
6.0.1 (2022-01-23)
==================
Bugfixes
--------
- Restored back ``MultiDict``, ``CIMultiDict``, ``MultiDictProxy``, and
``CIMutiDictProxy`` generic type arguments; they are parameterized by value type, but the
key type is fixed by container class.
``MultiDict[int]`` means ``MutableMultiMapping[str, int]``. The key type of
``MultiDict`` is always ``str``, while all str-like keys are accepted by API and
converted to ``str`` internally.
The same is true for ``CIMultiDict[int]`` which means ``MutableMultiMapping[istr,
int]``. str-like keys are accepted but converted to ``istr`` internally. (:issue:`682`)
Version 4.1.2
Changes
🚀 New Features
Invalid OCSP certificates should raise ConnectionError on failed validation
Added retry mechanism on socket timeouts when connecting to the server
🐛 Bug Fixes
LMOVE, BLMOVE return incorrect responses
Fixing AttributeError in UnixDomainSocketConnection
Fixing TypeError in GraphCommands.explain
🧰 Maintenance
For tests, increasing wait time for the cluster
Increased pubsub's wait_for_messages timeout to prevent flaky tests
README code snippets formatted to highlight properly
Fix link in the main page
Documentation fixes: JSON Example, SSL Connection Examples, RTD version
Direct link to readthedocs
Features
--------
- Use ``METH_FASTCALL`` where it makes sense.
``MultiDict.add()`` is 2.2 times faster now, ``CIMultiDict.add()`` is 1.5 times faster.
The same boost is applied to ``get*()``, ``setdefault()``, and ``pop*()`` methods. (:issue:`681`)
Bugfixes
--------
- Fixed type annotations for keys of multidict mapping classes. (:issue:`644`)
- Support Multidict[int] for pure-python version.
``__class_getitem__`` is already provided by C Extension, making it work with the pure-extension too. (:issue:`678`)
Deprecations and Removals
-------------------------
- Dropped Python 3.6 support (:issue:`680`)
December 03 2021 - v5.2
This maintenance release fixes issues reported by users since the last six
months with some improvements and new features.
New options and features:
* Allow to pass multiple files when using --inplace. Thanks to mieszko4
for the patch.
* Add a button to copy formatted text to clipboard in the CGI interface.
Thanks to Yan Sheng for the feature request.
Here is the complete list of changes and acknowledgments:
- Fix bad formatting of materialized view ddl with parameters. Thanks to
Wilson Lin for the report.
- Fix/unicode and --inplace argument causing an error. Thanks to mieszko4
for the patch.
- Fix anonymizing disabled in last version. Thanks to Nikolas Poniros for
the report.
- Correctly format casts to quoted types. Thanks to Adam Vartanian for the
patch.
Changes in MySQL 5.7.37
Audit Log Notes
Previously, each event logged by MySQL Enterprise Audit included the SQL statement literal text. To provide an alternative (because it is possible that statements contain sensitive information), the audit log filtering language now supports logging a statement's digest rather than its literal text. For example, instead of logging this statement:
SELECT * FROM orders WHERE some_sensitive_column=1234567
The audit log plugin can log this digest:
SELECT * FROM `orders` WHERE `some_sensitive_column` = ?
This is similar to what is already logged for prepared statements, for which parameter markers appear rather than actual data values.
To perform digest logging, use audit filter definitions that replace the statement literal text by its corresponding digest, as discussed in Replacement of Event Field Values.
Because text replacement occurs at an early auditing stage (during filtering), the choice of whether to log statement literal text or digest values applies regardless of log format written later (that is, whether the audit log plugin produces XML or JSON output).
Compilation Notes
Binary packages that include curl rather than linking to the system curl library have been upgraded to use curl 7.80.0.
SQL Function and Operator Notes
Queries making use of the MBRContains() function did not employ all available spatial indexes.
The FORMAT() function returned a formatted number without showing the thousands separator and grouping between separators when either the es_ES or es_MX locale was specified.
Packaging Notes
The GnuPG build key used to sign MySQL downloadable packages has been updated. The previous GnuPG build key is set to expire on 2022-02-16. For information about verifying the integrity and authenticity of MySQL downloadable packages using GnuPG signature checking, or to obtain a copy of our public GnuPG build key, see Signature Checking Using GnuPG.
Due to the GnuPG key update, systems configured to use repo.mysql.com may report a signature verification error when upgrading to MySQL 5.7.37 and higher or to MySQL 8.0.28 and higher using apt or yum. Use one of the following methods to resolve this issue:
Manually reinstall the MySQL APT or YUM repository setup package from https://dev.mysql.com/downloads/.
Download the MySQL GnuPG public key and add it your system GPG keyring.
For MySQL APT repository instructions, see Appendix A: Adding and Configuring the MySQL APT Repository Manually.
For MySQL YUM repository instructions, see Upgrading MySQL with the MySQL Yum Repository.
Bugs Fixed
InnoDB: The buf_validate() function in the InnoDB sources was optimized, improving performance on debug builds.
Thanks to Hobert Lu for the contribution.
Partitioning: Creating a table with nondeterministic functions in generated column expressions should not be possible, but this was not enforced in all cases; a series of one or more ALTER TABLE statements could be employed to arrive at a partitioned table with one or more such generated columns. When attempting to execute the CREATE TABLE statement obtained by running SHOW CREATE TABLE against this table, MySQL rejected the statement with a misleading error message referring to the partitioning expression rather than to the problematic column, despite the fact that the partitioning expression itself was legal.
This was caused by the result of a check for any unsafe expressions defined for a generated column (in the internal variable thd->safe_to_cache_query), which was later checked again without being cleared while parsing the partition expression, leading to an error even when the partition expression did not refer to the problematic generated column expression. Now in such cases, we reset thd->safe_to_cache_query before parsing the partition function.
The issue of allowing the use of certain nondeterminstic functions (AES_ENCRYPT(), AES_DECRYPT(), RANDOM_BYTES()) in generated columns is handled separately.
Partitioning: A query using an index other than the primary key of a partitioned table sometimes resulted in excessive CPU load.
Replication: When the PAD_CHAR_TO_FULL_LENGTH SQL mode was enabled on a replica server, trailing spaces could be added to a replication channel’s name in the replication metadata repository tables, resulting in errors in replication operations that identified the channel using that data. The issue has now been fixed in MySQL 8.0 by using VARCHAR for character columns, and in MySQL 5.7 by disabling the SQL mode when reading from those tables. Thanks to Brian Yue for the contribution.
MySQL 5.7 did not handle the thread_stack variable in the same manner as MySQL 5.6 or MySQL 8.0.
It was possible in some cases to create a generated column of type SERIAL, which is not allowed.
See Numeric Data Type Syntax, and CREATE TABLE and Generated Columns, for more information
Statements which commit a transaction implicitly or explicitly are not allowed inside a trigger or a stored function. Both CREATE TRIGGER and CREATE FUNCTION should report an error (ER_COMMIT_NOT_ALLOWED_IN_SF_OR_TRG) in this case, but did not correctly handle DROP TABLESPACE.
The MySQL session used for online keyring migration was not closed gracefully after the migration was complete, resulting in an “Aborted connection” note being printed to the error log.
SHOW PROCESSLIST could read freed memory when accessing the query string belonging to a connection that was in the process of deleting a prepared statement.
Privileges were not checked correctly for ALTER USER ... IDENTIFIED WITH ... BY.
OpenLDAP 2.6.1 Release (2022/01/20)
Fixed libldap to init client socket port
Fixed libldap with referrals
Added slapd config keyword for logfile format
Fixed slapd to allow objectClass edits with no net change
Fixed slapd configtable population
Fixed slapd to only set loglevel in server mode
Fixed slapd logfile-rotate use of uninitialized variable
Fixed slapd passwd scheme handling with slapd.conf
Fixed slapd postread support for modrdn
Fixed slapd syncrepl recreation of deleted entries
Fixed slapd syncrepl replication with ODSEE
Fixed slapd syncrepl to properly replicate glue entries
Fixed slapd syncrepl to reject REFRESH for precise resync
Fixed slapd syncrepl to avoid busy loop during refresh
Fixed slapd syncrepl when X-ORDERED is specified
Fixed slapd syncrepl to better handle out of order delete ops
Fixed slapd syncrepl to correctly close connections when config is deleted
Fixed slapd-mdb to update indices correctly on replace ops
Fixed slapd-wt to set correct flags
Fixed slapo-accesslog to fix assertion due to deprecated code
Fixed slapo-accesslog to fix inconsistently normalized minCSN
Fixed slapo-accesslog delete handling of multi-valued config attrs
Fixed slapo-autogroup to maintain values in insertion order
Fixed slapo-constraint to maintain values in insertion order
Fixed slapo-dyngroup to maintain values in insertion order
Fixed slapo-dynlist compare operation for static groups
Fixed slapo-dynlist static group filter with multiple members
Fixed slapo-ppolicy when not built modularly
Fixed slapo-refint to maintain values in insertion order
Fixed slapo-retcode to honor requested insert position
Fixed slapo-sock cn=config support
Fixed slapo-syncprov memory leak
Fixed slapo-syncprov to generate a more accurate accesslog query
Fixed slapo-syncprov to allow empty DB to host persistent syncrepl connections
Fixed slapo-syncprov to consider all deletes for sycnInfo messages
Fixed slapo-translucent to warn on invalid config
Fixed slapo-unique to warn on invalid config
Fixed slapo-valsort to maintain values in insertion order
Build Environment
Fix test022 to preserve DELAY search output
Fix slapd-watcher to allow startup when servers are down
Contrib
Fixed slapo-lastbind to work with 2.6 lastbind-precision configuration
Documentation
Fixed slapd.conf(5)/slapd-config(5) documentation on lastbind-precision
Fixed slapo-accesslog(5) to clarify logoldattr usage
Version 4.1.1
Changes
🚀 New Features
Add retries to connections in Sentinel Pools
OCSP Stapling Support
Define incr/decr as aliases of incrby/decrby
FT.CREATE - support MAXTEXTFIELDS, TEMPORARY, NOHL, NOFREQS, SKIPINITIALSCAN
🐛 Bug Fixes
Timeseries docs fix
get_connection: catch OSError too
Set keys var otherwise variable not created
Clusters should optionally require full slot coverage
🧰 Maintenance
Triple quote docstrings in client.py PEP 257
syncing requirements
Typo and typing in GraphCommands documentation
Allowing poetry and redis-py to install together
setup.py: Add project_urls for PyPI
Support test with redis unstable docker
Connection examples
Documentation cleanup
7.0.1 (2021-01-06)
* Change QueryMethods#in_order_of to drop records not listed in values.
in_order_of now filters down to the values provided, to match the
behavior of the Enumerable version.
Kevin Newton
* Allow named expression indexes to be revertible.
Previously, the following code would raise an error in a reversible
migration executed while rolling back, due to the index name not
being used in the index removal.
add_index(:settings, "(data->'property')", using: :gin, name: :index_settings_data_property)
Fixes#43331.
Oliver Günther
* Better error messages when association name is invalid in the
argument of ActiveRecord::QueryMethods::WhereChain#missing.
ykpythemind
* Fix ordered migrations for single db in multi db environment.
Himanshu
* Extract on update CURRENT_TIMESTAMP for mysql2 adapter.
Kazuhiro Masuda
* Fix incorrect argument in PostgreSQL structure dump tasks.
Updating the --no-comment argument added in Rails 7 to the correct
--no-comments argument.
Alex Dent
* Fix schema dumping column default SQL values for sqlite3.
fatkodima
* Correctly parse complex check constraint expressions for PostgreSQL.
fatkodima
* Fix timestamptz attributes on PostgreSQL handle blank inputs.
Alex Ghiculescu
Fix migration compatibility to create SQLite references/belongs_to
column as integer when migration version is 6.0.
Reference/belongs_to in migrations with version 6.0 were creating
columns as bigint instead of integer for the SQLite Adapter.
Marcelo Lauxen
* Fix joining through a polymorphic association.
Alexandre Ruban
* Fix QueryMethods#in_order_of to handle empty order list.
Post.in_order_of(:id, []).to_a Also more explicitly set the column
as secondary order, so that any other value is still ordered.
Jean Boussier
* Fix rails dbconsole for 3-tier config.
Eileen M. Uchitelle
* Fix quoting of column aliases generated by calculation methods.
Since the alias is derived from the table name, we can't assume the
result is a valid identifier.
class Test < ActiveRecord::Base
self.table_name = '1abc'
end
Test.group(:id).count
# syntax error at or near "1" (ActiveRecord::StatementInvalid)
# LINE 1: SELECT COUNT(*) AS count_all, "1abc"."id" AS 1abc_id FROM "1...
Jean Boussier
Add gdbm gem as ruby-gdbm-gem package version 2.1.0.
Note:
* Ruby 2.6 contains gdbm 2.0.0.
* Ruby 2.7 and Ruby 3.0 contains gdbm 2.1.0, but no conflict.
* Ruby 3.1 dose not contain gdbm.
GDBM
GNU dbm is a library for simple databases. A database is a file that stores
key-value pairs. Gdbm allows the user to store, retrieve, and delete data
by key. It furthermore allows a non-sorted traversal of all key-value
pairs. A gdbm database thus provides the same functionality as a hash. As
with objects of the Hash class, elements can be accessed with [].
Furthermore, GDBM mixes in the Enumerable module, thus providing convenient
methods such as #find, #collect, #map, etc.
A process is allowed to open several different databases at the same time.
A process can open a database as a "reader" or a "writer". Whereas a reader
has only read-access to the database, a writer has read- and write-access.
A database can be accessed either by any number of readers or by exactly one
writer at the same time.
Add dbm gem as ruby-dbm package version 1.1.0.
Note:
* Ruby 2.6 contains dbm 1.0.0.
* Ruby 2.7 and Ruby 3.0 contains dbm 1.1.0, but no conflict.
* Ruby 3.1 dose not contain dbm.
DBM
The DBM class provides a wrapper to a Unix-style dbm or Database Manager
library.
Dbm databases do not have tables or columns; they are simple key-value data
stores, like a Ruby Hash except not resident in RAM. Keys and values must
be strings.
The exact library used depends on how Ruby was compiled. It could be any of
the following:
* The original ndbm library is released in 4.3BSD. It is based on dbm
library in Unix Version 7 but has different API to support multiple
databases in a process.
* Berkeley DB versions 1 thru 5, also known as BDB and Sleepycat DB, now
owned by Oracle Corporation.
* Berkeley DB 1.x, still found in 4.4BSD derivatives (FreeBSD, OpenBSD,
etc).
* gdbm, the GNU implementation of dbm.
* qdbm, another open source reimplementation of dbm.
All of these dbm implementations have their own Ruby interfaces available,
which provide richer (but varying) APIs.
3.37.0-r1
Allow breaking of reference cycles between objects that contain a Connection or Cursor, and also use callbacks from that object (eg busy handler).
This is the last release supporting Python 2 and Python 3 before 3.7. If you still use those Python versions then you should pin to this APSW version. (More information).
Windows Python 3.10 binaries are available to download. The .exe format is no longer available with this Python version.
Fixed custom VFS extension loading failure could leave the error message unterminated.
Updated size of mutex array used by the fork checker
Connections are opened with SQLITE_OPEN_EXRESCODE so open errors will also include extended result codes.
Connection.changes() and Connection.totalchanges() use the new SQLite APIs that return 64 bit values (ie can now return values greater than 2 billion).
Added Connection.autovacuum_pages().
Added constants:
SQLITE_CONSTRAINT_DATATYPE, SQLITE_OPEN_EXRESCODE
Version 4.1.0
🚀 New Features
OCSP stapling support
Support for SELECT
Support for specifying error types with retry
Support for RESET command since Redis 6.2.0
Support CLIENT TRACKING
Support WRITE in CLIENT PAUSE
JSON set_file and set_path support
Allow ssl_ca_path with rediss:// urls
Support for password-encrypted SSL private keys
Support SYNC and PSYNC
🐛 Bug Fixes
Retry on error exception and timeout fixes
Fixing read race condition during pubsub
Fixing exception in listen
Fixed MovedError, and stopped iterating through startup nodes when slots are fully covered
Socket not closing after server disconnect
Single sourcing the package version
Ensure redis_connect_func is set on uds connection
🧰 Maintenance
SRTALGO - Skip for redis versions greater than 7.0.0
Documentation updates
Add CI action to install package from repository commit hash
Fix link in lmove docstring
Disabling JSON.DEBUG tests
Version 4.0.2
🐛 Bug Fixes
Restoring Sentinel commands to redis client
Better removal of hiredis warning
🧰 Maintenance
Adding links to redis documents in function calls
Version 4.0.1
🐛 Bug Fixes
Removing command on initial connections
Removing hiredis warning when not installed
Version 4.0.0
🚀 New Features
FT.EXPLAINCLI intentionally raising NotImplementedError
🐛 Bug Fixes
Restoring ZRANGE desc for Redis < 6.2.0
Response parsing occasionally fails to parse floats
Re-enabling read-the-docs
🧰 Maintenance
Call HSET after FT.CREATE to avoid keyspace scan
Unit tests fixes for compatibility
Improve documentation about Locks
Fixes to allow --redis-url to pass through all tests
Fix unit tests running against Redis 4.0.0
Search alias test fix
Adding RediSearch/RedisJSON tests
Updating codecov rules
Tests to validate custom JSON decoders
Added breaking icon to release drafter
Changes in version 3.37.2:
Fix a bug introduced in version 3.35.0 (2021-03-12) that can cause database corruption if a SAVEPOINT is rolled back while in PRAGMA temp_store=MEMORY mode, and other changes are made, and then the outer transaction commits. Check-in 73c2b50211d3ae26
Fix a long-standing problem with ON DELETE CASCADE and ON UPDATE CASCADE in which a cache of the bytecode used to implement the cascading change was not being reset following a local DDL change. Check-in 5232c9777fe4fb13.
Other minor fixes that should not impact production builds.
This flag should be set for packages that import pkg_resources
and thus need setuptools after the build step.
Set this flag for packages that need it and bump PKGREVISION.
version 3.37.1
Fix a bug introduced by the UPSERT enhancements of version 3.35.0 that can cause incorrect byte-code to be generated for some obscure but valid SQL, possibly resulting in a NULL-pointer dereference.
Fix an OOB read that can occur in FTS5 when reading corrupt database files.
Improved robustness of the --safe option in the CLI.
Other minor fixes to assert() statements and test cases.
pkgsrc changes:
---------------
* Add 14 to PGSQL_VERSION_ACCEPTABLE to fix a dependency failure of
geography/mapserver with PostgreSQL 14.
* Fix BUILDLINK_ABI_DEPENDS that contained a hardcoded dependency.
* Bump revision.
2021-04-19 v1.2.1
* Resolve CI bug where non-Linux wheels were not being published to PyPI.
2021-04-15 v1.2.0
* Update bundled LMDB to 0.9.29.
* Add non-bundled testing to CI.
* Remove wheel generation for 2.7 because the manylinux images no longer
support it.
* Allow passing None as a value to transaction.del in CFFI implementation
for parity with cpython implementation.
* Fix Cursor.put behavior on a dupsort DB with append=True.
* Add warning to docs about use of Environment.set_mapsize. This is currently
an unresolved issue with upstream LMDB.
* CFFI implementation: fix a seg fault when open_db returns map full.
* CFFI implementation: fix a bug in open_db in a read-only environment.
2021-02-05 v1.1.1
* Dowgrade underlying LMDB to 0.9.26. 0.9.27 has a minor defect that will
need to get resolved.
2021-02-04 v1.1.0
* Migrate CI pipeline from Travis and AppVeyor to Github Actions. Now
includes comprehensive testing across 4 dimensions (OS, Python version,
cpython/CFFI, pure/with mods). Also includes publishing to PyPI.
* Prevent invalid flag combinations when creating a database.
* Add a Cursor.getmulti method with optional buffer support. Contributed by
Will Thompson <willsthompson@gmail.com>.
* Upgrade underlying LMDB to 0.9.27.
2020-08-28 v1.0.0
* Start of new semantic versioning scheme. This would be a minor version
bump from the 0.99 release if it were semantically versioned.
* Allow environment copy to take a passed-in transaction. This is the
first released feature that requires a (very small) patch to the
underlying C library. By default, the patch will be applied unless
this module is built with LMDB_PURE environment variable set.
2020-08-13 v0.99
* Fix lmdb.tool encoding issues.
* Fix -l lmdb invocation issue.
* Minor documentation improvements.
* Update LMDB to version 0.9.24.
* Update for Python 3.9 (current release candidate) support.
* Resolve a bug when using cursor.putmulti and append=True on dupsort DBs.
* Allow _Database.flags method to take no arguments since the one argument
wasn't being used.
2019-11-06 v0.98
* Fix that a duplicate argument to a lmdb method would cause an assert.
* Solaris needs ```#include "python.h"``` as soon as possible. Fix
contributed by Jesús Cea.
* Fix crash under debug cpython when mdb_cursor_open failed
2019-08-11 v0.97
* Fix a missed GIL unlock sequence. Reported by ajschorr.
* Fix argv check in JEP (cpython under Java) environment. Contributed by
de-code.
2019-07-14 v0.96
* First release under new maintainer, Nic Watson.
* Doc updates.
* More removal of code for now-unsupported Python versions.
* Only preload the value with the GIL unlocked when the value is actually
requested. This significantly improves read performance to retrieve keys
with large values when the value isn't retrieved. Reported by Dan Patton.
2019-06-08 v0.95
* The minimum supported version of Python is now 2.7.
* The library is no longer tested on Python 3.2.
* The address-book.py example was updated for Python 3. Contributed by Jamie
Bliss.
* Development-related files were removed from the distribution tarball.
* Handling of the Environment(create=True) flag was improved. Fix contributed
by Nir Soffer.
* Database names may be reused after they are dropped on CFFI, without
reopening the environment. Fix contributed by Gareth Bult.
2018-04-09 v0.94
* CPython argument parsing now matches the behaviour of CFFI, and most sane
Python APIs: a bool parameter is considered to be true if it is any truthy
value, not just if it is exactly True. Reported by Nic Watson.
* Removed Python 2.6 support due to urllib3 warnings and pytest dropping it.
* Updared LMDB to version 0.9.22.
* Fixed several 2.7/3 bugs in command line tool.
fails on NetBSD because it assumes that the "libdl" library is required
to link with libssl (for some reason). Limit libdl usage to Linux and
Solaris, since libdl isn't a thing on BSD-likes.
5.3.4 (2021-03-24)
This release fixes a multi/pipeline segfault on apple silicon as well as
two small compression related bugs.
You can find a detailed list of changes in Changelog.md and package.xml
* Fix multi/pipeline segfault on Apple silicon [e0796d48] (Michael Grunder)
* Pass compression flag on HMGET in RedisCluster [edc724e6] (Adam Olley)
* Abide by ZSTD error return constants [8400ed1c] (Michael Grunder)
* Fix timing related CI session tests [9b986bf8] (Michael Grunder)
* Sponsors
~ Audiomack - https://audiomack.com
~ Open LMS - https://openlms.net
~ BlueHost - https://bluehost.com
~ Object Cache Pro for WordPress - https://objectcache.pro
~ Avtandil Kikabidze - https://github.com/akalongman
~ Zaher Ghaibeh - https://github.com/zaherg
~ BatchLabs - https://batch.com
5.3.5 (2021-12-18)
This release adds support for exponential backoff w/jitter, experimental
support for detecting a dirty connection, as well as many other fixes
and improvements.
You can find a detailed list of changes in Changelog.md and package.xml
or by inspecting the git commit logs.
--- Sponsors ---
Audiomack - https://audiomack.com
Open LMS - https://openlms.net
BlueHost - https://bluehost.com
Object Cache Pro for WordPress - https://objectcache.pro
Avtandil Kikabidze - https://github.com/akalongman
Zaher Ghaibeh - https://github.com/zaherg
BatchLabs - https://batch.com
Luis Zarate - https://github.com/jlzaratec
* Fixed segfault in redis_setoption_handler [692e4e84] (Pavlo Yatsukhnenko)
* Fix masters array in the event of a cluster failover [bce692962] (Bar Shaul)
* Fix 32 bit type error [672dec87f] (Remi Collet)
* Fix radix character in certain locales [89a871e24] (Pavlo Yatsukhnenko)
* ZSTD Validation fix [6a77ef5cd] (Michael Grunder)
* Remove superfluous typecast [b2871471f] (Remi Collet)
* Updated documentation [f84168657, d017788e7, 20ac84710, 0adf05260,
aee29bf73, 09a095e72, 12ffbf33a, ff331af98, a6bdb8731, 305c15840,
1aa10e93a, d78b0c79d, c6d37c27c, a6303f5b9, d144bd2c7, a6fb815ef, 9ef862bc6]
(neodisco, Clement Tessier, T. Todua, dengliming, Maxime Cornet,
Emanuele Filannino Michael Grunder)
* Travis CI Fixes
[a43f4586e, 4fde8178f, 7bd5415ac, fdb8c4bb7, d4f407470]
(Pavlo Yatsukhnenko)
* Minor fixes/cleanup
[2e190adc1, 99975b592, 9d0879fa5, 22b06457b]
(Pavlo Yatsukhnenko)
* Fix RedisArray constructor bug
[85dc883ba](https://github.com/phpredis/phpredis/commit/85dc883ba)
([Pavlo Yatsukhnenko](https://github.com/yatsukhnenko))
* Moved to GitHub Actions
[4d2afa786, 502d09fd5] (Pavlo Yatsukhnenko)
* Use more appropriate array iteration macro
[6008900c2] (Pavlo Yatsukhnenko)
* Clean up session tests
[ab25ae7f3] (Michael Grunder)
* RedisArray refactors [1250f0001, 017b2ea7f, 37ed3f079]
(Pavlo Yatsukhnenko)
* Use zend_parse_parameters_none helper
[a26b14dbe] (Remi Collet)
* Support for various exponential backoff strategies
[#1986, #1993, 732eb8dcb, 05129c3a3, 5bba6a7fc],
(Nathaniel Braun)
* Added experimental support for detecting a dirty connection
[d68579562] (Michael Grunder)
* Created distinct compression utility methods (pack/unpack)
[#1939, da2790aec] (Michael Grunder)
* SMISMEMBER Command
[#1894, ae2382472, ed283e1ab] (Pavlo Yatsukhnenko)
* Fix typo in cluster_scan_resp [44affad2] (Michael Grunder)
Active Record -- Object-relational mapping in Rails
Active Record connects classes to relational database tables to establish an
almost zero-configuration persistence layer for applications. The library
provides a base class that, when subclassed, sets up a mapping between the
new class and an existing table in the database. In the context of an
application, these classes are commonly referred to as *models*. Models can
also be connected to other models; this is done by defining *associations*.
Active Record relies heavily on naming in that it uses class and association
names to establish mappings between respective database tables and foreign
key columns. Although these mappings can be defined explicitly, it's
recommended to follow naming conventions, especially when getting started
with the library.
This is for Ruby on Rails 7.0.
Please consult the Solr Upgrade notes for details and links specified below:
https://github.com/apache/solr/blob/main/solr/solr-ref-guide/src/solr-upgrade-notes.adoc
====================================================================
Changelog:
Solr 8.11
See the 8.11 Release Notes for an overview of the main new features of Solr 8.11.
When upgrading to 8.11.x users should be aware of the following major changes from 8.10.
Support for Multiple Authentication Schemes
Two new authentication and authorization plugins provide support for configuring multiple authentication schemes.
The MultiAuthPlugin allows combining two or more authentication approaches, such as JWT and Basic authentication.
The MultiAuthRuleBasedAuthorizationPlugin is used when the MultiAuthPlugin is also in use, and combines the various roles defined for all plugins to determine the proper role assignment for the user account.
For information on configuring these plugins, see the following sections:
Combining Basic Authentication with Other Schemes
Multiple Authorization Plugins
ZooKeeper chroot
It’s now possible to create the ZooKeeper chroot at startup if it does not already exist. See the section Using the -z Parameter with bin/solr for an example.
Other Changes
A few other minor changes are worth noting:
The config-read pre-defined permission now correctly governs access for various configuration-related APIs. See also Predefined Permissions.
The S3BackupRepository supports configuring the AWS Profile, if necessary. See also S3BackupRepository.
Additionally, backups will now properly succeed after SPLITSHARD operations, and will correctly handle incremental backup purges.
SolrJ now supports uploading configsets.
Solr 8.10
See the 8.10 Release Notes for an overview of the main new features of Solr 8.10.
When upgrading to 8.10.x users should be aware of the following major changes from 8.9.
Schema Designer UI
A new screen has been added to the Admin UI that allows you to interactively design a Solr schema using your documents.
The designer screen provides a safe environment for you to:
Upload or paste sample documents to identify fields.
Get a "first" guess at what Solr thinks the field types in the fields should be.
Edit fields, field types, dynamic fields, and supporting files.
See how a field’s analysis will impact your text.
Test how schema changes will impact query-time behavior.
Save your changes to a configset to use with a new collection.
See the section Schema Designer for full details.
Backups in S3
Following the redesign of backups in Solr 8.8 that allowed storage of incremental backups in Google Cloud environments, Solr 8.10 provides support for storing backups in Amazon S3 buckets.
See the section S3BackupRepository for how to configure.
Security Admin UI
Solr’s Admin UI also got a new screen to support management of users, roles, and permissions.
The new UI works when authentication and/or authorization has been enabled with bin/solr auth or by manually installing a security.json file. Before this, it provides a warning that your Solr instance is unsecured.
See the section Security UI for details.
Solr SQL Improvements
A number of improvements have been made in Solr’s SQL functionality:
Support added for LIKE, IS NOT NULL, IS NULL, and wildcards (for simplistic LIKE functionality).
Two new aggregation functions, COUNT(DISTINCT field) and APPROX_COUNT_DISTINCT(field), have been added.
Queries using an ORDER BY clause can support OFFSET and FETCH operations.
Multi-valued fields can now be returned.
User permissions have been simplified so access to query endpoints /sql, /select, and /export is sufficient for full access for all SQL queries.
shards.preference
A new option for the shards.preference parameter allows selection of nodes based on whether or not the replica is a leader. Now adding shards.preference=replica.leader:false will limit queries only to replicas which are not currently their shard’s leader.
See the section shards.preference Parameter for details and examples.
Metrics & Prometheus Exporter
A new expr option in the Metrics API allows for more advanced filtering of metrics based on regular expressions. See the section Metrics API for examples.
The Prometheus Exporter’s default solr-exporter.config has been improved to use the new expr option in the Metrics API to get a smaller set of metrics. The default metrics exported still include most metrics, but the configuration will be easier to trim as needed. This should help provide performance improvements in busy clusters being monitored by Prometheus.
ZooKeeper Credentials
ZooKeeper credentials can now be stored in a file whose location is defined with a system property instead of being passed in plain-text. See Out of the Box Credential Implementations for how to set this up.
Solr 8.9
See the 8.9 Release Notes for an overview of the main new features of Solr 8.9.
When upgrading to 8.9.x users should be aware of the following major changes from 8.8.
Backup and Restore
Solr 8.9 introduces extensive changes to Solr’s backup and restore support.
A new backup format has been introduced in Solr 8.9 which replaces the previous snapshot-based backup. This new format enables ‘incremental’ backups. Repeated backups to a given location will take advantage of the data stored by their predecessors and will only operate on files that have changed since the previous backup. This is supported by default, simply by storing each backup file in the same location.
The old and new formats are not compatible, although backups in the old format, a full snapshot of all files, can still be used to restore to Solr for the time-being. The old format is officially deprecated, and support for it is likely to be removed in Solr 9.0.
For the time-being the old format can be created by defining a parameter incremental=false. Again, though, this support is likely to be removed in Solr 9.0.
More documentation on backups is available at Backup and Restore.
New Collections API commands for backups:
LISTBACKUP: Lists information about each backup stored at the specified repository location. See List Backups for more details.
DELETEBACKUP: Deletes specified backups from the repository. See Delete Backups for more details.
A new option for backup repository is also available in 8.9, which is to use Google Cloud Storage (GCS). This is a contrib (located in contrib/gcs-repository). See GCSBackupRepository for configuration details. The Solr community is working to add support for S3 buckets in the near future.
Nested Docs
Child Doc Transformer’s childFilter parameter no longer applies query syntax escaping because it’s inconsistent with the rest of Solr and was limiting. This refers to [child childFilter='field:value']. There was no escaping here prior to 8.0 either.
Collapse and Expand
BlockCollapse: If documents have been (or could be) indexed in a way where documents with the same collapse key have been indexed contiguously in the index, a new "block collapse" provides a significant speed improvement over traditional collapse.
See Block Collapsing for details.
Expand Null Groups: A new parameter expand.nullGroup allows an expanded group to be returned containing document with no value in the expanded field. See Expand Component for details.
In-Place Updates
A new request parameter update.partial.requireInPlace=true allows telling Solr to "fail fast" if all of the necessary conditions are not satisfied to allow an in-place update to succeed. See also In-Place Updates.
Metrics History
The Metrics History feature, which allowed long-term storage and aggregation of Solr’s metrics, has been deprecated and will be removed in 9.0.
Embedded Solr Server
When using EmbeddedSolrServer, it will no longer close CoreContainer instances that were passed to it.
Solr 8.8
When upgrading to 8.8.x users should be aware of the following major changes from 8.7.
Nested Documents
When doing atomic/partial updates to a child document:
Supply the _root_ field (the ID of the root document) so that Solr understands you are manipulating a child document and not a root document. In its absence, Solr looks at the _route_ parameter but this may change in the future because it’s not an ideal substitute. If neither are present, Solr assumes you are updating a root document. If this assumption is false, Solr will do a cheap check that usually detects the problem and will throw an exception to alert you of the need to specify the Root ID. This backwards incompatible change was done to increase performance and robustness.
This feature no longer requires stored=true or docValues=true on the _root_ field. You might have it for other purposes though (e.g., for uniqueBlock(…)).
This feature no longer requires the _nest_path_ field, although you probably ought to continue to define it as it’s useful for other things.
Removed Contribs
The search results clustering contrib (Carrot2) has been removed from 8.x Solr due to lack of Java 1.8 compatibility in the dependency that provides online clustering of search results. The contrib will be re-introduced in Solr 9.0.
Learning to Rank
Interleaving support has been added to Learning to Rank (LTR). Currently only the Team Draft Interleaving algorithm is supported. For examples using this feature, see the section Running a Rerank Query Interleaving Two Models.
Metrics
Two metrics have been added for SolrCloud’s Overseer:
solr_metrics_overseer_stateUpdateQueueSize
solr_metrics_overseer_collectionWorkQueueSize
Prometheus Exporter
The ./bin scripts included with the Prometheus Exporter contrib now allow use of custom java options with environment variables. See the section Environment Variable Options for more details.
The default Grafana dashboards now include panels for query performance monitoring. The default Prometheus Exporter configuration includes metrics like queries-per-second (QPS) and 95th percentiles (P95) to populate the new panels.
The default Prometheus Exporter configuration also includes the two new metrics mentioned in the Metrics above.
Solr Home
The internal logic for identifying 'Solr Home' (solr.solr.home) has been refactored to make testing less error prone. Plugin developers using SolrPaths.locateSolrHome() or 'new `SolrResourceLoader’ should check deprecation warnings as existing some existing functionality will be removed in 9.0. SOLR-14934 has more technical details about this change for those concerned.
base_url removed from stored state
As of Solr 8.8.0, the base_url property was removed from the stored state for replicas (SOLR-12182). If you’re able to upgrade SolrJ to 8.8.x for all of your client applications, then you can set -Dsolr.storeBaseUrl=false (introduced in Solr 8.8.1) to better align the stored state in ZooKeeper with future versions of Solr. However, if you are not able to upgrade SolrJ to 8.8.x for all client applications, then leave the default -Dsolr.storeBaseUrl=true so that Solr will continue to store the base_url in ZooKeeper.
You may also see some NPE in collection state updates during a rolling upgrade to 8.8.0 from a previous version of Solr. After upgrading all nodes in your cluster to 8.8.0, collections should fully recover. Trigger another rolling restart if there are any replicas that do not recover after the upgrade to re-elect leaders.
Solr 8.7
See the 8.7 Release Notes for an overview of the main new features of Solr 8.7.
When upgrading to 8.7.x users should be aware of the following major changes from 8.6.
Autoscaling
If upgrading from 8.6.0, please see the 8.6.1 Upgrade notes below for information on performance degradations introduced in 8.6.0 that require some intervention to resolve. If you are already on 8.6.1 or higher, you can ignore these instructions.
Deprecations
The autoscaling framework is now formally deprecated and will be removed in Solr 9.0. The Solr community is working on pluggable API to replace this functionality, with the goal for it to be ready by the time 9.0 is released. Deprecations include: autoscaling policy, triggers, withCollection support, simulation framework, autoscaling suggestions tab in the UI, autoAddReplicas and UTILIZENODE command.
Similarly, rule-based replica placement strategy has been deprecated and will be replaced in Solr 9.0 by APIs for replica placement and cluster events, with plugin-based implementations.
Support for detecting spinning disks has been removed in LUCENE-9576. Corresponding spins metrics in Solr still exist but now they always return false and will be removed in Solr 9.0.
User-Managed Cluster Terminology Updated
Solr has replaced the terms "master" and "slave" in the codebase and all documentation with "leader" and "follower".
This functionality has only changed in terms of parameter names changed, and we do not expect any back-compatibility issues on upgrade to 8.7 or even 9.0 later.
However, users should update their solrconfig.xml files after completing the upgrade on all nodes of a cluster. Comparing your configuration to the updated configuration examples in Index Replication will show examples of what needs to change, but here are the main changes:
On the replication leader, in the definition of the /replication request handler:
Replace "master" with "leader".
Replace "slave" with "follower" if the former term is used in the name of any follower solrconfig.xml file definitions. This file can be named anything, so you can change it to whatever you’d like to call it if you’d like.
Replace "slave" with "follower" if the former term is used in a replication repeater configuration.
On the replication follower, in the definition of the /replication request handler:
Replace "masterUrl" with "leaderUrl".
Replace "slave" with "follower" if the former term is used in a repeater configuration.
JSON Facets
Performance enhancements for the relatedness() statistics function are included with 8.7. These yield the highest benefits with high-cardinality fields and can be disabled if working with lower cardinality fields with a new sweep_collection parameter. See the section relatedness() Options for details.
solr.in.sh / solr.in.cmd
Solr has relied on the SOLR_STOP_WAIT parameter defined in solr.in.sh or solr.in.cmd to determine how long to wait on startup. A new parameter SOLR_START_WAIT allows defining how long Solr should wait for start up to complete.
If the time set by this parameter is exceeded, Solr will exit the startup process and return the last few lines of the solr.log file to the terminal.
By default, this parameter is set to the same value as SOLR_STOP_WAIT.
The default ZooKeeper client timeout (ZK_CLIENT_TIMEOUT) is now 30 seconds (30000 milliseconds) instead of 15.
Configsets
It’s now possible to overwrite an existing configset when uploading changes by supplying the overwrite=true parameter to the Configset API.
A related parameter is cleanup=true, which allows deleting any files from the old configset that are left behind after the overwrite.
The default for both of these parameters is false.
When deleting a collection that has an automatically created configset (i.e., the configset was copied from the _default collection when the collection was created), the configset will also be deleted if it is not in use by any other collection.
Logging
A request ID (rid) is now logged for all distributed search requests (in SolrCloud) which can be used to correlate query events across the system. A parameter disableRequestId=true can be added to disable this if desired.
Solr 8.6.1
See the 8.6.1 Release Notes for an overview of the fixes included in Solr 8.6.1.
When upgrading to 8.6.1 users should be aware of the following major changes from 8.6.0.
Autoscaling
As mentioned in the 8.6 upgrade notes, a default autoscaling policy was provided starting in 8.6.0. This default autoscaling policy resulted in increasingly slow collection creation calls in large clusters (50+ collections).
In 8.6.1 the default autoscaling policy has been removed, and clusters will not use autoscaling unless a policy has explicitly been created. If your cluster is running 8.6.0 and not using an explicit autoscaling policy, upgrade to 8.6.1 and remove the default cluster policy and preferences via the following command.
Replace localhost:8983 with your Solr endpoint.
curl -X POST -H 'Content-type:application/json' -d '{set-cluster-policy : [], set-cluster-preferences : []}' http://localhost:8983/api/cluster/autoscaling
This information is only relevant for users upgrading from 8.6.0. If upgrading from an earlier version to 8.6.1+, this warning can be ignored.
Solr 8.6
See the 8.6 Release Notes for an overview of the main new features of Solr 8.6.
When upgrading to 8.6.x users should be aware of the following major changes from 8.5.
Support for Block-Max WAND
Lucene added support for Block-Max WAND in 8.0, and 8.6 makes this available for Solr also.
This can provide significant performance enhancements by not calculating the score for results which are not likely to appear in the top set of results.
It is enabled when using a new query parameter minExactCount. This parameter tells Solr to accurately count the number of hits accurately until at least this value. Once this value is reached, Solr can skip over documents that don’t have a score high enough to be in the top set of documents, which has the potential for greatly speeding up searches.
It’s important to note that when using this parameter, the hit count of searches may not be accurate. It is guaranteed that the hit count is accurate up to the value of minExactCount, but any returned hit count higher than that may be an approximation.
A new boolean attribute numFoundExact is included in all responses to indicate if the hit count in the response is expected to be exact or not.
More information about this new feature is available in the section minExactCount Parameter.
Autoscaling
NOTE: The default autoscaling policy has been removed as of 8.6.1
Solr now includes a default autoscaling policy. This policy can be overridden with your custom rules or by specifying an empty policy to replace the default.
The ComputePlan action now supports a collection selector to identify collections based on collection properties to determine which collections should be operated on.
Security
Prior to Solr 8.6 Solr APIs which take a file system location, such as core creation, backup, restore, and others, did not validate the path and Solr would allow any absolute or relative path. Starting in 8.6 only paths that are relative to SOLR_HOME, SOLR_DATA_HOME and coreRootDir are allowed by default.
If you need to create a core or store a backup outside the default paths, you will need to tell Solr which paths to allow. A new element in solr.xml called allowPaths takes a comma-separated list of allowed paths.
When using the solr.xml file that ships with 8.6, you can configure the list of paths to allow through the system property solr.allowPaths. Please see bin/solr.in.sh or bin\solr.in.cmd for example usage. Using the value * will allow any path as in earlier versions.
For more on this, see the section Solr.xml Parameters.
Windows SMB shares on the UNC format, such as \\myhost\myshare\mypath are now always disallowed. Please use drive letter mounts instead, i.e., S:\mypath.
A new authorization plugin ExternalRoleRuleBasedAuthorizationPlugin is now available. This plugin allows an authentication plugin (such as JWT) to supply a user’s roles instead of maintaining a user-to-role mapping inside Solr.
When authentication is enabled, the Admin UI Dashboard (main screen) now includes a panel that shows the authentication and authorization plugins in use, the logged in username, and the roles assigned to this user. A new link will also appear in the left-hand navigation to allow a user to log out.
Streaming Expressions
The /export handler now supports streaming expressions to allow limiting the output of the export to only matching documents.
For more information about how to use this, see the section Specifying the Local Streaming Expression.
The stats, facet, and timeseries expressions now support percentiles and standard deviation aggregations.
Highlighting
For the Unified Highlighter: The setting hl.fragsizeIsMinimum now defaults to false because true was found to be a significant performance regression when highlighting lots of text. This will yield longer highlights on average compared to Solr 8.5 but relatively unchanged compared to previous releases. Furthermore, if your application highlights lots of text, you may want to experiment with lowering hl.fragAlignRatio to trade ideal fragment alignment for better performance.
Deprecations
A primary focus of the community is improving Solr’s stability and supportability. With the addition of the package manager system in 8.4, we now have the ability to move some features into plugins maintained by third-parties with the expertise to properly develop and support them. Our goal is to make running Solr easier and less prone to outages and other headaches. In this spirit, the following features have been deprecated and are slated to be removed in Solr 9.0.
Cross Data Center Replication (CDCR), in its current form, is deprecated and is scheduled to be removed in 9.0. This feature is unlikely to be replaced by an identical plugin. However, the community is working on figuring out a replacement feature for disaster recovery and failover.
The Data Import Handler (DIH) is deprecated and is scheduled to be removed in 9.0. Work to replace DIH with a community-supported plugin is underway and may be available soon.
Support to store indexes and backups in HDFS is deprecated and is scheduled to be removed in 9.0. A community-supported version of this may be available as a plugin in the future. For more details, please see SOLR-14021.
Users interested in maintaining a feature as a plugin are encouraged to join the developer mailing list to find out more about how to help.
See NEWS in sources for details. A summary of upstream NEWS:
* Enhancements and bugfixes
PostGIS 3.2.0 (Olivier Courtin Edition)
2021/12/17
This version requires PostgreSQL 9.6 or higher, GEOS 3.6 or higher
, and Proj 4.9+.
Additional features are enabled if you are running GEOS 3.9+
(and ST_MakeValid enhancements with 3.10+),
Proj 6.1+, and PostgreSQL 14+.
Due to some query performance degradation
with the new fast index build that requires PG14,
we have decided to disable the feature by default
until we get more user testing
as to the true impact of real-world queries.
If you are running PG14+, you can reenable it by doing
ALTER OPERATOR FAMILY gist_geometry_ops_2d USING gist
ADD FUNCTION 11 (geometry)
geometry_gist_sortsupport_2d (internal);
and then reindex your gist indexes.
To revert back to the old index behavior:
ALTER OPERATOR FAMILY gist_geometry_ops_2d using gist
DROP FUNCTION 11 (geometry);
and then reindex your gist indexes.
* Breaking changes *
- #5008, Empty geometries are not reported as being within Infinite
distance by ST_DWithin (Sandro Santilli)
- #4824, Removed `--without-wagyu` build option. Using Wagyu is now mandatory to build with MVT support.
- #4933, topology.GetFaceByPoint will not work with topologies having invalid edge linking.
- #4981, ST_StartPoint support any geometry. No longer returns null for non-linestrings.
- #4149, ST_AsMVTGeom now preserves more of original geometry's details at scale close to target extent.
If you need previous simplifying behaviour, you can ST_Simplify the geometry in advance.
(Darafei Praliaskouski)
- Proj 4.9 or higher is required
- #5000, Turn off Window support in ST_AsMVT aggregate
as no real use-case for it and it crashes with random input
(Paul Ramsey)
* New features*
- #4923, topology.ValidateTopologyRelation (Sandro Santilli)
- #4933, topology.GetFaceContainingPoint (Sandro Santilli)
- #2175, ST_Scroll (Sandro Santilli)
- #4841, FindTopology to quickly get a topology record (Sandro Santilli)
- #4869, FindLayer to quickly get a layer record (Sandro Santilli)
- #4851, TopoGeom_addTopoGeom function (Sandro Santilli)
- ST_MakeValid(geometry, options) allows alternative validity building
algorithms with GEOS 3.10 (Paul Ramsey)
- ST_InterpolateRaster() fills in raster cells between sample points
using one of a number of algorithms (inverse weighted distance, average, etc)
using algorithms from GDAL
(Paul Ramsey)
- ST_Contour() generates contour lines from raster values
using algorithms from GDAL (Paul Ramsey)
- ST_SetZ()/ST_SetM() fills in z/m coordinates of a geometry using data read
from a raster (Paul Ramsey)
- New postgis.gdal_vsi_options GUC allows out-db rasters on VSI network
services to be accessed with authentication keys, etc. (Paul Ramsey)
- ST_DumpSegments returns a set of segments of input geometry (Aliaksandr Kalenik)
- #4859, ST_Point, ST_PointZ, ST_PointM, ST_PointZM, constructors
with SRID parameter (Paul Ramsey)
- #4808, ST_ClusterKMeans now supports max_radius argument. Use it when you're not sure what is
the number of clusters but you know what the size of clusters should be. (Darafei Praliaskouski)
dbplyr is the database backend for dplyr. It allows you to use
remote database tables as if they are in-memory data frames by
automatically converting dplyr code into SQL.
Breaks build with python 2.7 because the paths are different there.
Still packages with python 3.6, 3.9, 3.10 on NetBSD.
Bump PKGREVISION for PLIST change.
Upstream changes:
0.143 2021-03-28
- fix RT #134147: Failed test 'SQL::Abstract - no limit' / Thanks to Slaven Rezić and Thibault Duponchelle
0.142 2020-11-22
- module adopted by Alexander Becker
- set up GitHub repository. See this repository for detailed changes.
- fix RT #34847: Avoid double quoting of table name. Fix provided by @dekimsey / Daniel Kimsey
- fix RT #50795: SQL::Abstract properties were ignored for select() if no limit was given. Thanks to Gunnar Hansson
- fix RT #73444: Offset argument wasn't handled properly if the value was 0. Thanks to ALEXBYK
- fix RT #62671: Properly handle 0 values in condition.
- remove Build.PL, switch to EUMM
Avoids build failure if monogb 4.x is installed.
Someone can always add a mongodb option to options.mk if they want, but
this is just to ensure reliable builds
- Updated all the live engine tests, aside from Oracle, to test with
unique registry names, so as to avoid conflicts when multiple
instances are being tested at once. Thanks to Slaven Rezić for the
report (#597).
- Removed `local` directory from the distribution, accidentally included
in v1.2.0. Thanks to gregor herrmann for the report (#600).
5.51.0 (2021-12-01)
* Make eager loading via tactical_eager_loading no longer modify objects who
already have a cached value for the association (jeremyevans)
* Make association cloning handle cases where clone association sets
different :class option than cloned association (jeremyevans)
* Make column schema entries on MySQL include an :extra entry for the Extra
column in DESCRIBE output (bschmeck) (#1791)
* Update mock PostgreSQL adapter to default to PostgreSQL 14 instead of
PostgreSQL 9.5 (jeremyevans)
* Support Dataset#with_recursive :search and :cycle options on PostgreSQL
14+ for result ordering and cycle detection (jeremyevans)
* Avoid method redefined verbose mode warnings in lazy_attributes plugin
(jeremyevans)
Version 1.3.0 - 2021-09-14
* Add GROUPS frame to Window
* Add exclude to Window
* Add method for each type of join
* Remove support for Python older than 3.5
* Support Select queries in conditionals and functions
Version 1.2.2 - 2021-05-16
* Escape table and column names containing double quote
* Use alias on the main table used in UPDATE and INSERT
* Add support for Python 3.9
Version 1.2.1 - 2020-10-08
* Fix AttributeError in Select.params with windows
Version 1.2.0 - 2020-10-05
* Add explicit Windows to Select
* Fix missing Windows definitions in nested expressions
Version 1.1.0 - 2020-01-25
* Add ORDER BY clause to aggregate functions
* Add support for Python 3.8
* Add distinct on Select
Version 1.0.0 - 2018-09-30
* Add Flavor filter_ to fallback to case expression
* Allow to use expression in AtTimeZone
* Fix Select query in returning
* Add comparison predicates
* Add COLLATE
kVS: ----------------------------------------------------------------------
SQLite Release 3.37.0 On 2021-11-27
STRICT tables provide a prescriptive style of data type management, for developers who prefer that kind of thing.
When adding columns that contain a CHECK constraint or a generated column containing a NOT NULL constraint, the ALTER TABLE ADD COLUMN now checks new constraints against preexisting rows in the database and will only proceed if no constraints are violated.
Added the PRAGMA table_list statement.
CLI enhancements:
Add the .connection command, allowing the CLI to keep multiple database connections open at the same time.
Add the --safe command-line option that disables dot-commands and SQL statements that might cause side-effects that extend beyond the single database file named on the command-line.
Performance improvements when reading SQL statements that span many lines.
Added the sqlite3_autovacuum_pages() interface.
The sqlite3_deserialize() does not and has never worked for the TEMP database. That limitation is now noted in the documentation.
The query planner now omits ORDER BY clauses on subqueries and views if removing those clauses does not change the semantics of the query.
The generate_series table-valued function extension is modified so that the first parameter ("START") is now required. This is done as a way to demonstrate how to write table-valued functions with required parameters. The legacy behavior is available using the -DZERO_ARGUMENT_GENERATE_SERIES compile-time option.
Added new sqlite3_changes64() and sqlite3_total_changes64() interfaces.
Added the SQLITE_OPEN_EXRESCODE flag option to sqlite3_open_v2().
Use less memory to hold the database schema.
Currently supports dBase III, Clipper, FoxPro, and Visual FoxPro
tables. Text is returned as unicode, and codepage settings in tables
are honored. Memos and Null fields are supported. Documentation
needs work, but author is very responsive to e-mails.
Not supported: index files (but can create tempory non-file indexes),
auto-incrementing fields, and Varchar fields.
- Fixed partitioned search for deployed changes on Oracle and Exasol to
correctly support plans with more than 250 changes. Thanks to @Nicqu
for the report (#521).
- DBI connections to the registry will now be set to trace level one
when a verbosity level of three or higher is passed to Sqitch
(i.e., `sqitch -vvv`). Thanks to @wkoszek for the suggestion (#155).
- Renamed the "master" branch to "main" and updated all relevant
references.
- Fixed the parsing of the Snowflake account name from the target URL
to that it no longer strips out the region and cloud platform parts.
Also deprecated the use of the region, host, and port config and
environment variables. This is to comply with the new account name
format. Thanks to @ldsingh00 for the report (#544).
- The checkout command will now show a usage statement when no branch
name is passed to it, rather than a series of warnings and a Git error.
Thanks to François Beausoleil for the report (#561).
- The checkout command now works when operating on a Sqitch project in
a subdirectory of a Git project. Thanks to François Beausoleil for the
report and suggested fix (#560).
- Fixed a failing bundle test when a top directory was configured in the
user or system configuration. Thanks to @CodingMinds for the spot
(#587).
- Added support to the Exasol engine for the `AUTHMETHOD` query parameter
(to allow Open ID authentication) and the `SSLCERTIFICATE=SSL_VERIFY_NONE`
query parameter to disable SSL verification. Thanks to Torsten Glunde
for the report (#588).
- Fixed "Use of uninitialized value $engine in concatenation" error when
Sqitch is unable to determine the engine when writing a plan file. Only
happens in the rare case that the core.engine value is not set.
- Improved the error message when attempting to update a plan file and no
project pragma is present. Thanks to Laurentiu Diaconu for the report
(#577).
- Fixed the error message when attempting to deploy a change that has
already been deployed to display the name of the change, rather than a
memory address. Thanks to Neil Mayhew for the report (#579).
- Added destination locking, currently implemented for PostgresQL and
MySQL. On starting a deploy or revert, Sqitch attempts to "lock the
destination" using advisory locks, to ensure that only one instance of
Sqitch makes changes to the database at any one time. This complements
the existing locking, which applies as each change is deployed or
reverted, as that pattern led to failures when multiple instances of
Sqitch were working at once. Thanks to Neil Mayhew for the report
(#579).
- Added the --lock-timeout option to the deploy, revert, rebase, and
checkout commands, to allow for shorter or longer times to wait for an
exclusive target database advisory lock for engines that support it.
Defaults to 60 seconds.
- Fixed the behavior of the plan command's `--max-count` option when used
with `--reverse` to show the proper items. Thanks to Adrian Klaver for
the report (#517).
- Fixed an issue that could cause bugs with the `check` command on
Firebird, Exasol, Oracle, and Vertica. Broken since the `check` command
was added in v1.1.0.
v0.25.0
Changes
Improve SSL option compatibility in URIs
Add Pool methods to determine its min, max, current and idle size
Make it possible to specify a statement name in Connection.prepare()
Implement support for multirange types
Fixes
Make sure timeout callbacks always get cleaned up
Update __all__ statements to a simpler form that is better supported by typecheckers
Fix test_timetz_encoding on Python 3.10
Fix a bunch of ResourceWarnings in the test suite
Fix SSLContext deprecation warnings
Fix the description of the database argument to connect()
Fix parsing of IPv6 addresses in the connection URI
Improve diagnostics of invalid executemany() input
PostgreSQL 14.1, 13.5, 12.9, 11.14, 10.19, and 9.6.24
Security Issues
CVE-2021-23214: Server processes unencrypted bytes from man-in-the-middle
Versions Affected: 9.6 - 14. The security team typically does not test unsupported versions, but this problem is quite old.
When the server is configured to use trust authentication with a clientcert requirement or to use cert authentication, a man-in-the-middle attacker can inject arbitrary SQL queries when a connection is first established, despite the use of SSL certificate verification and encryption.
The PostgreSQL project thanks Jacob Champion for reporting this problem.
CVE-2021-23222: libpq processes unencrypted bytes from man-in-the-middle
Versions Affected: 9.6 - 14. The security team typically does not test unsupported versions, but this problem is quite old.
A man-in-the-middle attacker can inject false responses to the client's first few queries, despite the use of SSL certificate verification and encryption.
If more preconditions hold, the attacker can exfiltrate the client's password or other confidential data that might be transmitted early in a session. The attacker must have a way to trick the client's intended server into making the confidential data accessible to the attacker. A known implementation having that property is a PostgreSQL configuration vulnerable to CVE-2021-23214.
As with any exploitation of CVE-2021-23214, the server must be using trust authentication with a clientcert requirement or using cert authentication. To disclose a password, the client must be in possession of a password, which is atypical when using an authentication configuration vulnerable to CVE-2021-23214. The attacker must have some other way to access the server to retrieve the exfiltrated data (a valid, unprivileged login account would be sufficient).
The PostgreSQL project thanks Jacob Champion for reporting this problem.
Bug Fixes and Improvements
This update fixes over 40 bugs that were reported in the last several months. The issues listed below affect PostgreSQL 14. Some of these issues may also affect other supported versions of PostgreSQL.
Some of these fixes include:
Fix physical replication for cases where the primary crashes after shipping a WAL segment that ends with a partial WAL record. When applying this update, update your standby servers before the primary so that they will be ready to handle the fix if the primary happens to crash.
Fix parallel VACUUM so that it will process indexes below the min_parallel_index_scan_size threshold if the table has at least two indexes that are above that size. This problem does not affect autovacuum. If you are affected by this issue, you should reindex any manually-vacuumed tables.
Fix causes of CREATE INDEX CONCURRENTLY and REINDEX CONCURRENTLY writing corrupt indexes. You should reindex any concurrently-built indexes.
Fix for attaching/detaching a partition that could allow certain INSERT/UPDATE queries to misbehave in active sessions.
Fix for creating a new range type with CREATE TYPE that could cause problems for later event triggers or subsequent executions of the CREATE TYPE command.
Fix updates of element fields in arrays of a domain that is a part of a composite.
Disallow the combination of FETCH FIRST WITH TIES and FOR UPDATE SKIP LOCKED.
Fix corner-case loss of precision in the numeric power() function.
Fix restoration of a Portal's snapshot inside a subtransaction, which could lead to a crash. For example, this could occur in PL/pgSQL when a COMMIT is immediately followed by a BEGIN ... EXCEPTION block that performs a query.
Clean up correctly if a transaction fails after exporting its snapshot. This could occur if a replication slot was created then rolled back, and then another replication slot was created in the same session.
Fix for "overflowed-subtransaction" wraparound tracking on standby servers that could lead to performance degradation.
Ensure that prepared transactions are properly accounted for during promotion of a standby server.
Ensure that the correct lock level is used when renaming a table.
Avoid crash when dropping a role that owns objects being dropped concurrently.
Disallow setting huge_pages to on when shared_memory_type is sysv
Fix query type checking in the PL/pgSQL RETURN QUERY.
Several fixes for pg_dump, including the ability to dump non-global default privileges correctly.
Use the CLDR project's data to map Windows time zone names to IANA time zones.
This update also contains tzdata release 2021e for DST law changes in Fiji, Jordan, Palestine, and Samoa, plus historical corrections for Barbados, Cook Islands, Guyana, Niue, Portugal, and Tonga.
Also, the Pacific/Enderbury zone has been renamed to Pacific/Kanton. Also, the following zones have been merged into nearby, more-populous zones whose clocks have agreed with them since 1970: Africa/Accra, America/Atikokan, America/Blanc-Sablon, America/Creston, America/Curacao, America/Nassau, America/Port_of_Spain, Antarctica/DumontDUrville, and Antarctica/Syowa. In all these cases, the previous zone name remains as an alias.
1.7.5
Released: November 11, 2021
bug
[bug] [tests]
Adjustments to the test suite to accommodate for error message changes occurring as of SQLAlchemy 1.4.27.
1.7.4
Released: October 6, 2021
bug
[bug] [regression]
Fixed a regression that prevented the use of post write hooks on python version lower than 3.9
[bug] [environment]
Fixed issue where the MigrationContext.autocommit_block() feature would fail to function when using a SQLAlchemy engine using 2.0 future mode.
1.7.3
Released: September 17, 2021
bug
[bug] [mypy]
Fixed type annotations for the “constraint_name” argument of operations create_primary_key(), create_foreign_key(). Pull request courtesy TilmanK.
1.7.2
Released: September 17, 2021
bug
[bug] [typing]
Added missing attributes from context stubs.
[bug] [mypy]
Fixed an import in one of the .pyi files that was triggering an assertion error in some versions of mypy.
[bug] [ops] [regression]
Fixed issue where registration of custom ops was prone to failure due to the registration process running exec() on generated code that as of the 1.7 series includes pep-484 annotations, which in the case of end user code would result in name resolution errors when the exec occurs. The logic in question has been altered so that the annotations are rendered as forward references so that the exec() can proceed.
1.7.1
Released: August 30, 2021
bug
[bug] [installation]
Corrected “universal wheel” directive in setup.cfg so that building a wheel does not target Python 2. The PyPi files index for 1.7.0 was corrected manually. Pull request courtesy layday.
[bug] [pep484]
Fixed issue in generated .pyi files where default values for Optional arguments were missing, thereby causing mypy to consider them as required.
[bug] [batch] [regression]
Fixed regression in batch mode due to 883 where the “auto” mode of batch would fail to accommodate any additional migration directives beyond encountering an add_column() directive, due to a mis-application of the conditional logic that was added as part of this change, leading to “recreate” mode not being used in cases where it is required for SQLite such as for unique constraints.
1.7.0
Released: August 30, 2021
changed
[changed] [installation]
Alembic 1.7 now supports Python 3.6 and above; support for prior versions including Python 2.7 has been dropped.
[changed] [installation]
Make the python-dateutil library an optional dependency. This library is only required if the timezone option is used in the Alembic configuration. An extra require named tz is available with pip install alembic[tz] to install it.
[changed] [installation]
The dependency on pkg_resources which is part of setuptools has been removed, so there is no longer any runtime dependency on setuptools. The functionality has been replaced with importlib.metadata and importlib.resources which are both part of Python std.lib, or via pypy dependency importlib-metadata for Python version < 3.8 and importlib-resources for Python version < 3.9 (while importlib.resources was added to Python in 3.7, it did not include the “files” API until 3.9).
feature
[feature] [environment]
Enhance version_locations parsing to handle paths containing spaces. The new configuration option version_path_separator specifies the character to use when splitting the version_locations string. The default for new configurations is version_path_separator = os, which will use os.pathsep (e.g., ; on Windows).
[feature] [tests]
Created a “test suite” similar to the one for SQLAlchemy, allowing developers of third-party dialects to test their code against a set of Alembic tests that have been specially selected to exercise back-end database operations. At the time of release, third-party dialects that have adopted the Alembic test suite to verify compatibility include CockroachDB and SAP ASE (Sybase).
[feature] [general]
pep-484 type annotations have been added throughout the library. Additionally, stub .pyi files have been added for the “dynamically” generated Alembic modules alembic.op and alembic.config, which include complete function signatures and docstrings, so that the functions in these namespaces will have both IDE support (vscode, pycharm, etc) as well as support for typing tools like Mypy. The files themselves are statically generated from their source functions within the source tree.
usecase
[usecase] [batch]
Named CHECK constraints are now supported by batch mode, and will automatically be part of the recreated table assuming they are named. They also can be explicitly dropped using op.drop_constraint(). For “unnamed” CHECK constraints, these are still skipped as they cannot be distinguished from the CHECK constraints that are generated by the Boolean and Enum datatypes.
Note that this change may require adjustments to migrations that drop or rename columns which feature an associated named check constraint, such that an additional op.drop_constraint() directive should be added for that named constraint as there will no longer be an associated column for it; for the Boolean and Enum datatypes, an existing_type keyword may be passed to BatchOperations.drop_constraint as well.
See also Changing the Type of Boolean, Enum and other implicit CHECK datatypes
Including CHECK constraints
bug
[bug] [operations]
Fixed regression due to 803 where the .info and .comment attributes of Table would be lost inside of the DropTableOp class, which when “reversed” into a CreateTableOp would then have lost these elements. Pull request courtesy Nicolas CANIART.
[bug] [batch] [sqlite]
Batch “auto” mode will now select for “recreate” if the add_column() operation is used on SQLite, and the column itself meets the criteria for SQLite where ADD COLUMN is not allowed, in this case a functional or parenthesized SQL expression or a Computed (i.e. generated) column.
[bug] [commands]
Re-implemented the python-editor dependency as a small internal function to avoid the need for external dependencies.
[bug] [postgresql]
Fixed issue where usage of the PostgreSQL postgresql_include option within a Operations.create_index() would raise a KeyError, as the additional column(s) need to be added to the table object used by the construct internally. The issue is equivalent to the SQL Server issue fixed in 513. Pull request courtesy Steven Bronson.
1.6.5
Released: May 27, 2021
bug
[bug] [autogenerate]
Fixed issue where dialect-specific keyword arguments within the DropIndex operation directive would not render in the autogenerated Python code. As support was improved for adding dialect specific arguments to directives as part of 803, in particular arguments such as “postgresql_concurrently” which apply to the actual create/drop of the index, support was needed for these to render even in a drop index operation. Pull request courtesy Jet Zhou.
1.6.4
Released: May 24, 2021
bug
[bug] [op directives] [regression]
Fixed regression caused by just fixed 844 that scaled back the filter for unique=True/index=True too far such that these directives no longer worked for the op.create_table() op, this has been fixed.
1.6.3
Released: May 21, 2021
bug
[bug] [autogenerate] [regression]
Fixed 1.6-series regression where UniqueConstraint and to a lesser extent Index objects would be doubled up in the generated model when the unique=True / index=True flags were used.
[bug] [autogenerate]
Fixed a bug where paths defined in post-write hook options would be wrongly escaped in non posix environment (Windows).
[bug] [regression] [versioning]
Fixed regression where a revision file that contained its own down revision as a dependency would cause an endless loop in the traversal logic.
1.6.2
Released: May 6, 2021
bug
[bug] [regression] [versioning]
Fixed additional regression nearly the same as that of 838 just released in 1.6.1 but within a slightly different codepath, where “alembic downgrade head” (or equivalent) would fail instead of iterating no revisions.
1.6.1
Released: May 6, 2021
bug
[bug] [regression] [versioning]
Fixed regression in new revisioning traversal where “alembic downgrade base” would fail if the database itself were clean and unversioned; additionally repairs the case where downgrade would fail if attempting to downgrade to the current head that is already present.
1.6.0
Released: May 3, 2021
feature
[feature] [autogenerate]
Fix the documentation regarding the default command-line argument position of the revision script filename within the post-write hook arguments. Implement a REVISION_SCRIPT_FILENAME token, enabling the position to be changed. Switch from str.split() to shlex.split() for more robust command-line argument parsing.
[feature]
Implement a .cwd (current working directory) suboption for post-write hooks (of type console_scripts). This is useful for tools like pre-commit, which rely on the working directory to locate the necessary config files. Add pre-commit as an example to the documentation. Minor change: rename some variables from ticket 819 to improve readability.
bug
[bug] [autogenerate]
Refactored the implementation of MigrateOperation constructs such as CreateIndexOp, CreateTableOp, etc. so that they no longer rely upon maintaining a persistent version of each schema object internally; instead, the state variables of each operation object will be used to produce the corresponding construct when the operation is invoked. The rationale is so that environments which make use of operation-manipulation schemes such as those those discussed in Fine-Grained Autogenerate Generation with Rewriters are better supported, allowing end-user code to manipulate the public attributes of these objects which will then be expressed in the final output, an example is some_create_index_op.kw["postgresql_concurrently"] = True.
Previously, these objects when generated from autogenerate would typically hold onto the original, reflected element internally without honoring the other state variables of each construct, preventing the public API from working.
[bug] [environment]
Fixed regression caused by the SQLAlchemy 1.4/2.0 compatibility switch where calling .rollback() or .commit() explicitly within the context.begin_transaction() context manager would cause it to fail when the block ended, as it did not expect that the transaction was manually closed.
[bug] [autogenerate]
Improved the rendering of op.add_column() operations when adding multiple columns to an existing table, so that the order of these statements matches the order in which the columns were declared in the application’s table metadata. Previously the added columns were being sorted alphabetically.
[bug] [versioning]
The algorithm used for calculating downgrades/upgrades/iterating revisions has been rewritten, to resolve ongoing issues of branches not being handled consistently particularly within downgrade operations, as well as for overall clarity and maintainability. This change includes that a deprecation warning is emitted if an ambiguous command such as “downgrade -1” when multiple heads are present is given.
In particular, the change implements a long-requested use case of allowing downgrades of a single branch to a branchpoint.
Huge thanks to Simon Bowly for their impressive efforts in successfully tackling this very difficult problem.
[bug] [batch] ¶
Added missing batch_op.create_table_comment(), batch_op.drop_table_comment() directives to batch ops.
DBD::MariaDB is the Perl5 Database Interface driver for MariaDB
and MySQL databases. In other words: DBD::MariaDB is an interface
between the Perl programming language and the MariaDB/MySQL
programming API that comes with the MariaDB/MySQL relational database
management system. Most functions provided by this programming API
are supported. Some rarely used functions are missing, mainly
because no-one ever requested them. :-)
=== 5.50.0 (2021-11-01)
* Make Migrator :allow_missing_migration_files also allow down migrations
where the current database version is greater than the last migration file
version (francisconeves97) (#1789)
* Fix Model#freeze in composition, serialization, and
serialization_modification_detection plugins to return self (jeremyevans)
(#1788)
* Fix typecasting of lazy columns when using lazy_attributes plugin in model
where dataset selects from subquery (jeremyevans)
* Add :before_preconnect Database option, for configuring extensions loaded
via :preconnect_extensions (MarcPer, jeremyevans) (#1786)
* Change Dataset#columns! to use a LIMIT 0 query instead of a LIMIT 1 query
(jeremyevans)
* Add sql_log_normalizer extension for normalizing logged SQL, helpful for
analytics and sensitive data (jeremyevans)
* Add support for range_merge, multirange, and unnest, and PGMultiRange#op
to pg_range_ops extension (jeremyevans)
* Add pg_multirange extension with support for PostgreSQL 14+ multirange
types (jeremyevans)
=== 5.49.0 (2021-10-01)
* Switch block_given? usage to defined?(yield) (jeremyevans)
* Support table aliases for JOIN USING columns on PostgreSQL 14+
(jeremyevans)
* Support calling PostgreSQL procedures without arguments (jeremyevans)
* Support hstore subscripts in pg_hstore_ops on PostgreSQL 14+, for updating
only part of an hstore value (jeremyevans)
* Support JSONB subscripts in pg_json_ops on PostgreSQL 14+, for updating
only part of a JSONB value (jeremyevans)
* Support SQL::Expression#sequel_ast_transform for custom AST transforms on
arbitrary expressions (jeremyevans)
* Add Database#create_trigger :replace option on PostgreSQL 14+ for CREATE
OR REPLACE TRIGGER (jeremyevans)
* Make auto_validations plugin automatically setup no_null_byte validations
(jeremyevans)
* Add Model#validates_no_null_byte to validation_helpers plugin
(jeremyevans)
A PostgreSQL SQL syntax beautifier that can work as a console program or as a
CGI.
This SQL formatter/beautifier supports keywords from SQL-92, SQL-99, SQL-2003,
SQL-2008, SQL-2011 and PostgreSQL specifics keywords. Works with any other
databases too.
Changes from the submitted version in the PR:
- p5 prefix for the package name
- clean up of github logic
- REPLACE_PERL
- updated distinfo with BLAKE2s
Version 1.22
* Fix file header validation
* Fix key verification in sequential access
* Fix testing with DejaGNU 1.6.3
* Fix stack overflow in print_usage
* Fix a leak of avail entry on pushing a new avail block
The leak would occur if the original avail table had odd number of entries.
* New gdbmtool variables: errorexit, errormask, trace, timing
"Errorexit" and "errormask" control which GDBM errors would cause the
program termination and emitting a diagnostic message,
correspondingly. Both variables are comma-delimited lists of error
codes.
The "trace" variable enables tracing of the gdbmtool commands.
The "timing" variable, when set, instructs gdbmtool to print time
spent in each command it runs.
* New gdbmtool options: -t (--trace), and -T (--timing)
3.14.8
Back-out all changes to automatically use RETURNING for `SqliteExtDatabase`,
`CSqliteExtDatabase` and `APSWDatabase`. The issue I found is that when a
RETURNING cursor is not fully-consumed, any parent SAVEPOINT (and possibly
transaction) would not be able to be released. Since this is a
backwards-incompatible change, I am going to back it out for now.
Returning clause can still be specified for Sqlite, however it just needs to be
done so manually rather than having it applied automatically.
3.14.7
Fix bug in APSW extension with Sqlite 3.35 and newer, due to handling of last
insert rowid with RETURNING.
3.14.6
Fix pesky bug in new `last_insert_id()` on the `SqliteExtDatabase`.
3.14.5
This release contains a number of bug-fixes and small improvements.
* Only raise `DoesNotExist` when `lazy_load` is enabled on ForeignKeyField.
* Add missing convenience method `ModelSelect.get_or_none()`
* Allow `ForeignKeyField` to specify a custom `BackrefAccessorClass`,
* Ensure foreign-key-specific conversions are applied on INSERT and UPDATE.
* Add handling of MySQL error 4031 (inactivity timeout) to the `ReconnectMixin`
helper class.
* Support specification of conflict target for ON CONFLICT/DO NOTHING.
* Add `encoding` parameter to the DataSet `freeze()` and `thaw()` methods.
* Fix bug which prevented `DeferredForeignKey` from being used as a model's
primary key.
* Ensure foreign key's related object cache is cleared when the foreign-key is
set to `None`.
* Allow specification of `(schema, table)` to be used with CREATE TABLE AS...,
* Allow reusing open connections with DataSet.
* Add `highlight()` and `snippet()` helpers to Sqlite `SearchField`, for use
with full-text search extension.
* Preserve user-provided aliases in column names.
* Add support for Sqlite 3.37 strict tables.
* Ensure database is inherited when using `ThreadSafeDatabaseMetadata`, and
also adds an implementation in `playhouse.shortcuts` along with basic unit
tests.
* Better handling of Model's dirty fields when saving.
* Add basic support for MariaDB connector driver in `playhouse.mysql_ext`.
* Begin a basic implementation for a psycopg3-compatible pg database.
* Add provisional support for RETURNING when using the appropriate versions of
Sqlite or MariaDB.
