1.1.0:
bug
[bug] [commands]
Fixed bug where the double-percent logic applied to some dialects such as psycopg2 would be rendered in --sql mode, by allowing dialect options to be passed through to the dialect used to generate SQL and then providing paramstyle="named" so that percent signs need not be doubled. For users having this issue, existing env.py scripts need to add dialect_opts={"paramstyle": "named"} to their offline context.configure(). See the alembic/templates/generic/env.py template for an example.
[bug] [py3k]
Fixed use of the deprecated “imp” module, which is used to detect pep3147 availability as well as to locate .pyc files, which started emitting deprecation warnings during the test suite. The warnings were not being emitted earlier during the test suite, the change is possibly due to changes in py.test itself but this is not clear. The check for pep3147 is set to True for any Python version 3.5 or greater now and importlib is used when available. Note that some dependencies such as distutils may still be emitting this warning. Tests are adjusted to accommodate for dependencies that emit the warning as well.
[bug] [mysql]
Fixed issue where emitting a change of column name for MySQL did not preserve the column comment, even if it were specified as existing_comment.
[bug] [setup]
Removed the “python setup.py test” feature in favor of a straight run of “tox”. Per Pypa / pytest developers, “setup.py” commands are in general headed towards deprecation in favor of tox. The tox.ini script has been updated such that running “tox” with no arguments will perform a single run of the test suite against the default installed Python interpreter.
usecase
[usecase] [commands]
The “alembic init” command will now proceed if the target directory exists as long as it’s still empty. Previously, it would not proceed if the directory existed. The new behavior is modeled from what git does, to accommodate for container or other deployments where an Alembic target directory may need to be already mounted instead of being created with alembic init. Pull request courtesy Aviskar KC.
misc
[change]
Alembic 1.1 bumps the minimum version of SQLAlchemy to 1.1. As was the case before, Python requirements remain at Python 2.7, or in the 3.x series Python 3.4.
[change] [internals]
The test suite for Alembic now makes use of SQLAlchemy’s testing framework directly. Previously, Alembic had its own version of this framework that was mostly copied from that of SQLAlchemy to enable testing with older SQLAlchemy versions. The majority of this code is now removed so that both projects can leverage improvements from a common testing framework.
1.3.8
orm
[orm] [bug]
Fixed bug where Load objects were not pickleable due to mapper/relationship state in the internal context dictionary. These objects are now converted to picklable using similar techniques as that of other elements within the loader option system that have long been serializable.
[orm] [usecase]
Added support for the use of an Enum datatype using Python pep-435 enumeration objects as values for use as a primary key column mapped by the ORM. As these values are not inherently sortable, as required by the ORM for primary keys, a new TypeEngine.sort_key_function attribute is added to the typing system which allows any SQL type to implement a sorting for Python objects of its type which is consulted by the unit of work. The Enum type then defines this using the database value of a given enumeration. The sorting scheme can be also be redefined by passing a callable to the Enum.sort_key_function parameter. Pull request courtesy Nicolas Caniart.
engine
[engine] [feature]
Added new parameter create_engine.hide_parameters which when set to True will cause SQL parameters to no longer be logged, nor rendered in the string representation of a StatementError object.
[engine] [bug]
Fixed an issue whereby if the dialect “initialize” process which occurs on first connect would encounter an unexpected exception, the initialize process would fail to complete and then no longer attempt on subsequent connection attempts, leaving the dialect in an un-initialized, or partially initialized state, within the scope of parameters that need to be established based on inspection of a live connection. The “invoke once” logic in the event system has been reworked to accommodate for this occurrence using new, private API features that establish an “exec once” hook that will continue to allow the initializer to fire off on subsequent connections, until it completes without raising an exception. This does not impact the behavior of the existing once=True flag within the event system.
postgresql
[postgresql] [bug]
Revised the approach for the just added support for the psycopg2 “execute_values()” feature added in 1.3.7. The approach relied upon a regular expression that would fail to match for a more complex INSERT statement such as one which had subqueries involved. The new approach matches exactly the string that was rendered as the VALUES clause.
[postgresql] [bug]
Fixed bug where Postgresql operators such as postgresql.ARRAY.Comparator.contains() and postgresql.ARRAY.Comparator.contained_by() would fail to function correctly for non-integer values when used against a postgresql.array object, due to an erroneous assert statement.
[postgresql] [usecase]
Added support for reflection of CHECK constraints that include the special PostgreSQL qualifier “NOT VALID”, which can be present for CHECK constraints that were added to an exsiting table with the directive that they not be applied to existing data in the table. The PostgreSQL dictionary for CHECK constraints as returned by Inspector.get_check_constraints() may include an additional entry dialect_options which within will contain an entry "not_valid": True if this symbol is detected. Pull request courtesy Bill Finn.
sqlite
[sqlite] [bug] [reflection]
Fixed bug where a FOREIGN KEY that was set up to refer to the parent table by table name only without the column names would not correctly be reflected as far as setting up the “referred columns”, since SQLite’s PRAGMA does not report on these columns if they weren’t given explicitly. For some reason this was harcoded to assume the name of the local column, which might work for some cases but is not correct. The new approach reflects the primary key of the referred table and uses the constraint columns list as the referred columns list, if the remote column(s) aren’t present in the reflected pragma directly.
1.3.7
Released: August 14, 2019
orm
[orm] [bug]
Fixed regression caused by new selectinload for many-to-one logic where a primaryjoin condition not based on real foreign keys would cause KeyError if a related object did not exist for a given key value on the parent object.
[orm] [bug]
Fixed bug where using Query.first() or a slice expression in conjunction with a query that has an expression based “offset” applied would raise TypeError, due to an “or” conditional against “offset” that did not expect it to be a SQL expression as opposed to an integer or None.
sql
[sql] [bug]
Fixed issue where Index object which contained a mixture of functional expressions which were not resolvable to a particular column, in combination with string-based column names, would fail to initialize its internal state correctly leading to failures during DDL compilation.
[sql] [bug]
Fixed bug where TypeEngine.column_expression() method would not be applied to subsequent SELECT statements inside of a UNION or other CompoundSelect, even though the SELECT statements are rendered at the topmost level of the statement. New logic now differentiates between rendering the column expression, which is needed for all SELECTs in the list, vs. gathering the returned data type for the result row, which is needed only for the first SELECT.
[sql] [bug]
Fixed issue where internal cloning of SELECT constructs could lead to a key error if the copy of the SELECT changed its state such that its list of columns changed. This was observed to be occurring in some ORM scenarios which may be unique to 1.3 and above, so is partially a regression fix.
postgresql
[postgresql] [usecase]
Added new dialect flag for the psycopg2 dialect, executemany_mode which supersedes the previous experimental use_batch_mode flag. executemany_mode supports both the “execute batch” and “execute values” functions provided by psycopg2, the latter which is used for compiled insert() constructs. Pull request courtesy Yuval Dinari.
See also
Psycopg2 Fast Execution Helpers
mysql
[mysql] [bug]
The MySQL dialects will emit “SET NAMES” at the start of a connection when charset is given to the MySQL driver, to appease an apparent behavior observed in MySQL 8.0 that raises a collation error when a UNION includes string columns unioned against columns of the form CAST(NULL AS CHAR(..)), which is what SQLAlchemy’s polymorphic_union function does. The issue seems to have affected PyMySQL for at least a year, however has recently appeared as of mysqlclient 1.4.4 based on changes in how this DBAPI creates a connection. As the presence of this directive impacts three separate MySQL charset settings which each have intricate effects based on their presense, SQLAlchemy will now emit the directive on new connections to ensure correct behavior.
[mysql] [bug]
Added another fix for an upstream MySQL 8 issue where a case sensitive table name is reported incorrectly in foreign key constraint reflection, this is an extension of the fix first added for 4344 which affects a case sensitive column name. The new issue occurs through MySQL 8.0.17, so the general logic of the 88718 fix remains in place.
See also
https://bugs.mysql.com/bug.php?id=96365 - upstream bug
[mysql] [usecase]
Added reserved words ARRAY and MEMBER to the MySQL reserved words list, as MySQL 8.0 has now made these reserved.
sqlite
[sqlite] [bug]
The dialects that support json are supposed to take arguments json_serializer and json_deserializer at the create_engine() level, however the SQLite dialect calls them _json_serilizer and _json_deserilalizer. The names have been corrected, the old names are accepted with a change warning, and these parameters are now documented as create_engine.json_serializer and create_engine.json_deserializer.
[sqlite] [bug]
Fixed bug where usage of “PRAGMA table_info” in SQLite dialect meant that reflection features to detect for table existence, list of table columns, and list of foreign keys, would default to any table in any attached database, when no schema name was given and the table did not exist in the base schema. The fix explicitly runs PRAGMA for the ‘main’ schema and then the ‘temp’ schema if the ‘main’ returned no rows, to maintain the behavior of tables + temp tables in the “no schema” namespace, attached tables only in the “schema” namespace.
mssql
[mssql] [usecase]
Added new mssql.try_cast() construct for SQL Server which emits “TRY_CAST” syntax. Pull request courtesy Leonel Atencio.
misc
[bug] [events]
Fixed issue in event system where using the once=True flag with dynamically generated listener functions would cause event registration of future events to fail if those listener functions were garbage collected after they were used, due to an assumption that a listened function is strongly referenced. The “once” wrapped is now modified to strongly reference the inner function persistently, and documentation is updated that using “once” does not imply automatic de-registration of listener functions.
1.3.6
orm
[orm] [feature]
Added new loader option method Load.options() which allows loader options to be constructed hierarchically, so that many sub-options can be applied to a particular path without needing to call defaultload() many times. Thanks to Alessio Bogon for the idea.
[orm] [bug]
Fixed regression caused by 4365 where a join from an entity to itself without using aliases no longer raises an informative error message, instead failing on an assertion. The informative error condition has been restored.
[orm] [bug]
Fixed an issue where the orm._ORMJoin.join() method, which is a not-internally-used ORM-level method that exposes what is normally an internal process of Query.join(), did not propagate the full and outerjoin keyword arguments correctly. Pull request courtesy Denis Kataev.
[orm] [bug]
Fixed bug where a many-to-one relationship that specified uselist=True would fail to update correctly during a primary key change where a related column needs to change.
[orm] [bug]
Fixed bug where the detection for many-to-one or one-to-one use with a “dynamic” relationship, which is an invalid configuration, would fail to raise if the relationship were configured with uselist=True. The current fix is that it warns, instead of raises, as this would otherwise be backwards incompatible, however in a future release it will be a raise.
[orm] [bug]
Fixed bug where a synonym created against a mapped attribute that does not exist yet, as is the case when it refers to backref before mappers are configured, would raise recursion errors when trying to test for attributes on it which ultimately don’t exist (as occurs when the classes are run through Sphinx autodoc), as the unconfigured state of the synonym would put it into an attribute not found loop.
[orm] [performance]
The optimization applied to selectin loading in 4340 where a JOIN is not needed to eagerly load related items is now applied to many-to-one relationships as well, so that only the related table is queried for a simple join condition. In this case, the related items are queried based on the value of a foreign key column on the parent; if these columns are deferred or otherwise not loaded on any of the parent objects in the collection, the loader falls back to the JOIN method.
engine
[engine] [bug]
Fixed bug where using reflection function such as MetaData.reflect() with an Engine object that had execution options applied to it would fail, as the resulting OptionEngine proxy object failed to include a .engine attribute used within the reflection routines.
sql
[sql] [bug]
Adjusted the initialization for Enum to minimize how often it invokes the .__members__ attribute of a given PEP-435 enumeration object, to suit the case where this attribute is expensive to invoke, as is the case for some popular third party enumeration libraries.
[sql] [bug] [postgresql]
oduce the correct operator precedence in combination with the array index operator.
[sql] [bug]
Fixed an unlikely issue where the “corresponding column” routine for unions and other CompoundSelect objects could return the wrong column in some overlapping column situtations, thus potentially impacting some ORM operations when set operations are in use, if the underlying select() constructs were used previously in other similar kinds of routines, due to a cached value not being cleared.
postgresql
[postgresql] [usecase]
Added support for reflection of indexes on PostgreSQL partitioned tables, which was added to PostgreSQL as of version 11.
[postgresql] [usecase]
Added support for multidimensional Postgresql array literals via nesting the postgresql.array object within another one. The multidimensional array type is detected automatically.
See also
postgresql.array
mysql
[mysql] [bug]
Fixed bug where the special logic to render “NULL” for the TIMESTAMP datatype when nullable=True would not work if the column’s datatype were a TypeDecorator or a Variant. The logic now ensures that it unwraps down to the original TIMESTAMP so that this special case NULL keyword is correctly rendered when requested.
[mysql] [bug]
Enhanced MySQL/MariaDB version string parsing to accommodate for exotic MariaDB version strings where the “MariaDB” word is embedded among other alphanumeric characters such as “MariaDBV1”. This detection is critical in order to correctly accommodate for API features that have split between MySQL and MariaDB such as the “transaction_isolation” system variable.
sqlite
[sqlite] [usecase]
Added support for composite (tuple) IN operators with SQLite, by rendering the VALUES keyword for this backend. As other backends such as DB2 are known to use the same syntax, the syntax is enabled in the base compiler using a dialect-level flag tuple_in_values. The change also includes support for “empty IN tuple” expressions for SQLite when using “in_()” between a tuple value and an empty set.
mssql
[mssql] [bug]
Ensured that the queries used to reflect indexes and view definitions will explicitly CAST string parameters into NVARCHAR, as many SQL Server drivers frequently treat string values, particularly those with non-ascii characters or larger string values, as TEXT which often don’t compare correctly against VARCHAR characters in SQL Server’s information schema tables for some reason. These CAST operations already take place for reflection queries against SQL Server information_schema. tables but were missing from three additional queries that are against sys. tables.
Version 3.9.1:
Bugfix for pg_error_field: make sure we do not feed null to newSVpv,
handle older versions of Postgres better.
Version 3.9.0:
- ShowErrorStatement works for "quickexec" do() calls
- Add :pg_limits to add constants such as PG_MAX_SMALLINT
- Add $dbh->pg_error_field() function
- Fix failing tests due to incorrect 'initdb' check
Version 3.8.1:
- Fix encoding of SQL_VARBINARY type in $dbh->quote() function
- Fix encoding in $dbh->do() function
- Fix E'' string escape handling on architectures with unsigned chars
- Minor fix to allow DBD::Pg to connect to PGbouncer's internal 'pgbouncer' database.
- Fix so table_info test works on non-empty databases
Version 3.8.0:
- Increase minimum supported PostgreSQL version to 8.0
- Add support for foreign tables in table_info() and column_info()
- Return the current database name as TABLE_CAT in info methods
- Handle backslash-escaped quotes in E'' strings
- Fix typo in Makefile.PL
- Fix parsing of PostgreSQL versions >= 10 on Debian/Ubuntu
- Fix client_min_messages=FATAL test when PostgreSQL caps it to ERROR
- Fix ->ping error detection on PostgreSQL 12
- Adjust tests for new pg_ctl output
- Adjust tests for removal of WITH OIDS in PostgreSQL 12
- Fix support for PostgreSQL versions back to 8.0
- Remove usage of deprecated pg_attrdef.adsrc and pg_constraint.consrc columns
- Fix typo in pg_placeholder_colons example
- Support GENERATED ... AS IDENTITY columns in last_insert_id()
* 3.3.8
* Fixed MONITOR parsing to properly parse IPv6 client addresses, unix
socket connections and commands issued from Lua.
* 3.3.7
* Fixed a regression introduced in 3.3.0 where socket.error exceptions
(or subclasses) could potentially be raised instead of
redis.exceptions.ConnectionError.
* 3.3.6
* Fixed a regression in 3.3.5 that caused PubSub.get_message() to raise
a socket.timeout exception when passing a timeout value.
* 3.3.5
* Fix an issue where socket.timeout errors could be handled by the wrong
exception handler in Python 2.7.
* 3.3.4
* More specifically identify nonblocking read errors for both SSL and
non-SSL connections. 3.3.1, 3.3.2 and 3.3.3 on Python 2.7 could
potentially mask a ConnectionError.
* 3.3.3
* The SSL module in Python < 2.7.9 handles non-blocking sockets
differently than 2.7.9+. This patch accommodates older versions
* 3.3.2
* Further fixed a regression introduced in 3.3.0 involving SSL and
non-blocking sockets.
* 3.3.1
* Fixed a regression introduced in 3.3.0 involving SSL and non-blocking
sockets.
* 3.3.0
* Resolve a race condition with the PubSubWorkerThread.
* Cleanup socket read error messages.
* Cleanup the Connection's selector correctly.
* Added a Monitor object to make working with MONITOR output easy.
* Internal cleanup: Removed the legacy Token class which was necessary
with older version of Python that are no longer supported.
* Response callbacks are now case insensitive. This allows users that
call Redis.execute_command() directly to pass lower-case command
names and still get reasonable responses.
* Added support for hiredis-py 1.0.0 encoding error support. This should
make the PythonParser and the HiredisParser behave identically
when encountering encoding errors.
* All authentication errors now properly raise AuthenticationError.
AuthenticationError is now a subclass of ConnectionError, which will
cause the connection to be disconnected and cleaned up appropriately.
* Add READONLY and READWRITE commands.
* Remove selectors in favor of nonblocking sockets. Selectors had
issues in some environments including eventlet and gevent. This should
resolve those issues with no other side effects.
* Fixed an issue with XCLAIM and previously claimed but not removed
messages.
* Allow for single connection client instances. These instances
are not thread safe but offer other benefits including a subtle
performance increase.
* Added extensive health checks that keep the connections lively.
Passing the "health_check_interval=N" option to the Redis client class
or to a ConnectionPool ensures that a round trip PING/PONG is successful
before any command if the underlying connection has been idle for more
than N seconds. ConnectionErrors and TimeoutErrors are automatically
retried once for health checks.
* Changed the PubSubWorkerThread to use a threading.Event object rather
than a boolean to control the thread's life cycle.
* Fixed a bug in Pipeline error handling that would incorrectly retry
ConnectionErrors.
v0.5.1:
* Add application_name as a label for pg_stat_replication metrics
v0.5.0:
It's been far too long since I've done a release and we have a lot of accumulated changes.
* Docker image now runs as a non-root user named "postgres_exporter"
* Add `--auto-discover-databases` option, which automatically discovers and scrapes all databases.
* Add support for boolean data types as metrics
* Replication lag is now expressed as a float and not truncated to an integer.
* When default metrics are disabled, no version metrics are collected anymore either.
* BUGFIX: Fix exporter panic when postgres server goes down.
* Add support for collecting metrics from multiple servers.
* PostgreSQL 11 is now supported in the integration tests.
2.12.0:
[FEATURE] Track currently active PromQL queries in a log file.
[FEATURE] Enable and provide binaries for mips64 / mips64le architectures.
[ENHANCEMENT] Improve responsiveness of targets web UI and API endpoint.
[ENHANCEMENT] Improve remote write desired shards calculation.
[ENHANCEMENT] Flush TSDB pages more precisely.
[ENHANCEMENT] Add prometheus_tsdb_retention_limit_bytes metric.
[ENHANCEMENT] Add logging during TSDB WAL replay on startup.
[ENHANCEMENT] Improve TSDB memory usage.
[BUGFIX] Check for duplicate label names in remote read.
[BUGFIX] Mark deleted rules' series as stale on next evaluation.
[BUGFIX] Fix JavaScript error when showing warning about out-of-sync server time.
[BUGFIX] Fix promtool test rules panic when providing empty exp_labels.
[BUGFIX] Only check last directory when discovering checkpoint number.
[BUGFIX] Fix error propagation in WAL watcher helper functions.
[BUGFIX] Correctly handle empty labels from alert templates.
2.11.0:
[CHANGE] Remove max_retries from queue_config (it has been unused since rewriting remote-write to utilize the write-ahead-log).
[CHANGE] The meta file BlockStats no longer holds size information. This is now dynamically calculated and kept in memory. It also includes the meta file size which was not included before.
[CHANGE] Renamed metric from prometheus_tsdb_wal_reader_corruption_errors to prometheus_tsdb_wal_reader_corruption_errors_total.
[FEATURE] Add option to use Alertmanager API v2.
[FEATURE] Added humanizePercentage function for templates.
[FEATURE] Include InitContainers in Kubernetes Service Discovery.
[FEATURE] Provide option to compress WAL records using Snappy.
[ENHANCEMENT] Create new clean segment when starting the WAL.
[ENHANCEMENT] Reduce allocations in PromQL aggregations.
[ENHANCEMENT] Add storage warnings to LabelValues and LabelNames API results.
[ENHANCEMENT] Add prometheus_http_requests_total metric.
[ENHANCEMENT] Enable openbsd/arm build.
[ENHANCEMENT] Remote-write allocation improvements.
[ENHANCEMENT] Query performance improvement: Efficient iteration and search in HashForLabels and HashWithoutLabels.
[ENHANCEMENT] Allow injection of arbitrary headers in promtool.
[ENHANCEMENT] Allow passing external_labels in alert unit tests groups.
[ENHANCEMENT] Allows globs for rules when unit testing.
[ENHANCEMENT] Improved postings intersection matching.
[ENHANCEMENT] Reduced disk usage for WAL for small setups.
[ENHANCEMENT] Optimize queries using regexp for set lookups.
[BUGFIX] resolve race condition in maxGauge.
[BUGFIX] Fix ZooKeeper connection leak.
[BUGFIX] Improved atomicity of .tmp block replacement during compaction for usual case.
[BUGFIX] Fix "unknown series references" after clean shutdown.
[BUGFIX] Re-calculate block size when calling block.Delete.
[BUGFIX] Fix unsafe snapshots with head block.
[BUGFIX] prometheus_tsdb_compactions_failed_total is now incremented on any compaction failure.
Changes since 1.9.0
2019-07-01 - PgBouncer 1.10.0 - "Afraid of the World"
Features
Add support for enabling and disabling TLS 1.3. (TLS 1.3 was
already supported, depending on the OpenSSL library, but now the
configuration settings to pick the TLS protocol versions also
support it.)
Fixes
Fix TLS 1.3 support. This was broken with OpenSSL 1.1.1 and
1.1.1a (but not before or after).
Fix a rare crash in SHOW FDS
(https://github.com/pgbouncer/pgbouncer/issues/311/).
Fix an issue that could lead to prolonged downtime if many cancel
requests arrive
(https://github.com/pgbouncer/pgbouncer/issues/329/).
Avoid "unexpected response from login query" after a postgres
reload
(https://github.com/pgbouncer/pgbouncer/issues/220/).
Fix idle_transaction_timeout calculation
(https://github.com/pgbouncer/pgbouncer/issues/125/). The
bug would lead to premature timeouts in specific situations.
Cleanups
Make various log and error messages more precise.
Fix issues found by Coverity (none had a significant impact in
practice).
Improve and document all test scripts.
Add additional SHOW commands to the documentation.
Convert the documentation from rst to Markdown.
Python scripts in the source tree are all compatible with Python 3
now.
1.1.5:
Python 3.x support fixes for whisper, carbon and graphite-web
REMOTE_BUFFER_SIZE = 0 now works properly
Multi-instance settings support
Increase performance of localdatabase tag db
avg_zero consolidation function fully supported now
Carbonlink queries working now for tagged series
Docker image migrated to Alpine, reducing the image size from ~530MB to 188MB
PostgreSQL 11.5, 10.10, 9.6.15, 9.5.19, 9.4.24
Security Issues
Four security vulnerabilities have been closed by this release:
CVE-2019-10208: TYPE in pg_temp executes arbitrary SQL during SECURITY DEFINER execution
Versions Affected: 9.4 - 11
Given a suitable SECURITY DEFINER function, an attacker can execute arbitrary SQL under the identity of the function owner. An attack requires EXECUTE permission on the function, which must itself contain a function call having inexact argument type match. For example, length('foo'::varchar) and length('foo') are inexact, while length('foo'::text) is exact. As part of exploiting this vulnerability, the attacker uses CREATE DOMAIN to create a type in a pg_temp schema. The attack pattern and fix are similar to that for CVE-2007-2138.
Writing SECURITY DEFINER functions continues to require
following the considerations noted in the documentation:
The PostgreSQL project thanks Tom Lane for reporting this problem.
CVE-2019-10209: Memory disclosure in cross-type comparison for hashed subplan
Versions Affected: 11
In a database containing hypothetical, user-defined hash equality operators, an attacker could read arbitrary bytes of server memory. For an attack to become possible, a superuser would need to create unusual operators. It is possible for operators not purpose-crafted for attack to have the properties that enable an attack, but we are not aware of specific examples.
The PostgreSQL project thanks Andreas Seltenreich for reporting this problem.
CVE-2019-10210: EnterpriseDB Windows installer writes PostgreSQL superuser password to unprotected temporary file
Versions Affected: The EnterpriseDB Windows installer for versions 9.4 - 11
The EnterpriseDB Windows installer writes a password to a temporary file in its installation directory, creates initial databases, and deletes the file. During those seconds while the file exists, a local attacker can read the PostgreSQL superuser password from the file.
The PostgreSQL project thanks Noah Misch for reporting this problem.
CVE-2019-10211: EnterpriseDB Windows installer bundled OpenSSL executes code from unprotected directory
Versions Affected: The EnterpriseDB Windows installer for versions 9.4 - 11
When the database server or libpq client library initializes SSL, libeay32.dll attempts to read configuration from a hard-coded directory. Typically, the directory does not exist, but any local user could create it and inject configuration. This configuration can direct OpenSSL to load and execute arbitrary code as the user running a PostgreSQL server or client. Most PostgreSQL client tools and libraries use libpq, and one can encounter this vulnerability by using any of them. This vulnerability is much like CVE-2019-5443, but it originated independently. One can work around the vulnerability by setting environment variable OPENSSL_CONF to "NUL:/openssl.cnf" or any other name that cannot exist as a file.
The PostgreSQL project thanks Daniel Gustafsson of the curl security team for reporting this problem.
Bug Fixes and Improvements
This update also fixes over 40 bugs that were reported in the last several months. Some of these issues affect only version 11, but many affect all supported versions.
Some of these fixes include:
Fix for ALTER TABLE ... ALTER COLUMN TYPE when multiple column types are modified in a single-command. This issue was introduced in the previous cumulative update (11.4, 10.9, 9.6.14, 9.5.18, 9.4.23, and 12 beta 2).
Ensure that partition key columns will not be dropped as the result of an "indirect drop," such as from a cascade from dropping the key column's data type (e.g. a custom data type). This fix is applied only to newly created partitioned tables: if you believe you have an affected partition table (e.g. one where the partition key uses a custom data type), you will need to either create a new table and move your data into it OR use pg_upgrade.
Prevent dropping a partitioned table's trigger if there are pending trigger events in child partitions. This particularly affects foreign key constraints, which are implemented by triggers.
Several additional fixes for partitioning, including a fix for partition pruning that could lead to inefficient queries.
Fix for parallel hash joins that could lead to duplicate result rows in EXISTS queries.
Several fixes for the query planner.
Several fixes for issues that would lead to query deadlocks.
Fix for multi-column foreign keys when rebuilding a foreign key constraint.
Prevent extended statistics from being built for inherited tables.
Fix for the canonicalization of date ranges that include -infinity/infinity endpoints to ensure the behavior matches the documentation.
Fix loss of fractional digits when converting very large money values to numeric.
Fix for PL/pgSQL functions that return composite types.
Make libpq ignore the \r carriage return in connection service files, which was causing connection failures in some edge cases.
Several fixes for psql, which includes avoiding incorrect tab completion options after SET variable =.
Improve reliability of contrib/amcheck's index verification.
Set initdb to prefer the timezone behavior defined by the C library instead of what is defined by localtime or posixrules. This ensures PostgreSQL uses the "real" timezone name instead of an artificial name.
Fix pg_dump to ensure that custom operator classes are dumped in the correct order to prevent creating an unrestorable dump.
Fix possible lockup in pgbench when using -R option.
Fix spinlock assembly code for MIPS CPUs so that it works on MIPS r6.
This update also contains tzdata release 2019b for DST law changes in Brazil, plus historical corrections for Hong Kong, Italy, and Palestine. This update also adds support for zic's new -b slim option to reduce the size of the installed zone files, though it is not currently being used by PostgreSQL.
The canonical form [1] of an R package Makefile includes the
following:
- The first stanza includes R_PKGNAME, R_PKGVER, PKGREVISION (as
needed), and CATEGORIES.
- HOMEPAGE is not present but defined in math/R/Makefile.extension to
refer to the CRAN web page describing the package. Other relevant
web pages are often linked from there via the URL field.
This updates all current R packages to this form, which will make
regular updates _much_ easier, especially using pkgtools/R2pkg.
[1] http://mail-index.netbsd.org/tech-pkg/2019/08/02/msg021711.html
Changes in MySQL 5.7.27:
Keyring Notes
The keyring_aws plugin has been updated to use the latest AWS SDK and so that it works with OpenSSL 1.1.
Packaging Notes
Binary packages that include curl rather than linking to the system curl library now use curl 7.64.0.
X Plugin Notes
On Windows, X Plugin logged some messages that were unnecessary or insufficiently informative. The messages have been removed or improved as appropriate.
Functionality Added or Changed
Microsoft Windows: A new warning message now reminds DBAs that connections made using the MySQL named pipe on Windows has limited the permissions a connector can request on the named pipe.
Previously, the named_pipe_full_access_group system variable was set to a value that maps to the built-in Windows Everyone group (SID S-1-1-0) by default. However, this group is not ideal and should be replaced with a group that restricts its membership for connectors that are unable to request fewer permissions on the MySQL named pipe.
The new warning is written to the error log at startup if the string value assigned to named_pipe_full_access_group is '*everyone*' (or the Windows System Language equivalent) and named pipes are enabled. In addition, the warning is written to the error log and raised to the client if the system variable is reset to the Everyone group at runtime.
Bugs Fixed
InnoDB: Insufficient memory barriers in the rw-lock implementation caused deadlocks on ARM.
Thanks to Yibo Cai from Arm Technology for the contribution.
InnoDB: Manually changing the system time while the MySQL server was running caused page cleaner thread delays.
InnoDB: During log application, after an OPTIMIZE TABLE operation, InnoDB did not populate virtual columns before checking for virtual column index updates.
InnoDB: An INSERT operation involving a generated virtual BLOB column resulted a secondary index being updated with an incorrect value.
InnoDB: A full-text cache lock taken when data is synchronized was not released if the full-text cache size exceeded the full-text cache size limit.
InnoDB: Client sessions using different auto_increment_increment values while performing concurrent insert operations could cause a duplicate key error.
Partitioning: ALTER TABLE ... EXCHANGE PARTITION failed with the error Non matching attribute 'ROW_FORMAT' between partition and table when the partitioned table had partitions using different row formats, even when the partition to be exchanged used the same row format as the non-partitioned table.
Replication: The error message that is issued for a discrepancy between the number of group members and the auto-increment interval incorrectly referred to the group_replication_auto_increment_increment system variable, instead of the auto_increment_increment system variable. The value of auto_increment_increment is changed to the value specified by group_replication_auto_increment_increment when Group Replication starts, but only if auto_increment_increment and auto_increment_offset have their default values, and from MySQL 8.0, only in multi-primary mode. The value of auto_increment_increment was always the value that was checked for the error message, and it has now been corrected to give the accurate system variable name.
Replication: When events generated by one MySQL server instance were written to the binary log of another instance, the second server implicitly assumed that the first server supported the same number of binary log event types as itself. Where this was not the case, the event header was handled incorrectly. The issue has now been fixed. Thanks to Facebook for the contribution.
Replication: In Group Replication, joining members could wrongly identify themselves as incompatible with an existing replication group even if there were members at the same version already in the group, because they checked against all other members, including the member at the highest version. Joining members also included their own version in the compatibility check. Now, joining members only compare themselves with the existing group member at the lowest version, and do not count their own version.
Replication: If a FLUSH LOGS statement was issued before the binary log file was initialized, the statement attempted to write a binary log rotation event to the uninitialized file. The server now checks first that a binary log file is available.
Replication: When a MEMORY table is implicitly deleted on a master following a server restart, the master writes a DELETE statement to the binary log so that slaves also empty the table. This generated event now includes a comment in the binary log so that the reason for the DELETE statement is easy to identify. Thanks to Daniël van Eeden for the contribution.
Replication: With statement-based replication in use, if super_read_only was set to ON for a server at the point when a no-op transaction was between its UPDATE and COMMIT operations, the transaction was written to the binary log and assigned a GTID. The transaction is now blocked in this situation. From MySQL 8.0, the value of super_read_only cannot be changed while a transaction is in progress.
Replication: The group communication engine for Group Replication (XCom, a Paxos variant) did not handle out of memory errors in an appropriate way. If memory could not be allocated to make a copy of the payload for a message, an error was logged but the message was still sent, with a null payload. The Group Communication System (GCS) on the receiving member discarded the message as empty, and the XCom instance on the receiving member accepted this action and did not retry, resulting in the message effectively being skipped. This caused the GTID set on the receiving member to diverge from the group, leading to replication errors. XCom now terminates gracefully if it experiences an out of memory error, so that this situation cannot occur.
Replication: In query log events in the binary log, the thread ID used for the execution of DROP TABLE and DELETE statements was identified incorrectly or not at all. On a multi-threaded replication slave, where temporary tables were involved (which require the correct thread ID as they are session specific), this omission resulted in errors when using mysqlbinlog to replay the binary log for point-in-time recovery. The thread ID is now set correctly.
Replication: When a slave server logs master status and connection information to a table (master_info_repository=TABLE), which is the default in MySQL 8.0, the mysql.slave_master_info table was not being updated on shutdown if the server was in super read only mode (super_read_only=ON). No error was written to the error log at this time, but replication failed after server startup because the master log file and master log position information was out of date. The thread that updates the master info log at shutdown is now excluded from read-only checks like other replication threads are, so it can update the table even if the server is in super read only mode. Error handling for a slave that is shutting down has also been improved so that any failure to write to the slave status logs results in an error in the error log.
An overly strict assertion could be raised during sorting of stored program local objects.
Installing from RPM packages could result in an error log with incorrect permissions.
Enabling audit log encryption could cause a server exit.
On Debian and Ubuntu, MySQL packages did not enable mysql.service after upgrades from native MySQL packages.
The server did not properly close shared-memory connections when an error occurred, which could result in unexpected server behavior.
MySQL Installer did not install OpenSSL DLL dependencies if the Development component was not selected.
The parser could leak memory for certain multiple-statement queries.
CREATE USER and ALTER USER did not check the validity of a hashed authentication string when used with IDENTIFIED WITH auth_plugin AS 'hash_string' syntax.
For InnoDB tables that contained an index on a VARCHAR column and were created prior to MySQL 5.7.23, some simple ALTER TABLE statements that should have been done in place were performed with a table rebuild after an upgrade to MySQL 5.7.23 or higher.
HANDLER statements did not always work correctly with tables having generated columns.
Session-tracking information in the client/server protocol could be mishandled.
With the PAD_CHAR_TO_FULL_LENGTH SQL mode enabled, password changes failed, with no warning or error reported.
The audit_log plugin did not log UNINSTALL PLUGIN audit_log statements.
audit_log filtering operations could leak memory.
An index defined on a virtual generated column could fail to be updated if the column had a base column in a foreign key relationship.
Privileges for dropping some Performance Schema tables were checked incorrectly.
A query that employed a derived table which included an ORDER BY was not always handled correctly.
Base columns were not excluded from index-only access by a generated column.
A thread pool group could be blocked when a thread process tick time exceeded the maximum permitted value. The tick time now uses a larger data type to permit larger values.
MySQL does not support OpenSSL session tickets, but did not set the SSL_OP_NO_TICKET flag to inform OpenSSL of that. The flag is now set.
The audit_null plugin did not properly check for a null event record.
UpdateXML() did not always free memory properly in certain cases.
Empty values in the name column of the mysql.plugin system table caused the server to exit during startup.
With the thread_pool plugin enabled, the Performance Schema status_by_thread table contained no data.
If an INSTALL PLUGIN statement contained invalid UTF-8 characters in the shared library name, it caused the server to hang (or to raise an assertion in debug builds).
Inner tables of different semijoin nests were interleaved during materialization, which could lead to a different result for the same query when it used a different query plan. To keep this from occurring, a check is added to prevent such interleaving.
A query involving GROUP BY on a TIMESTAMP column resulted in a duplicate entry for key (ER_DUP_ENTRY) error. This problem arose when TIMESTAMP values were inserted into a table using a given setting for the time zone and these values were later fetched after the time zone setting had been changed, such that at least some of the inserted TIMESTAMP values occurred during the hour that the time changed from standard to daylight time (DST) in the new time zone, during which time the same TIMESTAMP value can exist twice. Now, when the server would otherwise return the error DUPLICATE ENTRY FOR KEY 'group_key', if the grouping involves a TIMESTAMP column, it instead raises the error Grouping on temporal is non-deterministic for time zones having DST. Please consider switching to UTC for this query.
In addition, it is suggested to set explicit_defaults_for_timestamp to ON as well as one or more of MODE_NO_ZERO_IN_DATE, MODE_NO_ZERO_DATE, or MODE_INVALID_DATES as part of the server SQL mode to help avoid this issue.
Changes in MySQL 5.6.45:
Functionality Added or Changed
Microsoft Windows: A new warning message now reminds DBAs that connections made using the MySQL named pipe on Windows has limited the permissions a connector can request on the named pipe.
Previously, the named_pipe_full_access_group system variable was set to a value that maps to the built-in Windows Everyone group (SID S-1-1-0) by default. However, this group is not ideal and should be replaced with a group that restricts its membership for connectors that are unable to request fewer permissions on the MySQL named pipe.
The new warning is written to the error log at startup if the string value assigned to named_pipe_full_access_group is '*everyone*' (or the Windows System Language equivalent) and named pipes are enabled. In addition, the warning is written to the error log and raised to the client if the system variable is reset to the Everyone group at runtime.
Bugs Fixed
InnoDB: A query that scanned the primary key of a table did not return the expected result.
InnoDB: A full-text cache lock taken when data is synchronized was not released if the full-text cache size exceeded the full-text cache size limit.
InnoDB: Client sessions using different auto_increment_increment values while performing concurrent insert operations could cause a duplicate key error.
Replication: In query log events in the binary log, the thread ID used for the execution of DROP TABLE and DELETE statements was identified incorrectly or not at all. On a multi-threaded replication slave, where temporary tables were involved (which require the correct thread ID as they are session specific), this omission resulted in errors when using mysqlbinlog to replay the binary log for point-in-time recovery. The thread ID is now set correctly.
Installing from RPM packages could result in an error log with incorrect permissions.
Enabling audit log encryption could cause a server exit.
MySQL Installer did not install OpenSSL DLL dependencies if the Development component was not selected.
The parser could leak memory for certain multiple-statement queries.
MySQL does not support OpenSSL session tickets, but did not set the SSL_OP_NO_TICKET flag to inform OpenSSL of that. The flag is now set.
UpdateXML() did not always free memory properly in certain cases.
Empty values in the name column of the mysql.plugin system table caused the server to exit during startup.
Some PROXY grants were not replicated to slaves, causing incorrect replication.
If an INSTALL PLUGIN statement contained invalid UTF-8 characters in the shared library name, it caused the server to hang (or to raise an assertion in debug builds).
3.10.0
Add a helper to playhouse.mysql_ext for creating Match full-text search expressions.
Added date-part properties to TimestampField for accessing the year, month, day, etc., within a SQL expression.
Added to_timestamp() helper for DateField and DateTimeField that produces an expression returning a unix timestamp.
Add autoconnect parameter to Database classes. This parameter defaults to True and is compatible with previous versions of Peewee, in which executing a query on a closed database would open a connection automatically. To make it easier to catch inconsistent use of the database connection, this behavior can now be disabled by specifying autoconnect=False, making an explicit call to Database.connect() needed before executing a query.
Added database-agnostic interface for obtaining a random value.
Allow isolation_level to be specified when initializing a Postgres db.
Allow hybrid properties to be used on model aliases.
Support aggregates with FILTER predicates on the latest Sqlite.
Changes
-------
More aggressively slot row values into the appropriate field when building objects from the database cursor (rather than using whatever cursor.description tells us, which is buggy in older Sqlite).
Be more permissive in what we accept in the insert_many() and insert() methods.
When implicitly joining a model with multiple foreign-keys, choose the foreign-key whose name matches that of the related model. Previously, this would have raised a ValueError stating that multiple FKs existed.
Improved date truncation logic for Sqlite and MySQL to make more compatible with Postgres' date_trunc() behavior. Previously, truncating a datetime to month resolution would return '2019-08' for example. As of 3.10.0, the Sqlite and MySQL date_trunc implementation returns a full datetime, e.g. '2019-08-01 00:00:00'.
Apply slightly different logic for casting JSON values with Postgres. Previously, Peewee just wrapped the value in the psycopg2 Json() helper. In this version, Peewee now dumps the json to a string and applies an explicit cast to the underlying JSON data-type (e.g. json or jsonb).
Bug fixes
---------
Save hooks can now be called for models without a primary key.
Fixed bug in the conversion of Python values to JSON when using Postgres.
Fix for differentiating empty values from NULL values in model_to_dict.
Fixed a bug referencing primary-key values that required some kind of conversion (e.g., a UUID).
Add small jitter to the pool connection timestamp to avoid issues when multiple connections are checked-out at the same exact time.
OpenLDAP 2.4.48
Added libldap OpenSSL Elliptic Curve support
Added libldap Expose OpenLDAP specific interfaces via openldap.h
Added slapd-monitor support for slapd-mdb
Fixed liblber leaks
Fixed liblber with partial flush
Fixed libldap ASYNC TLS so it works
Fixed libldap ASYNC connections with Solaris 10
Fixed libldap with SASL_NOCANON=on and ldapi connections
Fixed libldap to be able to unset syncrepl TLS options
Fixed libldap race condition in ldap_int_initialize
Fixed libldap return code in ldap_create_assertion_control_value
Fixed libldap to correctly disable IPv6 when configured to do so
Fixed libldap to correctly close TLS connection
Fixed libldap_r handling of deprecated OpenSSL function
Fixed liblunicode case correspondance
Fixed slapd with an idletimeout of less than four seconds
Fixed slapd config parser variable for Windows64
Fixed slapd syncrepl fallback handling with delta-syncrepl
Fixed slapd telephoneNumberNormalize, cert DN validation
Fixed slapd syncrepl for relax with delta-syncrepl
Fixed slapd to restrict rootDN proxyauthz to its own databases
Fixed slapd to initialize SASL SSF per connection
Fixed slapo-accesslog with SLAP_MOD_SOFT modifications
Fixed slapd-ldap starttls connections timeout behavior
Fixed slapd-ldap segfault when entry result doesn't match filter
Fixed slapd-meta conversion from slapd.conf to cn=config
Fixed slapd-meta assertion when network interface goes down
Fixed slapd-mdb fix bitshift integer overflow
Fixed slapd-mdb index cleanup with cn=config
Fixed slapd-mdb to improve performance with alias deref
Fixed slapo-accesslog possible assert with exops
Fixed slapo-chain to correctly reject multiple chaining URIs
Fixed slapo-chain conversion from slapd.conf to cn=config
Fixed slapo-memberof conversion from slapd.conf to cn=config
Fixed slapo-memberof for group name change to itself
Fixed slapo-ppolicy behavior when pwdInHistory is changed
Fixed slapo-rwm to not free original filter
Fixed slapo-syncprov contextCSN generation
Build Environment
Fixed slapd to only link to BDB libraries with static build
Fixed libldap implicit declaration with LDAP_CONNECTIONLESS
Fixed libldap double inclusion of limits.h in cyrus.c
Documentation
General - Fixed minor typos
admin24 - Miscellaneous updates promoting mdb and fixing examples
slapd.access(5) - Note MDB is the primary backend
slapd.backends(5) - Note MDB is the recommended backend
slapd-ldap(5) - Document starttls parameter
Contrib
Added slapo-lastbind capability to forward authTimestamp updates
Update ruby-arel-helpers to 2.9.1.
2.9.1
* Remove has_rdoc from gemspec.
2.9.0
* Add support for Rails 6.
2.8.0
* Add support for Rails 5.2.1, which changed the arity of several internal
methods that we shamelessly use.
2.7.0
* Add support for Rails 5.2.
=== 5.22.0 (2019-07-01)
* Fix Dataset#multi_insert and #import with return: :primary_key on MSSQL when the dataset has a row_proc (jeremyevans) (#1627)
* Support Dataset#with :materialized option on PostgreSQL 12 for [NOT] MATERIALIZED (jeremyevans)
* Make Database#primary_key_sequence work on tables without serial sequences on PostgreSQL 12 (jeremyevans)
* Support ruby 2.7+ startless ranges in the pg_range extension (jeremyevans)
* Support ruby 2.7+ startless, endless ranges in filters, using an always true condition for them (jeremyevans)
* Support ruby 2.7+ startless ranges in filters, using just a <= or < operator for them (jeremyevans)
4.4.0:
repmgr client enhancements
--------------------------
repmgr standby clone: prevent a standby from being cloned from a witness server (PostgreSQL 9.6 and later only).
repmgr witness register: prevent a witness server from being registered on the replication cluster primary server (PostgreSQL 9.6 and later only).
Registering a witness on the primary node would defeat the purpose of having a witness server, which is intended to remain running even if the cluster's primary goes down.
repmgr standby follow: note that an active, reachable cluster primary is required for this command; and provide a more helpful error message if no reachable primary could be found.
repmgr: when executing repmgr standby switchover, if --siblings-follow is not supplied, list all nodes which repmgr considers to be siblings (this will include the witness server, if in use), and which will remain attached to the old primary.
repmgr: when executing repmgr standby switchover, ignore nodes which are unreachable and marked as inactive. Previously it would abort if any node was unreachable, as that means it was unable to check if repmgrd is running.
However if the node has been marked as inactive in the repmgr metadata, it's reasonable to assume the node is no longer part of the replication cluster and does not need to be checked.
repmgr standby switchover and repmgr standby promote: when executing with the --dry-run option, continue checks as far as possible even if errors are encountered.
repmgr standby promote: add --siblings-follow (similar to repmgr standby switchover).
repmgr daemon status: make output similar to that of repmgr cluster show for consistency and to make it easier to identify nodes not in the expected state.
repmgr cluster show: display each node's timeline ID (PostgreSQL 9.6 and later only).
repmgr cluster show and repmgr daemon status: show the upstream node name as reported by each individual node - this helps visualise situations where the cluster is in an unexpected state, and provide a better idea of the actual cluster state.
For example, if a cluster has divided somehow and a set of nodes are following a new primary, when running either of these commands, repmgr will now show the name of the primary those nodes are actually following, rather than the now outdated node name recorded on the other side of the "split". A warning will also be issued about the unexpected situation.
repmgr cluster show and repmgr daemon status: check if a node is attached to its advertised upstream node, and issue a warning if the node is not attached.
repmgrd enhancements
--------------------
On the primary node, repmgrd is now able to monitor standby connections and, if the number of nodes connected falls below a certain (configurable) value, execute a custom script.
This provides an additional method for fencing an isolated primary node, and/or taking other action if one or more standys become disconnected.
See section Monitoring standby disconnections on the primary node for more details.
In a failover situation, repmgrd nodes on the standbys of the failed primary are now able confirm among themselves that none can still see the primary before continuing with the failover.
The repmgr.conf option primary_visibility_consensus must be set to true to enable this functionality.
See section Primary visibility consensus for more details.
Bug fixes
---------
Ensure BDR2-specific functionality cannot be used on BDR3 and later.
The BDR support present in repmgr is for specific BDR2 use cases.
repmgr: when executing repmgr standby clone in --dry-run mode, ensure provision of the --force option does not result in an existing data directory being modified in any way.
repmgr: when executing repmgr primary register with the --force option, if another primary record exists but the associated node is unreachable (or running as a standby), set that node's record to inactive to enable the current node to be registered as a primary.
repmgr: when executing repmgr standby clone with the --upstream-conninfo, ensure that application_name is set correctly in primary_conninfo.
repmgr: when executing repmgr standby switchover, don't abort if one or more nodes are not reachable and they are marked as inactive.
repmgr: canonicalize the data directory path when parsing the configuration file, so the provided path matches the path PostgreSQL reports as its data directory. Otherwise, if e.g. the data directory is configured with a trailing slash, repmgr node check --data-directory-config will return a spurious error.
repmgrd: fix memory leak which occurs while the monitored PostgreSQL node is not running.
Other
-----
The repmgr documentation has been converted to DocBook XML format, as currently used by the main PostgreSQL project. This means it can now be built against any PostgreSQL version from 9.5 (previously it was not possible to build the documentation against PostgreSQL 10 or later), and makes it easier to provide the documentation in other formats such as PDF.
SQLite Release 3.29.0:
Added the SQLITE_DBCONFIG_DQS_DML and SQLITE_DBCONFIG_DQS_DDL actions to sqlite3_db_config() for activating and deactivating the double-quoted string literal misfeature. Both default to "on" for legacy compatibility, but developers are encouraged to turn them "off", perhaps using the -DSQLITE_DQS=0 compile-time option.
-DSQLITE_DQS=0 is now a recommended compile-time option.
Improvements to the query planner:
Improved optimization of AND and OR operators when one or the other operand is a constant.
Enhancements to the LIKE optimization for cases when the left-hand side column has numeric affinity.
Added the "sqlite_dbdata" virtual table for extracting raw low-level content from an SQLite database, even a database that is corrupt.
Enhancements to the CLI:
Add the ".recover" command which tries to recover as much content as possible from a corrupt database file.
Add the ".filectrl" command useful for testing.
Add the long-standing ".testctrl" command to the ".help" menu.
Added the ".dbconfig" command
3.4.21:
Issues fixed:
SERVER-39820: Add the client IP address to the successful authentication log message
SERVER-41213: Unique background index builds may produce inconsistent keys
SERVER-37765: Platform Support: Remove Ubuntu 14.04
SERVER-37774: Platform Support: Remove Enterprise Ubuntu 16.04 PPCLE
WT-4615: Sync backup file before returning backup cursor
3.4.20:
Issues fixed:
SERVER-39166: $graphLookup should force a pipeline to split in sharded cluster
SERVER-39723: Change listIndexes command behavior to show in-progress index builds
TOOLS-2158: mongodump failing on Windows with “error opening system CA store: Access is denied.”
3.4.19:
Issues fixed:
SERVER-18985: setParameter should log at level 0
SERVER-22766: Dynamic oplog sizing for WiredTiger nodes
SERVER-33469: Make syslog log lines consistent with mongod log lines
SERVER-37132: Negation of $in with regex can incorrectly plan from the cache, leading to missing query results
SERVER-37182: Different values when referencing whole object vs. a field of that object after $arrayToObject
SERVER-37616: Implement tuneable batch size for the rangedeleter
3.4.18:
Issues fixed:
SERVER-25175: listIndexes shouldn’t include in-progress background indexes
SERVER-35418: Allow specifying CAs for incoming and outgoing connections separately.
SERVER-36944: applyOps does not permit unknown field names when creating a v:1 index.
SERVER-37058: Update with numeric field names inside an array can cause validation to fail.
3.4.17:
Issues fixed:
SERVER-33857 Missing log redaction due to confusion with Command::redactForLogging().
SERVER-34846 Covered index with collated field returns incorrect result when collation not involved in match or sort.
SERVER-36010 Change log messages for Windows stacktraces to use error() or severe() rather than log().
SERVER-34558: Add SSL_version to client metadata logging.
3.4.16:
Issues fixed:
SERVER-32999: Platform Support: Remove Debian 7 builds.
SERVER-29301: Upgrade MozJS to ESR 45.9.0
SERVER-5461: Add syncSourceHost field to replSetGetStatus output.
3.4.15:
Issues fixed:
SERVER-31535: Platform Support: remove Ubuntu 12.04 builds.
SERVER-32923: Platform Support: remove SLES11 builds.
SERVER-20056: Log a startup warning if wiredTigerCacheSizeGB is > 80% of RAM.
SERVER-28981: Sharding balancer prefers shards in a specific order when moving chunks.
3.4.14:
Issues fixed:
SERVER-28151: Authentication database should be synced first during an initial sync.
SERVER-32886: Unnecessary sleeps during chunk migration.
SERVER-32933: Allow mongod to start when unable to reach LDAP server.
3.4.13:
Issues fixed:
SERVER-33238: Prevent WiredTiger read ticket count from going negative.
3.4.12:
Issues fixed:
SERVER-31437: Fix parsing of mongo host/db connect string.
SERVER-32875: Don’t stall ftdc due to running out of tickets.
SERVER-33089: Unable to start queryable mongod because it failed to regenerate index for admin.system.users.
3.4.11:
Issues fixed:
SERVER-19605: Oplog timeout should be configurable.
SERVER-19919: Chunks that exceed 250000 docs but are under half chunk size get marked as jumbo.
SERVER-29293: Recipient shard fails to abort migration on stepdown.
SERVER-32001: unindexing a key in a partial unique index may cause server crash
3.4.10:
Issues fixed:
SERVER-15723: Avoid G_X lock for rename_collection within database
SERVER-31049: View with collation doesn’t work as expected in sharded cluster
SERVER-29287: Upgrade pcre to 8.41
SERVER-31204: Calling shardCollection after enableSharding may fail if executed against different mongos
3.4.9:
Issues fixed:
SERVER-20392: Sharding an existing small collection results in large number of chunks
SERVER-30189: Reduce calls to allocator for large $in expressions
SERVER-30487: RangeDeleter holds WT transaction open while waiting for majority
SERVER-30636: Range deleter assert failed because of replication lag
3.4.7:
Issues fixed:
SERVER-29282: BSON Document Size can be exceeded when grouping inserts on SECONDARY nodes
SERVER-29568: Enable configuration of OpenSSL cipher suite via setParameter
SERVER-29817: Optimize incremental update performance of ChunkManager and CollectionMetadata
3.4.6:
Issues fixed:
SERVER-27347: Only close idle cached cursors on the WiredTiger ident that is busy.
SERVER-29618: $geoWithin in aggregation pipeline after $lookup and $unwind returns incorrect results.
WT-3362: Cursor opens should never block for the duration of a checkpoint.
3.4.5:
Issues fixed:
SERVER-28952: Multikey indexes should not be eligible for DISTINCT_SCAN if distinct key is an array component.
SERVER-28427: GlobalLock with timeout can still block indefinitely.
SERVER-29018 mongos can segfault in getMore on views with find batchSize of 0.
From David Weller-Fahy in PR pkg/54340.
DBF is a file format used by databases such dBase, Visual FoxPro, and
FoxBase+. This library reads DBF files and returns the data as native
Python data types for further processing. It is primarily intended for
batch jobs and one-off scripts.
0.9.3
* cryptography dependency is optional now.
* Fix old_password (used before MySQL 4.1) support.
* Deprecate old_password.
* Stop sending ``sys.argv[0]`` for connection attribute "program_name".
* Close connection when unknown error is happened.
* Deprecate context manager API of Connection object.
version: 1.0.11
SQLite server default reflection will ensure parenthesis are surrounding a
column default expression that is detected as being a non-constant
expression, such as a datetime() default, to accommodate for the
requirement that SQL expressions have to be parenthesized when being sent
as DDL. Parenthesis are not added to constant expressions to allow for
maximum cross-compatibility with other dialects and existing test suites
(such as Alembic's), which necessarily entails scanning the expression to
eliminate for constant numeric and string values. The logic is added to the
two "reflection->DDL round trip" paths which are currently autogenerate and
batch migration. Within autogenerate, the logic is on the rendering side,
whereas in batch the logic is installed as a column reflection hook.
Improved SQLite server default comparison to accommodate for a text()
construct that added parenthesis directly vs. a construct that relied
upon the SQLAlchemy SQLite dialect to render the parenthesis, as well
as improved support for various forms of constant expressions such as
values that are quoted vs. non-quoted.
Fixed bug where the "literal_binds" flag was not being set when
autogenerate would create a server default value, meaning server default
comparisons would fail for functions that contained literal values.
Added support for MySQL "DROP CHECK", which is added as of MySQL 8.0.16,
separate from MariaDB's "DROP CONSTRAINT" for CHECK constraints. The MySQL
Alembic implementation now checks for "MariaDB" in server_version_info to
decide which one to use.
Fixed issue where MySQL databases need to use CHANGE COLUMN when altering a
server default of CURRENT_TIMESTAMP, NOW() and probably other functions
that are only usable with DATETIME/TIMESTAMP columns. While MariaDB
supports both CHANGE and ALTER COLUMN in this case, MySQL databases only
support CHANGE. So the new logic is that if the server default change is
against a DateTime-oriented column, the CHANGE format is used
unconditionally, as in the vast majority of cases the server default is to
be CURRENT_TIMESTAMP which may also be potentially bundled with an "ON
UPDATE CURRENT_TIMESTAMP" directive, which SQLAlchemy does not currently
support as a distinct field. The fix addiionally improves the server
default comparison logic when the "ON UPDATE" clause is present and
there are parenthesis to be adjusted for as is the case on some MariaDB
versions.
Warnings emitted by Alembic now include a default stack level of 2, and in
some cases it's set to 3, in order to help warnings indicate more closely
where they are originating from. Pull request courtesy Ash Berlin-Taylor.
Replaced the Python compatbility routines for getargspec() with a fully
vendored version based on getfullargspec() from Python 3.3.
Originally, Python was emitting deprecation warnings for this function in
Python 3.8 alphas. While this change was reverted, it was observed that
Python 3 implementations for getfullargspec() are an order of magnitude
slower as of the 3.4 series where it was rewritten against Signature.
While Python plans to improve upon this situation, SQLAlchemy projects for
now are using a simple replacement to avoid any future issues.
1.3.5
orm
[orm] [bug]
Fixed a series of related bugs regarding joined table inheritance more than two levels deep, in conjunction with modification to primary key values, where those primary key columns are also linked together in a foreign key relationship as is typical for joined table inheritance. The intermediary table in a three-level inheritance hierachy will now get its UPDATE if only the primary key value has changed and passive_updates=False (e.g. foreign key constraints not being enforced), whereas before it would be skipped; similarly, with passive_updates=True (e.g. ON UPDATE CASCADE in effect), the third-level table will not receive an UPDATE statement as was the case earlier which would fail since CASCADE already modified it. In a related issue, a relationship linked to a three-level inheritance hierarchy on the primary key of an intermediary table of a joined-inheritance hierarchy will also correctly have its foreign key column updated when the parent object’s primary key is modified, even if that parent object is a subclass of the linked parent class, whereas before these classes would not be counted.
[orm] [bug]
Fixed bug where the Mapper.all_orm_descriptors accessor would return an entry for the Mapper itself under the declarative __mapper___ key, when this is not a descriptor. The .is_attribute flag that’s present on all InspectionAttr objects is now consulted, which has also been modified to be True for an association proxy, as it was erroneously set to False for this object.
[orm] [bug]
Fixed regression in Query.join() where the aliased=True flag would not properly apply clause adaptation to filter criteria, if a previous join were made to the same entity. This is because the adapters were placed in the wrong order. The order has been reversed so that the adapter for the most recent aliased=True call takes precedence as was the case in 1.2 and earlier. This broke the “elementtree” examples among other things.
[orm] [bug] [py3k]
Replaced the Python compatbility routines for getfullargspec() with a fully vendored version from Python 3.3. Originally, Python was emitting deprecation warnings for this function in Python 3.8 alphas. While this change was reverted, it was observed that Python 3 implementations for getfullargspec() are an order of magnitude slower as of the 3.4 series where it was rewritten against Signature. While Python plans to improve upon this situation, SQLAlchemy projects for now are using a simple replacement to avoid any future issues.
[orm] [bug]
Reworked the attribute mechanics used by AliasedClass to no longer rely upon calling __getattribute__ on the MRO of the wrapped class, and to instead resolve the attribute normally on the wrapped class using getattr(), and then unwrap/adapt that. This allows a greater range of attribute styles on the mapped class including special __getattr__() schemes; but it also makes the code simpler and more resilient in general.
sql
[sql] [bug]
Fixed a series of quoting issues which all stemmed from the concept of the literal_column() construct, which when being “proxied” through a subquery to be referred towards by a label that matches its text, the label would not have quoting rules applied to it, even if the string in the Label were set up as a quoted_name construct. Not applying quoting to the text of the Label is a bug because this text is strictly a SQL identifier name and not a SQL expression, and the string should not have quotes embedded into it already unlike the literal_column() which it may be applied towards. The existing behavior of a non-labeled literal_column() being propagated as is on the outside of a subquery is maintained in order to help with manual quoting schemes, although it’s not clear if valid SQL can be generated for such a construct in any case.
postgresql
[postgresql] [bug]
Fixed bug where PostgreSQL dialect could not correctly reflect an ENUM datatype that has no members, returning a list with None for the get_enums() call and raising a TypeError when reflecting a column which has such a datatype. The inspection now returns an empty list.
[postgresql] [usecase]
Added support for column sorting flags when reflecting indexes for PostgreSQL, including ASC, DESC, NULLSFIRST, NULLSLAST. Also adds this facility to the reflection system in general which can be applied to other dialects in future releases. Pull request courtesy Eli Collins.
mysql
[mysql] [bug]
Fixed bug where MySQL ON DUPLICATE KEY UPDATE would not accommodate setting a column to the value NULL.
