All checksums have been double-checked against existing RMD160 and
SHA512 hashes
The following distfiles could not be fetched (some may be only fetched
conditionally):
./databases/cstore/distinfo D6.data.ros.gz
./databases/cstore/distinfo cstore0.2.tar.gz
./databases/cstore/distinfo data4.tar.gz
Changes in MySQL 5.7.36
Security Notes
Bugs Fixed
Security Notes
The linked OpenSSL library for MySQL Server has been updated to version 1.1.1l. Issues fixed in the new OpenSSL version are described at https://www.openssl.org/news/cl111.txt and and http://www.openssl.org/news/vulnerabilities.html.
Bugs Fixed
Incompatible Change: For all SELECT statements on a view, the query digest was based on the view definition. As a result, different queries had the same digest and aggregated together in the Performance Schema table events_statements_summary_by_digest, so statistics in that table were not usable for distinguishing distinct SELECT statements.
The query digest for each SELECT statement on a view now is based on the SELECT, not the view definition. This enables distinguishing distinct SELECT statements in the events_statements_summary_by_digest table. However, tools that use query digests may need some adjustment to account for this change. For example, MySQL Enterprise Firewall and query rewrite plugins rely on query digests and existing rules for them that are associated with views may need to be updated.
InnoDB: With undo log truncation enabled (innodb_undo_log_truncate=ON), it was possible for a deadlock and eventual failure to occur when an undo log truncate operation was initiated after a version upgrade from MySQL 5.6 to MySQL 5.7.34 or earlier. A patch introduced in MySQL 5.7.35
[Note] InnoDB: Found duplicate reference rseg: 33 space: 1 page: 3
[Note] InnoDB: Reset pre-5.7.2 rseg: 1 after duplicate is found.
If pre-5.7.2 rollback segment slots have no undo data to purge, a message similar to the following is emitted:
[Note] InnoDB: Successfully reset 32 pre-5.7.2 rseg slots.
If undo data is found in pre-5.7.2 rollback segment slots, a message similar to the following is emitted recommending a slow shutdown and restart:
[Note] InnoDB: pre-5.7.2 rseg: 2 holds data to be purged.
History length: 1. Recommend slow shutdown with innodb_fast_shutdown=0 and restart
InnoDB: Truncation of an undo tablespace during use by an active transaction raised an assertion failure. The transaction was prematurely marked as complete, permitting the truncation operation.
InnoDB: Deleting or updating a row from a parent table initiated a cascading SET NULL operation on the child table that set a virtual column value to NULL. The virtual column value should have been derived from the base column value.
Thanks to Yin Peng at Tencent for the contribution.
InnoDB: The InnoDB recovery process did not recognize that page compression had been applied to data that was being recovered, causing the tablespace data file to increase in size during the redo log apply phase, which could lead to a recovery failure for systems approaching a disk-full state.
Replication: The error messages issued by MySQL Replication when GTIDs required for auto-positioning have been purged could be incorrectly assigned or scrambled in some situations.
Replication: The contents of the gtid_executed and gtid_purged GTID sets were not persisted after restoring a dump taken using mysqldump. The dump file sequence has now been changed so that the mysql schema (which contains the mysql.gtid_executed table) is not dropped after the gtid_purged GTID set is written. A new option --skip-mysql-schema is added for mysqldump which lets you choose not to drop the mysql schema at all.
JSON: Conversion of JSON values to text caused linear growth of the destination string, resulting in an unnecessarily high number of reallocations. Now this process uses exponential growth instead, to reduce the number of allocations required.
This fix originally appeared in MySQL 8.0 and was backported to MySQL 5.7 by Annirudh Prasad, whom we thank for the contribution.
Concurrent insert operations on multiple tables with full-text indexes caused a large number of full-text index synchronization requests, resulting in an out of memory condition.
When a query uses a temporary table for aggregation, the group by item is used as a unique constraint on the temporary table: If the item value is already present, the row is updated; otherwise, a new row is inserted into the temporary table. If the item has a result field or reference item, it it evaluated twice, once to check whether the result exists in the temporary table and, if not, again while constructing the row to be inserted. When the group by item was nondeterministic, the result value used to check for existence differed from that with which an insert was attempted, causing the insert to be rejected if the value already existed in the table.
We fix this by using the hash of any nondeterministic items as the unique constraint, so that the hash is evaluated once only.
Quote handling was improved for the SHOW GRANTS statement.
Changes in MySQL 5.7.35 (2021-07-20, General Availability)
Audit Log Notes
Deprecation and Removal Notes
Packaging Notes
Bugs Fixed
Audit Log Notes
For MySQL Enterprise Audit, the new audit_log_format_unix_timestamp system variable enables inclusion of a time field in each audit record. The field value is an integer that represents the UNIX timestamp value indicating the date and time when the audit event was generated. The time field is supported only for JSON-format log files.
Deprecation and Removal Notes
The TLSv1 and TLSv1.1 connection protocols now are deprecated and support for them is subject to removal in a future MySQL version. (For background, refer to the IETF memo Deprecating TLSv1.0 and TLSv1.1.) It is recommended that connections be made using the more-secure TLSv1.2 and TLSv1.3 protocols. TLSv1.3 requires that both the MySQL server and the client application be compiled with OpenSSL 1.1.1 or higher.
On the server side, this deprecation has the following effects:
If the tls_version system variable is assigned a value containing a deprecated TLS protocol during server startup, the server writes a warning for each deprecated protocol to the error log.
If a client successfully connects using a deprecated TLS protocol, the server writes a warning to the error log.
On the client side, the deprecation has no visible effect. Clients do not issue a warning if configured to permit a deprecated TLS protocol. This includes:
Client programs that support a --tls-version option for specifying TLS protocols for connections to the MySQL server.
Statements that enable replicas to specify TLS protocols for connections to the source server. (CHANGE MASTER TO has a MASTER_TLS_VERSION option.)
Packaging Notes
Binary packages that include curl rather than linking to the system curl library have been upgraded to use curl 7.77.0.
The bundled lz4 library was upgraded to version 1.9.3.
Bugs Fixed
InnoDB: A deadlock between a user thread and purge thread involving a undo log page and rollback segment page occurred after an undo tablespace truncate operation was initiated. The deadlock caused a long semaphore wait and an eventual failure.
InnoDB: An integer underflow issue was addressed in the InnoDB mecached plugin sources.
InnoDB: An index with a key prefix length greater than 767 bytes was permitted on a table defined with the REDUNDANT row format, exceeding the index key prefix length limit for that row format. The ALTER TABLE operation that added the index validated the index key prefix length for the row format defined by the innodb_default_row_format variable instead of the actual row format of the table. The fix ensures that index key prefix length is validated for the correct row format.
InnoDB: An online buffer pool resizing operation freed the previous buffer pool page hash, conflicting with a concurrent buffer pool lookup that required the previous page hash.
InnoDB: Numerous system temporary table pages at the tail of the buffer pool flush list caused a performance degradation. The flush_list_mutex was held while the flush list scan traversed over system temporary table pages. The flush list scan now excludes system temporary table pages.
InnoDB: A binary log rotation deadlock occurred on a system using statement-based replication where there was high number of concurrent update operations and low innodb_thread_concurrency setting.
Replication: When the system variable replication_optimize_for_static_plugin_config was set, the plugins for Group Replication and semi-synchronous replication could not be uninstalled cleanly on server shutdown.
Replication: A deadlock could occur when START GROUP_REPLICATION and STOP GROUP_REPLICATION statements were issued at the same time that a view change was taking place for the group.
Replication: A deadlock could occur if a STOP GROUP_REPLICATION statement was issued when a replication channel on a group member was attempting to commit a transaction. The server now rolls back the transaction immediately if it cannot acquire the relevant lock, rather than waiting for the lock and the commit to complete and causing the deadlock.
Replication: On a multithreaded replica, the reference to the active event was sometimes managed incorrectly when retrying a transaction.
Replication: Replica servers now check and validate the transaction ID part of a GTID before applying and committing the transaction associated with it.
Replication: Replication could stop on a multithreaded replica if a unique secondary key was omitted from the writeset hashes used to compute transaction dependencies, leading to errors when executing the transactions on the multithreaded replica. Write set hashes now always include unique secondary keys even if they are not included in the read set and write set.
JSON: Passing NULL to a stored procedure expecting a JSON parameter led to an assertion failure in debug builds.
Replication could fail if a DML statement was executed immediately after an XA transaction was rejected or forced to rollback due to a deadlock.
The mysql_change_user() C API function did not properly parse the COM_CHANGE_USER packet, which could result in silent failure to process optional query attributes that may have been supplied prior to the mysql_change_user() call. Thanks for René Cannaò for the contribution.
Repreparation of a prepared statement at the beginning of an implicit transaction could cause an ER_GTID_NEXT_TYPE_UNDEFINED_GROUP error.
An out-of-memory error occurred when loading large amounts of data into tables with full-text search indexes. Not all of the memory allocated to the full-text search cache was accounted for when inserting data into the full-text search auxiliary tables.
A secondary index over a virtual column became corrupted when the index was built online.
For UPDATE statements, we fix this as follows: If the virtual column value of the index record is set to NULL, then we generate this value from the cluster index record.
Boolean system variables could be assigned a negative value.
Changes in MySQL 5.7.34 (2021-04-20, General Availability)
Configuration Notes
The CMAKE_BUILD_TYPE CMake option now supports a Release build type, which is like the RelWithDebInfo build type but omits debugging information to reduce the build size.
Packaging Notes
The bundled libedit library was upgraded to version 20190324-3.1.
Binary packages that include curl rather than linking to the system curl library have been upgraded to use curl 7.74.0.
Security Notes
The linked OpenSSL library for MySQL Server has been updated to version 1.1.1k. Issues fixed in the new OpenSSL version are described at https://www.openssl.org/news/cl111.txt and https://www.openssl.org/news/vulnerabilities.html.
Functionality Added or Changed
Microsoft Windows: The named_pipe_full_access_group system variable now defaults to an empty string (''), making named pipe connections secure. Previously, '*everyone*' was the default value. A valid Windows local group name may be substituted.
Bugs Fixed
InnoDB: The introduction of sharded rw_lock_stats counters in MySQL 5.7 caused a regression in CPU cache efficiency. To address this issue in MySQL 5.7, the sharding method was changed. For optimal performance, the rw_lock_stats counter is removed in MySQL 8.0.
InnoDB: A delete operation on a parent table that initiated a cascading update on a child table with an indexed virtual column and indexed foreign key constraint column caused a virtual column corruption.
InnoDB: An adaptive hash index (AHI) latch was held in shared mode by a thread truncating a large table, causing mutex waits for other threads. The AHI latch was not required and has been removed.
InnoDB: The open and close sequence for table share instances (m_share objects) and dictionary table instances was modified to prevent accessing old m_share objects that could point to stale dictionary indexes.
Thanks to Yuxiang Jiang for the contribution.
InnoDB: An online ALTER TABLE operation failed with an “Incorrect key file for table” error due to an unnecessary encryption status check that was performed when reading online DDL row logs.
Replication: If all previous binary log files were purged at startup because their retention period had expired, the new binary log file contained an empty Previous_gtids event, which could cause errors in replication. The order of initialization has now been changed so that previous binary log files are only purged after the previous GTID set has been written to the new binary log file that is created at startup.
Replication: An assertion was raised in debug builds relating to lost GTIDs if binary log files were removed at startup because their retention period had expired.
Replication: A deadlock could occur if the binary log file was rotated while system variables were being updated and read by different clients.
Replication: The output of a SHOW PROCESSLIST statement for a replica’s SQL thread sometimes showed the last query as currently being applied when the replica was actually caught up.
A query string was displayed before it had been rewritten.
For builds compiled using the libedit library, if the mysql client was invoked with the --default-character-set=utf8 option, libedit rejected input of multibyte characters.
On Windows, large result sets could cause the mysql client to exit unexpectedly.
Improper locking on an internal queue could cause mysqlpump to exit unexpectedly.
It is now possible to use START REPLICA SQL_THREAD and STOP REPLICA SQL_THREAD statements for the group_replication_applier channel when Group Replication is stopped. This enables an operator to apply any remaining unapplied transactions on a server that left the group, without having to rejoin the server to the group.
When the mysql client was used in batch mode, its parser could be confused by USE followed by DROP DATABASE when the USE database name was quoted.
While optimizing the ORDER BY clause of a subquery there was a possibility of cleaning up a subquery tree referenced in the outer SELECT, which could lead to a premature exit.
A malformed name in the mysql.func system table could cause unexpected server behavior.
Sessions could disable their own auditing.
Mishandling of stored program local variables could lead to unexpected server behavior.
Uninstalling a plugin could affect subsequent execution of prepared statements.
Changes in MySQL 5.7.33
Optimizer Notes
MySQL attempts to use an ordered index for any ORDER BY or GROUP BY query that has a LIMIT clause, overriding any other choices made by the optimizer, whenever it determines that this would result in faster execution. Because the algorithm for making this determination makes certain assumptions about data distribution and other conditions, it may not always be completely correct, and it is possible in some cases that choosing a different optimization for such queries can provide better performance. To handle such occurrences, it is now possible to disable this optimization by setting the optimizer_switch system variable's prefer_ordering_index flag to off.
For more information about this flag and examples of its use, see Switchable Optimizations, and LIMIT Query Optimization.
Our thanks to Jeremy Cole for the contribution.
Security Notes
The linked OpenSSL library for MySQL Server has been updated to version 1.1.1i. Issues fixed in the new OpenSSL version are described at https://www.openssl.org/news/cl111.txt and https://www.openssl.org/news/vulnerabilities.html.
Functionality Added or Changed
When invoked with the --all-databases option, mysqldump now dumps the mysql database first, so that when the dump file is reloaded, any accounts named in the DEFINER clause of other objects will already have been created.
Bugs Fixed
InnoDB: The full-text search synchronization thread attempted to read a previously-freed word from the index cache.
InnoDB: Calls to numa_all_nodes_ptr were replaced by the numa_get_mems_allowed() function. Thanks to Daniel Black for the contribution.
Replication: When the system variable transaction_write_set_extraction=XXHASH64 is set, which is the default in MySQL 8.0 and a requirement for Group Replication, the collection of writes for a transaction previously had no upper size limit. Now, for standard source to replica replication, the numeric limit on write sets specified by binlog_transaction_dependency_history_size is applied, after which the write set information is discarded but the transaction continues to execute. Because the write set information is then unavailable for the dependency calculation, the transaction is marked as non-concurrent, and is processed sequentially on the replica. For Group Replication, the process of extracting the writes from a transaction is required for conflict detection and certification on all group members, so the write set information cannot be discarded if the transaction is to complete. The byte limit set by group_replication_transaction_size_limit is applied instead of the numeric limit, and if the limit is exceeded, the transaction fails to execute.
Replication: As the number of replicas replicating from a semisynchronous source server increased, locking contention could result in a performance degradation. The locking mechanisms used by the plugins have been changed to use shared locks where possible, avoid unnecessary lock acquisitions, and limit callbacks. The new behaviors can be implemented by enabling the following system variables:
replication_sender_observe_commit_only=1 limits callbacks.
replication_optimize_for_static_plugin_config=1 adds shared locks and avoids unnecessary lock acquisitions. This system variable must be disabled if you want to uninstall the plugin.
Both system variables can be enabled before or after installing the semisynchronous replication plugin, and can be enabled while replication is running. Semisynchronous replication source servers can also get performance benefits from enabling these system variables, because they use the same locking mechanisms as the replicas.
Replication: On a multi-threaded replica where the commit order is preserved, worker threads must wait for all transactions that occur earlier in the relay log to commit before committing their own transactions. If a deadlock occurs because a thread waiting to commit a transaction later in the commit order has locked rows needed by a transaction earlier in the commit order, a deadlock detection algorithm signals the waiting thread to roll back its transaction. Previously, if transaction retries were not available, the worker thread that rolled back its transaction would exit immediately without signalling other worker threads in the commit order, which could stall replication. A worker thread in this situation now waits for its turn to call the rollback function, which means it signals the other threads correctly.
Replication: GTIDs are only available on a server instance up to the number of non-negative values for a signed 64-bit integer (2 to the power of 63 minus 1). If you set the value of gtid_purged to a number that approaches this limit, subsequent commits can cause the server to run out of GTIDs and take the action specified by binlog_error_action. From MySQL 8.0.23, a warning message is issued when the server instance is approaching the limit.
Microsoft Windows: On Windows, running the MySQL server as a service caused shared-memory connections to fail.
The server did not handle all cases of the WHERE_CONDITION optimization correctly.
For the engines which support primary key extension, when the total key length exceeded MAX_KEY_LENGTH or the number of key parts exceeded MAX_REF_PARTS, key parts of primary keys which did not fit within these limits were not added to the secondary key, but key parts of primary keys were unconditionally marked as part of secondary keys.
This led to a situation in which the secondary key was treated as a covering index, which meant sometimes the wrong access method was chosen.
This is fixed by modifying the way in which key parts of primary keys are added to secondary keys so that those which do not fit within which do not fit within the limits mentioned previously mentioned are cleared.
Privileges for some INFORMATION_SCHEMA tables were checked incorrectly.
In certain cases, the server did not handle multiply-nested subqueries correctly.
Certain accounts could cause server startup failure if the skip_name_resolve system variable was enabled.
Client programs could unexpectedly exit if communication packets contained bad data.
A buffer overflow in the client library was fixed.
mysql_config_editor incorrectly treated # in password values as a comment character.
Changes in MySQL 5.7.32 (2020-10-19, General Availability)
Functionality Added or Changed
Bugs Fixed
Functionality Added or Changed
LOCK TABLES privilege checking for views was improved.
Bugs Fixed
InnoDB: A query that updated the clustered index of an internal temporary table returned an incorrect result. The modified pages of the clustered index were not added to the flush list resulting in lost changes when the modified pages were evicted from the buffer pool.
References: This issue is a regression of:
InnoDB: An ALTER TABLE ... IMPORT TABLESPACE operation on a large encrypted and compressed table failed with a Page decompress failed after reading from disk error. The decryption operation did not use the encryption block size used during encryption. Also, the encryption process did not consider compressed length, while the decryption process decrypts data by compressed length only.
InnoDB: A failure occurred during a concurrent update operation. The failure was due to an invalid previous record value.
InnoDB: The function used to process the SHOW ENGINE INNODB MUTEX statement was insufficiently isolated from other threads adding new mutexes concurrently.
InnoDB: The buffer control block structure (buf_block_t) was freed while reducing the size of the buffer pool, causing an assertion failure. The fix for this bug also backports important aspects of the fix for
InnoDB: In session started with START TRANSACTION WITH CONSISTENT SNAPSHOT, a range query returned a truncated result. The end range flag was not reset at the beginning of the index read resulting in an aborted read and missing rows.
References: This issue is a regression of:
InnoDB: A full-text phrase search raised an assertion failure.
Thanks to TXSQL (Tencent MySQL) for the contribution.
References: This issue is a regression of:
InnoDB: A long running statistics calculation operation on a large table blocked other operations requiring access to the table's statistics, causing those operations to fail. A new statistics calculation mutex was introduced, which permits concurrent access table statistics.
Thanks to Kamil Holubicki for the contribution.
InnoDB: Two connections attempted to use the same transaction handler object resulting in a stalled query.
Replication: When a replication source server shuts down and restarts, its MEMORY tables become empty. To replicate this effect to replicas, the first time that the source uses a given MEMORY table after startup, it logs an event that notifies replicas that the table must be emptied by writing a statement to the binary log to that effect. Previously, this was a DELETE statement, but it is now a TRUNCATE TABLE statement. A replica server also writes this statement to its own binary log when it shuts down and restarts. The statement is always logged in statement format, even if the binary logging format is set to ROW, and it is written even if read_only or super_read_only mode is set on the server.
Replication: When the system variable session_track_gtids was set to OWN_GTID on a multithreaded replica, the replica’s performance would degrade over time and begin to lag behind the master. The cause was the buildup of the GTIDs recorded by the replica’s worker threads at each transaction commit, which increased the time taken by the worker threads to insert new ones. Session state tracking is now disabled for worker threads on a multithreaded replica. Thanks to Facebook for the contribution.
Certain cases of successful LDAP authentication could cause the server to hang.
In bootstrapping mode, certain multiple-statement transactions could cause unexpected server behavior.
Sensitive LDAP authentication plugin system variables now display as asterisks when retrieved in SQL statements.
After the fix for
References: This issue is a regression of:
Some INSERT statements were not handled correctly.
Certain prepared statements could cause an unexpected server exit.
mysqlpump object validation included objects in excluded databases.
LDAP authentication plugins enforced CA verification incorrectly, which could result in use of an incorrect CA.
ORDER BY queries were not executed correctly when sort_buffer_size and max_sort_length were set to values which caused the internal limit on the maximum number of keys allowed per sort buffer to be set to 0.
A large number of nested arguments in full-text search query caused an error.
When explicit_defaults_for_timestamp was disabled and a NULL was inserted into a generated column declared as TIMESTAMP NOT NULL, the server would attempt to convert the inserted value to CURRENT_TIMESTAMP. Such an insertion is now rejected with ER_BAD_NULL_ERROR.
An assertion could be raised when the SQL layer passed incorrect information to InnoDB about the type of operation to be performed on a temporary table.
Changes in MySQL 5.7.31
Configuration Notes
tcmalloc is no longer a permitted value for the mysqld_safe --malloc-lib option.
Packaging Notes
The libevent library bundled with MySQL was upgraded to version 2.1.11. In addition, for the WITH_LIBEVENT CMake option, the following two changes were made:
yes is no longer permitted as a synonym for system. Use system instead.
If system is specified but no system libevent is found, the bundled version is no longer used in place of the missing system library, and an error occurs instead.
Security Notes
Incompatible Change: Access to the INFORMATION_SCHEMA.FILES table now requires the PROCESS privilege.
This change affects users of the mysqldump command, which accesses tablespace information in the FILES table, and thus now requires the PROCESS privilege as well. Users who do not need to dump tablespace information can work around this requirement by invoking mysqldump with the --no-tablespaces option.
The linked OpenSSL library for MySQL Server has been updated to version 1.1.1g. Issues fixed in the new OpenSSL version are described at https://www.openssl.org/news/cl111.txt and https://www.openssl.org/news/vulnerabilities.html.
Bugs Fixed
InnoDB: The INNODB_METRICS table AVG_COUNT_RESET value for a counter defined as a module owner reported NULL. The METRIC_AVG_VALUE_RESET field was incorrectly marked as NULL.
InnoDB: Purge thread activity was excessive when the history list length approached zero, wasting CPU resource and causing mutex contention.
InnoDB: The server failed intermittently with an “ibuf cursor restoration fails” error.
InnoDB: A fatal “page still fixed or dirty” error occurred during shutdown.
Partitioning: A query against a partitioned table, which used an ORDER BY, returned unordered results under the following conditions:
The table had a composite index with a prefix on one of the columns.
The query's WHERE clause contained an equality condition on the prefixed column.
The column with the prefix was the leftmost column in the index.
The column used in the ORDER BY was the rightmost column in the index.
The index was used for handling the ORDER BY.
Our thanks to Quanan Han for the contribution.
Replication: When a replication source server shuts down and restarts, its MEMORY tables become empty. To replicate this effect to replicas, the first time that the source uses a given MEMORY table after startup, it notifies replicas that the table must be emptied by writing a DELETE statement for that table to the binary log. Previously, the generated DELETE statement was written to the binary log statement cache for the current session, which could result in it being logged together with other statements under the same GTID, or logged without BEGIN and COMMIT statements. Also, in some situations, the generated DELETE statement could consume the GTID intended for the transaction that triggered it. The generated DELETE statement is now logged with accompanying BEGIN and COMMIT statements, and the resulting transaction is flushed to the binary log immediately after it is written to the statement cache, so that it always receives its own GTID and is kept separate from other transactions.
Replication: Internal queries from Group Replication to the Performance Schema for statistics on local group members failed if they occurred simultaneously with changes to the group's membership. Locking for the internal queries has been improved to fix the issue.
Replication: A fix made in MySQL 8.0.14 and MySQL 5.7.25 for a deadlock scenario involving the system variables binlog_transaction_dependency_tracking and binlog_transaction_dependency_history_size had the side effect of leaving the writeset history used for transaction dependency tracking unprotected from concurrent update. The writeset history and tracking mode are now locked correctly whenever they are accessed.
Configuring with -DWITH_EDITLINE=system resulted in compilation failures for older library versions.
The upgrade of the bundled libedit library in the previous MySQL distribution caused a problem for builds using that library such that CTRL+C (SIGINT) in the mysql client required a following Enter to take effect in some circumstances.
Assigning CONCAT('') or CONCAT_WS('') to a variable set the variable to NULL, not the empty string.
The gen_range() user-defined function could mishandle its arguments, causing a server exit.
During UPDATE processing, conversion of an internal in-memory table to InnoDB could result in a key-length error.
It is possible to define a column named * (single asterisk character), but SELECT `*` was treated identically to SELECT *, making it impossible to select only this column in a query; in other words, the asterisk character was expanded to a list of all table columns even when it was surrounded by backticks.
An internal packet-length function returned values of the wrong integer type.
Calculations by mysqldump for the length of INSERT statements did not take into account the _binary character set introducer used for VARBINARY strings.
mysql_store_result() could fail to detect invalid data packets.
In a multiple-table UPDATE that updated the key of the first table, if a temporary table strategy was used, duplicate entries could be written to the temporary table, followed by occurrence of a Can't find record error.
The optimizer could attempt to access a pseudo table as a table, resulting in unexpected server behavior.
The server sometimes mistakenly removed a subquery with a GROUP BY when optimizing a query, even in some cases when this subquery was used by an outer select. This could occur when the subquery also used an aggregate function.
Coercibility of the NAME_CONST() function was assessed incorrectly.
Incorrect results could occur when the optimizer chose an index on a generated column to fetch values.
Changes in MySQL 5.7.30:
JSON Notes
The rapidjson library included with MySQL has been upgraded to the GitHub snapshot of 16 January 2020. A fix for a compiler error encountered when building from the snapshot on Mac OS X has been added.
Packaging Notes
Binary packages that include curl rather than linking to the system curl library have been upgraded to use curl 7.69.0.
The bundled libedit library was upgraded to version 3.1.
Bugs Fixed
InnoDB: The row_upd_clust_rec_by_insert function, which marks a clustered index record as deleted and inserts an updated version of the record into the clustered index, passed an incorrect n_ext value (the total number of external fields) to lower level functions, causing an assertion failure.
InnoDB: An operation performed with the innodb_buffer_pool_evict debug variable set to uncompressed caused an assertion failure.
InnoDB: An add column operation caused an assertion failure. The failure was due to a dangling pointer.
InnoDB: Updating certain InnoDB system variables that take string values raised invalid read errors during Valgrind testing.
InnoDB: An insert statement on a table with a spatial index raised a record type mismatch assertion due to a tuple corruption.
InnoDB: A function that calculates undo log record size could calculate an incorrect length value in the case of a corrupted undo log record, resulting in a malloc failure. Assertion code was added to detect incorrect calculations.
Replication: While an SQL statement was in the process of being rewritten for the binary log so that sensitive information did not appear in plain text, if a SHOW PROCESSLIST statement was used to inspect the query, the query could become corrupted when it was written to the binary log, causing replication to stop. The process of rewriting the query is now kept private, and the query thread is updated only when rewriting is complete.
Replication: When a GRANT or REVOKE statement is only partially executed, an incident event is logged in the binary log, which makes the replication slave's applier thread stop so that the slave can be reconciled manually with the master. Previously, if a failed GRANT or REVOKE statement was the first statement executed in the session, no GTID was applied to the incident event (because the cache manager did not yet exist for the session), causing an error on the replication slave. Also, no incident event was logged in the situation where a GRANT statement created a user but then failed because the privileges had been specified incorrectly, again causing an error on the replication slave. Both these issues have now been fixed.
Replication: When a replication slave has a generated column that the master does not have in that table, with a secondary index on the generated column, the generated expression should be evaluated and the value stored by the storage engine in the secondary index. When row-based binary logging is in use, the replication slave assigns default values to any fields that are not in the master's definition of the table. In the case of a generated column, which does not have a default value, the slave was previously assigning a null or a zero value to the column. This value was then stored by the storage engine in the secondary index, causing both the table and the index to become corrupted. To fix this issue, generated columns in a table on a replication slave are now re-evaluated before the values are sent to the storage engine.
Replication: In the event of an unplanned disconnection of a replication slave from the master, the reference to the master's dump thread might not be removed from the list of registered slaves, in which case statements that accessed the list of slaves would fail. The issue has now been fixed.
Replication: With the settings binlog_format=MIXED, tx_isolation=READ-COMMITTED, and binlog_row_image=FULL, an INSERT ... SELECT query involving a transactional storage engine omitted any columns with a null value from the row image written to the binary log. This happened because when processing INSERT ... SELECT statements, the columns were marked for inserts before the binary logging format was selected. The issue has now been fixed.
The -libs-compat RPM package is now built with system zlib to avoid problems with unrestricted export of symbols in libmysqlclient.so.18.
The Event Scheduler had a memory leak.
Under certain circumstances, a memcached command could result in reading an uninitialized memory buffer, causing a failure.
Using ALTER USER to reset an account MAX_USER_CONNECTIONS value did not take effect until all current account connections terminated, if there were any.
A materialized subquery including a condition in which a column value was used as input to a nondeterministic function produced incorrect results.
CONCAT() and CONCAT_WS() could produce incorrect results in rare cases due to incorrect substring handling.
Scheduling of events could be disturbed by removing events.
Client programs could load authentication plugins from outside the plugin library.
The server did not handle correctly a UNION in which one of the queries contained a subquery that used ORDER BY.
A query with a WHERE clause whose predicate contained a numeric value in scientific notation was not handled correctly.
In addition, attempting to insert a particular integer specified as a string caused a server exit when the string-to-integer conversion was not successful.
Previously, mysqlpump read the [mysql_dump] and [client] groups from option files. mysqlpump now additionally reads the [mysqlpump] group. The [mysql_dump] group is still accepted but is deprecated.
A multi-table UPDATE statement which updated a table joined to a derived table that joined two other tables was not optimized properly as it had been in MySQL 5.6, instead being treated as if STRAIGHT_JOIN had been used with the subquery creating the derived table.
Changes in MySQL 5.7.29:
Audit Log Notes
ANALYZE TABLE statements now produce read audit events.
Packaging Notes
Binary packages that include curl rather than linking to the system curl library have been upgraded to use curl 7.66.0.
Bugs Fixed
InnoDB: os_file_get_parent_dir warnings were encountered when compiling MySQL with GCC 9.2.0.
InnoDB: An internal function (btr_push_update_extern_fields()) used to fetch newly added externally stored fields and update them during a pessimistic update or when going back to a previous version of a record was no longer required. Newly added externally stored fields are updated by a different function. Also, the method used to determine the number of externally stored fields was corrected.
InnoDB: A comparison function found two records to be equal when attempting to merge non-leaf pages of a spatial index. The function was unable to handle this unexpected condition, which resulted in a long semaphore wait and an eventual assertion failure.
InnoDB: A tablespace import operation that failed due to the source and destination tables being defined with different DATA DIRECTORY clauses reported an insufficiently descriptive schema mismatch error. Moreover, if a .cfg file was not present, the same operation would raise an assertion failure. A more informative error message is now reported in both cases before the import operation is terminated due to the data directory mismatch.
InnoDB: Criteria used by the btr_cur_will_modify_tree() function, which detects whether a modifying record needs a modifying tree structure, was insufficient.
InnoDB: An ALTER TABLE ... DISCARD TABLESPACE operation caused a hang condition.
InnoDB: A code regression was addressed by prohibiting unnecessary implicit to explicit secondary index lock conversions for session temporary tables.
InnoDB: A tablespace import operation raised an assertion when the cursor was positioned on a corrupted page while purging delete-marked records. Instead of asserting when encountering a corrupted page, the import operation is now terminated and an error is reported.
Replication: When a member is joining or rejoining a replication group, if Group Replication detects an error in the distributed recovery process (during which the joining member receives state transfer from an existing online member), it automatically switches over to a new donor, and retries the state transfer. The number of times the joining member retries before giving up is set by the group_replication_recovery_retry_count system variable. The Performance Schema table replication_applier_status_by_worker displays the error that caused the last retry. Previously, this error was only shown if the group member was configured with parallel replication applier threads (as set by the slave_parallel_workers system variable). If the group member was configured with a single applier thread, the error was cleared after each retry by an internal RESET SLAVE operation, so it could not be viewed. This was also the case for the output of the SHOW SLAVE STATUS command whether there were single or multiple applier threads. The RESET SLAVE operation is now no longer carried out after retrying distributed recovery, so the error that caused the last retry can always be viewed.
Replication: A memory leak could occur when a failed replication group member tried to rejoin a minority group and was disallowed from doing so.
Replication: If a replication slave was set up using a CHANGE MASTER TO statement that did not specify the master log file name and master log position, then shut down before START SLAVE was issued, then restarted with the option --relay-log-recovery set, replication did not start. This happened because the receiver thread had not been started before relay log recovery was attempted, so no log rotation event was available in the relay log to provide the master log file name and master log position. In this situation, the slave now skips relay log recovery and logs a warning, then proceeds to start replication.
Replication: When GTIDs are enabled on a replication master and slave, and the slave connects to the master with the MASTER_AUTO_POSITION=1 option set, the master must send the slave all the transactions that the slave has not already received, committed, or both. If any of the transactions that should be sent by the master have been already purged from the master's binary log, the master sends the error ER_MASTER_HAS_PURGED_REQUIRED_GTIDS (1789) to the slave, and replication does not start.
The message provided for the error ER_MASTER_HAS_PURGED_REQUIRED_GTIDS has been changed to provide advice on the correct action in this situation, which is for the slave to replicate the missing transactions from another source, or for the slave to be replaced by a new slave created from a more recent backup. The message advises that the master's binary log expiration period can be revised to avoid the situation in future. In addition, the master now identifies the GTIDs of the purged transactions and supplies them in its error log in the warning message ER_FOUND_MISSING_GTIDS (11809), so that you do not need to calculate the missing GTIDs manually.
macOS: On macOS, configuring MySQL with -DWITH_SSL=system caused mysql_config output to incorrectly include internal CMake names for the static SSL libraries.
There could be a mismatch between the version of OpenSSL used to build the server and the version used for other parts of MySQL such as libraries or plugins. This could cause certain features not to work, such as the LDAP authentication plugins. Now the same version of OpenSSL is used for building everything.
Docker packages were missing the LDAP authentication plugins.
The original table name for a field in a derived table was not always displayed correctly.
MySQL Installer was unable to uninstall MySQL 5.7 on Windows Server 2012.
With multiple sessions executing concurrent INSERT ... ON DUPLICATE KEY UPDATE statements into a table with an AUTO_INCREMENT column but not specifying the AUTO_INCREMENT value, inserts could fail with a unique index violation.
With lower_case_table_names=2, SHOW TABLES could fail to display tables with uppercase names.
With keyring_encrypted_file_password set on the command line at server startup, the password value could be visible to system utilities.
With a LOCK TABLES statement in effect, a metadata change for the locked table could cause Performance Schema or SHOW queries for session variables to hang in the opening_tables state.
A SELECT using a WHERE condition of the form A AND (B OR C [OR ...]) resulting in an impossible range led to an unplanned exit of the server.
For JSON-format audit logging, the id field now may contain values larger than 65535. Previously, with heaving logging activity, more than 65536 queries per second could be executed, exceeding the 16 bits permitted for id values.
An incomplete connection packet could cause clients not to properly initialize the authentication plugin name.
Client programs that used the libmysqlclient C client library could exit upon receipt of an OK packet containing malformed session-tracking information.
Under certain conditions, enabling the read_only or super_read_only system variable did not block concurrent DDL statements executed by users without the SUPER privilege.
mysqlpump exits rather than dumping databases that contain an invalid view, by design, but it also failed if an invalid view existed but was not in any of the databases to be dumped.
The current code uses functions that are only available in newer releases of
Oracle Solaris, so disable for now to fix illumos and others. This could be
improved with proper feature tests.
Changes in MySQL 5.7.28
Configuration Notes
It is now possible to compile MySQL 5.7 using OpenSSL 1.1.1, enabling compilation support for MySQL 5.7 against OpenSSL even when OpenSSL 1.0.2 reaches End of Life status at the end of 2019.
All MySQL 5.7 builds now use OpenSSL. MySQL no longer supports using yaSSL as the SSL library, and source distributions no longer include yaSSL.
The WITH_SSL CMake option no longer permits bundled (use yaSSL) as a valid value, and the default option value has changed from bundled to system (use the version of OpenSSL installed on the host system).
sys Schema Notes
The sys.schema_unused_indexes view now filters out unique indexes.
The sys.ps_is_consumer_enabled() function now produces an error rather than returning NULL if the argument is an unknown non-NULL consumer name.
Previously, sys schema sources were maintained in a separate Git repository. sys schema sources now are included with and maintained within MySQL source distributions (under scripts/sys_schema).
The sys.version view is deprecated and will be removed in a future MySQL version. Affected applications should be adjusted to use an alternative instead. For example, use the VERSION() function to retrieve the MySQL server version.
Bugs Fixed
InnoDB: An ALTER TABLE ... ENCRYPTION = 'Y' operation on a MyISAM table failed to raise an error indicating that the storage engine does not support encryption. The INPLACE algorithm did not check encryption support before updating metadata in the .frm file. To address this issue, an encryption support flag was added. The ALTER TABLE ... ENCRYPTION = 'Y' operation now checks the flag and reports an error if the storage engine does not support encryption.
InnoDB: Delete marked rows were able to acquire an external read lock before a partial rollback was completed. The external read lock prevented conversion of an implicit lock to an explicit lock during the partial rollback, causing an assertion failure.
InnoDB: A long running ALTER TABLE ... ADD INDEX operation with concurrent inserts caused semaphore waits.
Replication: A deadlock involving three threads could occur if a START SLAVE statement was issued to start the SQL thread on a slave while the SQL thread was still in the process of being stopped, and a request for the slave status was made at the same time. The issue has now been fixed by releasing a lock earlier while the SQL thread is being stopped.
Replication: A locking issue in the WAIT_FOR_EXECUTED_GTID_SET() function could cause the server to hang in certain circumstances. The issue has now been corrected.
Replication: On a multi-threaded slave with GTIDs in use and MASTER_AUTO_POSITION set to ON, following an unexpected halt the slave would attempt relay log recovery, which failed if relay logs had been lost, preventing replication from starting. However, this step was unnecessary as GTID auto-positioning can be used to restore any missing transactions. In a recovery situation, the slave now checks first whether MASTER_AUTO_POSITION is set to ON, and if it is, skips relay log recovery.
When generating C source from SQL scripts, Some utf8-encoded characters were split across lines. Thanks to Przemysław Skibiński for the patch.
For Solaris, mysqld.cc contained a prototype for memcntl() that is no longer needed. The prototype has been removed.
For Solaris, -DWITH_SSL=system did not work when compiling with GCC.
MySQL builds configured with -DWITHOUT_SERVER=1 failed.
For the keyring_aws plugin, some valid region values for the keyring_aws_region system variable were rejected.
For debug builds, an assertion could be raised during UNION queries when computing the combined data type of a GEOMETRY column and SELECT * FROM (SELECT NULL).
For authentication using an LDAP authentication plugin, if the user DN portion was empty and group mapping was configured, authentication assigned an incorrect user DN and skipped the user search.
mysqlpump produced an error when run against a server older than MySQL 5.7.
A possible integer overflow due to unsigned integer type casting could lead to later buffer overflow due to arbitrary size memory allocation.
Attempted use of a freed object during MeCab plugin initialization caused a segmentation fault.
For MySQL installed using RPM packages, an initialization script that tested server connectivity misbehaved if the client account authenticated using an LDAP authentication plugin.
Improper locking during storage engine initialization could cause a server exit.
On a GTID-enabled server, concurrent execution of DROP USER and a prepared statement that accessed a view could deadlock.
A query with a WHERE clause whose predicate contained a numeric value in scientific notation was not handled correctly.
VS2019 produced compilation errors with debug compilation selected due to use of the /ZI flag. Now /Z7 is used instead.
The client library could dereference a null pointer while fetching result set metadata from the server.
In READ UNCOMMITTED isolation level, a segmentation fault occurred under heavy load from memcached clients. An externally stored BLOB column that was being updated by one transaction was read by another transaction as having a NULL value and a non-zero data length.
Arguments for the TIMESTAMPADD() function could be reversed for prepared statements.
With the thread_pool plugin enabled, the sys.processlist and sys.session views displayed a thread name rather than the actual user name.
The delete_latency column in the sys.schema_index_statistics view incorrectly referred to the SUM_TIMER_INSERT column of the Performance Schema table_io_waits_summary_by_index_usage table rather than the SUM_TIMER_DELETE column.
In output from the sys.diagnostics() procedure, the latency column for the user_summary_by_file_io_type view was incorrectly displayed in raw picoseconds rather than as a formatted value.
MySQL Enterprise Encryption functions could apply Diffie-Hellman (DH) methods to non-DH keys, resulting in unpredictable results or server exit.
Password masking was incomplete for SHOW PROCESSLIST and some INFORMATION_SCHEMA and Performance Schema tables.
The -DWITH_EXAMPLE_STORAGE_ENGINE=1 CMake option was ignored but should not have been. If -DWITH_EXAMPLE_STORAGE_ENGINE=0 is given, the EXAMPLE storage engine is built as a plugin.
Changes in MySQL 5.7.27:
Keyring Notes
The keyring_aws plugin has been updated to use the latest AWS SDK and so that it works with OpenSSL 1.1.
Packaging Notes
Binary packages that include curl rather than linking to the system curl library now use curl 7.64.0.
X Plugin Notes
On Windows, X Plugin logged some messages that were unnecessary or insufficiently informative. The messages have been removed or improved as appropriate.
Functionality Added or Changed
Microsoft Windows: A new warning message now reminds DBAs that connections made using the MySQL named pipe on Windows has limited the permissions a connector can request on the named pipe.
Previously, the named_pipe_full_access_group system variable was set to a value that maps to the built-in Windows Everyone group (SID S-1-1-0) by default. However, this group is not ideal and should be replaced with a group that restricts its membership for connectors that are unable to request fewer permissions on the MySQL named pipe.
The new warning is written to the error log at startup if the string value assigned to named_pipe_full_access_group is '*everyone*' (or the Windows System Language equivalent) and named pipes are enabled. In addition, the warning is written to the error log and raised to the client if the system variable is reset to the Everyone group at runtime.
Bugs Fixed
InnoDB: Insufficient memory barriers in the rw-lock implementation caused deadlocks on ARM.
Thanks to Yibo Cai from Arm Technology for the contribution.
InnoDB: Manually changing the system time while the MySQL server was running caused page cleaner thread delays.
InnoDB: During log application, after an OPTIMIZE TABLE operation, InnoDB did not populate virtual columns before checking for virtual column index updates.
InnoDB: An INSERT operation involving a generated virtual BLOB column resulted a secondary index being updated with an incorrect value.
InnoDB: A full-text cache lock taken when data is synchronized was not released if the full-text cache size exceeded the full-text cache size limit.
InnoDB: Client sessions using different auto_increment_increment values while performing concurrent insert operations could cause a duplicate key error.
Partitioning: ALTER TABLE ... EXCHANGE PARTITION failed with the error Non matching attribute 'ROW_FORMAT' between partition and table when the partitioned table had partitions using different row formats, even when the partition to be exchanged used the same row format as the non-partitioned table.
Replication: The error message that is issued for a discrepancy between the number of group members and the auto-increment interval incorrectly referred to the group_replication_auto_increment_increment system variable, instead of the auto_increment_increment system variable. The value of auto_increment_increment is changed to the value specified by group_replication_auto_increment_increment when Group Replication starts, but only if auto_increment_increment and auto_increment_offset have their default values, and from MySQL 8.0, only in multi-primary mode. The value of auto_increment_increment was always the value that was checked for the error message, and it has now been corrected to give the accurate system variable name.
Replication: When events generated by one MySQL server instance were written to the binary log of another instance, the second server implicitly assumed that the first server supported the same number of binary log event types as itself. Where this was not the case, the event header was handled incorrectly. The issue has now been fixed. Thanks to Facebook for the contribution.
Replication: In Group Replication, joining members could wrongly identify themselves as incompatible with an existing replication group even if there were members at the same version already in the group, because they checked against all other members, including the member at the highest version. Joining members also included their own version in the compatibility check. Now, joining members only compare themselves with the existing group member at the lowest version, and do not count their own version.
Replication: If a FLUSH LOGS statement was issued before the binary log file was initialized, the statement attempted to write a binary log rotation event to the uninitialized file. The server now checks first that a binary log file is available.
Replication: When a MEMORY table is implicitly deleted on a master following a server restart, the master writes a DELETE statement to the binary log so that slaves also empty the table. This generated event now includes a comment in the binary log so that the reason for the DELETE statement is easy to identify. Thanks to Daniël van Eeden for the contribution.
Replication: With statement-based replication in use, if super_read_only was set to ON for a server at the point when a no-op transaction was between its UPDATE and COMMIT operations, the transaction was written to the binary log and assigned a GTID. The transaction is now blocked in this situation. From MySQL 8.0, the value of super_read_only cannot be changed while a transaction is in progress.
Replication: The group communication engine for Group Replication (XCom, a Paxos variant) did not handle out of memory errors in an appropriate way. If memory could not be allocated to make a copy of the payload for a message, an error was logged but the message was still sent, with a null payload. The Group Communication System (GCS) on the receiving member discarded the message as empty, and the XCom instance on the receiving member accepted this action and did not retry, resulting in the message effectively being skipped. This caused the GTID set on the receiving member to diverge from the group, leading to replication errors. XCom now terminates gracefully if it experiences an out of memory error, so that this situation cannot occur.
Replication: In query log events in the binary log, the thread ID used for the execution of DROP TABLE and DELETE statements was identified incorrectly or not at all. On a multi-threaded replication slave, where temporary tables were involved (which require the correct thread ID as they are session specific), this omission resulted in errors when using mysqlbinlog to replay the binary log for point-in-time recovery. The thread ID is now set correctly.
Replication: When a slave server logs master status and connection information to a table (master_info_repository=TABLE), which is the default in MySQL 8.0, the mysql.slave_master_info table was not being updated on shutdown if the server was in super read only mode (super_read_only=ON). No error was written to the error log at this time, but replication failed after server startup because the master log file and master log position information was out of date. The thread that updates the master info log at shutdown is now excluded from read-only checks like other replication threads are, so it can update the table even if the server is in super read only mode. Error handling for a slave that is shutting down has also been improved so that any failure to write to the slave status logs results in an error in the error log.
An overly strict assertion could be raised during sorting of stored program local objects.
Installing from RPM packages could result in an error log with incorrect permissions.
Enabling audit log encryption could cause a server exit.
On Debian and Ubuntu, MySQL packages did not enable mysql.service after upgrades from native MySQL packages.
The server did not properly close shared-memory connections when an error occurred, which could result in unexpected server behavior.
MySQL Installer did not install OpenSSL DLL dependencies if the Development component was not selected.
The parser could leak memory for certain multiple-statement queries.
CREATE USER and ALTER USER did not check the validity of a hashed authentication string when used with IDENTIFIED WITH auth_plugin AS 'hash_string' syntax.
For InnoDB tables that contained an index on a VARCHAR column and were created prior to MySQL 5.7.23, some simple ALTER TABLE statements that should have been done in place were performed with a table rebuild after an upgrade to MySQL 5.7.23 or higher.
HANDLER statements did not always work correctly with tables having generated columns.
Session-tracking information in the client/server protocol could be mishandled.
With the PAD_CHAR_TO_FULL_LENGTH SQL mode enabled, password changes failed, with no warning or error reported.
The audit_log plugin did not log UNINSTALL PLUGIN audit_log statements.
audit_log filtering operations could leak memory.
An index defined on a virtual generated column could fail to be updated if the column had a base column in a foreign key relationship.
Privileges for dropping some Performance Schema tables were checked incorrectly.
A query that employed a derived table which included an ORDER BY was not always handled correctly.
Base columns were not excluded from index-only access by a generated column.
A thread pool group could be blocked when a thread process tick time exceeded the maximum permitted value. The tick time now uses a larger data type to permit larger values.
MySQL does not support OpenSSL session tickets, but did not set the SSL_OP_NO_TICKET flag to inform OpenSSL of that. The flag is now set.
The audit_null plugin did not properly check for a null event record.
UpdateXML() did not always free memory properly in certain cases.
Empty values in the name column of the mysql.plugin system table caused the server to exit during startup.
With the thread_pool plugin enabled, the Performance Schema status_by_thread table contained no data.
If an INSTALL PLUGIN statement contained invalid UTF-8 characters in the shared library name, it caused the server to hang (or to raise an assertion in debug builds).
Inner tables of different semijoin nests were interleaved during materialization, which could lead to a different result for the same query when it used a different query plan. To keep this from occurring, a check is added to prevent such interleaving.
A query involving GROUP BY on a TIMESTAMP column resulted in a duplicate entry for key (ER_DUP_ENTRY) error. This problem arose when TIMESTAMP values were inserted into a table using a given setting for the time zone and these values were later fetched after the time zone setting had been changed, such that at least some of the inserted TIMESTAMP values occurred during the hour that the time changed from standard to daylight time (DST) in the new time zone, during which time the same TIMESTAMP value can exist twice. Now, when the server would otherwise return the error DUPLICATE ENTRY FOR KEY 'group_key', if the grouping involves a TIMESTAMP column, it instead raises the error Grouping on temporal is non-deterministic for time zones having DST. Please consider switching to UTC for this query.
In addition, it is suggested to set explicit_defaults_for_timestamp to ON as well as one or more of MODE_NO_ZERO_IN_DATE, MODE_NO_ZERO_DATE, or MODE_INVALID_DATES as part of the server SQL mode to help avoid this issue.
Changes in MySQL 5.7.25
Deprecation and Removal Notes
The resolveip and resolve_stack_dump utilities are now deprecated and will be removed in MySQL 8.0. nslookup, host, or dig can be used instead of resolveip. Stack traces from official MySQL builds are always symbolized, so there is no need to use resolve_stack_dump.
Pluggable Authentication
If the LDAP port number is configured as 636 or 3269, the plugin now uses LDAPS (LDAP over SSL) instead of LDAP. The port number is settable using the authentication_ldap_sasl_server_port or authentication_ldap_simple_server_port system variable. (LDAPS differs from startTLS.)
Previously, for LDAP authentication with proxying, LDAP authentication plugins used the first group name returned by the LDAP server as the MySQL proxy user account name. The authentication string for a MySQL account now can specify a list of groups to match, in preference order, and can optionally map the matching group name to a specified MySQL proxy user name. See LDAP Pluggable Authentication.
Security Notes
The linked OpenSSL library for the MySQL Commercial Server has been updated to version 1.0.2q. Issues fixed in the new OpenSSL version are described at http://www.openssl.org/news/vulnerabilities.html.
This change does not affect the Oracle-produced MySQL Community build of MySQL Server, which uses the yaSSL library instead.
Functionality Added or Changed
Microsoft Windows: The access control granted to clients on the named pipe created by the MySQL server now is set to the minimum necessary for successful communication on Windows. Newer MySQL client software can open named pipe connections without any additional configuration. If older client software cannot be upgraded immediately, the new named_pipe_full_access_group server system variable can be used to give a Windows group the necessary permissions to open a named pipe connection. Membership in the full-access group should be restricted and temporary.
Bugs Fixed
InnoDB: A dangling pointer caused a memory leak.
InnoDB: An ON DELETE CASCADE operation on table with a foreign key constraint and an indexed virtual column caused the server to exit.
InnoDB: An incorrectly written DML log involving a virtual column value raised an assertion.
InnoDB: Using the O_DIRECT_NO_FSYNC innodb_flush_method setting could cause the system to hang due to file system metadata becoming unsynchronized. To prevent this issue from occurring in O_DIRECT_NO_FSYNC mode, InnoDB now calls fsync() after creating a new file, after increasing file size, and after closing a file. The fsync() system call is still skipped after each write operation.
With the changes described above, O_DIRECT_NO_FSYNC mode can now be safely used on EXT4 and XFS file systems.
InnoDB: An assertion was raised when attempting to write to a tablespace file greater than 4GB in size on a 64-bit Windows system. The failure was due to a narrowing cast.
Partitioning: Repeated ALTER TABLE statements on partitioned tables containing BLOB or TEXT columns were not always handled correctly.
Partitioning: ALTER TABLE ... EXCHANGE PARTITION did not work when the partitioned table had one or more partition definitions using the DATA DIRECTORY option. This fix supports partitioned tables using the InnoDB storage engine only.
Replication: A patch to correct the handling of quotes for identifiers in ROLLBACK TO SAVEPOINT statements in the binary log was not correctly applied to subsequent MySQL versions.
Replication: Following a patch in MySQL 5.7.23, LOAD DATA statements stopped statement-based replication from a MySQL 5.7.22 master to a replication slave at a later release. The problem has now been fixed.
Replication: In some circumstances, the CHANGE MASTER TO statement could not be used on a replication slave if the master info log had been changed from a table (master_info_repository=TABLE) into a file (master_info_repository=FILE).
Replication: When the system variables binlog_transaction_dependency_tracking and binlog_transaction_dependency_history_size were set or read, the types of lock that were required could result in a deadlock scenario, because the same locks were also required for working with the active binary logs. A new lock type is now used instead for access to the transaction dependency tracking system variables, so that this deadlock cannot occur.
Replication: The PURGE BINARY LOGS TO 'log_name' statement failed for binary log files that had been moved to another location using mysqlbinlogmove. Such files are still listed in the binary log index file, but they are listed using an absolute path, rather than a path relative to the directory where the binary log files are normally stored. MySQL Server can now locate and purge moved binary log files successfully.
Replication: If autocommit was set to 0 for a replication slave or Group Replication group member where GTIDs were in use and super_read_only=ON was set, server shutdown was prevented by a transaction that did not complete. The transaction was attempting to save GTIDs to the mysql.gtid_executed table, but the update failed because super_read_only=ON was set. (With autocommit set to 1, the transaction would complete in this situation, and the mysql.gtid_executed table would instead be updated at server startup.) Now, the check for the super_read_only setting is skipped for this task, so the transaction is able to save the GTIDs to the mysql.gtid_executed table and complete regardless of the combination of super_read_only and autocommit settings.
Replication: An assertion was raised in debug builds if an XA ROLLBACK statement was issued for an unknown transaction identifier when the gtid_next value had been set manually. The server now does not attempt to update the GTID state if an XA ROLLBACK statement fails with an error.
Replication: The value returned by a SHOW SLAVE STATUS statement for the total combined size of all existing relay log files (Relay_Log_Space) could become much larger than the actual disk space used by the relay log files. The I/O thread did not lock the variable while it updated the value, so the SQL thread could automatically delete a relay log file and write a reduced value before the I/O thread finished updating the value. The I/O thread then wrote its original size calculation, ignoring the SQL thread's update and so adding back the space for the deleted file. The Relay_Log_Space value is now locked during updates to prevent concurrent updates and ensure an accurate calculation.
Replication: If the relay log index file was temporarily locked for viewing by a backup process for a replication slave, and MySQL Server also attempted to access the file at that time for rename or delete operations, the backup completed with warnings, but MySQL Server experienced an unexpected halt. MySQL Server now retries the file access operation a number of times in case this or a similar scenario is the explanation and the file becomes available again before long.
Replication: With sync_binlog=1 set, if the binary log was rotated during a commit before the binary log end position was updated, replication stopped on the slave because the server attempted to use the old binary log end position with the new binary log file. The server now compares the binary log file name with the active binary log file when updating the binary log end position, so that the issue does not occur.
Replication: When adding a new member to a group, if the certification information was too big to transmit, an event was generated that caused failures in all group members. To avoid this situation, now if the certification information is too large an error is generated which makes the joining member leave the group.
Replication: When stopping replication, any channels that had pending transactions could cause a deadlock in Group Replication.
The keyring_aws plugin was missing from Commercial packages for macOS.
The supported macOS versions for this plugin now are macOS 10.13 and 10.14.
MySQL Enterprise Firewall did not work well if the audit_log plugin was installed.
The server permitted creation of databases with the same name as redo log files, which could result in unexpected server behavior. Such names are no longer permitted as database names.
Removal of Sun RPC and XDR from glibc into a separate libtirpc library caused problems with libasan on some platforms.
In LDAP group search filter values, special characters were not escaped. Special characters in the user DN now are escaped with their hexadecimal equivalant as follows:
* => \2a
( => \28
) => \29
\ => \5c
\0 => \00
A memory leak was caused by GET_LOCK() calls with a zero timeout that failed due to concurrent connections holding the same user-level lock.
mysqlpump did not free all allocated resources when it encountered an error, resulting in memory leaks.
For debug builds, the server could exit when attempting to roll back CREATE USER statements.
Mishandling of deprecated system variables could cause output from queries on the Performance Schema variables_by_thread table to be incorrect.
When a subquery contained a UNION, the count of the number of subquery columns was calculated incorrectly.
On a GTID-enabled server, concurrent statements on the INFORMATION_SCHEMA.COLUMNS table could deadlock.
Comparing log file names as strings using the memcmp() function resulted in uninitialized memory read errors. The comparison now uses the strncmp() function. Thanks to Zsolt Parragi and Laurynas Biveinis for their contributions.
The optimizer skipped the second column in a composite index when executing an inner join with a LIKE clause against the second column.
CREATE TABLE ... SELECT could create date columns with “zero” date default values when it should have created them without a default value.
The transformation of IN subquery predicates into semijoins was not handled correctly for a very large number of tables.
Server mishandling of SIGHUP signals could result in a server exit.
When the character set of one string comparison operand was a superset of the character set of the other operand, some comparisons were disallowed that should be permitted by converting the operand with the “smaller” character set to the “larger” character set. utf8mb4 and utf32 are considered to be a superset of any other encoding.
Improper memory handling by account management statements could result in server misbehavior.
Executing a prepared statement to do a multiple-row insert with large number of placeholders consumed excessive memory and could execute slowly.
The parser accepted invalid SET statement syntax in trigger definitions that could result in a server exit.
The server failed to start if the keyring_encrypted_file plugin keyring file was invalid.
Keyring migration failed with source and destination keyring plugins of keyring_okv and keyring_encrypted_file, respectively.
When executing a prepared statement with a procedure call with the CURSOR_TYPE_READ_ONLY flag set, the client library hung if the procedure performed a SELECT that returned an empty result set.
The parser performed some out-of-memory checks incorrectly.
When binlog_format is ROW or MIXED, operations on temporary tables are not logged. Previously, the exception to this rule was that when the connection was terminated at the end of the session, the statement DROP TEMPORARY TABLE IF EXISTS was logged for any temporary tables that had been opened in the session. For row-based replication, this behavior caused an unnecessary write to the binary log, and added a transaction sequence number for the GTID where these were enabled.
Now, when a temporary table is created in a session, the binary logging format is tracked. The DROP TEMPORARY TABLE IF EXISTS statement is only logged at the end of the session if statement-based format was in effect when the temporary table was created, so the CREATE TEMPORARY TABLE statement was logged. If row-based or mixed-format binary logging was in use when the table was created, the DROP TEMPORARY TABLE IF EXISTS statement is not logged.
Thanks to Laurynas Biveinis for the patch.
DML statements using IGNORE were not always handled correctly on tables having generated columns.
A query employing a dynamic range and an index merge could use more memory than expected.
Changes in MySQL 5.7.24
Deprecation and Removal Notes
* InnoDB; Partitioning: Support for placing table partitions in shared tablespaces is deprecated and will be removed in a future version of MySQL. Shared tablespaces include the system tablespace and general tablespaces. For information about identifying partitions in shared tablespaces and moving them to file-per-table tablespaces, see Preparing Your Installation for Upgrade.
* InnoDB: Support for TABLESPACE = innodb_file_per_table and TABLESPACE = innodb_temporary clauses with CREATE TEMPORARY TABLE is deprecated and will be removed in a future MySQL version.
Functionality Added or Changed
Replication: Use the group_replication_exit_state_action option to configure how Group Replication behaves when a member leaves the group involuntarily, for example when it is expelled from the group due to an unstable network connection. When group_replication_exit_state_action is set to ABORT_SERVER, upon exiting the group unintentionally, the instance shuts MySQL down, and when group_replication_exit_state_action is set to READ_ONLY the instance sets MySQL to super read only mode instead and its state is set to ERROR.
Previously, file I/O performed in the I/O cache in the mysys library was not instrumented, affecting in particular file I/O statistics reported by the Performance Schema about the binary log index file. Now, this I/O is instrumented and Performance Schema statistics are accurate. Thanks to Yura Sorokin for the contribution.
The zlib library version bundled with MySQL was raised from version 1.2.3 to version 1.2.11. MySQL implements compression with the help of the zlib library.
The zlib compressBound() function in zlib 1.2.11 returns a slightly higher estimate of the buffer size required to compress a given length of bytes than it did in zlib version 1.2.3. The compressBound() function is called by InnoDB functions that determine the maximum row size permitted when creating compressed InnoDB tables or inserting rows into compressed InnoDB tables. As a result, CREATE TABLE ... ROW_FORMAT=COMPRESSED or INSERT operations with row sizes very close to the maximum row size that were successful in earlier releases could now fail.
Bugs Fixed
* InnoDB: An ALTER TABLE operation that added a primary key produced a segmentation fault.
* InnoDB: A query that scanned the primary key of a table did not return the expected result.
* InnoDB: A query interruption during a lock wait caused an error.
* InnoDB: An index record was not found when updating a secondary index defined on a generated column.
* InnoDB: The update log applied as part of an online ALTER TABLE operation did not take into account the computed value of the generated column in the old row while updating the secondary index.
* InnoDB: An unsupported DDL operation involving a foreign key constraint raised an assertion.
* InnoDB: An attempted foreign key check on a discarded table caused a segmentation fault.
* InnoDB: An assertion was raised during an OPTIMIZE TABLE operation.
* InnoDB: A foreign key constraint name was duplicated during a rename table operation, causing a failure during later query execution.
* InnoDB: In a function called before the execution of a statement in a stored procedure, a read and write operation on trx->lock.start_stmt was not protected by a mutex.
* InnoDB: An error occurred during a DDL operation due to a mismatch in a REDUNDANT row format calculation that determines the length of the online log.
* InnoDB: The location of the Innodb Merge Temp File that reported by the wait/io/file/innodb/innodb_temp_file Performance Schema instrument was incorrect.
* Partitioning: When a CREATE TABLE ... PARTITION BY ... statement failed due to an invalid partition definition, the server did not remove any partition files which might have been created prior to encountering the invalid PARTITION clause.
* Partitioning: It was possible to perform FLUSH TABLES FOR EXPORT on a partitioned table created with innodb_file_per_table=1 after discarding its tablespace. Attempting to do so now raises ER_TABLESPACE_DISCARDED.
* Replication: When the binlog_group_commit_sync_delay system variable is set to a wait time to delay synchronization of transactions to disk, and the binlog_group_commit_sync_no_delay_count system variable is also set to a number of transactions, the MySQL server exits the wait procedure if the specified number of transactions is reached before the specified wait time is reached. The server manages this process by checking on the transaction count after a delta of one tenth of the time specified by binlog_group_commit_sync_delay has elapsed, then subtracting that interval from the remaining wait time.
more...
