Python 3.9.5 final
Security
bpo-43434: Creating a sqlite3.Connection object now also produces a sqlite3.connect auditing event. Previously this event was only produced by sqlite3.connect() calls. Patch by Erlend E. Aasland.
bpo-43882: The presence of newline or tab characters in parts of a URL could allow some forms of attacks.
Following the controlling specification for URLs defined by WHATWG urllib.parse() now removes ASCII newlines and tabs from URLs, preventing such attacks.
bpo-43472: Ensures interpreter-level audit hooks receive the cpython.PyInterpreterState_New event when called through the _xxsubinterpreters module.
bpo-36384: ipaddress module no longer accepts any leading zeros in IPv4 address strings. Leading zeros are ambiguous and interpreted as octal notation by some libraries. For example the legacy function socket.inet_aton() treats leading zeros as octal notatation. glibc implementation of modern inet_pton() does not accept any leading zeros. For a while the ipaddress module used to accept ambiguous leading zeros.
bpo-43075: Fix Regular Expression Denial of Service (ReDoS) vulnerability in urllib.request.AbstractBasicAuthHandler. The ReDoS-vulnerable regex has quadratic worst-case complexity and it allows cause a denial of service when identifying crafted invalid RFCs. This ReDoS issue is on the client side and needs remote attackers to control the HTTP server.
bpo-42800: Audit hooks are now fired for frame.f_code, traceback.tb_frame, and generator code/frame attribute access.
Core and Builtins
bpo-43105: Importlib now resolves relative paths when creating module spec objects from file locations.
bpo-42924: Fix bytearray repetition incorrectly copying data from the start of the buffer, even if the data is offset within the buffer (e.g. after reassigning a slice at the start of the bytearray to a shorter byte string).
Library
bpo-43993: Update bundled pip to 21.1.1.
bpo-43937: Fixed the turtle module working with non-default root window.
bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0
bpo-43920: OpenSSL 3.0.0: load_verify_locations() now returns a consistent error message when cadata contains no valid certificate.
bpo-43607: urllib can now convert Windows paths with \\?\ prefixes into URL paths.
bpo-43284: platform.win32_ver derives the windows version from sys.getwindowsversion().platform_version which in turn derives the version from kernel32.dll (which can be of a different version than Windows itself). Therefore change the platform.win32_ver to determine the version using the platform module’s _syscmd_ver private function to return an accurate version.
bpo-42248: [Enum] ensure exceptions raised in _missing__ are released
bpo-43799: OpenSSL 3.0.0: define OPENSSL_API_COMPAT 1.1.1 to suppress deprecation warnings. Python requires OpenSSL 1.1.1 APIs.
bpo-43794: Add ssl.OP_IGNORE_UNEXPECTED_EOF constants (OpenSSL 3.0.0)
bpo-43789: OpenSSL 3.0.0: Don’t call the password callback function a second time when first call has signaled an error condition.
bpo-43788: The header files for ssl error codes are now OpenSSL version-specific. Exceptions will now show correct reason and library codes. The make_ssl_data.py script has been rewritten to use OpenSSL’s text file with error codes.
bpo-43655: tkinter dialog windows are now recognized as dialogs by window managers on macOS and X Window.
bpo-43534: turtle.textinput() and turtle.numinput() create now a transient window working on behalf of the canvas window.
bpo-43522: Fix problem with hostname_checks_common_name. OpenSSL does not copy hostflags from struct SSL_CTX to struct SSL.
bpo-42967: Allow bytes separator argument in urllib.parse.parse_qs and urllib.parse.parse_qsl when parsing str query strings. Previously, this raised a TypeError.
bpo-43176: Fixed processing of a dataclass that inherits from a frozen dataclass with no fields. It is now correctly detected as an error.
bpo-41735: Fix thread locks in zlib module may go wrong in rare case. Patch by Ma Lin.
bpo-36470: Fix dataclasses with InitVars and replace(). Patch by Claudiu Popa.
bpo-32745: Fix a regression in the handling of ctypes’ ctypes.c_wchar_p type: embedded null characters would cause a ValueError to be raised. Patch by Zackery Spytz.
Documentation
bpo-43959: The documentation on the PyContextVar C-API was clarified.
bpo-43938: Update dataclasses documentation to express that FrozenInstanceError is derived from AttributeError.
bpo-43755: Update documentation to reflect that unparenthesized lambda expressions can no longer be the expression part in an if clause in comprehensions and generator expressions since Python 3.9.
bpo-43739: Fixing the example code in Doc/extending/extending.rst to declare and initialize the pmodule variable to be of the right type.
Tests
bpo-43961: Fix test_logging.test_namer_rotator_inheritance() on Windows: use os.replace() rather than os.rename(). Patch by Victor Stinner.
bpo-43842: Fix a race condition in the SMTP test of test_logging. Don’t close a file descriptor (socket) from a different thread while asyncore.loop() is polling the file descriptor. Patch by Victor Stinner.
bpo-43811: Tests multiple OpenSSL versions on GitHub Actions. Use ccache to speed up testing.
bpo-43791: OpenSSL 3.0.0: Disable testing of legacy protocols TLS 1.0 and 1.1. Tests are failing with TLSV1_ALERT_INTERNAL_ERROR.
Windows
bpo-35306: Avoid raising errors from pathlib.Path.exists() when passed an invalid filename.
bpo-38822: Fixed os.stat() failing on inaccessible directories with a trailing slash, rather than falling back to the parent directory’s metadata. This implicitly affected os.path.exists() and os.path.isdir().
bpo-26227: Fixed decoding of host names in socket.gethostbyaddr() and socket.gethostbyname_ex().
bpo-40432: Updated pegen regeneration script on Windows to find and use Python 3.8 or higher. Prior to this, pegen regeneration already required 3.8 or higher, but the script may have used lower versions of Python.
bpo-43745: Actually updates Windows release to OpenSSL 1.1.1k. Earlier releases were mislabelled and actually included 1.1.1i again.
bpo-43492: Upgrade Windows installer to use SQLite 3.35.5.
macOS
bpo-42119: Fix check for macOS SDK paths when building Python. Narrow search to match contents of SDKs, namely only files in /System/Library, /System/IOSSupport, and /usr other than /usr/local. Previously, anything under /System was assumed to be in an SDK which causes problems with the new file system layout in 10.15+ where user file systems may appear to be mounted under /System. Paths in /Library were also incorrectly treated as SDK locations.
bpo-44009: Provide “python3.x-intel64” executable to allow reliably forcing macOS universal2 framework builds to run under Rosetta 2 Intel-64 emulation on Apple Silicon Macs. This can be useful for testing or when universal2 wheels are not yet available.
bpo-43492: Update macOS installer to use SQLite 3.35.4.
IDLE
bpo-43655: IDLE dialog windows are now recognized as dialogs by window managers on macOS and X Window.
Python 3.8.10 final
Security
bpo-43434: Creating a sqlite3.Connection object now also produces a sqlite3.connect auditing event. Previously this event was only produced by sqlite3.connect() calls. Patch by Erlend E. Aasland.
bpo-43472: Ensures interpreter-level audit hooks receive the cpython.PyInterpreterState_New event when called through the _xxsubinterpreters module.
bpo-43075: Fix Regular Expression Denial of Service (ReDoS) vulnerability in urllib.request.AbstractBasicAuthHandler. The ReDoS-vulnerable regex has quadratic worst-case complexity and it allows cause a denial of service when identifying crafted invalid RFCs. This ReDoS issue is on the client side and needs remote attackers to control the HTTP server.
Core and Builtins
bpo-43105: Importlib now resolves relative paths when creating module spec objects from file locations.
bpo-42924: Fix bytearray repetition incorrectly copying data from the start of the buffer, even if the data is offset within the buffer (e.g. after reassigning a slice at the start of the bytearray to a shorter byte string).
Library
bpo-43993: Update bundled pip to 21.1.1.
bpo-43937: Fixed the turtle module working with non-default root window.
bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0
bpo-43920: OpenSSL 3.0.0: load_verify_locations() now returns a consistent error message when cadata contains no valid certificate.
bpo-43607: urllib can now convert Windows paths with \\?\ prefixes into URL paths.
bpo-43284: platform.win32_ver derives the windows version from sys.getwindowsversion().platform_version which in turn derives the version from kernel32.dll (which can be of a different version than Windows itself). Therefore change the platform.win32_ver to determine the version using the platform module’s _syscmd_ver private function to return an accurate version.
bpo-42248: [Enum] ensure exceptions raised in _missing__ are released
bpo-43799: OpenSSL 3.0.0: define OPENSSL_API_COMPAT 1.1.1 to suppress deprecation warnings. Python requires OpenSSL 1.1.1 APIs.
bpo-43794: Add ssl.OP_IGNORE_UNEXPECTED_EOF constants (OpenSSL 3.0.0)
bpo-43789: OpenSSL 3.0.0: Don’t call the password callback function a second time when first call has signaled an error condition.
bpo-43788: The header files for ssl error codes are now OpenSSL version-specific. Exceptions will now show correct reason and library codes. The make_ssl_data.py script has been rewritten to use OpenSSL’s text file with error codes.
bpo-43655: tkinter dialog windows are now recognized as dialogs by window managers on macOS and X Window.
bpo-43534: turtle.textinput() and turtle.numinput() create now a transient window working on behalf of the canvas window.
bpo-43522: Fix problem with hostname_checks_common_name. OpenSSL does not copy hostflags from struct SSL_CTX to struct SSL.
bpo-42967: Allow bytes separator argument in urllib.parse.parse_qs and urllib.parse.parse_qsl when parsing str query strings. Previously, this raised a TypeError.
bpo-43176: Fixed processing of a dataclass that inherits from a frozen dataclass with no fields. It is now correctly detected as an error.
bpo-34463: Fixed discrepancy between traceback and the interpreter in formatting of SyntaxError with lineno not set (traceback was changed to match interpreter).
bpo-41735: Fix thread locks in zlib module may go wrong in rare case. Patch by Ma Lin.
bpo-26053: Fixed bug where the pdb interactive run command echoed the args from the shell command line, even if those have been overridden at the pdb prompt.
bpo-36470: Fix dataclasses with InitVars and replace(). Patch by Claudiu Popa.
bpo-28577: The hosts method on 32-bit prefix length IPv4Networks and 128-bit prefix IPv6Networks now returns a list containing the single Address instead of an empty list.
bpo-32745: Fix a regression in the handling of ctypes’ ctypes.c_wchar_p type: embedded null characters would cause a ValueError to be raised. Patch by Zackery Spytz.
Documentation
bpo-43959: The documentation on the PyContextVar C-API was clarified.
bpo-43938: Update dataclasses documentation to express that FrozenInstanceError is derived from AttributeError.
bpo-43739: Fixing the example code in Doc/extending/extending.rst to declare and initialize the pmodule variable to be of the right type.
Tests
bpo-43842: Fix a race condition in the SMTP test of test_logging. Don’t close a file descriptor (socket) from a different thread while asyncore.loop() is polling the file descriptor. Patch by Victor Stinner.
bpo-43811: Tests multiple OpenSSL versions on GitHub Actions. Use ccache to speed up testing.
bpo-43791: OpenSSL 3.0.0: Disable testing of legacy protocols TLS 1.0 and 1.1. Tests are failing with TLSV1_ALERT_INTERNAL_ERROR.
Windows
bpo-35306: Avoid raising errors from pathlib.Path.exists() when passed an invalid filename.
bpo-38822: Fixed os.stat() failing on inaccessible directories with a trailing slash, rather than falling back to the parent directory’s metadata. This implicitly affected os.path.exists() and os.path.isdir().
bpo-26227: Fixed decoding of host names in socket.gethostbyaddr() and socket.gethostbyname_ex().
bpo-43745: Actually updates Windows release to OpenSSL 1.1.1k. Earlier releases were mislabelled and actually included 1.1.1i again.
bpo-43492: Upgrade Windows installer to use SQLite 3.35.5.
macOS
bpo-44009: Provide “python3.x-intel64” executable to allow reliably forcing macOS universal2 framework builds to run under Rosetta 2 Intel-64 emulation on Apple Silicon Macs. This can be useful for testing or when universal2 wheels are not yet available.
bpo-41100: As of 3.8.10, Python now supports building and running on macOS 11 (Big Sur) and on Apple Silicon Macs (based on the ARM64 architecture). A new universal build variant, universal2, is also available to natively support both ARM64 and Intel 64 in one set of executables. This backport from Python 3.9 does not include support for “weaklinking”; to support a range of macOS versions, continue to target for and build on the oldest version in the range; for 3.8.x, the universal2 variant is only useful on macOS 11 or later.
bpo-43492: Update macOS installer to use SQLite 3.35.4.
IDLE
bpo-43655: IDLE dialog windows are now recognized as dialogs by window managers on macOS and X Window.
C API
bpo-43962: _PyInterpreterState_IDIncref() now calls _PyInterpreterState_IDInitref() and always increments id_refcount. Previously, calling _xxsubinterpreters.get_current() could create an id_refcount inconsistency when a _xxsubinterpreters.InterpreterID object was deallocated. Patch by Victor Stinner.
Vala 0.52.3
===========
* Various improvements and bug fixes:
- codegen:
+ Apply gconstpointer to gpointer cast to GenericType only
+ Fix access to captured generics in async method of interfaces (2)
+ Use if-clause for is_in_destructor() condition to be more clear
+ Add missing "_return" label and "_inner_error*_" declaration in dtors
+ Don't use G_GNUC_INTERNAL on implicit type specific fields
- vala:
+ length-type of arrays must not be nullable
+ Report a warning for unhandled errors in destructors
- parser:
+ Minor semantic checks to improve error messages
+ Allow empty member-initializer and accept trailing comma
+ Include INTERR token in source_reference of parsed types
Our tool wrappers don't look into response files, so common mistakes
such as forgetting to declare dependencies have been overlooked by
our buildlink framework. With the change that will no longer happen.
29 Apr 2021, PHP 8.0.5
- Core:
. Fixed bug #75776 (Flushing streams with compression filter is broken). (cmb)
. Fixed bug #80811 (Function exec without $output but with $restult_code
parameter crashes). (Nikita)
. Fixed bug #80814 (threaded mod_php won't load on FreeBSD: No space
available for static Thread Local Storage). (Dmitry)
. Changed PowerPC CPU registers used by Zend VM to work around GCC bug.
Old registers (r28/r29) might be clobbered by _restgpr routine used for
return from C function compiled with -Os. (Dmitry)
- Dba:
. Fixed bug #80817 (dba_popen() may cause segfault during RSHUTDOWN). (cmb)
- DOM:
. Fixed bug #66783 (UAF when appending DOMDocument to element). (cmb)
- FFI:
. Fixed bug #80847 (CData structs with fields of type struct can't be passed
as C function argument). (Nickolas Daniel da Silva, Dmitry)
- FPM:
. Fixed bug #80024 (Duplication of info about inherited socket after pool
removing). (Jakub Zelenka)
- FTP:
. Fixed bug #80880 (SSL_read on shutdown, ftp/proc_open). (cmb, Jakub
Zelenka)
- IMAP:
. Fixed bug #80800 (imap_open() fails when the flags parameter includes
CL_EXPUNGE). (girgias)
. Fixed bug #80710 (imap_mail_compose() header injection). (cmb, Stas)
- Intl:
. Fixed bug #80763 (msgfmt_format() does not accept DateTime references).
(cmb)
- LibXML:
. Fixed bug #73533 (Invalid memory access in php_libxml_xmlCheckUTF8). (cmb)
. Fixed bug #51903 (simplexml_load_file() doesn't use HTTP headers). (cmb)
- MySQLnd:
. Fixed bug #80837 (Calling stmt_store_result after fetch doesn't throw an
error). (Kamil Tekiela)
- Opcache:
. Fixed bug #80839 (PHP problem with JIT). (Dmitry)
. Fixed bug #80861 (erronous array key overflow in 2D array with JIT).
(Dmitry)
. Fixed bug #80786 (PHP crash using JIT). (Nikita)
. Fixed bug #80782 (DASM_S_RANGE_VREG on PHP_INT_MIN-1). (Dmitry)
- Pcntl:
. Fixed bug #79812 (Potential integer overflow in pcntl_exec()). (cmb)
- PCRE:
. Fixed bug #80866 (preg_split ignores limit flag when pattern with \K has
0-width fullstring match). (Kamil Tekiela)
- PDO_ODBC:
. Fixed bug #80783 (PDO ODBC truncates BLOB records at every 256th byte).
(cmb)
- PDO_pgsql:
. Fixed bug #80892 (PDO::PARAM_INT is treated the same as PDO::PARAM_STR).
(Matteo)
- Session:
. Fixed bug #80889 (Cannot set save handler when save_handler is invalid).
(cmb)
. Fixed bug #80774 (session_name() problem with backslash). (cmb)
- SOAP:
. Fixed bug #69668 (SOAP special XML characters in namespace URIs not
encoded). (cmb)
- Standard:
. Fixed bug #80915 (Taking a reference to $_SERVER hides its values from
phpinfo()). (Rowan Tommins)
. Fixed bug #80914 ('getdir' accidentally defined as an alias of 'dir').
(Rowan Tommins)
. Fixed bug #80771 (phpinfo(INFO_CREDITS) displays nothing in CLI). (cmb)
. Fixed bug #78719 (http wrapper silently ignores long Location headers).
(cmb)
. Fixed bug #80838 (HTTP wrapper waits for HTTP 1 response after HTTP 101).
(manuelm)
- Zip:
. Fixed bug #80825 (ZipArchive::isCompressionMethodSupported does not exist).
(cmb)
New in version 2.1.4
* platform support:
* work around address-space randomization causing instability
on new versions of MinGW. (#1921141)
* bug fix: RANDOM on floats returns values strictly less than the
float argument.
* bug fix: compiler error on x86-64 resulting from attempting to
zero a memory location with xor. (reported by Eric Marsden)
* optimization: extended loops updating iteration variables with
THEN can perform specialized arithmetic for those updates.
* optimization: in some cases, the jump table resulting from a
compilation of TYPECASE is simpler.
* optimization: on x86-64, IF BOUNDP followed by SYMBOL-VALUE can
elide some memory loads and tests.
New in version 2.1.3
* minor incompatible change: support for the
:SB-SAFEPOINT-STRICTLY, :SB-THRUPTION, and :SB-WTIMER build
features has been removed
* platform support:
* support for :SB-CORE-COMPRESSION on Darwin/ARM64
* support ARM v8.1 atomic and compare-and-swap instructions
* x86, x86-64: microoptimizations in multiple type-checking routines
* bug fix: structures and conditions are now TYPEP all
classes in the class precedence list of their
class. (reported by Luis Oliveira)
* bug fix: derivation of the result type from subtraction
sometimes erroneously excluded zero. (#1916895)
* bug fix: reduce the number of places where the system
permissively accepts the symbol * as a type specifier where
it should not be accepted. (#1860919)
* bug fix: the code-walker used by the system's
implementation of CLOS can handle defuns declared
inline. (reported by Don Cohen)
* optimization: EQUALP on specialized vectors and arrays is faster.
* optimization: support routines for EQUALP hash tables
generate less garbage.
New in version 2.1.2
* platform support:
* support for ARM64 macOS;
* improvement in coverage mark implementation on non-x86oid backends,
approaching the existing x86oid support;
* more empirically-robust retrieval of the program counter from
illegal instruction traps on SPARC;
* retain fewer dead objects when saving cores with precise collectors.
* incompatible change: MAP-ALL-SAMPLES and MAP-TRACE-SAMPLES
are no longer present in the SB-SPROF contrib module.
* minor incompatible change: SB-SPROF:WITH-PROFILING defaults
to all threads. SB-SPROF:START-PROFILING no longer accepts
a :SAMPLING keyword.
* enhancement: the sb-introspect contrib now supports finding
the lambda lists of method combinations. (thanks to Didier
Verna)
* enhancement: short-form DEFSETF now stores a source-location.
* bug fix: canonical unions of CONS types were being
incorrectly computed. (#1912863, reported by James
Kalenius)
* bug fix: better understanding of array simplicity (or
otherwise) in the type system. (#1903241)
* bug fix: unions of rational and integer types now have a
single canonical form, allowing more correct reasoning
about them in the type system.
* bug fix: less likely to overclaim certainty about type
equality of union types. (#1916040)
* bug fix: HANDLER-BIND evaluates the forms producing handler
functions only once. (#1916302, reported by Christophe
Junke)
* optimization: FIND on constant sequences can be compiled
into a jump table, in a similar manner to POSITION
* optimization: the compiler's awareness of numeric contagion
rules for operations on pairs of floating point numbers is
improved. (#1914094, thanks to Andrew Berkley)
New in version 2.1.1
* platform support:
* restore non-threaded NetBSD builds;
* adjust how the finalizer thread is started; (#1906571, #1907872)
* fix the encoding of PEXTR on x86-64;
* minor incompatible change: emit warnings for list iteration
forms when the object being iterated over is known not to be a
list. (#1908819, reported by Michael Fiano)
* bug fix: detect 2 or 1 as an invalid number of arguments passed
to optimized slot reading or writing effective method
respectively. (#1909659, reported by Michal Herda)
* bug fix: division by zero errors were in some cases not being
signalled. (#1910098, reported by il71)
* bug fix: erroneous coercions in the type system could lose
precision. (#1910294)
* bug fix: literal (read-time evaluated) NaNs in source code no
longer cause compiler crashes. (#1909881, reported by Michal
Herda)
* bug fix: detect more erroneous syntax in method
bodies. (#1912362, reported by Paul M. Rodriguez)
* optimization: the compiler's understanding of EXPT is improved,
reducing the introduction of COMPLEX types. (#1908830, reported
by Michael Fiano)
* optimization: the compiler is better at computing numeric
contagion when (COMPLEX FLOAT) types are involved.
* micro-optimizations:
* moving from slightly-bigger-than-fixnum ranges is more
efficient on x86-64;
* encode character comparisons with smaller operands on x86-64;
* truncating (and related operations) on floats can be inlined
in more cases on 64-bit platforms;
* rounding can use specialized instructions on ARM64 and on
x86-64 when SSE4 is available;
Our binary packages have been broken because of bad "REQUIRES"
entries: GHC uses $ORIGIN in rpath but mk/pkgformat/pkg/metadata.mk
uses ldd(1) to extract the list of required .so objects, which causes
them to be wrongly expanded.
- Libraries and binaries don't have the .note.GNU-stack section stripped
anymore. Previously, "make strip" would strip that section, which
would sometimes (depending on the toolchain) cause binaries to be
incorrectly tagged as needing an executable stack. This is not a
security issue in itself, but an executable stack makes it easier for
an attacker to turn bugs into exploits, so it should be avoided
whenever possible. Thanks to Xavier Stonestreet for reporting and
finding the cause of the problem.
- Link tests are now performed with a regular file as their
output, instead of /dev/null, which makes them more portable to
old/buggy linkers.
Bump default BUILDLINK_API_DEPENDS to match.
most of these simply extend matching from "aarch64" to "aarch64eb"
in various forms of code. most remaining uses in pkgsrc of
"MACHINE_ARCH == aarch64" are because of missing aarch64eb support,
such as most of the binary-bootstrap requiring languages like rust,
go, and java.
no pkg-bump because this shouldn't change packages on systems that
could already build all of these.
- Fix malformed preprocessor directive: ``#ifdef FOO && BAR''
- Use V8_OS_NETBSD instead of defined(__NetBSD__) consistently where appropriate
XXX
Unfortunately, nodejs does not work for aarch64eb yet.
We need to add big-endian support to built-in assembler.
