Python 3.6.15 final
Security
bpo-44394: Update the vendored copy of libexpat to 2.4.1 (from 2.2.8) to get the fix for the CVE-2013-0340 “Billion Laughs” vulnerability. This copy is most used on Windows and macOS.
bpo-43124: Made the internal putcmd function in smtplib sanitize input for presence of \r and \n characters to avoid (unlikely) command injection.
Library
bpo-45001: Made email date parsing more robust against malformed input, namely a whitespace-only Date: header. Patch by Wouter Bolsterlee.
Tests
bpo-38965: Fix test_faulthandler on GCC 10. Use the “volatile” keyword in faulthandler._stack_overflow() to prevent tail call optimization on any compiler, rather than relying on compiler specific pragma.
Python 3.6.14 final
Security
bpo-44022: mod:http.client now avoids infinitely reading potential HTTP headers after a 100 Continue status response from the server.
bpo-43882: The presence of newline or tab characters in parts of a URL could allow some forms of attacks.
Following the controlling specification for URLs defined by WHATWG urllib.parse() now removes ASCII newlines and tabs from URLs, preventing such attacks.
bpo-42988: CVE-2021-3426: Remove the getfile feature of the pydoc module which could be abused to read arbitrary files on the disk (directory traversal vulnerability). Moreover, even source code of Python modules can contain sensitive data like passwords. Vulnerability reported by David Schwörer.
bpo-43285: ftplib no longer trusts the IP address value returned from the server in response to the PASV command by default. This prevents a malicious FTP server from using the response to probe IPv4 address and port combinations on the client network.
Code that requires the former vulnerable behavior may set a trust_server_pasv_ipv4_address attribute on their ftplib.FTP instances to True to re-enable it.
bpo-43075: Fix Regular Expression Denial of Service (ReDoS) vulnerability in urllib.request.AbstractBasicAuthHandler. The ReDoS-vulnerable regex has quadratic worst-case complexity and it allows cause a denial of service when identifying crafted invalid RFCs. This ReDoS issue is on the client side and needs remote attackers to control the HTTP server.
Python 3.6.13 final
Security
bpo-42967: Fix web cache poisoning vulnerability by defaulting the query args separator to &, and allowing the user to choose a custom separator.
bpo-42938: Avoid static buffers when computing the repr of ctypes.c_double and ctypes.c_longdouble values.
bpo-42103: Prevented potential DoS attack via CPU and RAM exhaustion when processing malformed Apple Property List files in binary format.
bpo-42051: The plistlib module no longer accepts entity declarations in XML plist files to avoid XML vulnerabilities. This should not affect users as entity declarations are not used in regular plist files.
bpo-40791: Add volatile to the accumulator variable in hmac.compare_digest, making constant-time-defeating optimizations less likely.
Core and Builtins
bpo-35560: Fix an assertion error in format() in debug build for floating point formatting with “n” format, zero padding and small width. Release build is not impacted. Patch by Karthikeyan Singaravelan.
Library
bpo-42103: InvalidFileException and RecursionError are now the only errors caused by loading malformed binary Plist file (previously ValueError and TypeError could be raised in some specific cases).
Tests
bpo-42794: Update test_nntplib to use offical group name of news.aioe.org for testing. Patch by Dong-hee Na.
bpo-41944: Tests for CJK codecs no longer call eval() on content received via HTTP.
Like the other test, because /dev/shm isn't available in the build
environment doesn't mean it won't be available on the destination
machine for the packages.
This attempts to build and run a program that uses POSIX semaphores.
This fails in a pbulk sandbox that doesn't contain /dev/shm, resulting
in a broken package where the idea that the platform doesn't support
POSIX semaphores is baked in forever. In newer Python versions,
this means Python doesn't even build properly.
XXX: We might want to avoid it on other platforms too...
Python 3.6.12 final
Security
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is embedded (CVE-2020-15523).
bpo-41004: CVE-2020-14422: The __hash__() methods of ipaddress.IPv4Interface and ipaddress.IPv6Interface incorrectly generated constant hash values of 32 and 128 respectively. This resulted in always causing hash collisions. The fix uses hash() to generate hash values for the tuple of (address, mask length, network address).
bpo-39603: Prevent http header injection by rejecting control characters in http.client.putrequest(…).
Library
bpo-41288: Unpickling invalid NEWOBJ_EX opcode with the C implementation raises now UnpicklingError instead of crashing.
bpo-39017: Avoid infinite loop when reading specially crafted TAR files using the tarfile module (CVE-2019-20907).
1. LD_LIBRARY_PATH does _not_ take precedence over DT_RPATH
(e.g. Linux)
2. A previous libpython with the same major.minor is already installed
(e.g. a previous version of this package)
hold, the built python will be linked with the installed libpython,
causing it to report an old teeny version in sys.version_info while
staging the install. Then "make package" fails with PLIST mismatches for
{,Pattern}Grammar.*.pickle.
pkgsrc knows which version we're building. Pass that down instead.
For platforms that weren't having this problem, no functional change
intended. For platforms that were, this simply restores "make package",
so no PKGREVISION bump.
Python 3.6.11 final
There were no new changes in version 3.6.11.
Python 3.6.11 release candidate 1
Security
bpo-39073: Disallow CR or LF in email.headerregistry.Address arguments to guard against header injection attacks.
bpo-38576: Disallow control characters in hostnames in http.client, addressing CVE-2019-18348. Such potentially malicious header injection URLs now cause a InvalidURL to be raised.
bpo-39503: CVE-2020-8492: The AbstractBasicAuthHandler class of the urllib.request module uses an inefficient regular expression which can be exploited by an attacker to cause a denial of service. Fix the regex to prevent the catastrophic backtracking. Vulnerability reported by Ben Caller and Matt Schwager.
bpo-39401: Avoid unsafe load of api-ms-win-core-path-l1-1-0.dll at startup on Windows 7.
Core and Builtins
bpo-39510: Fix segfault in readinto() method on closed BufferedReader.
bpo-39421: Fix possible crashes when operating with the functions in the heapq module and custom comparison operators.
Library
bpo-39503: AbstractBasicAuthHandler of urllib.request now parses all WWW-Authenticate HTTP headers and accepts multiple challenges per header: use the realm of the first Basic challenge.
Python 3.6.9 final
Library
bpo-37437: Update vendorized expat version to 2.2.7.
macOS
bpo-34602: Avoid test suite failures on macOS by no longer calling resource.setrlimit to increase the process stack size limit at runtime. The runtime change is no longer needed since the interpreter is being built with a larger default stack size.
Python 3.6.9 release candidate 1
Security
bpo-35907: CVE-2019-9948: Avoid file reading by disallowing local-file:// and local_file:// URL schemes in URLopener().open() and URLopener().retrieve() of urllib.request.
bpo-36742: Fixes mishandling of pre-normalization characters in urlsplit().
bpo-30458: Address CVE-2019-9740 by disallowing URL paths with embedded whitespace or control characters through into the underlying http client request. Such potentially malicious header injection URLs now cause an http.client.InvalidURL exception to be raised.
bpo-36216: Changes urlsplit() to raise ValueError when the URL contains characters that decompose under IDNA encoding (NFKC-normalization) into characters that affect how the URL is parsed.
bpo-33529: Prevent fold function used in email header encoding from entering infinite loop when there are too many non-ASCII characters in a header.
bpo-35746: [CVE-2019-5010] Fix a NULL pointer deref in ssl module. The cert parser did not handle CRL distribution points with empty DP or URI correctly. A malicious or buggy certificate can result into segfault. Vulnerability (TALOS-2018-0758) reported by Colin Read and Nicolas Edet of Cisco.
bpo-35121: Don’t send cookies of domain A without Domain attribute to domain B when domain A is a suffix match of domain B while using a cookiejar with http.cookiejar.DefaultCookiePolicy policy. Patch by Karthikeyan Singaravelan.
Library
bpo-35643: Fixed a SyntaxWarning: invalid escape sequence in Modules/_sha3/cleanup.py. Patch by Mickaël Schoentgen.
bpo-35121: Don’t set cookie for a request when the request path is a prefix match of the cookie’s path attribute but doesn’t end with “/”. Patch by Karthikeyan Singaravelan.
Documentation
bpo-35605: Fix documentation build for sphinx<1.6. Patch by Anthony Sottile.
bpo-35564: Explicitly set master_doc variable in conf.py for compliance with Sphinx 2.0
Tests
bpo-36816: Update Lib/test/selfsigned_pythontestdotnet.pem to match self-signed.pythontest.net’s new TLS certificate.
bpo-35925: Skip specific nntplib and ssl networking tests when they would otherwise fail due to a modern OS or distro with a default OpenSSL policy of rejecting connections to servers with weak certificates or disabling TLS below TLSv1.2.
bpo-27313: Avoid test_ttk_guionly ComboboxTest failure with macOS Cocoa Tk.
bpo-32947: test_ssl fixes for TLS 1.3 and OpenSSL 1.1.1.
macOS
bpo-34602: Avoid failures setting macOS stack resource limit with resource.setrlimit. This reverts an earlier fix for bpo-18075 which forced a non-default stack size when building the interpreter executable on macOS.
Because python won't even try to build it.
You only see nis_failed.so if there is an error building it, when
it wanted to build it, and that should be fixed accordingly.
In a nutshell, if the yp headers are installed, python will build
the nis module, otherwise it won't.
On netbsd systems at least, if you have the yp headers installed and
subsequently re-install over the top with MKYP=no you get into the state
where the headers are installed, but the functions are no longer in libc.
This is an error with *your* system - either rebuild with MKYP=yes OR
remove the yp headers from include/rpc and include/rpcsvc.
Follow on fix for PR pkg/53673.
Match the logic used by setup.py: it looks for two headers in the default
include path. This helps newer glibc linux.
Omit PLIST.dll on python3* because it doesn't appear in the PLIST.
Make PLIST.dll true on all non-IRIX.
tested: NetBSD-current, FreeBSD 11.2, Ubuntu 18.10, CentOS 6.9, Source Mage
From Dr. Thomas Orgis, myself, and with pointers to a change from leot.
PR pkg/53673
PkgSrc changes:
* Make find_library() consitant for all Python versions:
- Fallback to clang, when gcc is not installed.
- Find libraries in PkgSrc prefix.
* Do not use -stack_size; it ends up in 'pythonNN-config --ldflags', and some
modules fail to build.
Python 3.6.8 final
Library
- bpo-31715: Associate .mjs file extension with application/javascript MIME Type.
Build
- bpo-35499: make profile-opt no longer replaces CFLAGS_NODIST with CFLAGS. It now adds profile-guided optimization (PGO) flags to CFLAGS_NODIST: existing CFLAGS_NODIST flags are kept.
- bpo-35257: Avoid leaking the linker flags from Link Time Optimizations (LTO) into distutils when compiling C extensions.
C API
- bpo-35259: Conditionally declare Py_FinalizeEx() (new in 3.6) based on Py_LIMITED_API.
Python 3.6.8 release candidate 1
Security
- bpo-34812: The -I command line option (run Python in isolated mode) is now also copied by the multiprocessing and distutils modules when spawning child processes. Previously, only -E and -s options (enabled by -I) were copied.
- bpo-34791: The xml.sax and xml.dom.domreg no longer use environment variables to override parser implementations when sys.flags.ignore_environment is set by -E or -I arguments.
Core and Builtins
- bpo-35444: Fixed error handling in pickling methods when fail to look up builtin “getattr”.
- bpo-35436: Fix various issues with memory allocation error handling. Patch by Zackery Spytz.
- bpo-35357: Internal attributes’ names of unittest.mock._Call and unittest.mock.MagicProxy (name, parent & from_kall) are now prefixed with _mock_ in order to prevent clashes with widely used object attributes. Fixed minor typo in test function name.
- bpo-35372: Fixed the code page decoder for input longer than 2 GiB containing undecodable bytes.
- bpo-33954: For str.format(), float.__format__() and complex.__format__() methods for non-ASCII decimal point when using the “n” formatter.
- bpo-35214: Fixed an out of bounds memory access when parsing a truncated unicode escape sequence at the end of a string such as '\N'. It would read one byte beyond the end of the memory allocation.
- bpo-35214: The interpreter and extension modules have had annotations added so that they work properly under clang’s Memory Sanitizer. A new configure flag –with-memory-sanitizer has been added to make test builds of this nature easier to perform.
- bpo-35193: Fix an off by one error in the bytecode peephole optimizer where it could read bytes beyond the end of bounds of an array when removing unreachable code. This bug was present in every release of Python 3.6 until now.
- bpo-29341: Clarify in the docstrings of os methods that path-like objects are also accepted as input parameters.
- bpo-35050: socket: Fix off-by-one bug in length check for AF_ALG name and type.
- bpo-34974: bytes and bytearray constructors no longer convert unexpected exceptions (e.g. MemoryError and KeyboardInterrupt) to TypeError.
- bpo-34973: Fixed crash in bytes() when the list argument is mutated while it is iterated.
- bpo-34824: Fix a possible null pointer dereference in Modules/_ssl.c. Patch by Zackery Spytz.
- bpo-1621: Do not assume signed integer overflow behavior (C undefined behavior) when performing set hash table resizing.
Library
- bpo-35052: Fix xml.dom.minidom cloneNode() on a document with an entity: pass the correct arguments to the user data handler of an entity.
- bpo-35330: When a Mock instance was used to wrap an object, if side_effect is used in one of the mocks of it methods, don’t call the original implementation and return the result of using the side effect the same way that it is done with return_value.
- bpo-34172: Revert the fix for this issue previously released in 3.6.7 pending further investigation: Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly.
- bpo-10496: posixpath.expanduser() now returns the input path unchanged if the HOME environment variable is not set and the current user has no home directory (if the current user identifier doesn’t exist in the password database). This change fix the site module if the current user doesn’t exist in the password database (if the user has no home directory).
- bpo-35310: Fix a bug in select.select() where, in some cases, the file descriptor sequences were returned unmodified after a signal interruption, even though the file descriptors might not be ready yet. select.select() will now always return empty lists if a timeout has occurred. Patch by Oran Avraham.
- bpo-35380: Enable TCP_NODELAY on Windows for proactor asyncio event loop.
- bpo-35371: Fixed possible crash in os.utime() on Windows when pass incorrect arguments.
- bpo-27903: Fix ResourceWarning in platform.dist() on SuSE and Caldera OpenLinux. Patch by Ville Skyttä.
- bpo-28604: locale.localeconv() now sets temporarily the LC_CTYPE locale to the LC_MONETARY locale if the two locales are different and monetary strings are non-ASCII. This temporary change affects other threads.
- bpo-35277: Update ensurepip to install pip 18.1 and setuptools 40.6.2.
- bpo-35226: Recursively check arguments when testing for equality of unittest.mock.call objects and add note that tracking of parameters used to create ancestors of mocks in mock_calls is not possible.
- bpo-35189: Modify the following fnctl function to retry if interrupted by a signal (EINTR): flock, lockf, fnctl
- bpo-35062: Fix incorrect parsing of _io.IncrementalNewlineDecoder’s translate argument.
- bpo-35079: Improve difflib.SequenceManager.get_matching_blocks doc by adding ‘non-overlapping’ and changing ‘!=’ to ‘<’.
- bpo-35017: socketserver.BaseServer.serve_forever() now exits immediately if it’s shutdown() method is called while it is polling for new events.
- bpo-31047: Fix ntpath.abspath regression where it didn’t remove a trailing separator on Windows. Patch by Tim Graham.
- bpo-34794: Fixed a leak in Tkinter when pass the Python wrapper around Tcl_Obj back to Tcl/Tk.
- bpo-35008: Fixed references leaks when call the __setstate__() method of xml.etree.ElementTree.Element in the C implementation for already initialized element.
- bpo-23420: Verify the value for the parameter ‘-s’ of the cProfile CLI. Patch by Robert Kuska
- bpo-16965: The 2to3 execfile fixer now opens the file with mode 'rb'. Patch by Zackery Spytz.
- bpo-34966: pydoc now supports aliases not only to methods defined in the end class, but also to inherited methods. The docstring is not duplicated for aliases.
- bpo-34941: Methods find(), findtext() and findall() of the Element class in the xml.etree.ElementTree module are now able to find children which are instances of Element subclasses.
- bpo-34936: Fix TclError in tkinter.Spinbox.selection_element(). Patch by Juliette Monsel.
- bpo-34900: Fixed unittest.TestCase.debug() when used to call test methods with subtests. Patch by Bruno Oliveira.
- bpo-34866: Adding max_num_fields to cgi.FieldStorage to make DOS attacks harder by limiting the number of MiniFieldStorage objects created by FieldStorage.
- bpo-34738: ZIP files created by distutils will now include entries for directories.
- bpo-31177: Fix bug that prevented using reset_mock on mock instances with deleted attributes
- bpo-34604: Fix possible mojibake in the error message of pwd.getpwnam and grp.getgrnam using string representation because of invisible characters or trailing whitespaces. Patch by William Grzybowski.
- bpo-34574: OrderedDict iterators are not exhausted during pickling anymore. Patch by Sergey Fedoseev.
- bpo-34052: sqlite3.Connection.create_aggregate(), sqlite3.Connection.create_function(), sqlite3.Connection.set_authorizer(), sqlite3.Connection.set_progress_handler() methods raises TypeError when unhashable objects are passed as callable. These methods now don’t pass such objects to SQLite API. Previous behavior could lead to segfaults. Patch by Sergey Fedoseev.
- bpo-29877: compileall: import ProcessPoolExecutor only when needed, preventing hangs on low resource platforms
- bpo-22005: Implemented unpickling instances of datetime, date and time pickled by Python 2. encoding='latin1' should be used for successful decoding.
Version 3.6.6:
Core and Builtins
* Fix asynchronous generators to handle GeneratorExit in athrow() correctly
* Fixed reset of the SIGINT handler to SIG_DFL on interpreter shutdown even when there was a custom handler set previously.
* Fixed a leak when the garbage collector fails to add an object with the __del__ method or referenced by it into the gc.garbage list. PyGC_Collect() can now be called when an exception is set and preserves it.
* Fix signed/unsigned comparison warning in pyhash.c.
* Fix a leak in set_symmetric_difference().
* Fix unaligned accesses in siphash24().
* Fix potential memory leak in normalizestring().
* Improved error messages in ‘async with’ when __aenter__() or __aexit__() return non-awaitable object.
* Fix ma_version_tag in dict implementation is uninitialized when copying from key-sharing dict.
* Fixed jumping when the function contains an async for loop.
* Fix an unnecessary ifdef in the include of VersionHelpers.h in socketmodule on Windows.
* Fix a crash in ctypes.cast() in case the type argument is a ctypes structured data type.
Library
* Prevent site.main() exception if PYTHONSTARTUP is set.
* Datetime instance d with non-None tzinfo, but with d.tzinfo.utcoffset(d) returning None is now treated as naive by the astimezone() method.
* Avoid race condition with debug logging
* The concatenation (+) and repetition (*) sequence operations now raise TypeError instead of SystemError when performed on mmap.mmap objects.
* Fix gather to propagate cancellation of itself even with return_exceptions.
* Fix a race condition in SSLProtocol.connection_made() of asyncio.sslproto: start immediately the handshake instead of using call_soon(). Previously, data_received() could be called before the handshake started, causing the handshake to hang or fail.
* Fixed bug where calling write_eof() on a _SelectorSocketTransport after it’s already closed raises AttributeError.
* Fix Task.__repr__ crash with Cython’s bogus coroutines
* Fix RuntimeError after closing loop that used run_in_executor
* Use a better regex when breaking usage into wrappable parts. Avoids bogus assertion errors from custom metavar strings.
* Fixed a bug in the Python implementation of the JSON decoder that prevented the cache of parsed strings from clearing after finishing the decoding. Based on patch by c-fos.
* tempfile._candidate_tempdir_list should consider common TEMP locations
* Prevent uuid.get_node from using a DUID instead of a MAC on Windows.
* Fix race condition with ReadTransport.resume_reading in Windows proactor event loop.
* Minor fixes in typing module: add annotations to NamedTuple.__new__, pass *args and **kwds in Generic.__new__.
* Updated alias mapping with glibc 2.27 supported locales.
* Fix trailing quotation marks getting deleted when looking up byte/string literals on pydoc.
* Update error message when constructing invalid inspect.Parameters
* Fixed crash in the get() method of the dbm.ndbm database object when it is called with a single argument.
* Fix multiprocessing regression on newer glibcs
* Fix quoting of the Comment attribute of http.cookies.SimpleCookie.
* Upgrade bundled version of pip to 10.0.1.
* Fixed a crash in the parser module when converting an ST object to a tree of tuples or lists with line_info=False and col_info=True.
* Fix FD leak in _SelectorSocketTransport
* Fix display of <module> call in the html produced by cgitb.html().
* random.Random.choice() now raises IndexError for empty sequences consistently even when called from subclasses without a getrandbits() implementation.
* Update difflib.mdiff() for PEP 479. Convert an uncaught StopIteration in a generator into a return-statement.
* End framing at the end of C implementation of pickle.Pickler.dump().
* The urllib.robotparser’s __str__ representation now includes wildcard entries and the “Crawl-delay” and “Request-rate” fields.
* Allow ttk.Treeview.insert to insert iid that has a false boolean value. Note iid=0 and iid=False would be same.
* The ssl module now compiles with LibreSSL 2.7.1.
* Release the GIL during fstat() calls, avoiding hang of all threads when calling mmap.mmap(), os.urandom(), and random.seed().
* Fix a regression in ipaddress that result of hosts() is empty when the network is constructed by a tuple containing an integer mask and only 1 bit left for addresses.
* Fix wrong redirection of a low descriptor (0 or 1) to stderr in subprocess if another low descriptor is closed.
* Fix output of cover files for trace module command-line tool. Previously emitted cover files only when --missing option was used.
* If nested log adapters are used, the inner process() methods are no longer omitted.
* Support arrays >=2GiB in ctypes.
* pydoc: the stop() method of the private ServerThread class now waits until DocServer.serve_until_quit() completes and then explicitly sets its docserver attribute to None to break a reference cycle.
Python 3.6.5:
Security
* Minimal fix to prevent buffer overrun in os.symlink on Windows
* Regexes in difflib and poplib were vulnerable to catastrophic backtracking. These regexes formed potential DOS vectors (REDOS). They have been refactored. This resolves CVE-2018-1060 and CVE-2018-1061.
Core and Builtins
* Fixed jumping out of “with” block by setting f_lineno.
* Prevent jumps from ‘return’ and ‘exception’ trace events.
* Update Valgrind suppression list to account for the rename of Py_ADDRESS_IN_RANG to address_in_range.
* Pdb and other debuggers dependent on bdb.py will correctly step over (next command) native coroutines.
* Improve suggestion when the Python 2 form of print statement is either present on the same line as the header of a compound statement or else terminated by a semi-colon instead of a newline.
* Fix possible crashing in builtin Unicode decoders caused by write out-of- bound errors when using customized decode error handlers.
* Improved frozenset() hash to create more distinct hash values when faced with datasets containing many similar values.
* The __debug__ constant is now optimized out at compile time. This fixes also bpo-22091.
* sys.flags.hash_randomization is now properly set to 0 when hash randomization is turned off by PYTHONHASHSEED=0.
* The optimizer is now protected from spending much time doing complex calculations and consuming much memory for creating large constants in constant folding.
* repr() on a dict containing its own values() or items() no longer raises RecursionError; OrderedDict similarly. Instead, use ..., as for other recursive structures.
* Leading whitespace is now correctly ignored when generating suggestions for converting Py2 print statements to Py3 builtin print function calls.
* The repr of deeply nested dict now raises a RecursionError instead of crashing due to a stack overflow.
Library
* lib2to3 now properly supports trailing commas after *args and **kwargs in function signatures.
* Avoid failing in multiprocessing.Process if the standard streams are closed or None at exit.
* Skip sending/receiving data after SSL transport closing.
* Fix ctypes pass-by-value for structs on 64-bit Cygwin/MinGW.
* Fix inspect.signature() for single-parameter partialmethods.
* Expose several missing constants in zlib and fix corresponding documentation.
* Fixed tarfile.itn handling of out-of-bounds float values.
* The ssl module now detects missing NPN support in LibreSSL.
* dbm.open() now encodes filename with the filesystem encoding rather than default encoding.
* In os.dup2, don’t check every call whether the dup3 syscall exists or not.
* Rewrite confusing message from setup.py upload from “No dist file created in earlier command” to the more helpful “Must create and upload files in one command”.
* In tkinter, after_cancel(None) now raises a ValueError instead of canceling the first scheduled function.
* Make sure sys.argv remains as a list when running trace.
* Fixed asyncio.Condition issue which silently ignored cancellation after notifying and cancelling a conditional lock.
* Fixed refleaks of __init__() methods in various modules. (Contributed by Oren Milman)
* Fixed guessing quote and delimiter in csv.Sniffer.sniff() when only the last field is quoted.
* socket: Remove TCP_FASTOPEN, TCP_KEEPCNT flags on older version Windows during run-time.
* Fix a rare but potential pre-exec child process deadlock in subprocess on POSIX systems when marking file descriptors inheritable on exec in the child process. This bug appears to have been introduced in 3.4.
* The ctypes module used to depend on indirect linking for dlopen. The shared extension is now explicitly linked against libdl on platforms with dl.
* Fixed asyncio.Lock() safety issue which allowed acquiring and locking the same lock multiple times, without it being free.
* Do not include name field in SMTP envelope from address.
* Fix email address header parsing error when the username is an empty quoted string.
* distutils’ upload command no longer corrupts tar files ending with a CR byte, and no longer tries to convert CR to CRLF in any of the upload text fields.
* uuid.uuid1 no longer raises an exception if a 64-bit hardware address is encountered.
* Fix the error handling in Aifc_read.initfp() when the SSND chunk is not found.
* On FreeBSD and Solaris, os.strerror() now always decode the byte string from the current locale encoding, rather than using ASCII/surrogateescape in some cases.
* The nis module is now compatible with new libnsl and headers location.
* Improve ABCMeta._dump_registry() output readability
* glibc has removed Sun RPC. Use replacement libtirpc headers and library in nis module.
* Ensure that truncate() preserves the file position (as reported by tell()) after writes longer than the buffer size.
* Don’t unsubscribe signals in asyncio UNIX event loop on interpreter shutdown.
* The SSL module no longer sends IP addresses in SNI TLS extension on platforms with OpenSSL 1.0.2+ or inet_pton.
* urllib.parse.urlsplit() does not convert zone-id (scope) to lower case for scoped IPv6 addresses in hostnames now.
* Fix bdist_wininst of distutils for CRT v142: it binary compatible with CRT v140.
* A single empty field is now always quoted when written into a CSV file. This allows to distinguish an empty row from a row consisting of a single empty field.
* Raise NotImplementedError instead of SystemError on platforms where chmod(..., follow_symlinks=False) is not supported.
* The getnode() ip getter now uses ‘ip link’ instead of ‘ip link list’.
* Ensure TCP_NODELAY is set on Linux. Tests by Victor Stinner.
* The locale.localeconv() function now sets temporarily the LC_CTYPE locale to the LC_NUMERIC locale to decode decimal_point and thousands_sep byte strings if they are non-ASCII or longer than 1 byte, and the LC_NUMERIC locale is different than the LC_CTYPE locale. This temporary change affects other threads.
Same change for the str.format() method when formatting a number (int, float, float and subclasses) with the n type (ex: '{:n}'.format(1234)).
* Importing native path module (posixpath, ntpath) now works even if the os module still is not imported.
Python 3.6.4 release candidate 1:
Core and Builtins
-----------------
- bpo-32176: co_flags.CO_NOFREE is now always set correctly by the code
object constructor based on freevars and cellvars, rather than needing to
be set correctly by the caller. This ensures it will be cleared
automatically when additional cell references are injected into a modified
code object and function.
- bpo-31949: Fixed several issues in printing tracebacks
(PyTraceBack_Print()).
* Setting sys.tracebacklimit to 0 or less now suppresses printing tracebacks.
* Setting sys.tracebacklimit to None now causes using the default limit.
* Setting sys.tracebacklimit to an integer larger than LONG_MAX now means using
the limit LONG_MAX rather than the default limit.
* Fixed integer overflows in the case of more than 2**31 traceback items on
Windows.
* Fixed output errors handling.
- bpo-30696: Fix the interactive interpreter looping endlessly when no
memory.
- bpo-20047: Bytearray methods partition() and rpartition() now accept only
bytes-like objects as separator, as documented. In particular they now
raise TypeError rather of returning a bogus result when an integer is
passed as a separator.
- bpo-31852: Fix a segmentation fault caused by a combination of the async
soft keyword and continuation lines.
- bpo-21720: BytesWarning no longer emitted when the *fromlist* argument of
__import__() or the __all__ attribute of the module contain bytes
instances.
- bpo-31825: Fixed OverflowError in the 'unicode-escape' codec and in
codecs.escape_decode() when decode an escaped non-ascii byte.
- bpo-28603: Print the full context/cause chain of exceptions on interpreter
exit, even if an exception in the chain is unhashable or compares equal to
later ones. Patch by Zane Bitter.
- bpo-31786: Fix timeout rounding in the select module to round correctly
negative timeouts between -1.0 and 0.0. The functions now block waiting
for events as expected. Previously, the call was incorrectly non-blocking.
Patch by Pablo Galindo.
- bpo-31642: Restored blocking "from package import module" by setting
sys.modules["package.module"] to None.
- bpo-31626: Fixed a bug in debug memory allocator. There was a write to
freed memory after shrinking a memory block.
- bpo-31619: Fixed a ValueError when convert a string with large number of
underscores to integer with binary base.
- bpo-31592: Fixed an assertion failure in Python parser in case of a bad
unicodedata.normalize(). Patch by Oren Milman.
- bpo-31588: Raise a TypeError with a helpful error message when class
creation fails due to a metaclass with a bad __prepare__() method.
Patch by Oren Milman.
- bpo-31566: Fix an assertion failure in _warnings.warn() in case of a bad
__name__ global. Patch by Oren Milman.
- bpo-31505: Fix an assertion failure in json, in case
_json.make_encoder() received a bad encoder() argument. Patch by Oren
Milman.
- bpo-31492: Fix assertion failures in case of failing to import from a
module with a bad __name__ attribute, and in case of failing to access
an attribute of such a module. Patch by Oren Milman.
- bpo-31490: Fix an assertion failure in ctypes class definition, in case
the class has an attribute whose name is specified in _anonymous_ but
not in _fields_. Patch by Oren Milman.
- bpo-31478: Fix an assertion failure in _random.Random.seed() in case the
argument has a bad __abs__() method. Patch by Oren Milman.
- bpo-31315: Fix an assertion failure in imp.create_dynamic(), when
spec.name is not a string. Patch by Oren Milman.
- bpo-31311: Fix a crash in the __setstate__() method of
ctypes._CData, in case of a bad __dict__. Patch by Oren Milman.
- bpo-31293: Fix crashes in true division and multiplication of a timedelta
object by a float with a bad as_integer_ratio() method. Patch by Oren
Milman.
- bpo-31285: Fix an assertion failure in warnings.warn_explicit, when the
return value of the received loader's get_source() has a bad splitlines()
method. Patch by Oren Milman.
- bpo-30817: PyErr_PrintEx() clears now the ignored exception that may be
raised by _PySys_SetObjectId(), for example when no memory.
Library
-------
- bpo-28556: Two minor fixes for typing module: allow shallow copying
instances of generic classes, improve interaction of __init_subclass__
with generics.
- bpo-27240: The header folding algorithm for the new email policies has
been rewritten, which also fixes bpo-30788, bpo-31831, and bpo-32182. In
particular, RFC2231 folding is now done correctly.
- bpo-32186: io.FileIO.readall() and io.FileIO.read() now release the GIL
when getting the file size. Fixed hang of all threads with inaccessible
NFS server. Patch by Nir Soffer.
- bpo-12239: Make :meth:msilib.SummaryInformation.GetProperty return
None when the value of property is VT_EMPTY. Initial patch by
Mark Mc Mahon.
- bpo-31325: Fix wrong usage of :func:collections.namedtuple in the
:meth:RobotFileParser.parse() <urllib.robotparser.RobotFileParser.parse>
method.
- bpo-12382: :func:msilib.OpenDatabase now raises a better exception
message when it couldn't open or create an MSI file. Initial patch by
William Tisäter.
- bpo-32110: codecs.StreamReader.read(n) now returns not more than *n*
characters/bytes for non-negative *n*. This makes it compatible with
read() methods of other file-like objects.
- bpo-32072: Fixed issues with binary plists:
* Fixed saving bytearrays.
* Identical objects will be saved only once.
* Equal references will be load as identical objects.
* Added support for saving and loading recursive data structures.
- bpo-32034: Make asyncio.IncompleteReadError and LimitOverrunError
pickleable.
- bpo-32015: Fixed the looping of asyncio in the case of reconnection the
socket during waiting async read/write from/to the socket.
- bpo-32011: Restored support of loading marshal files with the TYPE_INT64
code. These files can be produced in Python 2.7.
- bpo-31970: Reduce performance overhead of asyncio debug mode.
- bpo-9678: Fixed determining the MAC address in the uuid module:
* Using ifconfig on NetBSD and OpenBSD.
* Using arp on Linux, FreeBSD, NetBSD and OpenBSD.
Based on patch by Takayuki Shimizukawa.
- bpo-30057: Fix potential missed signal in signal.signal().
- bpo-31933: Fix Blake2 params leaf_size and node_offset on big endian
platforms. Patch by Jack O'Connor.
- bpo-31927: Fixed compilation of the socket module on NetBSD 8. Fixed
assertion failure or reading arbitrary data when parse a AF_BLUETOOTH
address on NetBSD and DragonFly BSD.
- bpo-27666: Fixed stack corruption in curses.box() and curses.ungetmouse()
when the size of types chtype or mmask_t is less than the size of C long.
curses.box() now accepts characters as arguments. Based on patch by Steve
Fink.
- bpo-31897: plistlib now catches more errors when read binary plists and
raises InvalidFileException instead of unexpected exceptions.
- bpo-25720: Fix the method for checking pad state of curses WINDOW. Patch
by Masayuki Yamamoto.
- bpo-31893: Fixed the layout of the kqueue_event structure on OpenBSD and
NetBSD. Fixed the comparison of the kqueue_event objects.
- bpo-31891: Fixed building the curses module on NetBSD.
- bpo-28416: Instances of pickle.Pickler subclass with the persistent_id()
method and pickle.Unpickler subclass with the persistent_load() method no
longer create reference cycles.
- bpo-28326: Fix multiprocessing.Process when stdout and/or stderr is closed
or None.
- bpo-31457: If nested log adapters are used, the inner process()
methods are no longer omitted.
- bpo-31457: The manager property on LoggerAdapter objects is now
properly settable.
- bpo-31806: Fix timeout rounding in time.sleep(), threading.Lock.acquire()
and socket.socket.settimeout() to round correctly negative timeouts
between -1.0 and 0.0. The functions now block waiting for events as
expected. Previously, the call was incorrectly non-blocking. Patch by
Pablo Galindo.
- bpo-28603: traceback: Fix a TypeError that occurred during printing of
exception tracebacks when either the current exception or an exception in
its context/cause chain is unhashable. Patch by Zane Bitter.
- bpo-30058: Fixed buffer overflow in select.kqueue.control().
- bpo-31770: Prevent a crash when calling the __init__() method of a
sqlite3.Cursor object more than once. Patch by Oren Milman.
- bpo-31672: idpattern in string.Template matched some non-ASCII
characters. Now it uses -i regular expression local flag to avoid non-
ASCII characters.
- bpo-31764: Prevent a crash in sqlite3.Cursor.close() in case the
Cursor object is uninitialized. Patch by Oren Milman.
- bpo-31752: Fix possible crash in timedelta constructor called with custom
integers.
- bpo-31701: On Windows, faulthandler.enable() now ignores MSC and COM
exceptions.
- bpo-31728: Prevent crashes in _elementtree due to unsafe cleanup of
Element.text and Element.tail. Patch by Oren Milman.
- bpo-31620: an empty asyncio.Queue now doesn't leak memory when queue.get
pollers timeout
- bpo-31632: Fix method set_protocol() of class _SSLProtocolTransport in
asyncio module. This method was previously modifying a wrong reference to
the protocol.
- bpo-31675: Fixed memory leaks in Tkinter's methods splitlist() and split()
when pass a string larger than 2 GiB.
- bpo-31673: Fixed typo in the name of Tkinter's method adderrorinfo().
- bpo-30806: Fix the string representation of a netrc object.
- bpo-15037: Added a workaround for getkey() in curses for ncurses 5.7 and
earlier.
- bpo-25351: Avoid venv activate failures with undefined variables
- bpo-25532: inspect.unwrap() will now only try to unwrap an object
sys.getrecursionlimit() times, to protect against objects which create a
new object on every attribute access.
- bpo-30347: Stop crashes when concurrently iterate over itertools.groupby()
iterators.
- bpo-31516: threading.current_thread() should not return a dummy thread
at shutdown.
- bpo-31351: python -m ensurepip now exits with non-zero exit code if pip
bootstrapping has failed.
- bpo-31482: random.seed() now works with bytes in version=1
- bpo-31334: Fix poll.poll([timeout]) in the select module for
arbitrary negative timeouts on all OSes where it can only be a non-
negative integer or -1. Patch by Riccardo Coccioli.
- bpo-31310: multiprocessing's semaphore tracker should be launched again if
crashed.
- bpo-31308: Make multiprocessing's forkserver process immune to Ctrl-C and
other user interruptions. If it crashes, restart it when necessary.
Documentation
-------------
- bpo-32105: Added asyncio.BaseEventLoop.connect_accepted_socket
versionaddded marker.
- bpo-31537: Fix incorrect usage of get_history_length in readline
documentation example code. Patch by Brad Smith.
- bpo-30085: The operator functions without double underscores are preferred
for clarity. The one with underscores are only kept for back-
compatibility.
Tests
-----
- bpo-31380: Skip test_httpservers test_undecodable_file on macOS: fails on
APFS.
- bpo-31705: Skip test_socket.test_sha256() on Linux kernel older than 4.5.
The test fails with ENOKEY on kernel 3.10 (on ppc64le). A fix was merged
into the kernel 4.5.
- bpo-31174: Fix test_tools.test_unparse: DirectoryTestCase now stores the
names sample to always test the same files. It prevents false alarms when
hunting reference leaks.
- bpo-30695: Add the set_nomemory(start, stop) and remove_mem_hooks()
functions to the _testcapi module.
Build
-----
- bpo-32059: detect_modules() in setup.py now also searches the
sysroot paths when cross-compiling.
- bpo-31957: Fixes Windows SDK version detection when building for Windows.
- bpo-31609: Fixes quotes in PCbuild/clean.bat
- bpo-31934: Abort the build when building out of a not clean source tree.
- bpo-31926: Fixed Argument Clinic sometimes causing compilation errors when
there was more than one function and/or method in a .c file with the same
name.
- bpo-28791: Update Windows builds to use SQLite 3.21.0.
- bpo-28791: Update OS X installer to use SQLite 3.21.0.
- bpo-22140: Prevent double substitution of prefix in python-config.sh.
- bpo-31536: Avoid wholesale rebuild after make regen-all if nothing
changed.
Windows
-------
- bpo-1102: Return None when View.Fetch() returns
ERROR_NO_MORE_ITEMS instead of raising MSIError.
- bpo-31944: Fixes Modify button in Apps and Features dialog.
macOS
-----
- bpo-31392: Update macOS installer to use OpenSSL 1.0.2m
This is a useless library (we've built it incorrectly for a long time so it
contains no valid symbols) that only creates CONFLICTS with other python3
packages. No objection on tech-pkg. Bump PKGREVISION.
Python 3.6.3 final
Library
* bpo-31641: Re-allow arbitrary iterables in concurrent.futures.as_completed(). Fixes regression in 3.6.3rc1.
Build
* bpo-31662: Fix typos in Windows uploadrelease.bat script. Fix Windows Doc build issues in Doc/make.bat.
* bpo-31423: Fix building the PDF documentation with newer versions of Sphinx.
More here https://docs.python.org/3.6/whatsnew/changelog.html
as an unresolved undefined symbol, causing the install to fail due to
PLIST issues. Change from -std=c99 to -std=gnu99 to work around this
problem, based on hint from joerg@.
Bump PKGREVISION.
