Python 3.6.4 release candidate 1:
Core and Builtins
-----------------
- bpo-32176: co_flags.CO_NOFREE is now always set correctly by the code
object constructor based on freevars and cellvars, rather than needing to
be set correctly by the caller. This ensures it will be cleared
automatically when additional cell references are injected into a modified
code object and function.
- bpo-31949: Fixed several issues in printing tracebacks
(PyTraceBack_Print()).
* Setting sys.tracebacklimit to 0 or less now suppresses printing tracebacks.
* Setting sys.tracebacklimit to None now causes using the default limit.
* Setting sys.tracebacklimit to an integer larger than LONG_MAX now means using
the limit LONG_MAX rather than the default limit.
* Fixed integer overflows in the case of more than 2**31 traceback items on
Windows.
* Fixed output errors handling.
- bpo-30696: Fix the interactive interpreter looping endlessly when no
memory.
- bpo-20047: Bytearray methods partition() and rpartition() now accept only
bytes-like objects as separator, as documented. In particular they now
raise TypeError rather of returning a bogus result when an integer is
passed as a separator.
- bpo-31852: Fix a segmentation fault caused by a combination of the async
soft keyword and continuation lines.
- bpo-21720: BytesWarning no longer emitted when the *fromlist* argument of
__import__() or the __all__ attribute of the module contain bytes
instances.
- bpo-31825: Fixed OverflowError in the 'unicode-escape' codec and in
codecs.escape_decode() when decode an escaped non-ascii byte.
- bpo-28603: Print the full context/cause chain of exceptions on interpreter
exit, even if an exception in the chain is unhashable or compares equal to
later ones. Patch by Zane Bitter.
- bpo-31786: Fix timeout rounding in the select module to round correctly
negative timeouts between -1.0 and 0.0. The functions now block waiting
for events as expected. Previously, the call was incorrectly non-blocking.
Patch by Pablo Galindo.
- bpo-31642: Restored blocking "from package import module" by setting
sys.modules["package.module"] to None.
- bpo-31626: Fixed a bug in debug memory allocator. There was a write to
freed memory after shrinking a memory block.
- bpo-31619: Fixed a ValueError when convert a string with large number of
underscores to integer with binary base.
- bpo-31592: Fixed an assertion failure in Python parser in case of a bad
unicodedata.normalize(). Patch by Oren Milman.
- bpo-31588: Raise a TypeError with a helpful error message when class
creation fails due to a metaclass with a bad __prepare__() method.
Patch by Oren Milman.
- bpo-31566: Fix an assertion failure in _warnings.warn() in case of a bad
__name__ global. Patch by Oren Milman.
- bpo-31505: Fix an assertion failure in json, in case
_json.make_encoder() received a bad encoder() argument. Patch by Oren
Milman.
- bpo-31492: Fix assertion failures in case of failing to import from a
module with a bad __name__ attribute, and in case of failing to access
an attribute of such a module. Patch by Oren Milman.
- bpo-31490: Fix an assertion failure in ctypes class definition, in case
the class has an attribute whose name is specified in _anonymous_ but
not in _fields_. Patch by Oren Milman.
- bpo-31478: Fix an assertion failure in _random.Random.seed() in case the
argument has a bad __abs__() method. Patch by Oren Milman.
- bpo-31315: Fix an assertion failure in imp.create_dynamic(), when
spec.name is not a string. Patch by Oren Milman.
- bpo-31311: Fix a crash in the __setstate__() method of
ctypes._CData, in case of a bad __dict__. Patch by Oren Milman.
- bpo-31293: Fix crashes in true division and multiplication of a timedelta
object by a float with a bad as_integer_ratio() method. Patch by Oren
Milman.
- bpo-31285: Fix an assertion failure in warnings.warn_explicit, when the
return value of the received loader's get_source() has a bad splitlines()
method. Patch by Oren Milman.
- bpo-30817: PyErr_PrintEx() clears now the ignored exception that may be
raised by _PySys_SetObjectId(), for example when no memory.
Library
-------
- bpo-28556: Two minor fixes for typing module: allow shallow copying
instances of generic classes, improve interaction of __init_subclass__
with generics.
- bpo-27240: The header folding algorithm for the new email policies has
been rewritten, which also fixes bpo-30788, bpo-31831, and bpo-32182. In
particular, RFC2231 folding is now done correctly.
- bpo-32186: io.FileIO.readall() and io.FileIO.read() now release the GIL
when getting the file size. Fixed hang of all threads with inaccessible
NFS server. Patch by Nir Soffer.
- bpo-12239: Make :meth:msilib.SummaryInformation.GetProperty return
None when the value of property is VT_EMPTY. Initial patch by
Mark Mc Mahon.
- bpo-31325: Fix wrong usage of :func:collections.namedtuple in the
:meth:RobotFileParser.parse() <urllib.robotparser.RobotFileParser.parse>
method.
- bpo-12382: :func:msilib.OpenDatabase now raises a better exception
message when it couldn't open or create an MSI file. Initial patch by
William Tisäter.
- bpo-32110: codecs.StreamReader.read(n) now returns not more than *n*
characters/bytes for non-negative *n*. This makes it compatible with
read() methods of other file-like objects.
- bpo-32072: Fixed issues with binary plists:
* Fixed saving bytearrays.
* Identical objects will be saved only once.
* Equal references will be load as identical objects.
* Added support for saving and loading recursive data structures.
- bpo-32034: Make asyncio.IncompleteReadError and LimitOverrunError
pickleable.
- bpo-32015: Fixed the looping of asyncio in the case of reconnection the
socket during waiting async read/write from/to the socket.
- bpo-32011: Restored support of loading marshal files with the TYPE_INT64
code. These files can be produced in Python 2.7.
- bpo-31970: Reduce performance overhead of asyncio debug mode.
- bpo-9678: Fixed determining the MAC address in the uuid module:
* Using ifconfig on NetBSD and OpenBSD.
* Using arp on Linux, FreeBSD, NetBSD and OpenBSD.
Based on patch by Takayuki Shimizukawa.
- bpo-30057: Fix potential missed signal in signal.signal().
- bpo-31933: Fix Blake2 params leaf_size and node_offset on big endian
platforms. Patch by Jack O'Connor.
- bpo-31927: Fixed compilation of the socket module on NetBSD 8. Fixed
assertion failure or reading arbitrary data when parse a AF_BLUETOOTH
address on NetBSD and DragonFly BSD.
- bpo-27666: Fixed stack corruption in curses.box() and curses.ungetmouse()
when the size of types chtype or mmask_t is less than the size of C long.
curses.box() now accepts characters as arguments. Based on patch by Steve
Fink.
- bpo-31897: plistlib now catches more errors when read binary plists and
raises InvalidFileException instead of unexpected exceptions.
- bpo-25720: Fix the method for checking pad state of curses WINDOW. Patch
by Masayuki Yamamoto.
- bpo-31893: Fixed the layout of the kqueue_event structure on OpenBSD and
NetBSD. Fixed the comparison of the kqueue_event objects.
- bpo-31891: Fixed building the curses module on NetBSD.
- bpo-28416: Instances of pickle.Pickler subclass with the persistent_id()
method and pickle.Unpickler subclass with the persistent_load() method no
longer create reference cycles.
- bpo-28326: Fix multiprocessing.Process when stdout and/or stderr is closed
or None.
- bpo-31457: If nested log adapters are used, the inner process()
methods are no longer omitted.
- bpo-31457: The manager property on LoggerAdapter objects is now
properly settable.
- bpo-31806: Fix timeout rounding in time.sleep(), threading.Lock.acquire()
and socket.socket.settimeout() to round correctly negative timeouts
between -1.0 and 0.0. The functions now block waiting for events as
expected. Previously, the call was incorrectly non-blocking. Patch by
Pablo Galindo.
- bpo-28603: traceback: Fix a TypeError that occurred during printing of
exception tracebacks when either the current exception or an exception in
its context/cause chain is unhashable. Patch by Zane Bitter.
- bpo-30058: Fixed buffer overflow in select.kqueue.control().
- bpo-31770: Prevent a crash when calling the __init__() method of a
sqlite3.Cursor object more than once. Patch by Oren Milman.
- bpo-31672: idpattern in string.Template matched some non-ASCII
characters. Now it uses -i regular expression local flag to avoid non-
ASCII characters.
- bpo-31764: Prevent a crash in sqlite3.Cursor.close() in case the
Cursor object is uninitialized. Patch by Oren Milman.
- bpo-31752: Fix possible crash in timedelta constructor called with custom
integers.
- bpo-31701: On Windows, faulthandler.enable() now ignores MSC and COM
exceptions.
- bpo-31728: Prevent crashes in _elementtree due to unsafe cleanup of
Element.text and Element.tail. Patch by Oren Milman.
- bpo-31620: an empty asyncio.Queue now doesn't leak memory when queue.get
pollers timeout
- bpo-31632: Fix method set_protocol() of class _SSLProtocolTransport in
asyncio module. This method was previously modifying a wrong reference to
the protocol.
- bpo-31675: Fixed memory leaks in Tkinter's methods splitlist() and split()
when pass a string larger than 2 GiB.
- bpo-31673: Fixed typo in the name of Tkinter's method adderrorinfo().
- bpo-30806: Fix the string representation of a netrc object.
- bpo-15037: Added a workaround for getkey() in curses for ncurses 5.7 and
earlier.
- bpo-25351: Avoid venv activate failures with undefined variables
- bpo-25532: inspect.unwrap() will now only try to unwrap an object
sys.getrecursionlimit() times, to protect against objects which create a
new object on every attribute access.
- bpo-30347: Stop crashes when concurrently iterate over itertools.groupby()
iterators.
- bpo-31516: threading.current_thread() should not return a dummy thread
at shutdown.
- bpo-31351: python -m ensurepip now exits with non-zero exit code if pip
bootstrapping has failed.
- bpo-31482: random.seed() now works with bytes in version=1
- bpo-31334: Fix poll.poll([timeout]) in the select module for
arbitrary negative timeouts on all OSes where it can only be a non-
negative integer or -1. Patch by Riccardo Coccioli.
- bpo-31310: multiprocessing's semaphore tracker should be launched again if
crashed.
- bpo-31308: Make multiprocessing's forkserver process immune to Ctrl-C and
other user interruptions. If it crashes, restart it when necessary.
Documentation
-------------
- bpo-32105: Added asyncio.BaseEventLoop.connect_accepted_socket
versionaddded marker.
- bpo-31537: Fix incorrect usage of get_history_length in readline
documentation example code. Patch by Brad Smith.
- bpo-30085: The operator functions without double underscores are preferred
for clarity. The one with underscores are only kept for back-
compatibility.
Tests
-----
- bpo-31380: Skip test_httpservers test_undecodable_file on macOS: fails on
APFS.
- bpo-31705: Skip test_socket.test_sha256() on Linux kernel older than 4.5.
The test fails with ENOKEY on kernel 3.10 (on ppc64le). A fix was merged
into the kernel 4.5.
- bpo-31174: Fix test_tools.test_unparse: DirectoryTestCase now stores the
names sample to always test the same files. It prevents false alarms when
hunting reference leaks.
- bpo-30695: Add the set_nomemory(start, stop) and remove_mem_hooks()
functions to the _testcapi module.
Build
-----
- bpo-32059: detect_modules() in setup.py now also searches the
sysroot paths when cross-compiling.
- bpo-31957: Fixes Windows SDK version detection when building for Windows.
- bpo-31609: Fixes quotes in PCbuild/clean.bat
- bpo-31934: Abort the build when building out of a not clean source tree.
- bpo-31926: Fixed Argument Clinic sometimes causing compilation errors when
there was more than one function and/or method in a .c file with the same
name.
- bpo-28791: Update Windows builds to use SQLite 3.21.0.
- bpo-28791: Update OS X installer to use SQLite 3.21.0.
- bpo-22140: Prevent double substitution of prefix in python-config.sh.
- bpo-31536: Avoid wholesale rebuild after make regen-all if nothing
changed.
Windows
-------
- bpo-1102: Return None when View.Fetch() returns
ERROR_NO_MORE_ITEMS instead of raising MSIError.
- bpo-31944: Fixes Modify button in Apps and Features dialog.
macOS
-----
- bpo-31392: Update macOS installer to use OpenSSL 1.0.2m
This is a useless library (we've built it incorrectly for a long time so it
contains no valid symbols) that only creates CONFLICTS with other python3
packages. No objection on tech-pkg. Bump PKGREVISION.
Python 3.6.3 final
Library
* bpo-31641: Re-allow arbitrary iterables in concurrent.futures.as_completed(). Fixes regression in 3.6.3rc1.
Build
* bpo-31662: Fix typos in Windows uploadrelease.bat script. Fix Windows Doc build issues in Doc/make.bat.
* bpo-31423: Fix building the PDF documentation with newer versions of Sphinx.
More here https://docs.python.org/3.6/whatsnew/changelog.html
as an unresolved undefined symbol, causing the install to fail due to
PLIST issues. Change from -std=c99 to -std=gnu99 to work around this
problem, based on hint from joerg@.
Bump PKGREVISION.
Major new features of the 3.6 series, compared to 3.5
Among the new major new features in Python 3.6 are:
PEP 468, Preserving Keyword Argument Order
PEP 487, Simpler customization of class creation
PEP 495, Local Time Disambiguation
PEP 498, Literal String Formatting
PEP 506, Adding A Secrets Module To The Standard Library
PEP 509, Add a private version to dict
PEP 515, Underscores in Numeric Literals
PEP 519, Adding a file system path protocol
PEP 520, Preserving Class Attribute Definition Order
PEP 523, Adding a frame evaluation API to CPython
PEP 524, Make os.urandom() blocking on Linux (during system startup)
PEP 525, Asynchronous Generators (provisional)
PEP 526, Syntax for Variable Annotations (provisional)
PEP 528, Change Windows console encoding to UTF-8
PEP 529, Change Windows filesystem encoding to UTF-8
PEP 530, Asynchronous Comprehensions
According to Python 3.6 changelog:
The undocumented IN, CDROM, DLFCN, TYPES, CDIO, and STROPTS modules have been
removed. They had been available in the platform specific Lib/plat-*/
directories, but were chronically out of date, inconsistently available across
platforms, and unmaintained. The script that created these modules is still
available in the source distribution at Tools/scripts/h2py.py.
No PKGREVISION bump since they failed to install on these platforms.
TODO: PLIST.IRIX entries seems to still contains plat-* modules but the ones
TODO: contained are not explicitly documented in the changelog, so they can
TODO: maybe still installed.
Pointed out by Joyent's Linux and SmartOS bulk builds and thanks to <jperkin>
for testing it!
Summary – Release highlights
New syntax features:
PEP 498, formatted string literals.
PEP 515, underscores in numeric literals.
PEP 526, syntax for variable annotations.
PEP 525, asynchronous generators.
PEP 530: asynchronous comprehensions.
New library modules:
secrets: PEP 506 – Adding A Secrets Module To The Standard Library.
CPython implementation improvements:
The dict type has been reimplemented to use a more compact representation based on a proposal by Raymond Hettinger and similar to the PyPy dict implementation. This resulted in dictionaries using 20% to 25% less memory when compared to Python 3.5.
Customization of class creation has been simplified with the new protocol.
The class attribute definition order is now preserved.
The order of elements in **kwargs now corresponds to the order in which keyword arguments were passed to the function.
DTrace and SystemTap probing support has been added.
The new PYTHONMALLOC environment variable can now be used to debug the interpreter memory allocation and access errors.
Significant improvements in the standard library:
The asyncio module has received new features, significant usability and performance improvements, and a fair amount of bug fixes. Starting with Python 3.6 the asyncio module is no longer provisional and its API is considered stable.
A new file system path protocol has been implemented to support path-like objects. All standard library functions operating on paths have been updated to work with the new protocol.
The datetime module has gained support for Local Time Disambiguation.
The typing module received a number of improvements and is no longer provisional.
The tracemalloc module has been significantly reworked and is now used to provide better output for ResourceWarning as well as provide better diagnostics for memory allocation errors. See the PYTHONMALLOC section for more information.
Security improvements:
The new secrets module has been added to simplify the generation of cryptographically strong pseudo-random numbers suitable for managing secrets such as account authentication, tokens, and similar.
On Linux, os.urandom() now blocks until the system urandom entropy pool is initialized to increase the security. See the PEP 524 for the rationale.
The hashlib and ssl modules now support OpenSSL 1.1.0.
The default settings and feature set of the ssl module have been improved.
The hashlib module received support for the BLAKE2, SHA-3 and SHAKE hash algorithms and the scrypt() key derivation function.
