Python 3.5.7 final
Security
bpo-36216: Changes urlsplit() to raise ValueError when the URL contains characters that decompose under IDNA encoding (NFKC-normalization) into characters that affect how the URL is parsed.
bpo-35121: Don’t send cookies of domain A without Domain attribute to domain B when domain A is a suffix match of domain B while using a cookiejar with http.cookiejar.DefaultCookiePolicy policy.
Library
bpo-35121: Don’t set cookie for a request when the request path is a prefix match of the cookie’s path attribute but doesn’t end with “/”.
Python 3.5.7 release candidate 1
Security
bpo-35746: [CVE-2019-5010] Fix a NULL pointer deref in ssl module. The cert parser did not handle CRL distribution points with empty DP or URI correctly. A malicious or buggy certificate can result into segfault. Vulnerability (TALOS-2018-0758).
bpo-34791: The xml.sax and xml.dom.domreg no longer use environment variables to override parser implementations when sys.flags.ignore_environment is set by -E or -I arguments.
bpo-34623: CVE-2018-14647: The C accelerated _elementtree module now initializes hash randomization salt from _Py_HashSecret instead of libexpat’s default CSPRNG.
Library
bpo-33329: Fix multiprocessing regression on newer glibcs
bpo-33127: The ssl module now compiles with LibreSSL 2.7.1.
PkgSrc changes:
* Make find_library() consitant for all Python versions:
- Fallback to clang, when gcc is not installed.
- Find libraries in PkgSrc prefix.
* Do not use -stack_size; it ends up in 'pythonNN-config --ldflags', and some
modules fail to build.
This is a useless library (we've built it incorrectly for a long time so it
contains no valid symbols) that only creates CONFLICTS with other python3
packages. No objection on tech-pkg. Bump PKGREVISION.
Security
* bpo-30730: Prevent environment variables injection in subprocess on Windows. Prevent passing other environment variables and command arguments.
* bpo-30694: Upgrade expat copy from 2.2.0 to 2.2.1 to get fixes of multiple security vulnerabilities including: CVE-2017-9233 (External entity infinite loop DoS), CVE-2016-9063 (Integer overflow, re-fix), CVE-2016-0718 (Fix regression bugs from 2.2.0’s fix to CVE-2016-0718) and CVE-2012-0876 (Counter hash flooding with SipHash). Note: the CVE-2016-5300 (Use os- specific entropy sources like getrandom) doesn’t impact Python, since Python already gets entropy from the OS to set the expat secret using XML_SetHashSalt().
* bpo-30500: Fix urllib.parse.splithost() to correctly parse fragments. For example, splithost('//127.0.0.1#@evil.com/') now correctly returns the 127.0.0.1 host, instead of treating @evil.com as the host in an authentification (login@host).
* bpo-29591: Update expat copy from 2.1.1 to 2.2.0 to get fixes of CVE-2016-0718 and CVE-2016-4472. See https://sourceforge.net/p/expat/bugs/537/ for more information.
Upstream changes:
What's New in Python 3.5.3?
===========================
Release date: 2017-01-16
There were no code changes between 3.5.3rc1 and 3.5.3 final.
What's New in Python 3.5.3 release candidate 1?
===============================================
Release date: 2017-01-02
Core and Builtins
-----------------
- Issue #29073: bytearray formatting no longer truncates on first null byte.
- Issue #28932: Do not include <sys/random.h> if it does not exist.
- Issue #28147: Fix a memory leak in split-table dictionaries: setattr()
must not convert combined table into split table.
- Issue #25677: Correct the positioning of the syntax error caret for
indented blocks. Based on patch by Michael Layzell.
- Issue #29000: Fixed bytes formatting of octals with zero padding in alternate
form.
- Issue #28512: Fixed setting the offset attribute of SyntaxError by
PyErr_SyntaxLocationEx() and PyErr_SyntaxLocationObject().
- Issue #28991: functools.lru_cache() was susceptible to an obscure reentrancy
bug caused by a monkey-patched len() function.
- Issue #28648: Fixed crash in Py_DecodeLocale() in debug build on Mac OS X
when decode astral characters. Patch by Xiang Zhang.
- Issue #19398: Extra slash no longer added to sys.path components in case of
empty compile-time PYTHONPATH components.
- Issue #28426: Fixed potential crash in PyUnicode_AsDecodedObject() in debug
build.
- Issue #23782: Fixed possible memory leak in _PyTraceback_Add() and exception
loss in PyTraceBack_Here().
- Issue #28379: Added sanity checks and tests for PyUnicode_CopyCharacters().
Patch by Xiang Zhang.
- Issue #28376: The type of long range iterator is now registered as Iterator.
Patch by Oren Milman.
- Issue #28376: The constructor of range_iterator now checks that step is not 0.
Patch by Oren Milman.
- Issue #26906: Resolving special methods of uninitialized type now causes
implicit initialization of the type instead of a fail.
- Issue #18287: PyType_Ready() now checks that tp_name is not NULL.
Original patch by Niklas Koep.
- Issue #24098: Fixed possible crash when AST is changed in process of
compiling it.
- Issue #28350: String constants with null character no longer interned.
- Issue #26617: Fix crash when GC runs during weakref callbacks.
- Issue #27942: String constants now interned recursively in tuples and frozensets.
- Issue #21578: Fixed misleading error message when ImportError called with
invalid keyword args.
- Issue #28203: Fix incorrect type in error message from
``complex(1.0, {2:3})``. Patch by Soumya Sharma.
- Issue #27955: Fallback on reading /dev/urandom device when the getrandom()
syscall fails with EPERM, for example when blocked by SECCOMP.
- Issue #28131: Fix a regression in zipimport's compile_source(). zipimport
should use the same optimization level as the interpreter.
- Issue #25221: Fix corrupted result from PyLong_FromLong(0) when
Python is compiled with NSMALLPOSINTS = 0.
- Issue #25758: Prevents zipimport from unnecessarily encoding a filename
(patch by Eryk Sun)
- Issue #28189: dictitems_contains no longer swallows compare errors.
(Patch by Xiang Zhang)
- Issue #27812: Properly clear out a generator's frame's backreference to the
generator to prevent crashes in frame.clear().
- Issue #27811: Fix a crash when a coroutine that has not been awaited is
finalized with warnings-as-errors enabled.
- Issue #27587: Fix another issue found by PVS-Studio: Null pointer check
after use of 'def' in _PyState_AddModule().
Initial patch by Christian Heimes.
- Issue #26020: set literal evaluation order did not match documented behaviour.
- Issue #27782: Multi-phase extension module import now correctly allows the
``m_methods`` field to be used to add module level functions to instances
of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang.
- Issue #27936: The round() function accepted a second None argument
for some types but not for others. Fixed the inconsistency by
accepting None for all numeric types.
- Issue #27487: Warn if a submodule argument to "python -m" or
runpy.run_module() is found in sys.modules after parent packages are
imported, but before the submodule is executed.
- Issue #27558: Fix a SystemError in the implementation of "raise" statement.
In a brand new thread, raise a RuntimeError since there is no active
exception to reraise. Patch written by Xiang Zhang.
- Issue #27419: Standard __import__() no longer look up "__import__" in globals
or builtins for importing submodules or "from import". Fixed handling an
error of non-string package name.
- Issue #27083: Respect the PYTHONCASEOK environment variable under Windows.
- Issue #27514: Make having too many statically nested blocks a SyntaxError
instead of SystemError.
- Issue #27473: Fixed possible integer overflow in bytes and bytearray
concatenations. Patch by Xiang Zhang.
- Issue #27507: Add integer overflow check in bytearray.extend(). Patch by
Xiang Zhang.
- Issue #27581: Don't rely on wrapping for overflow check in
PySequence_Tuple(). Patch by Xiang Zhang.
- Issue #27443: __length_hint__() of bytearray iterators no longer return a
negative integer for a resized bytearray.
- Issue #27942: Fix memory leak in codeobject.c
Library
-------
- Issue #15812: inspect.getframeinfo() now correctly shows the first line of
a context. Patch by Sam Breese.
- Issue #29094: Offsets in a ZIP file created with extern file object and modes
"w" and "x" now are relative to the start of the file.
- Issue #13051: Fixed recursion errors in large or resized
curses.textpad.Textbox. Based on patch by Tycho Andersen.
- Issue #29119: Fix weakrefs in the pure python version of
collections.OrderedDict move_to_end() method.
Contributed by Andra Bogildea.
- Issue #9770: curses.ascii predicates now work correctly with negative
integers.
- Issue #28427: old keys should not remove new values from
WeakValueDictionary when collecting from another thread.
- Issue 28923: Remove editor artifacts from Tix.py.
- Issue #28871: Fixed a crash when deallocate deep ElementTree.
- Issue #19542: Fix bugs in WeakValueDictionary.setdefault() and
WeakValueDictionary.pop() when a GC collection happens in another
thread.
- Issue #20191: Fixed a crash in resource.prlimit() when pass a sequence that
doesn't own its elements as limits.
- Issue #28779: multiprocessing.set_forkserver_preload() would crash the
forkserver process if a preloaded module instantiated some
multiprocessing objects such as locks.
- Issue #28847: dbm.dumb now supports reading read-only files and no longer
writes the index file when it is not changed.
- Issue #25659: In ctypes, prevent a crash calling the from_buffer() and
from_buffer_copy() methods on abstract classes like Array.
- Issue #28732: Fix crash in os.spawnv() with no elements in args
- Issue #28485: Always raise ValueError for negative
compileall.compile_dir(workers=...) parameter, even when multithreading is
unavailable.
- Issue #28387: Fixed possible crash in _io.TextIOWrapper deallocator when
the garbage collector is invoked in other thread. Based on patch by
Sebastian Cufre.
- Issue #27517: LZMA compressor and decompressor no longer raise exceptions if
given empty data twice. Patch by Benjamin Fogle.
- Issue #28549: Fixed segfault in curses's addch() with ncurses6.
- Issue #28449: tarfile.open() with mode "r" or "r:" now tries to open a tar
file with compression before trying to open it without compression. Otherwise
it had 50% chance failed with ignore_zeros=True.
- Issue #23262: The webbrowser module now supports Firefox 36+ and derived
browsers. Based on patch by Oleg Broytman.
- Issue #27939: Fixed bugs in tkinter.ttk.LabeledScale and tkinter.Scale caused
by representing the scale as float value internally in Tk. tkinter.IntVar
now works if float value is set to underlying Tk variable.
- Issue #28255: calendar.TextCalendar().prmonth() no longer prints a space
at the start of new line after printing a month's calendar. Patch by
Xiang Zhang.
- Issue #20491: The textwrap.TextWrapper class now honors non-breaking spaces.
Based on patch by Kaarle Ritvanen.
- Issue #28353: os.fwalk() no longer fails on broken links.
- Issue #25464: Fixed HList.header_exists() in tkinter.tix module by addin
a workaround to Tix library bug.
- Issue #28488: shutil.make_archive() no longer add entry "./" to ZIP archive.
- Issue #24452: Make webbrowser support Chrome on Mac OS X.
- Issue #20766: Fix references leaked by pdb in the handling of SIGINT
handlers.
- Issue #26293: Fixed writing ZIP files that starts not from the start of the
file. Offsets in ZIP file now are relative to the start of the archive in
conforming to the specification.
- Issue #28321: Fixed writing non-BMP characters with binary format in plistlib.
- Issue #28322: Fixed possible crashes when unpickle itertools objects from
incorrect pickle data. Based on patch by John Leitch.
- Fix possible integer overflows and crashes in the mmap module with unusual
usage patterns.
- Issue #1703178: Fix the ability to pass the --link-objects option to the
distutils build_ext command.
- Issue #28253: Fixed calendar functions for extreme months: 0001-01
and 9999-12.
Methods itermonthdays() and itermonthdays2() are reimplemented so
that they don't call itermonthdates() which can cause datetime.date
under/overflow.
- Issue #28275: Fixed possible use after free in the decompress()
methods of the LZMADecompressor and BZ2Decompressor classes.
Original patch by John Leitch.
- Issue #27897: Fixed possible crash in sqlite3.Connection.create_collation()
if pass invalid string-like object as a name. Patch by Xiang Zhang.
- Issue #18893: Fix invalid exception handling in Lib/ctypes/macholib/dyld.py.
Patch by Madison May.
- Issue #27611: Fixed support of default root window in the tkinter.tix module.
- Issue #27348: In the traceback module, restore the formatting of exception
messages like "Exception: None". This fixes a regression introduced in
3.5a2.
- Issue #25651: Allow falsy values to be used for msg parameter of subTest().
- Issue #27932: Prevent memory leak in win32_ver().
- Fix UnboundLocalError in socket._sendfile_use_sendfile.
- Issue #28075: Check for ERROR_ACCESS_DENIED in Windows implementation of
os.stat(). Patch by Eryk Sun.
- Issue #25270: Prevent codecs.escape_encode() from raising SystemError when
an empty bytestring is passed.
- Issue #28181: Get antigravity over HTTPS. Patch by Kaartic Sivaraam.
- Issue #25895: Enable WebSocket URL schemes in urllib.parse.urljoin.
Patch by Gergely Imreh and Markus Holtermann.
- Issue #27599: Fixed buffer overrun in binascii.b2a_qp() and binascii.a2b_qp().
- Issue #19003:m email.generator now replaces only \r and/or \n line
endings, per the RFC, instead of all unicode line endings.
- Issue #28019: itertools.count() no longer rounds non-integer step in range
between 1.0 and 2.0 to 1.
- Issue #25969: Update the lib2to3 grammar to handle the unpacking
generalizations added in 3.5.
- Issue #14977: mailcap now respects the order of the lines in the mailcap
files ("first match"), as required by RFC 1542. Patch by Michael Lazar.
- Issue #24594: Validates persist parameter when opening MSI database
- Issue #17582: xml.etree.ElementTree nows preserves whitespaces in attributes
(Patch by Duane Griffin. Reviewed and approved by Stefan Behnel.)
- Issue #28047: Fixed calculation of line length used for the base64 CTE
in the new email policies.
- Issue #27445: Don't pass str(_charset) to MIMEText.set_payload().
Patch by Claude Paroz.
- Issue #22450: urllib now includes an "Accept: */*" header among the
default headers. This makes the results of REST API requests more
consistent and predictable especially when proxy servers are involved.
- lib2to3.pgen3.driver.load_grammar() now creates a stable cache file
between runs given the same Grammar.txt input regardless of the hash
randomization setting.
- Issue #27570: Avoid zero-length memcpy() etc calls with null source
pointers in the "ctypes" and "array" modules.
- Issue #22233: Break email header lines *only* on the RFC specified CR and LF
characters, not on arbitrary unicode line breaks. This also fixes a bug in
HTTP header parsing.
- Issue 27988: Fix email iter_attachments incorrect mutation of payload list.
- Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name
fields in X.509 certs.
- Issue #27850: Remove 3DES from ssl module's default cipher list to counter
measure sweet32 attack (CVE-2016-2183).
- Issue #27766: Add ChaCha20 Poly1305 to ssl module's default ciper list.
(Required OpenSSL 1.1.0 or LibreSSL).
- Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0.
- Remove support for passing a file descriptor to os.access. It never worked but
previously didn't raise.
- Issue #12885: Fix error when distutils encounters symlink.
- Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level.
Based on patch by Xiang Zhang.
- Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory
creates not a cursor. Patch by Xiang Zhang.
- Issue #19884: Avoid spurious output on OS X with Gnu Readline.
- Issue #27706: Restore deterministic behavior of random.Random().seed()
for string seeds using seeding version 1. Allows sequences of calls
to random() to exactly match those obtained in Python 2.
Patch by Nofar Schnider.
- Issue #10513: Fix a regression in Connection.commit(). Statements should
not be reset after a commit.
- A new version of typing.py from https://github.com/python/typing:
- Collection (only for 3.6) (Issue #27598)
- Add FrozenSet to __all__ (upstream #261)
- fix crash in _get_type_vars() (upstream #259)
- Remove the dict constraint in ForwardRef._eval_type (upstream #252)
- Issue #27539: Fix unnormalised ``Fraction.__pow__`` result in the case
of negative exponent and negative base.
- Issue #21718: cursor.description is now available for queries using CTEs.
- Issue #2466: posixpath.ismount now correctly recognizes mount points which
the user does not have permission to access.
- Issue #27773: Correct some memory management errors server_hostname in
_ssl.wrap_socket().
- Issue #26750: unittest.mock.create_autospec() now works properly for
subclasses of property() and other data descriptors.
- In the curses module, raise an error if window.getstr() or window.instr() is
passed a negative value.
- Issue #27783: Fix possible usage of uninitialized memory in
operator.methodcaller.
- Issue #27774: Fix possible Py_DECREF on unowned object in _sre.
- Issue #27760: Fix possible integer overflow in binascii.b2a_qp.
- Issue #27758: Fix possible integer overflow in the _csv module for large
record lengths.
- Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the
HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates
that the script is in CGI mode.
- Issue #27656: Do not assume sched.h defines any SCHED_* constants.
- Issue #27130: In the "zlib" module, fix handling of large buffers
(typically 4 GiB) when compressing and decompressing. Previously, inputs
were limited to 4 GiB, and compression and decompression operations did not
properly handle results of 4 GiB.
- Issue #27533: Release GIL in nt._isdir
- Issue #17711: Fixed unpickling by the persistent ID with protocol 0.
Original patch by Alexandre Vassalotti.
- Issue #27522: Avoid an unintentional reference cycle in email.feedparser.
- Issue #26844: Fix error message for imp.find_module() to refer to 'path'
instead of 'name'. Patch by Lev Maximov.
- Issue #23804: Fix SSL zero-length recv() calls to not block and not raise
an error about unclean EOF.
- Issue #27466: Change time format returned by http.cookie.time2netscape,
confirming the netscape cookie format and making it consistent with
documentation.
- Issue #26664: Fix activate.fish by removing mis-use of ``$``.
- Issue #22115: Fixed tracing Tkinter variables: trace_vdelete() with wrong
mode no longer break tracing, trace_vinfo() now always returns a list of
pairs of strings, tracing in the "u" mode now works.
- Fix a scoping issue in importlib.util.LazyLoader which triggered an
UnboundLocalError when lazy-loading a module that was already put into
sys.modules.
- Issue #27079: Fixed curses.ascii functions isblank(), iscntrl() and ispunct().
- Issue #26754: Some functions (compile() etc) accepted a filename argument
encoded as an iterable of integers. Now only strings and byte-like objects
are accepted.
- Issue #27048: Prevents distutils failing on Windows when environment
variables contain non-ASCII characters
- Issue #27330: Fixed possible leaks in the ctypes module.
- Issue #27238: Got rid of bare excepts in the turtle module. Original patch
by Jelle Zijlstra.
- Issue #27122: When an exception is raised within the context being managed
by a contextlib.ExitStack() and one of the exit stack generators
catches and raises it in a chain, do not re-raise the original exception
when exiting, let the new chained one through. This avoids the PEP 479
bug described in issue25782.
- [Security] Issue #27278: Fix os.urandom() implementation using getrandom() on
Linux. Truncate size to INT_MAX and loop until we collected enough random
bytes, instead of casting a directly Py_ssize_t to int.
- Issue #26386: Fixed ttk.TreeView selection operations with item id's
containing spaces.
- [Security] Issue #22636: Avoid shell injection problems with
ctypes.util.find_library().
- Issue #16182: Fix various functions in the "readline" module to use the
locale encoding, and fix get_begidx() and get_endidx() to return code point
indexes.
- Issue #27392: Add loop.connect_accepted_socket().
Patch by Jim Fulton.
- Issue #27930: Improved behaviour of logging.handlers.QueueListener.
Thanks to Paulo Andrade and Petr Viktorin for the analysis and patch.
- Issue #21201: Improves readability of multiprocessing error message. Thanks
to Wojciech Walczak for patch.
- Issue #27456: asyncio: Set TCP_NODELAY by default.
- Issue #27906: Fix socket accept exhaustion during high TCP traffic.
Patch by Kevin Conway.
- Issue #28174: Handle when SO_REUSEPORT isn't properly supported.
Patch by Seth Michael Larson.
- Issue #26654: Inspect functools.partial in asyncio.Handle.__repr__.
Patch by iceboy.
- Issue #26909: Fix slow pipes IO in asyncio.
Patch by INADA Naoki.
- Issue #28176: Fix callbacks race in asyncio.SelectorLoop.sock_connect.
- Issue #27759: Fix selectors incorrectly retain invalid file descriptors.
Patch by Mark Williams.
- Issue #28368: Refuse monitoring processes if the child watcher has
no loop attached.
Patch by Vincent Michel.
- Issue #28369: Raise RuntimeError when transport's FD is used with
add_reader, add_writer, etc.
- Issue #28370: Speedup asyncio.StreamReader.readexactly.
Patch by ▒<9A>о▒<80>енбе▒<80>г ▒<9C>а▒<80>к.
- Issue #28371: Deprecate passing asyncio.Handles to run_in_executor.
- Issue #28372: Fix asyncio to support formatting of non-python coroutines.
- Issue #28399: Remove UNIX socket from FS before binding.
Patch by ▒<9A>о▒<80>енбе▒<80>г ▒<9C>а▒<80>к.
- Issue #27972: Prohibit Tasks to await on themselves.
- Issue #26923: Fix asyncio.Gather to refuse being cancelled once all
children are done.
Patch by Johannes Ebke.
- Issue #26796: Don't configure the number of workers for default
threadpool executor.
Initial patch by Hans Lawrenz.
- Issue #28600: Optimize loop.call_soon().
- Issue #28613: Fix get_event_loop() return the current loop if
called from coroutines/callbacks.
- Issue #28639: Fix inspect.isawaitable to always return bool
Patch by Justin Mayfield.
- Issue #28652: Make loop methods reject socket kinds they do not support.
- Issue #28653: Fix a refleak in functools.lru_cache.
- Issue #28703: Fix asyncio.iscoroutinefunction to handle Mock objects.
- Issue #24142: Reading a corrupt config file left the parser in an
invalid state. Original patch by Florian Höch.
- Issue #28990: Fix SSL hanging if connection is closed before handshake
completed.
(Patch by HoHo-Ho)
IDLE
----
- Issue #15308: Add 'interrupt execution' (^C) to Shell menu.
Patch by Roger Serwy, updated by Bayard Randel.
- Issue #27922: Stop IDLE tests from 'flashing' gui widgets on the screen.
- Add version to title of IDLE help window.
- Issue #25564: In section on IDLE -- console differences, mention that
using exec means that __builtins__ is defined for each statement.
- Issue #27714: text_textview and test_autocomplete now pass when re-run
in the same process. This occurs when test_idle fails when run with the
-w option but without -jn. Fix warning from test_config.
- Issue #25507: IDLE no longer runs buggy code because of its tkinter imports.
Users must include the same imports required to run directly in Python.
- Issue #27452: add line counter and crc to IDLE configHandler test dump.
- Issue #27365: Allow non-ascii chars in IDLE NEWS.txt, for contributor names.
- Issue #27245: IDLE: Cleanly delete custom themes and key bindings.
Previously, when IDLE was started from a console or by import, a cascade
of warnings was emitted. Patch by Serhiy Storchaka.
C API
-----
- Issue #28808: PyUnicode_CompareWithASCIIString() now never raises exceptions.
- Issue #26754: PyUnicode_FSDecoder() accepted a filename argument encoded as
an iterable of integers. Now only strings and bytes-like objects are accepted.
Documentation
-------------
- Issue #28513: Documented command-line interface of zipfile.
Tests
-----
- Issue #28950: Disallow -j0 to be combined with -T/-l/-M in regrtest
command line arguments.
- Issue #28666: Now test.support.rmtree is able to remove unwritable or
unreadable directories.
- Issue #23839: Various caches now are cleared before running every test file.
- Issue #28409: regrtest: fix the parser of command line arguments.
- Issue #27787: Call gc.collect() before checking each test for "dangling
threads", since the dangling threads are weak references.
- Issue #27369: In test_pyexpat, avoid testing an error message detail that
changed in Expat 2.2.0.
Tools/Demos
-----------
- Issue #27952: Get Tools/scripts/fixcid.py working with Python 3 and the
current "re" module, avoid invalid Python backslash escapes, and fix a bug
parsing escaped C quote signs.
- Issue #27332: Fixed the type of the first argument of module-level functions
generated by Argument Clinic. Patch by Petr Viktorin.
- Issue #27418: Fixed Tools/importbench/importbench.py.
Windows
-------
- Issue #28251: Improvements to help manuals on Windows.
- Issue #28110: launcher.msi has different product codes between 32-bit and
64-bit
- Issue #25144: Ensures TargetDir is set before continuing with custom
install.
- Issue #27469: Adds a shell extension to the launcher so that drag and drop
works correctly.
- Issue #27309: Enabled proper Windows styles in python[w].exe manifest.
Build
-----
- Issue #29080: Removes hard dependency on hg.exe from PCBuild/build.bat
- Issue #23903: Added missed names to PC/python3.def.
- Issue #10656: Fix out-of-tree building on AIX. Patch by Tristan Carel and
Michael Haubenwallner.
- Issue #26359: Rename --with-optimiations to --enable-optimizations.
- Issue #28444: Fix missing extensions modules when cross compiling.
- Issue #28248: Update Windows build and OS X installers to use OpenSSL 1.0.2j.
- Issue #28258: Fixed build with Estonian locale (python-config and distclean
targets in Makefile). Patch by Arfrever Frehtes Taifersar Arahesis.
- Issue #26661: setup.py now detects system libffi with multiarch wrapper.
- Issue #28066: Fix the logic that searches build directories for generated
include files when building outside the source tree.
- Issue #15819: Remove redundant include search directory option for building
outside the source tree.
- Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach)
- Issue #27705: Update message in validate_ucrtbase.py
- Issue #27983: Cause lack of llvm-profdata tool when using clang as
required for PGO linking to be a configure time error rather than
make time when --with-optimizations is enabled. Also improve our
ability to find the llvm-profdata tool on MacOS and some Linuxes.
- Issue #26307: The profile-opt build now applies PGO to the built-in modules.
- Issue #26359: Add the --with-optimizations configure flag.
- Issue #27713: Suppress spurious build warnings when updating importlib's
bootstrap files. Patch by Xiang Zhang
- Issue #25825: Correct the references to Modules/python.exp and ld_so_aix,
which are required on AIX. This updates references to an installation path
that was changed in 3.2a4, and undoes changed references to the build tree
that were made in 3.5.0a1.
- Issue #27453: CPP invocation in configure must use CPPFLAGS. Patch by
Chi Hsuan Yen.
- Issue #27641: The configure script now inserts comments into the makefile
to prevent the pgen and _freeze_importlib executables from being cross-
compiled.
- Issue #26662: Set PYTHON_FOR_GEN in configure as the Python program to be
used for file generation during the build.
- Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X.
Also update FreedBSD version checks for the original ctype UTF-8 workaround.
- Issue #28676: Prevent missing 'getentropy' declaration warning on macOS.
Patch by Gareth Rees.
WINDOW structure.
Include <term.h> when needed and rename lines and columns vars to avoid
conflicts.
Builds and works with NetBSD-8 curses, so use mk/curses.buildlink3.mk
and test for getsyx(3) in curses rather than indescriminately linking
to ncurses.
Original changes in python27/Makefile r1.62-r1.67
=================================================
Add an option for Python: x11
Fix dlopen(3) calls in _ctypes.so for X11BASE libraries
A Python code can dynamically load shared libraries and it's wrapped with
a plain dlopen(3) call. The holder of this interface (_ctypes module)
without rpath set to X11BASE cannot detect libs like 'GL'.
Fixing find_library() on POSIX-like (excluding Darwin) systems.
This isn't addressing Python's wrapper for dlopen(3).
This possibily addresses mostly NetBSD as other popular OSes have ldconfig.
Testing commands:
- before applying the patch
>>> from ctypes.util import find_library
>>> find_library("m")
'libm.so.0'
>>> find_library("crypto")
'libcrypto.so.11'
>>> find_library("GL")
>>> find_library("curl")
- after applying the patch
>>> from ctypes.util import find_library
>>> find_library("m")
'libm.so.0'
>>> find_library("crypto")
'libcrypto.so.11'
>>> find_library("GL")
'libGL.so.2'
>>> find_library("curl")
'libcurl.so.4'
This patch doesn't solve the case of custom dirs like $PREFIX/qt5/lib.
However it's solving most common cases of using this call.
A possible solution is to parse the output "pkg_info -La"... however it's
very slow. In other words a cache with libraries might be needed to handle
it efficiently.
Issue 27066: Fixed SystemError if a custom opener (for open()) returns a negative number without setting an exception.
Issue 20041: Fixed TypeError when frame.f_trace is set to None. Patch by Xavier de Gaye.
Issue 26168: Fixed possible refleaks in failing Py_BuildValue() with the “N” format unit.
Issue 26991: Fix possible refleak when creating a function with annotations.
Issue 27039: Fixed bytearray.remove() for values greater than 127. Patch by Joe Jevnik.
Issue 23640: int.from_bytes() no longer bypasses constructors for subclasses.
Issue 26811: gc.get_objects() no longer contains a broken tuple with NULL pointer.
Issue 20120: Use RawConfigParser for .pypirc parsing, removing support for interpolation unintentionally added with move to Python 3. Behavior no longer does any interpolation in .pypirc files, matching behavior in Python 2.7 and Setuptools 19.0.
Issue 26659: Make the builtin slice type support cycle collection.
Issue 26718: super.__init__ no longer leaks memory if called multiple times. NOTE: A direct call of super.__init__ is not endorsed!
Issue 25339: PYTHONIOENCODING now has priority over locale in setting the error handler for stdin and stdout.
Issue 26494: Fixed crash on iterating exhausting iterators. Affected classes are generic sequence iterators, iterators of str, bytes, bytearray, list, tuple, set, frozenset, dict, OrderedDict, corresponding views and os.scandir() iterator.
Issue 26581: If coding cookie is specified multiple times on a line in Python source code file, only the first one is taken to account.
Issue 26464: Fix str.translate() when string is ASCII and first replacements removes character, but next replacement uses a non-ASCII character or a string longer than 1 character. Regression introduced in Python 3.5.0.
Issue 22836: Ensure exception reports from PyErr_Display() and PyErr_WriteUnraisable() are sensible even when formatting them produces secondary errors. This affects the reports produced by sys.__excepthook__() and when __del__() raises an exception.
Issue 26302: Correct behavior to reject comma as a legal character for cookie names.
Issue 4806: Avoid masking the original TypeError exception when using star (*) unpacking in function calls. Based on patch by Hagen Fürstenau and Daniel Urban.
Issue 27138: Fix the doc comment for FileFinder.find_spec().
Issue 26154: Add a new private _PyThreadState_UncheckedGet() function to get the current Python thread state, but don’t issue a fatal error if it is NULL. This new function must be used instead of accessing directly the _PyThreadState_Current variable. The variable is no more exposed since Python 3.5.1 to hide the exact implementation of atomic C types, to avoid compiler issues.
Issue 26194: Deque.insert() gave odd results for bounded deques that had reached their maximum size. Now an IndexError will be raised when attempting to insert into a full deque.
Issue 25843: When compiling code, don’t merge constants if they are equal but have a different types. For example, f1, f2 = lambda: 1, lambda: 1.0 is now correctly compiled to two different functions: f1() returns 1 (int) and f2() returns 1.0 (int), even if 1 and 1.0 are equal.
Issue 22995: [UPDATE] Comment out the one of the pickleability tests in _PyObject_GetState() due to regressions observed in Cython-based projects.
Issue 25961: Disallowed null characters in the type name.
Issue 25973: Fix segfault when an invalid nonlocal statement binds a name starting with two underscores.
Issue 22995: Instances of extension types with a state that aren’t subclasses of list or dict and haven’t implemented any pickle-related methods (__reduce__, __reduce_ex__, __getnewargs__, __getnewargs_ex__, or __getstate__), can no longer be pickled. Including memoryview.
Issue 20440: Massive replacing unsafe attribute setting code with special macro Py_SETREF.
Issue 25766: Special method __bytes__() now works in str subclasses.
Issue 25421: __sizeof__ methods of builtin types now use dynamic basic size. This allows sys.getsize() to work correctly with their subclasses with __slots__ defined.
Issue 25709: Fixed problem with in-place string concatenation and utf-8 cache.
Issue 27147: Mention PEP 420 in the importlib docs.
Issue 24097: Fixed crash in object.__reduce__() if slot name is freed inside __getattr__.
Issue 24731: Fixed crash on converting objects with special methods __bytes__, __trunc__, and __float__ returning instances of subclasses of bytes, int, and float to subclasses of bytes, int, and float correspondingly.
Issue 26478: Fix semantic bugs when using binary operators with dictionary views and tuples.
Issue 26171: Fix possible integer overflow and heap corruption in zipimporter.get_data().
Issue 25660: Fix TAB key behaviour in REPL with readline.
Issue 25887: Raise a RuntimeError when a coroutine object is awaited more than once.
Issue 27243: Update the __aiter__ protocol: instead of returning an awaitable that resolves to an asynchronous iterator, the asynchronous iterator should be returned directly. Doing the former will trigger a PendingDeprecationWarning.
more...
all platforms that use GNU ld. This is of course completely wrong. Some
platforms have always implemented the sane semantic for DT_RPATH already
and newer saw a reason for implementing the then-redundant DT_RUNPATH.
Unbreak them by trusting the compiler to actually know what the platform
default should be. Bump revision.
New syntax features:
* PEP 492, coroutines with async and await syntax.
* PEP 465, a new matrix multiplication operator: a @ b.
* PEP 448, additional unpacking generalizations.
New library modules:
* typing: PEP 484 – Type Hints.
* zipapp: PEP 441 Improving Python ZIP Application Support.
New built-in features:
* bytes % args, bytearray % args: PEP 461 – Adding % formatting to bytes and bytearray.
* New bytes.hex(), bytearray.hex() and memoryview.hex() methods. (Contributed by Arnon Yaari in issue 9951.)
* memoryview now supports tuple indexing (including multi-dimensional). (Contributed by Antoine Pitrou in issue 23632.)
* Generators have a new gi_yieldfrom attribute, which returns the object being iterated by yield from expressions. (Contributed by Benno Leslie and Yury Selivanov in issue 24450.)
* A new RecursionError exception is now raised when maximum recursion depth is reached. (Contributed by Georg Brandl in issue 19235.)
CPython implementation improvements:
* When the LC_TYPE locale is the POSIX locale (C locale), sys.stdin and sys.stdout now use the surrogateescape error handler, instead of the strict error handler. (Contributed by Victor Stinner in issue 19977.)
* .pyo files are no longer used and have been replaced by a more flexible scheme that includes the optimization level explicitly in .pyc name. (See PEP 488 overview.)
* Builtin and extension modules are now initialized in a multi-phase process, which is similar to how Python modules are loaded. (See PEP 489 overview.)
Significant improvements in the standard library:
* collections.OrderedDict is now implemented in C, which makes it 4 to 100 times faster.
* The ssl module gained support for Memory BIO, which decouples SSL protocol handling from network IO.
* The new os.scandir() function provides a better and significantly faster way of directory traversal.
* functools.lru_cache() has been mostly reimplemented in C, yielding much better performance.
* The new subprocess.run() function provides a streamlined way to run subprocesses.
* The traceback module has been significantly enhanced for improved performance and developer convenience.
Security improvements:
* SSLv3 is now disabled throughout the standard library. It can still be enabled by instantiating a ssl.SSLContext manually. (See issue 22638 for more details; this change was backported to CPython 3.4 and 2.7.)
* HTTP cookie parsing is now stricter, in order to protect against potential injection attacks. (Contributed by Antoine Pitrou in issue 22796.)
Windows improvements:
* A new installer for Windows has replaced the old MSI. See Using Python on Windows for more information.
* Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same.
