Python 3.9.4
Core and Builtins
bpo-43710: Reverted the fix for https://bugs.python.org/issue42500 as it changed the PyThreadState struct size and broke the 3.9.x ABI in the 3.9.3 release (visible on 32-bit platforms using binaries compiled using an earlier version of Python 3.9.x headers).
Library
bpo-26053: Fixed bug where the pdb interactive run command echoed the args from the shell command line, even if those have been overridden at the pdb prompt.
Python 3.9.3 final
Security
bpo-42988: CVE-2021-3426: Remove the getfile feature of the pydoc module which could be abused to read arbitrary files on the disk (directory traversal vulnerability). Moreover, even source code of Python modules can contain sensitive data like passwords. Vulnerability reported by David Schwörer.
bpo-43285: ftplib no longer trusts the IP address value returned from the server in response to the PASV command by default. This prevents a malicious FTP server from using the response to probe IPv4 address and port combinations on the client network.
Code that requires the former vulnerable behavior may set a trust_server_pasv_ipv4_address attribute on their ftplib.FTP instances to True to re-enable it.
bpo-43439: Add audit hooks for gc.get_objects(), gc.get_referrers() and gc.get_referents(). Patch by Pablo Galindo.
Core and Builtins
bpo-43660: Fix crash that happens when replacing sys.stderr with a callable that can remove the object while an exception is being printed. Patch by Pablo Galindo.
bpo-43555: Report the column offset for SyntaxError for invalid line continuation characters. Patch by Pablo Galindo.
bpo-43517: Fix misdetection of circular imports when using from pkg.mod import attr, which caused false positives in non-trivial multi-threaded code.
bpo-35883: Python no longer fails at startup with a fatal error if a command line argument contains an invalid Unicode character. The Py_DecodeLocale() function now escapes byte sequences which would be decoded as Unicode characters outside the [U+0000; U+10ffff] range.
bpo-43406: Fix a possible race condition where PyErr_CheckSignals tries to execute a non-Python signal handler.
bpo-42500: Improve handling of exceptions near recursion limit. Converts a number of Fatal Errors in RecursionErrors.
Library
bpo-43433: xmlrpc.client.ServerProxy no longer ignores query and fragment in the URL of the server.
bpo-35930: Raising an exception raised in a “future” instance will create reference cycles.
bpo-43577: Fix deadlock when using ssl.SSLContext debug callback with ssl.SSLContext.sni_callback().
bpo-43521: ast.unparse can now render NaNs and empty sets.
bpo-43423: subprocess.communicate() no longer raises an IndexError when there is an empty stdout or stderr IO buffer during a timeout on Windows.
bpo-27820: Fixed long-standing bug of smtplib.SMTP where doing AUTH LOGIN with initial_response_ok=False will fail.
The cause is that SMTP.auth_login _always_ returns a password if provided with a challenge string, thus non-compliant with the standard for AUTH LOGIN.
Also fixes bug with the test for smtpd.
bpo-43332: Improves the networking efficiency of http.client when using a proxy via set_tunnel(). Fewer small send calls are made during connection setup.
bpo-43399: Fix ElementTree.extend not working on iterators when using the Python implementation
bpo-43316: The python -m gzip command line application now properly fails when detecting an unsupported extension. It exits with a non-zero exit code and prints an error message to stderr.
bpo-43260: Fix TextIOWrapper can not flush internal buffer forever after very large text is written.
bpo-42782: Fail fast in shutil.move() to avoid creating destination directories on failure.
bpo-37193: Fixed memory leak in socketserver.ThreadingMixIn introduced in Python 3.7.
Documentation
bpo-43199: Answer “Why is there no goto?” in the Design and History FAQ.
bpo-43407: Clarified that a result from time.monotonic(), time.perf_counter(), time.process_time(), or time.thread_time() can be compared with the result from any following call to the same function - not just the next immediate call.
bpo-27646: Clarify that ‘yield from <expr>’ works with any iterable, not just iterators.
bpo-36346: Update some deprecated unicode APIs which are documented as “will be removed in 4.0” to “3.12”. See PEP 623 for detail.
Tests
bpo-37945: Fix test_getsetlocale_issue1813() of test_locale: skip the test if setlocale() fails. Patch by Victor Stinner.
bpo-41561: Add workaround for Ubuntu’s custom OpenSSL security level policy.
bpo-43288: Fix test_importlib to correctly skip Unicode file tests if the fileystem does not support them.
Build
bpo-43631: Update macOS, Windows, and CI to OpenSSL 1.1.1k.
bpo-43617: Improve configure.ac: Check for presence of autoconf-archive package and remove our copies of M4 macros.
macOS
bpo-41837: Update macOS installer build to use OpenSSL 1.1.1j.
IDLE
bpo-42225: Document that IDLE can fail on Unix either from misconfigured IP masquerage rules or failure displaying complex colored (non-ascii) characters.
bpo-43283: Document why printing to IDLE’s Shell is often slower than printing to a system terminal and that it can be made faster by pre-formatting a single string before printing.
Python 3.9.2 final
Release date: 2021-02-19
Windows
bpo-43155: PyCMethod_New() is now present in python3.lib.
Python 3.9.2 release candidate 1
Release date: 2021-02-16
Security
bpo-42967: Fix web cache poisoning vulnerability by defaulting the query args separator to &, and allowing the user to choose a custom separator.
bpo-42938: Avoid static buffers when computing the repr of ctypes.c_double and ctypes.c_longdouble values.
Core and Builtins
bpo-42819: readline: Explicitly disable bracketed paste in the interactive interpreter, even if it’s set in the inputrc, is enabled by default (eg GNU Readline 8.1), or a user calls readline.read_init_file(). The Python REPL has not implemented bracketed paste support. Also, bracketed mode writes the "\x1b[?2004h" escape sequence into stdout which causes test failures in applications that don’t support it. It can still be explicitly enabled by calling readline.parse_and_bind("set enable-bracketed-paste on"). Patch by Dustin Rodrigues.
bpo-42806: Fix the column offsets for f-strings ast nodes surrounded by parentheses and for nodes that spawn multiple lines. Patch by Pablo Galindo.
bpo-40631: Fix regression where a single parenthesized starred expression was a valid assignment target.
bpo-32381: Fix encoding name when running a .pyc file on Windows: PyRun_SimpleFileExFlags() now uses the correct encoding to decode the filename.
bpo-42536: Several built-in and standard library types now ensure that their internal result tuples are always tracked by the garbage collector:
collections.OrderedDict.items()
dict.items()
enumerate()
functools.reduce()
itertools.combinations()
itertools.combinations_with_replacement()
itertools.permutations()
itertools.product()
itertools.zip_longest()
zip()
Previously, they could have become untracked by a prior garbage collection. Patch by Brandt Bucher.
bpo-42195: The __args__ of the parameterized generics for typing.Callable and collections.abc.Callable are now consistent. The __args__ for collections.abc.Callable are now flattened while typing.Callable’s have not changed. To allow this change, types.GenericAlias can now be subclassed and collections.abc.Callable’s __class_getitem__ will now return a subclass of types.GenericAlias. Tests for typing were also updated to not subclass things like Callable[..., T] as that is not a valid base class. Finally, both types no longer validate their argtypes, in Callable[[argtypes], resulttype] to prepare for PEP 612. Patch by Ken Jin.
Library
bpo-43102: The namedtuple __new__ method had its __builtins__ set to None instead of an actual dictionary. This created problems for introspection tools.
bpo-43108: Fixed a reference leak in the curses module. Patch by Pablo Galindo
bpo-42944: Fix random.Random.sample when counts argument is not None.
bpo-42931: Add randbytes() to random.__all__.
bpo-42780: Fix os.set_inheritable() for O_PATH file descriptors on Linux.
bpo-42851: remove __init_subclass__ support for Enum members
bpo-41748: Fix HTMLParser parsing rules for element attributes containing commas with spaces. Patch by Karl Dubost.
bpo-42759: Fixed equality comparison of tkinter.Variable and tkinter.font.Font. Objects which belong to different Tcl interpreters are now always different, even if they have the same name.
bpo-42756: Configure LMTP Unix-domain socket to use socket global default timeout when a timeout is not explicitly provided.
bpo-23328: Allow / character in username, password fields on _PROXY envars.
bpo-42655: subprocess extra_groups is now correctly passed into setgroups() system call.
bpo-42727: EnumMeta.__prepare__ now accepts **kwds to properly support __init_subclass__
bpo-42681: Fixed range checks for color and pair numbers in curses.
bpo-37961: Fix crash in tracemalloc.Traceback.__repr__() (regressed in Python 3.9).
bpo-42630: tkinter functions and constructors which need a default root window raise now RuntimeError with descriptive message instead of obscure AttributeError or NameError if it is not created yet or cannot be created automatically.
bpo-42644: logging.disable will now validate the types and value of its parameter. It also now accepts strings representing the levels (as does loging.setLevel) instead of only the numerical values.
bpo-36541: Fixed lib2to3.pgen2 to be able to parse PEP-570 positional only argument syntax.
bpo-42517: Enum: private names will raise a DeprecationWarning; in 3.10 they will become normal attributes
bpo-42678: Enum: call __init_subclass__ after members have been added
bpo-42532: Remove unexpected call of __bool__ when passing a spec_arg argument to a Mock.
bpo-42388: Fix subprocess.check_output(…, input=None) behavior when text=True to be consistent with that of the documentation and universal_newlines=True.
bpo-34463: Fixed discrepancy between traceback and the interpreter in formatting of SyntaxError with lineno not set (traceback was changed to match interpreter).
bpo-42375: subprocess module update for DragonFlyBSD support.
bpo-42384: Make pdb populate sys.path[0] exactly the same as regular python execution.
bpo-42383: Fix pdb: previously pdb would fail to restart the debugging target if it was specified using a relative path and the current directory changed.
bpo-42318: Fixed support of non-BMP characters in tkinter on macOS.
bpo-42163: Restore compatibility for uname_result around deepcopy and _replace.
bpo-39825: Windows: Change sysconfig.get_config_var('EXT_SUFFIX') to the expected full platform_tag.extension format. Previously it was hard-coded to .pyd, now it is compatible with distutils.sysconfig and will result in something like .cp38-win_amd64.pyd. This brings windows into conformance with the other platforms.
bpo-42059: typing.TypedDict types created using the alternative call-style syntax now correctly respect the total keyword argument when setting their __required_keys__ and __optional_keys__ class attributes.
bpo-39101: Fixed tests using IsolatedAsyncioTestCase from hanging on BaseExceptions.
bpo-42005: Fix CLI of cProfile and profile to catch BrokenPipeError.
bpo-41907: fix format() behavior for IntFlag
bpo-41889: Enum: fix regression involving inheriting a multiply-inherited enum
bpo-41891: Ensure asyncio.wait_for waits for task completion
bpo-41604: Don’t decrement the reference count of the previous user_ptr when set_panel_userptr fails.
bpo-40219: Lowered tkinter.ttk.LabeledScale dummy widget to prevent hiding part of the content label.
bpo-40084: Fix Enum.__dir__: dir(Enum.member) now includes attributes as well as methods.
bpo-39068: Fix initialization race condition in a85encode() and b85encode() in base64. Patch by Brandon Stansbury.
bpo-33289: Correct call to tkinter.colorchooser to return RGB triplet of ints instead of floats. Patch by Cheryl Sabella.
Documentation
bpo-40304: Fix doc for type(name, bases, dict). Patch by Boris Verkhovskiy and Éric Araujo.
bpo-42811: Updated importlib.utils.resolve_name() doc to use __spec__.parent instead of __package__. (Thanks Yair Frid.)
bpo-17140: Add documentation for the multiprocessing.pool.ThreadPool class.
Tests
bpo-42794: Update test_nntplib to use offical group name of news.aioe.org for testing. Patch by Dong-hee Na.
bpo-40810: In sqlite3, fix CheckTraceCallbackContent for SQLite pre 3.7.15.
Build
bpo-43174: Windows build now uses /utf-8 compiler option.
bpo-42692: Fix __builtin_available check on older compilers. Patch by Joshua Root.
bpo-42604: Now all platforms use a value for the “EXT_SUFFIX” build variable derived from SOABI (for instance in freeBSD, “EXT_SUFFIX” is now “.cpython-310d.so” instead of “.so”). Previosuly only Linux, Mac and VxWorks were using a value for “EXT_SUFFIX” that included “SOABI”.
bpo-42598: Fix implicit function declarations in configure which could have resulted in incorrect configuration checks. Patch contributed by Joshua Root.
bpo-29076: Add fish shell support to macOS installer.
Windows
bpo-41837: Updated Windows installer to include OpenSSL 1.1.1i
bpo-42584: Upgrade Windows installer to use SQLite 3.34.0.
macOS
bpo-42504: Ensure that the value of sysconfig.get_config_var(‘MACOSX_DEPLOYMENT_TARGET’) is always a string, even in when the value is parsable as an integer.
bpo-42361: Update macOS installer build to use Tcl/Tk 8.6.11 (rc2, expected to be final release).
bpo-41837: Update macOS installer build to use OpenSSL 1.1.1i.
bpo-42584: Update macOS installer to use SQLite 3.34.0.
IDLE
bpo-43008: Make IDLE invoke sys.excepthook() in normal, 2-process mode. Patch by Ken Hilton.
bpo-33065: Fix problem debugging user classes with __repr__ method.
bpo-23544: Disable Debug=>Stack Viewer when user code is running or Debugger is active, to prevent hang or crash. Patch by Zackery Spytz.
bpo-32631: Finish zzdummy example extension module: make menu entries work; add docstrings and tests with 100% coverage.
Tools/Demos
bpo-42726: Fixed Python 3 compatibility issue with gdb/libpython.py handling of attribute dictionaries.
bpo-42613: Fix freeze.py tool to use the prope config and library directories. Patch by Victor Stinner.
C API
bpo-43030: Fixed a compiler warning in Py_UNICODE_ISSPACE() on platforms with signed wchar_t.
bpo-42591: Export the Py_FrozenMain() function: fix a Python 3.9.0 regression. Python 3.9 uses -fvisibility=hidden and the function was not exported explicitly and so not exported.
bpo-40052: Fix an alignment build warning/error in function PyVectorcall_Function(). Patch by Andreas Schneider, Antoine Pitrou and Petr Viktorin.
Python 3.9.1 final
Core and Builtins
bpo-42576: types.GenericAlias will now raise a TypeError when attempting to initialize with a keyword argument. Previously, this would cause the interpreter to crash if the interpreter was compiled with debug symbols. This does not affect interpreters compiled for release. Patch by Ken Jin.
Library
bpo-5054: CGIHTTPRequestHandler.run_cgi() HTTP_ACCEPT improperly parsed. Replace the special purpose getallmatchingheaders with generic get_all method and add relevant tests.
bpo-17735: inspect.findsource() now raises OSError instead of IndexError when co_lineno of a code object is greater than the file length. This can happen, for example, when a file is edited after it was imported.
bpo-42116: Fix handling of trailing comments by inspect.getsource().
bpo-42487: ChainMap.__iter__ no longer calls __getitem__ on underlying maps
bpo-42482: TracebackException no longer holds a reference to the exception’s traceback object. Consequently, instances of TracebackException for equivalent but non-equal exceptions now compare as equal.
bpo-42406: We fixed an issue in pickle.whichmodule in which importing multiprocessing could change the how pickle identifies which module an object belongs to, potentially breaking the unpickling of those objects.
bpo-34215: Clarify the error message for asyncio.IncompleteReadError when expected is None.
bpo-12800: Extracting a symlink from a tarball should succeed and overwrite the symlink if it already exists. The fix is to remove the existing file or symlink before extraction. Based on patch by Chris AtLee, Jeffrey Kintscher, and Senthil Kumaran.
Tests
bpo-41473: Reenable test_gdb on gdb 9.2 and newer: https://bugzilla.redhat.com/show_bug.cgi?id=1866884 bug is fixed in gdb 10.1.
bpo-42553: Fix test_asyncio.test_call_later() race condition: don’t measure asyncio performance in the call_later() unit test. The test failed randomly on the CI.
macOS
bpo-41116: If no explicit macOS SDK was specified, setup.py should check for Tcl and TK frameworks in /Library/Frameworks; the previous commit inadvertently broke that test.
bpo-42504: Fix build on macOS Big Sur when MACOSX_DEPLOYMENT_TARGET=11
IDLE
bpo-42508: Keep IDLE running on macOS. Remove obsolete workaround that prevented running files with shortcuts when using new universal2 installers built on macOS 11.
3.9.0 Release highlights
New syntax features:
PEP 584, union operators added to dict;
PEP 585, type hinting generics in standard collections;
PEP 614, relaxed grammar restrictions on decorators.
New built-in features:
PEP 616, string methods to remove prefixes and suffixes.
New features in the standard library:
PEP 593, flexible function and variable annotations;
os.pidfd_open() added that allows process management without races and signals.
Interpreter improvements:
PEP 573, fast access to module state from methods of C extension types;
PEP 617, CPython now uses a new parser based on PEG;
a number of Python builtins (range, tuple, set, frozenset, list, dict) are now sped up using PEP 590 vectorcall;
garbage collection does not block on resurrected objects;
a number of Python modules (_abc, audioop, _bz2, _codecs, _contextvars, _crypt, _functools, _json, _locale, math, operator, resource, time, _weakref) now use multiphase initialization as defined by PEP 489;
a number of standard library modules (audioop, ast, grp, _hashlib, pwd, _posixsubprocess, random, select, struct, termios, zlib) are now using the stable ABI defined by PEP 384.
New library modules:
PEP 615, the IANA Time Zone Database is now present in the standard library in the zoneinfo module;
an implementation of a topological sort of a graph is now provided in the new graphlib module.
Release process changes:
PEP 602, CPython adopts an annual release cycle.
adam