3.6.4:
plateau
The main focus in this release was fix some of the more glaring problems creapt in from the last release due to that refactor.
uncompyle6 code is at a plateau where what is most needed is a code refactoring. In doing this, until everything refactored and replaced, decomplation may get worse.
Therefore, this release largely serves as a checkpoint before more major upheaval.
The upheaval, in started last release, I believe the pinnicle was around c90ff51 which wasn't a release. I suppose I should tag that.
After c90ff5, I started down the road of redoing control flow in a more comprehensible, debuggable, and scalable way. See The Control Flow Mess.
The bulk of the refactoring going on in the decompyle3 project, but I try to trickle down the changes.
It is tricky because the changes are large and I have to figure decompose things so that little testable pieces can be done. And there is also the problem that what is in decompyle3 is incomplete as well.
Other than control flow, another change that will probably happen in the next release is to redo the grammar for lambda expressions. Right now, we treat them as Python statements, you know, things with compound statements in them. But lambdas aren't that. And so there is hackery to paper over difference making a statement out of an expression the wrong thing to do. For example, a return of an "and" expression can be expressed as nested "if" statements with return inside them, but the "if" variant of the bytecode is not valid in a lambda.
In the decompyle3 code, I've gone down the road making the grammar goal symbol be an expression. This also offers the opportunity to split the grammar making parsing inside lambda not only more reliable because the wrong choices don't exist, but also simpler and faster because all those rules just need don't need to exist in parsing.
I cringe in thinking about how the code has lived for so long without noticing such a simple stupidity, and lapse of sufficient thought.
3.6.3:
Martin and Susanne
Of late, every release fixes major gaps and embarrassments of the last release....
And in some cases, like this one, exposes lacuna and rot.
I now have [control] flow under control, even if it isn't the most optimal way.
I now have greatly expanded automated testing.
On the most recent Python versions I regularly decompile thousands of Python programs that are distributed with Python. when it is possible, I then decompile Python's standard test suite distributed with Python and run the decompiled source code which basically checks itself. This amounts to about 250 test programs per version. This is in addition to the 3 CI testing services which do different things.
Does this mean the decompiler works perfectly? No. There are still a dozen or so failing programs, although the actual number of bugs is probably smaller though.
However, in perparation of a more major refactoring of the parser grammar, this release was born.
In many cases, decompilation is better. But there are some cases where decompilation has gotten worse. For lack of time (and interest) 3.0 bytecode suffered a hit. Possibly some code in the 3.x range did too. In time and with cleaner refactored code, this will come back.
Commit c90ff51 was a local maxiumum before, I started reworking the grammar to separate productions that were specific to loops versus those that are not in loops.
In the middle of that I added another grammar simplication to remove singleton productions of the form sstmts-> stmts. These were always was a bit ugly, and complicated output.
At any rate if decompilation fails, you can try c90ff51. Or another decompiler. unpyc37 is pretty good for 3.7. wibiti uncompyle2 is great for 2.7. pycdc is mediocre for Python before 3.5 or so, and not that good for the most recent Python. Generally these programs will give some sort of answer even if it isn't correct.
decompyle3 isn't that good for 3.7 and worse for 3.8, but right now it does things no other Python decompiler like unpyc37 or pycdc does. For example, decompyle3 handles variable annotations. As always, the issue trackers for the various programs will give you a sense for what needs to be done. For now, I've given up on reporting issues in the other decompilers because there are already enough issues reported, and they are just not getting fixed anyway.
v1.10.0
Support for Erlang/OTP 21+
Releases improvements
Elixir v1.9 introduced releases as a mechanism to package self-contained applications. Elixir v1.10 further improves releases with bug fixes and new enhancements based on feedback we got from the community. The highlights are:
* Allow the dual boot system of releases to be disabled on environments that are boot-time sensitive, such as embedded devices
* Track and raise if compile-time configuration is set or changes at runtime (more in the next section)
* Support for easily adding extra files to releases via overlays
* Allow RELEASE_DISTRIBUTION to be set to none in order to fully disable it
* Add a built-in :tar step that automatically packages releases
Changelog:
Highlights
Various Bugfixes
In Depth
Runtime
WebAssembly
We continue to work on making our WebAssembly support better. Various sets of issues have been resolved in this release and general performance and feature work is happening as well.
Community improvements for AIX/PASE and Haiku
The ports for these systems received a bunch of improvements from community contributor Calvin Buckley (@NattyNarwhal).
Class Libraries
CoreFX integration
We continued to replace some of our classes with the implementation from CoreFX to improve performance and compatibility with .NET.
In this release we tightened up our networking with a few bug fixes.
Tools
Resolved Issues
7377 [Mono.Debugger.Soft] Connections are not properly closed
10641 New arm64_32 architecture (watchOS 5) [arm64_32] make “Debug Mode” work on Watch series 4 with –interpreter
12421 Task calls SynchronizationContext.Post on cancelling
12586 DataContractSerializer is not available
12917 [Blazor] IL Linker not working correctly with IQueryable extensions
14080 condition ‘ji’ not met, with ‘dynamic’ and multithreading
14725 mono 5.20 hangs when called from native thread
14772 Null Pointer Exception after debugger pauses and then continues
15261 [iOS][Tests] There are a number of System.Numeric tests that fail on iOS 32b devices
15262 [iOS][Tests] A number of linq tests fail on iOS 32b devices
15263 [iOS][Tests] A number of tests from Microsoft.CSharp fail on iOS 32b devices.
15307 [iOS][Tests] Several corelib tests crash on iOS32b devices.
15308 [netcore] Make OverlappedTests.PackNegTest Pass
15310 [netcore] Make OverlappedTests.UnPackTest Pass
15646 MERP indefinitely hangs on application quit
15687 [debugger] Update client thread frames after SetIP.
15805 Mono.Net.Security.MonoTlsStream: Crash when stream appears to have already been disposed
15931 1024 hard limit of open file descriptors
15992 Mono crashes during precompiling dll in fullaot mode
15994 Coverity: derefenrecing null in profiler (CID: 1307043) Skip degenerate qsort: num < 2, size == 0, base == 0
15999 [mini] run regression tests with fullaot+llvm
16010 typeof(object).GetMember(“”) does not return an empty array
16024 Native crash in ves_icall_System_Net_Sockets_Socket_Connect_internal
16032 IOException: Sharing violation on path …
16046 [netcore] AOT with unresolved dependencies hits asserts
16122 VS 16.2.0 regression: CopyToAsync from DeflateStream to GzipStream throws NotImplementedException
16172 Interpreter stack size is too large
16192 Embedded Mono hangs when using native threads
16308 Change netcore package to be using release builds
16310 LLVM: ““.AsSpan()[0] doesn’t throw IndexOutOfRangeException
16319 [wasm] Timers do not fire when run inside WebWorkers.
16369 Relocations in .text on arm
16380 [netcore] Regession with AOTed System.Private.CoreLib
16381 [llvm] Assertion: should not be reached at mini-llvm.c:7104
16395 [Mobile] DateTime.Now returns incorrect local time for one hour after entering DST in 39 locales.
16411 JitTests.Float fails on iOS devices in release mode on 2019-08
16415 Timezone data not handled correctly on FreeBSD
16460 System.IO.Path.GetFullPath(“\\.\pipe\pipename”): different results on Mono and .NET
16486 FileSystemEventArgs.FullPath throws ArgumentNullException
16605 make distclean fails after build w/ embedded zlib
16616 [netcore] Vector.CopyTo w/ LLVM emits wrong exception
16626 [WinForms] ControlBox property didn`t hide Close button and icon
16689 [merp] missing data in crashing reporting json native frames
16701 C# REPL Unusable on Mono 6 / Ubuntu 19
16709 Disposing the FileSystemWatcher does not close associated file descriptors
16712 SIGSEGV in mono_class_setup_vtable_general
16742 Date.Time parse issue (year 1948) Avoid creating un-representable DateTime for transition point
16759 Stream.ReadAsync continues on different thread
16803 mono_gc_finalize_notify spinning after process exit
16808 LLVM failed for String.GetHashCode
16819 [coop][interp] mono_threads_enter_gc_safe_region Cannot transition thread 0x1952b000 from STATE_BLOCKING with DO_BLOCKING
16824 Crash when debugging iOS application that throws on DispatchQueue
16864 [arm64_32] fix remaining regression tests
16876 [Wasm] Switch to PNSE FileWatcher & Exclude Tests
16879 [Wasm] Determine if System.Net.Http.UnitTests Failures are Relevant
16918 XmlSerializer Deserializing Property with same name as its enum type no longer works
16943 LLVM: recursive calls have overhead in JIT mode
16950 Starting in Mono 6.0 (2019-02), NotImplementedException prevents using StreamWriter with GZipStream for encodings that use a preamble
16974 File.GetCreationTimeUtc() returns the time last modified, not time created
17004 [merp] crash reports have Darwin kernel version in OSVersion field
17017 Out of bounds unhandled exception
17038 [wasm][xunit tests] RemoteExecutorTests.RemoteInvokeWritesToFile fails
17040 [wasm][xunit tests] System.Net.Security.Tests.SslStreamAlpnTests - MissingMethodException: Default constructor not found
17083 Regression: when debugger encounters a not-unhandled exception Marshalling managed exception contains cut off stacktraces
17133 SystemNative_CopyFile() call to fchmod() introduced in Mono 6.0 is not compatible with the external storage location on some Android OS versions and devices
17139 mono_aot_register_n_debug_info
17278 System.BadImageFormatException when reflecting on BCL shipped with Mono
17334 mono_magic_trampoline being called more than expected
17335 Investigate heavy stack from System_Threading_Interlocked_Exchange_T_REF_T_REF__T_REF
tem.Reflection.Emit + ConstructorInfo doesn’t work as expected
17687 [Regression] Profiler’s calls log SIGABRTs when using GDI+ loader
17718 Delegate does not invoke target virtually
17737 Q: is using loader lock for domain seq_points access correct? (mono runtime) [runtime] Fix locking in mono_get_seq_points ().
17790 [merp] SendMicrosoftTelemetry icall doesn’t do any MERP logging when inducing a Managed Crash
17833 threadpool hang due to unlimited hill climbing
17878 Deadlock in GC during bridge callback from native thread on Android
17924 Mtouch argument interpreter crashes EntityFrameworkCore 3.0 iOS app
17926 [Bug] Segmentation Fault receiving message on WatchOS with debugger attached
17931 Native linking fails for armv7s
18011 [embedding] Chaining Mach exceptions to Mono signals using utility threads
* Don't use DATE as a variable, as it's used elsewhere, use QJS_DATE instead
* Don't use the "bignum" PLIST variable anymore, the "bn" variants are no
longer being built
* Add "lto" PLIST variable, supported by and used on Linux
* Move the setting of CONFIG_BIGNUM to the pkg Makefile instead of
always setting it, should support 32-bit NetBSD ports.
Bump PKGREVISION.
Version 13.8.0 (Current):
Notable Changes
This is a security release.
Vulnerabilities fixed:
CVE-2019-15606: HTTP header values do not have trailing OWS trimmed.
CVE-2019-15605: HTTP request smuggling using malformed Transfer-Encoding header.
CVE-2019-15604: Remotely trigger an assertion on a TLS server with a malformed certificate string.
Also, HTTP parsing is more strict to be more secure. Since this may cause problems in interoperability with some non-conformant HTTP implementations, it is possible to disable the strict checks with the --insecure-http-parser command line flag, or the insecureHTTPParser http option. Using the insecure HTTP parser should be avoided.
Version 12.15.0 'Erbium' (LTS):
Notable changes
This is a security release.
Vulnerabilities fixed:
CVE-2019-15606: HTTP header values do not have trailing OWS trimmed.
CVE-2019-15605: HTTP request smuggling using malformed Transfer-Encoding header.
CVE-2019-15604: Remotely trigger an assertion on a TLS server with a malformed certificate string.
Also, HTTP parsing is more strict to be more secure. Since this may cause problems in interoperability with some non-conformant HTTP implementations, it is possible to disable the strict checks with the --insecure-http-parser command line flag, or the insecureHTTPParser http option. Using the insecure HTTP parser should be avoided.
Version 10.19.0 'Dubnium' (LTS):
Notable changes
This is a security release.
Vulnerabilities fixed:
CVE-2019-15606: HTTP header values do not have trailing OWS trimmed.
CVE-2019-15605: HTTP request smuggling using malformed Transfer-Encoding header.
CVE-2019-15604: Remotely trigger an assertion on a TLS server with a malformed certificate string.
Also, HTTP parsing is more strict to be more secure. Since this may cause problems in interoperability with some non-conformant HTTP implementations, it is possible to disable the strict checks with the --insecure-http-parser command line flag, or the insecureHTTPParser http option. Using the insecure HTTP parser should be avoided.
Changelog:
Updated BSD port of JDK 11
Additional features include:
Update to 11.0.6 GA
Fix remote HotSpot debugging on BSD
Add support for setting thread names on BSD (useful for top -H)
Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
functions of golang.org/x/crypto/cryptobyte can lead to a panic.
The malformed certificate can be delivered via a crypto/tls connection to a
client, or to a server that accepts client certificates. net/http clients can
be made to crash by an HTTPS server, while net/http servers that accept client
certificates will recover the panic and are unaffected.
Thanks to Project Wycheproof for providing the test cases that led to the
discovery of this issue.
The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.
This is also fixed in version v0.0.0-20200124225646-8b5121be2f68 of
golang.org/x/crypto/cryptobyte.
Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
functions of golang.org/x/crypto/cryptobyte can lead to a panic.
The malformed certificate can be delivered via a crypto/tls connection to a
client, or to a server that accepts client certificates. net/http clients can
be made to crash by an HTTPS server, while net/http servers that accept client
certificates will recover the panic and are unaffected.
Thanks to Project Wycheproof for providing the test cases that led to the
discovery of this issue.
The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.
This is also fixed in version v0.0.0-20200124225646-8b5121be2f68 of
golang.org/x/crypto/cryptobyte.
I created the package (based on lang/gcc8) and made the PLIST check
intentionally strict, to document which files are installed under which
circumstances. Therefore it's only fair that I get all the bug reports
directly.
Pkgsrc changes:
* adapt patches
* If NetBSD platform misses 64-bit atomics, turn off all atomics
(Trying to just disable BIGNUM fails; workaround suggested by
Fabrice Bellard)
Upstream changes:
- keep CONFIG_BIGNUM in the makefile
- added os.chdir()
- qjs: added -I option
- more memory checks in the bignum operations
- modified operator overloading semantics to be closer to the TC39
proposal
- suppressed "use bigint" mode. Simplified "use math" mode
- BigDecimal: changed suffix from 'd' to 'm'
- misc bug fixes
pkglint -r --network --only "migrate"
As a side-effect of migrating the homepages, pkglint also fixed a few
indentations in unrelated lines. These and the new homepages have been
checked manually.
Changes include:
- native 63-bit machine integers;
- a new sort of definitionally proof-irrelevant propositons: SProp;
- private universes for opaque polymorphic constants;
- string notations and numeral notations;
- a new simplex-based proof engine for the tactics lia, nia, lra and nra;
- new introduction patterns for SSReflect;
- a tactic to rewrite under binders: under;
- easy input of non-ASCII symbols in CoqIDE, which now uses GTK3.
and many small improvements and bugfixes.
