0.10.1:
Backwards Compatibility Notes
* ZstdCompressor.stream_reader().closed is now a property instead of a
method.
* ZstdDecompressor.stream_reader().closed is now a property instead of a
method.
Changes
* Stop attempting to package Python 3.6 for Miniconda. The latest version of
Miniconda is using Python 3.7. The Python 3.6 Miniconda packages were a lie
since this were built against Python 3.7.
* ZstdCompressor.stream_reader()'s and ZstdDecompressor.stream_reader()'s
closed attribute is now a read-only property instead of a method. This now
properly matches the IOBase API and allows instances to be used in more
places that accept IOBase instances.
0.10.0:
Backwards Compatibility Notes
* ZstdDecompressor.stream_reader().read() now consistently requires an
argument in both the C and CFFI backends. Before, the CFFI implementation
would assume a default value of -1, which was later rejected.
* The compress_literals argument and attribute has been removed from
zstd.ZstdCompressionParameters because it was removed by the zstd 1.3.5
API.
* ZSTD_CCtx_setParametersUsingCCtxParams() is no longer called on every
operation performed against ZstdCompressor instances. The reason for this
change is that the zstd 1.3.5 API no longer allows this without calling
ZSTD_CCtx_resetParameters() first. But if we called
ZSTD_CCtx_resetParameters() on every operation, we'd have to redo
potentially expensive setup when using dictionaries. We now call
ZSTD_CCtx_reset() on every operation and don't attempt to change
compression parameters.
* Objects returned by ZstdCompressor.stream_reader() no longer need to be
used as a context manager. The context manager interface still exists and its
behavior is unchanged.
* Objects returned by ZstdDecompressor.stream_reader() no longer need to be
used as a context manager. The context manager interface still exists and its
behavior is unchanged.
Bug Fixes
* ZstdDecompressor.decompressobj().decompress() should now return all data
from internal buffers in more scenarios. Before, it was possible for data to
remain in internal buffers. This data would be emitted on a subsequent call
to decompress(). The overall output stream would still be valid. But if
callers were expecting input data to exactly map to output data (say the
producer had used flush(COMPRESSOBJ_FLUSH_BLOCK) and was attempting to
map input chunks to output chunks), then the previous behavior would be
wrong. The new behavior is such that output from
flush(COMPRESSOBJ_FLUSH_BLOCK) fed into decompressobj().decompress()
should produce all available compressed input.
* ZstdDecompressor.stream_reader().read() should no longer segfault after
a previous context manager resulted in error.
* ZstdCompressor.compressobj().flush(COMPRESSOBJ_FLUSH_BLOCK) now returns
all data necessary to flush a block. Before, it was possible for the
flush() to not emit all data necessary to fully represent a block. This
would mean decompressors wouldn't be able to decompress all data that had been
fed into the compressor and flush()ed.
New Features
* New module constants BLOCKSIZELOG_MAX, BLOCKSIZE_MAX,
TARGETLENGTH_MAX that expose constants from libzstd.
* New ZstdCompressor.chunker() API for manually feeding data into a
compressor and emitting chunks of a fixed size. Like compressobj(), the
API doesn't impose restrictions on the input or output types for the
data streams. Unlike compressobj(), it ensures output chunks are of a
fixed size. This makes this API useful when the compressed output is being
fed into an I/O layer, where uniform write sizes are useful.
* ZstdCompressor.stream_reader() no longer needs to be used as a context
manager.
* ZstdDecompressor.stream_reader() no longer needs to be used as a context
manager.
* Bundled zstandard library upgraded from 1.3.4 to 1.3.6.
Changes
* Added zstd_cffi.py and NEWS.rst to MANIFEST.in.
* zstandard.__version__ is now defined.
* Upgrade pip, setuptools, wheel, and cibuildwheel packages to latest versions.
* Upgrade various packages used in CI to latest versions. Notably tox (in
order to support Python 3.7).
* Use relative paths in setup.py to appease Python 3.7.
* Added CI for Python 3.7.
Zstandard v1.3.7
perf: slightly better decompression speed on clang (depending on hardware target)
fix: ratio for dictionary compression at levels 9 and 10, reported by @indygreg
build: no longer build backtrace by default in release mode; restrict further automatic mode
build: control backtrace support through build macro BACKTRACE
misc: added man pages for zstdless and zstdgrep, by @samrussell
Zstandard v1.3.6 release is focused on intensive dictionary compression for database scenarios.
This is a new environment we are experimenting. The success of dictionary compression on small data, of which databases tend to store plentiful, led to increased adoption, and we now see scenarios where literally thousands of dictionaries are being used simultaneously, with permanent generation or update of new dictionaries.
== 1.0.0 (2018-05-20)
* *BreakingChange* The XZ module's methods now take any parameters
beyond the IO object as real Ruby keyword arguments rather than
a long argument list.
* *BreakingChange* XZ.decompress_stream now honours Ruby's
external and internal encoding concept instead of just
returning BINARY-tagged strings.
* *BreakingChange* Remove deprecated API on stream reader/writer
class and instead sync the API with Ruby's zlib library
(Ticket #12 by me).
* *BreakingChange* StreamWriter.new and StreamReader.new do not accept
a block anymore. This is part of syncing with Ruby's zlib API.
* *BreakingChange* StreamReader.open and StreamWriter.open always
return the new instance, even if a block is given to the method
(previous behaviour was to return the return value of the block).
This is part of the syncing with Ruby's zlib API.
* *BreakingChange* StreamReader.new and StreamWriter.new as well as
the ::open variants take additional arguments as real Ruby keyword
arguments now instead of a long parameter list plus options hash.
This is different from Ruby's own zlib API as that one takes both
a long parameter list and a hash of additional options. ruby-xz
is meant to follow zlib's semantics mostly, but not as a drop-in
replacement, so this divergence from zlib's API is okay (also
given that it isn't possible to replicate all possible options
1:1 anyway, since liblzma simply accepts different options as
libz). If you've never used these methods' optional arguments,
you should be fine.
* *BreakingChange* Stream#close now returns nil instead of the
number of bytes written. This syncs Stream#close with Ruby's
own IO#close, which also returns nil.
* *BreakingChange* Remove Stream#pos=, Stream#seek, Stream#stat. These
methods irritated the minitar gem, which doesn't expect them to
raise NotImplementedError, but directly to be missing if the object
does not support seeking.
* *BreakingChange* StreamReader and StreamWriter now honour Ruby's
encoding system instead of returning only BINARY-tagged strings.
* *Dependency* Remove dependency on ffi. ruby-xz now uses fiddle from
the stdlib instead.
* *Dependency* Remove dependency on io-like. ruby-xz now implements
all the IO mechanics itself. (Ticket #10 by me)
* *Dependency* Bump required Ruby version to 2.3.0.
* *Fix* libzlma.dylib not being found on OS X (Ticket #15 by
s0nspark).
- perf: minor decompression speed improvement (~+2%) with gcc
- fix : corruption in v1.8.2 at level 9 for files > 64KB under rare
conditions (#560)
- cli : new command --fast, by @jennifermliu
- api : LZ4_decompress_safe_partial() now decodes exactly the nb of
bytes requested (feature request #566)
- build : added Haiku target, by @fbrosson, and MidnightBSD, by @laffer1
- doc : updated documentation regarding dictionary compression
This is based on the decision The NetBSD Foundation made in 2008 to
do so, which was already applied to src.
This change has been applied to code which is likely not in other
repositories.
ok board@, reviewed by riastradh@
1.61 Sat 18 Aug 2018
- File::Find will not untaint [github/ThisUsedToBeAnEmail]
- Prevent from traversing symlinks and parent directories when extracting [github/ppisar]
Changes:
improve q=1 compression on small files
inverse Bazel workspace tree
add rolling-composite-hasher for large-window mode
add tools to download and transform static dictionary data
Changes:
2018-03-15 guidod <guidod@gmx.de>
* fix a number of CVEs reported with special *.zip PoC files
* man-pages are generated with new dbk2man.py - docbook xmlto is optional now
* completing some doc strings while checking the new man-pages to look good
* allow the zziptests.py testsuite to run with an installed /bin path
* try to fix some issues on testing with non-installed binaries on non-linux platfors
* update autotools to allow compiling on some newer Mac / Win machines
* a zip-program is still required for testing, but some errors are gone when not there
* complete the approximation of fnmatch for the test binaries (on platforms without)
* allow windows __mmap.h to be simpler, helping with some problems on MingW
* integrate 'fopen("wb")' from TexLive to be more portable across
* more portability as well for helpers like strnlen being used in the sources
* update doc refs to point to github instead of sf.net
* update the sf.net pages to have a prominent hint on newer github.com location
* release v0.13.69
2018-04-26 Stuart Caie <kyzer@cabextract.org.uk>
* read_chunk(): the test that chunk numbers are in bounds was off
by one, so read_chunk() returned a pointer taken from outside
allocated memory that usually crashes libmspack when accessed.
Thanks to Hanno Böck for finding the issue and providing a sample.
* chmd_read_headers(): reject files with blank filenames. Thanks
again to Hanno Böck for finding the issue and providing a sample file.
2018-02-06 Stuart Caie <kyzer@cabextract.org.uk>
* chmd.c: fixed an off-by-one error in the TOLOWER() macro, reported
by Dmitry Glavatskikh. Thanks Dmitry!
2017-11-26 Stuart Caie <kyzer@cabextract.org.uk>
* kwajd_read_headers(): fix up the logic of reading the filename and
extension headers to avoid a one or two byte overwrite. Thanks to
Jakub Wilk for finding the issue.
* test/kwajd_test.c: add tests for KWAJ filename.ext handling
2017-10-16 Stuart Caie <kyzer@cabextract.org.uk>
* test/cabd_test.c: update the short string tests to expect not only
MSPACK_ERR_DATAFORMAT but also MSPACK_ERR_READ, because of the recent
change to cabd_read_string(). Thanks to maitreyee43 for spotting this.
* test/msdecompile_md5: update the setup instructions for this script,
and also change the script so it works with current Wine. Again, thanks
to maitreyee43 for trying to use it and finding it not working.
2017-08-13 Stuart Caie <kyzer@cabextract.org.uk>
* src/chmextract.c: support MinGW one-arg mkdir(). Thanks to AntumDeluge
for reporting this.
2017-08-13 Stuart Caie <kyzer@cabextract.org.uk>
* read_spaninfo(): a CHM file can have no ResetTable and have a
negative length in SpanInfo, which then feeds a negative output length
to lzxd_init(), which then sets frame_size to a value of your choosing,
the lower 32 bits of output length, larger than LZX_FRAME_SIZE. If the
first LZX block is uncompressed, this writes data beyond the end of the
window. This issue was raised by ClamAV as CVE-2017-6419. Thanks to
Sebastian Andrzej Siewior for finding this by chance!
* lzxd_init(), lzxd_set_output_length(), mszipd_init(): due to the issue
mentioned above, these functions now reject negative lengths
2017-08-05 Stuart Caie <kyzer@cabextract.org.uk>
* cabd_read_string(): add missing error check on result of read().
If an mspack_system implementation returns an error, it's interpreted
as a huge positive integer, which leads to reading past the end of the
stack-based buffer. Thanks to Sebastian Andrzej Siewior for explaining
the problem. This issue was raised by ClamAV as CVE-2017-11423
2016-04-20 Stuart Caie <kyzer@cabextract.org.uk>
* configure.ac: change my email address to kyzer@cabextract.org.uk
2015-05-10 Stuart Caie <kyzer@4u.net>
* cabd_read_string(): correct rejection of empty strings. Thanks to
Hanno Böck for finding the issue and providing a sample file.
2015-05-10 Stuart Caie <kyzer@4u.net>
* Makefile.am: Add subdir-objects option as suggested by autoreconf.
* configure.ac: Add AM_PROG_AR as suggested by autoreconf.
2015-01-29 Stuart Caie <kyzer@4u.net>
* system.h: if C99 inttypes.h exists, use its PRI{d,u}{32,64} macros.
Thanks to Johnathan Kollasch for the suggestion.
New in 1.7
* cabextract now supports an --encoding parameter, to specify the character
encoding of CAB filenames if they are not ASCII or UTF8
* cabextract -L now lowercases non-ASCII characters
Performing substitutions during post-patch breaks tools such as mkpatches,
making it very difficult to regenerate correct patches after making changes,
and often leading to substituted string replacements being committed.
Update LICENSE
Upstream changes:
0.26 (2018/06/09)
Implemented refactoring due warnings from Perl::Critic.
0.25 (2018/06/04)
Implemented refactoring due warnings from Perl::Critic.
Merge pull request #3 from manwar/suggest-code-tidy
0.24 (2018/06/02)
Added a LICENSE file (GNU GPL v3).
Removed MYMETA files (see https://rt.cpan.org/Ticket/Display.html?id=108171).
Improved Kwalitee by adding information to Makefile.PL
Fixed tests under OpenBSD
Added some code to check for OpenBSD tar, which is not quite compatible to the command line options passed by this module.
Also made the method is_gnu() more robust, testing the return code and properly handling STDOUT and STDERR when trying "tar --version".
Dependencies added are those already available on standard perl (Config and IPC::Open3).
Added a README.md for better formatting in Github project page.
Small refactorings and code formating with perltidy.
Upstream changes:
2.30 19/06/2018
- skip white_space test on MSWin32 as Windows will report that both
files exist, which is obviously a 'feature'
2.28 08/06/2018 (madroach, ARC, OCBNET, ppisar)
- fix creating file with trailing whitespace on filename - fixes 103279
- allow archiving with absolute pathnames - fixes 97748
- small POD fix
- Speed up extract when archive contains lots of files
- CVE-2018-12015 directory traversal vulnerability [RT#125523]
2.0.1:
This release fixes: tests failed when run under python setup.py test, but passed when running under tox.
2.0.0:
It's now possible to specify a compession dictionary for block compression.
The bundled LZ4 libraries have been updated to 1.8.2
A compatibility fix for 2.x memoryview objects has been added.
Various flake8 cleanups and test additions.
This Go language package supports the reading and writing of xz
compressed streams. It includes also a gxz command for compressing and
decompressing data. The package is completely written in Go and
doesn't have any dependency on any C code.
Changes 2.8:
add support for setting atime, ctime, mtime and birthtime
tell libarchive when writing an archive is aborted due to an exception
add support for getting uid and gid
add support for high resolution timestamps
add two new archive readers: stream_reader and custom_reader
add missing archive extraction flags
add the lz4 and warc formats
add support for write options and uid/gid lookup
innoextract 1.7 (2018-06-12)
- Added support for Inno Setup 5.6.0 installers
- Added support for new GOG installers with GOG Galaxy file parts
- Added support for encrypted installers with the --password (-P) and --password-file options
- Added a --show-password option to print password check information
- Added a --check-password option to abort if the provided password does not match the stored checksum
- Added a --info (-i) convenience option to print information about the installer
- Added a --list-sizes option to print file sizes even with --quiet or --silent
- Added a --list-checksums option to print file checksums
- Added a --data-version (-V) option to print the data version and exit
- Added a --no-extract-unknown (-n) option to abort on unknown Inno Setup data versions
- Fixed building in paths that contain regex expressions
- Fixed case-sensitivity in parent directory when creating subdirectories
- Fixed .bin slice file names used with Inno Setup versions older than 4.1.7
- Fixed build with newer libc++ versions
- Made loading of .bin slice files case-insensitive
- The --test option can now be combined with --extract to abort on file checksum errors
- Now compiles in C++17 mode if supported
