Improvements
------------
- setup.py detects conda env and uses installed conda (hdf5, bzip2, lzo
and/or blosc) packages when building from source.
Bugs fixed
----------
- Linux wheels now built against built-in blosc.
- Fixed windows absolute paths in ptrepack, ptdump, ptree.
Changes from 3.2.2 to 3.2.3
Improvements
It is now possible to use HDF5 with the new shared library naming scheme (>= 1.8.10, hdf5.dll instead of hdf5dll.dll) on Windows (gh-540). Thanks to Tadeu Manoel.
Now :program: ptdump sorts output by node name and does not print a backtrace if file cannot be opened. Thanks to Zbigniew Jędrzejewski-Szmek.
Bugs fixed
Only run tables.tests.test_basics.UnicodeFilename if the filesystem encoding is utf-8. Closes gh-485.
Add lib64 to posix search path. (closes gh-507) Thanks to Mehdi Sadeghi.
Ensure cache entries are removed if fewer than 10 (closes gh-529). Thanks to Graham Jones.
Fix segmentation fault in a number of test cases that use index.Index (closes gh-532 and gh-533). Thanks to Diane Trout.
Fixed the evaluation of transcendental functions when numexpr is compiled with VML support (closes gh-534, PR #536). Thanks to Tom Kooij.
Make sure that index classes use buffersizes that are a multiple of chunkshape[0] (closes gh-538, PR #538). Thanks to Tom Kooij.
Ensure benchmark paths exist before benchmarks are executed (PR #544). Thanks to rohitjamuar.
Other changes
Minimum Cython version is now v0.21
Changes from 3.2.1.1 to 3.2.2
Bug fixed
Fix AssertionError in Row.__init_loop. See gh-477.
Fix issues with Cython 0.23. See gh-481.
Only run tables.tests.test_basics.UnicodeFilename if the filesystem encoding is utf-8. Closes gh-485.
Fix missing missing PyErr_Clear. See gh-#486.
Fix the C type of some numpy attributes. See gh-494.
Cast selection indices to integer. See gh-496.
Fix indexesextension._keysort_string. Closes gh-497 and gh-498.
Changes from 3.2.1 to 3.2.1.1
Fix permission on distributed source distribution
Other changes
Minimum Cython version is now v0.21
Changes from 3.2.0 to 3.2.1
Bug fixed
Fix indexesextension._keysort. Fixes gh-455. Thanks to Andrew Lin.
Changes from 3.1.1 to 3.2.0
Improvements
The nrowsinbuf is better computed now for EArray/CArray having a small chunkshape in the main dimension. Fixes#285.
PyTables should be installable very friendly via pip, including NumPy being installed automatically in the unlikely case it is not yet installed in the system. Thanks to Andrea Bedini.
setup.py has been largely simplified and now it requires setuptools. Although we think this is a good step, please keep us informed this is breaking some installation in a very bad manner.
setup.py now is able to used pkg-config, if available, to locate required libraries (hdf5, bzip2, etc.). The use of pkg-config can be controlled via setup.py command line flags or via environment variables. Please refer to the installation guide (in the User Manual) for details. Closes gh-442.
It is now possible to create a new node whose parent is a softlink to another group (see gh-422). Thanks to Alistair Muldal.
link.SoftLink objects no longer need to be explicitly dereferenced. Methods and attributes of the linked object are now automatically accessed when the user acts on a soft-link (see gh-399). Thanks to Alistair Muldal.
Now ptrepack recognizes hardlinks and replicates them in the output (repacked) file. This saves disk space and makes repacked files more conformal to the original one. Closes gh-380.
New pttree script for printing HDF5 file contents as a pretty ASCII tree (closes gh-400). Thanks to Alistair Muldal.
The internal Blosc library has been downgraded to version 1.4.4. This is in order to still allow using multiple threads inside Blosc, even on multithreaded applications (see gh-411, gh-412, gh-437 and gh-448).
The print_versions() function now also reports the version of compression libraries used by Blosc.
Now the setup.py tries to use the ‘-march=native’ C flag by default. In falls back on ‘-msse2’ if ‘-march=native’ is not supported by the compiler. Closes gh-379.
Fixed a spurious unicode comparison warning (closes gh-372 and gh-373).
Improved handling of empty string attributes. In previous versions of PyTables empty string were stored as scalar HDF5 attributes having size 1 and value ‘0’ (an empty null terminated string). Now empty string are stored as HDF5 attributes having zero size
Added a new cookbook recipe and a couple of examples for simple threading with PyTables.
The redundant utilsextension.get_indices() function has been eliminated (replaced by slice.indices()). Closes gh-195.
Allow negative indices in point selection (closes gh-360)
Index wasn’t being used if it claimed there were no results. Closes gh-351 (see also gh-353)
Atoms and Col types are no longer generated dynamically so now it is easier for IDEs and static analysis tool to handle them (closes gh-345)
The keysort functions in idx-opt.c have been cythonised using fused types. The perfomance is mostly unchanged, but the code is much more simpler now. Thanks to Andrea Bedini.
Small unit tests re-factoring:
print_versions() and tests.common.print_heavy() functions
moved to the tests.common module
always use print_versions() when test modules are called as scripts
use the unittest2 package in Python 2.6.x
removed internal machinery used to replicate unittest2 features
always use tests.common.PyTablesTestCase as base class for all test cases
code of the old tasts.common.cleanup() function has been moved to tests.common.PyTablesTestCase.tearDown() method
new implementation of tests.common.PyTablesTestCase.assertWarns() compatible with the one provided by the standard unittest module in Python >= 3.2
use tests.common.PyTablesTestCase.assertWarns() as context manager when appropriate
use the unittest.skipIf() decorator when appropriate
new :class:tests.comon.TestFileMixin: class
Bugs fixed
Fixed compatibility problems with numpy 1.9 and 1.10-dev (closes gh-362 and gh-366)
Fixed compatibility with Cython >= 0.20 (closes gh-386 and gh-387)
Fixed support for unicode node names in LRU cache (only Python 2 was affected). Closes gh-367 and gh-369.
Fixed support for unicode node titles (only Python 2 was affected). Closes gh-370 and gh-374.
Fixed a bug that caused the silent truncation of unicode attributes containing the ‘0’ character. Closes gh-371.
Fixed descr_from_dtype() to work as expected with complex types. Closes gh-381.
Fixed the tests.test_basics.ThreadingTestCase test case. Closes gh-359.
Fix incomplete results when performing the same query twice and exhausting the second iterator before the first. The first one writes incomplete results to seqcache (gh-353)
Fix false results potentially going to seqcache if tableextension.Row.update() is used during iteration (see gh-353)
Fix Column.create_csindex() when there’s NaNs
Fixed handling of unicode file names on windows (closes gh-389)
No longer not modify sys.argv at import time (closes gh-405)
Fixed a performance issue on NFS (closes gh-402)
Fixed a nasty problem affecting results of indexed queries. Closes gh-319 and probably gh-419 too.
Fixed another problem affecting results of indexed queries too. Closes gh-441.
Replaced “len(xrange(start, stop, step))” -> “len(xrange(0, stop - start, step))” to fix issues with large row counts with Python 2.x. Fixes#447.
Other changes
Cython is not a hard dependency anymore (although developers will need it so as to generated the C extension code).
The number of threads used by default for numexpr and Blosc operation that was set to the number of available cores have been reduced to 2. This is a much more reasonable setting for not creating too much overhead.
We are happy to announce PyTables 3.1.1.
This is a bug-fix release that addresses a critical bug that make PyTables
unusable on some platforms.
What's new
==========
- Fixed a critical bug that caused an exception at import time.
The error was triggered when a bug in long-double detection is detected
in the HDF5 library (see :issue:`275`) and numpy_ does not expose
`float96` or `float128`. Closes :issue:`344`.
- The internal Blosc_ library has been updated to version 1.3.5.
This fixes a false buffer overrun condition that made c-blosc to fail,
even if the problem was not real.
As always, a large amount of bugs have been addressed and squashed as well.
into options.mk.
Changes from 2.4 to 3.0
=======================
New features
------------
- Since this release PyTables provides full support to Python_ 3
(closes :issue:`188`).
- The entire code base is now more compliant with coding style guidelines
describe in the PEP8_ (closes :issue:`103` and :issue:`224`).
See `API changes`_ for more details.
- Basic support for HDF5 drivers. Now it is possible to open/create an
HDF5 file using one of the SEC2, DIRECT, LOG, WINDOWS, STDIO or CORE
drivers. Users can also set the main driver parameters (closes
:issue:`166`).
Thanks to Michal Slonina.
- Basic support for in-memory image files. An HDF5 file can be set from or
copied into a memory buffer (thanks to Michal Slonina). This feature is
only available if PyTables is built against HDF5 1.8.9 or newer.
Closes :issue:`165` and :issue:`173`.
- New :meth:`File.get_filesize` method for retrieving the HDF5 file size.
- Implemented methods to get/set the user block size in a HDF5 file
(closes :issue:`123`)
- Improved support for PyInstaller_. Now it is easier to pack frozen
applications that use the PyTables package (closes: :issue:`177`).
Thanks to Stuart Mentzer and Christoph Gohlke.
- All read methods now have an optional *out* argument that allows to pass a
pre-allocated array to store data (closes :issue:`192`)
- Added support for the floating point data types with extended precision
(Float96, Float128, Complex192 and Complex256). This feature is only
available if numpy_ provides it as well.
Closes :issue:`51` and :issue:`214`. Many thanks to Andrea Bedini.
- Consistent ``create_xxx()`` signatures. Now it is possible to create all
data sets :class:`Array`, :class:`CArray`, :class:`EArray`,
:class:`VLArray`, and :class:`Table` from existing Python objects (closes
:issue:`61` and :issue:`249`). See also the `API changes`_ section.
- Complete rewrite of the :mod:`nodes.filenode` module. Now it is fully
compliant with the interfaces defined in the standard :mod:`io` module.
Only non-buffered binary I/O is supported currently.
See also the `API changes`_ section. Closes :issue:`244`.
- New :program:`pt2to3` tool is provided to help users to port their
applications to the new API (see `API changes`_ section).
Improvements
------------
- Improved runtime checks on dynamic loading of libraries: meaningful error
messages are generated in case of failure.
Also, now PyTables no more alters the system PATH.
Closes :issue:`178` and :issue:`179` (thanks to Christoph Gohlke).
- Improved list of search paths for libraries as suggested by Nicholaus
Halecky (see :issue:`219`).
- Removed deprecated Cython_ include (.pxi) files. Contents of
:file:`convtypetables.pxi` have been moved in :file:`utilsextension.pyx`.
Closes :issue:`217`.
- The internal Blosc_ library has been upgraded to version 1.2.3.
- Pre-load the bzip2_ library on windows (closes :issue:`205`)
- The :meth:`File.get_node` method now accepts unicode paths
(closes :issue:`203`)
- Improved compatibility with Cython_ 0.19 (see :issue:`220` and
:issue:`221`)
- Improved compatibility with numexpr_ 2.1 (see also :issue:`199` and
:issue:`241`)
- Improved compatibility with development versions of numpy_
(see :issue:`193`)
- Packaging: since this release the standard tar-ball package no more includes
the PDF version of the "PyTables User Guide", so it is a little bit smaller
now. The complete and pre-build version of the documentation both in HTML
and PDF format is available on the file `download area`_ on SourceForge.net.
Closes: :issue:`172`.
- Now PyTables also uses `Travis-CI`_ as continuous integration service.
All branches and all pull requests are automatically tested with different
Python_ versions. Closes :issue:`212`.
Other changes
-------------
- PyTables now requires Python 2.6 or newer.
- Minimum supported version of Numexpr_ is now 2.0.
API changes
-----------
The entire PyTables API as been made more PEP8_ compliant (see :issue:`224`).
This means that many methods, attributes, module global variables and also
keyword parameters have been renamed to be compliant with PEP8_ style
guidelines (e.g. the ``tables.hdf5Version`` constant has been renamed into
``tables.hdf5_version``).
We made the best effort to maintain compatibility to the old API for existing
applications. In most cases, the old 2.x API is still available and usable
even if it is now deprecated (see the Deprecations_ section).
The only important backwards incompatible API changes are for names of
function/methods arguments. All uses of keyword arguments should be
checked and fixed to use the new naming convention.
The new :program:`pt2to3` tool can be used to port PyTables based applications
to the new API.
Many deprecated features and support for obsolete modules has been dropped:
- The deprecated :data:`is_pro` module constant has been removed
- The nra module and support for the obsolete numarray module has been removed.
The *numarray* flavor is no more supported as well (closes :issue:`107`).
- Support for the obsolete Numeric module has been removed.
The *numeric* flavor is no longer available (closes :issue:`108`).
- The tables.netcdf3 module has been removed (closes :issue:`68`).
- The deprecated :exc:`exceptions.Incompat16Warning` exception has been
removed
- The :meth:`File.create_external_link` method no longer has a keyword
parameter named *warn16incompat*. It was deprecated in PyTables 2.4.
Moreover:
- The :meth:`File.create_array`, :meth:`File.create_carray`,
:meth:`File.create_earray`, :meth:`File.create_vlarray`, and
:meth:`File.create_table` methods of the :class:`File` objects gained a
new (optional) keyword argument named ``obj``. It can be used to initialize
the newly created dataset with an existing Python object, though normally
these are numpy_ arrays.
The *atom*/*descriptor* and *shape* parameters are now optional if the
*obj* argument is provided.
- The :mod:`nodes.filenode` has been completely rewritten to be fully
compliant with the interfaces defined in the :mod:`io` module.
The FileNode classes currently implemented are intended for binary I/O.
Main changes:
* the FileNode base class is no more available,
* the new version of :class:`nodes.filenode.ROFileNode` and
:class:`nodes.filenode.RAFileNode` objects no more expose the *offset*
attribute (the *seek* and *tell* methods can be used instead),
* the *lineSeparator* property is no more available end the ``\n``
character is always used as line separator.
- The `__version__` module constants has been removed from almost all the
modules (it was not used after the switch to Git). Of course the package
level constant (:data:`tables.__version__`) still remains.
Closes :issue:`112`.
- The :func:`lrange` has been dropped in favor of xrange (:issue:`181`)
- The :data:`parameters.MAX_THREADS` configuration parameter has been dropped
in favor of :data:`parameters.MAX_BLOSC_THREADS` and
:data:`parameters.MAX_NUMEXPR_THREADS` (closes :issue:`147`).
- The :func:`conditions.compile_condition` function no more has a *copycols*
argument, it was no more necessary since Numexpr_ 1.3.1.
Closes :issue:`117`.
- The *expectedsizeinMB* parameter of the :meth:`File.create_vlarray` and of
the :meth:`VLArrsy.__init__` methods has been replaced by *expectedrows*.
See also (:issue:`35`).
- The :meth:`Table.whereAppend` method has been renamed into
:meth:`Table.append_where` (closes :issue:`248`).
Please refer to the :doc:`../MIGRATING_TO_3.x` document for more details about
API changes and for some useful hint about the migration process from the 2.X
API to the new one.
Other possibly incompatible changes
-----------------------------------
- All methods of the :class:`Table` class that take *start*, *stop* and
*step* parameters (including :meth:`Table.read`, :meth:`Table.where`,
:meth:`Table.iterrows`, etc) have been redesigned to have a consistent
behaviour. The meaning of the *start*, *stop* and *step* and their default
values now always work exactly like in the standard :class:`slice` objects.
Closes :issue:`44` and :issue:`255`.
- Unicode attributes are not stored in the HDF5 file as pickled string.
They are now saved on the HDF5 file as UTF-8 encoded strings.
Although this does not introduce any API breakage, files produced are
different (for unicode attributes) from the ones produced by earlier
versions of PyTables.
- System attributes are now stored in the HDF5 file using the character set
that reflects the native string behaviour: ASCII for Python 2 and UTF8 for
Python 3. In any case, system attributes are represented as Python string.
- The :meth:`iterrows` method of :class:`*Array` and :class:`Table` as well
as the :meth:`Table.itersorted` now behave like functions in the standard
:mod:`itertools` module.
If the *start* parameter is provided and *stop* is None then the
array/table is iterated from *start* to the last line.
In PyTables < 3.0 only one element was returned.
Deprecations
------------
- As described in `API changes`_, all functions, methods and attribute names
that was not compliant with the PEP8_ guidelines have been changed.
Old names are still available but they are deprecated.
- The use of upper-case keyword arguments in the :func:`open_file` function
and the :class:`File` class initializer is now deprecated. All parameters
defined in the :file:`tables/parameters.py` module can still be passed as
keyword argument to the :func:`open_file` function just using a lower-case
version of the parameter name.
Bugs fixed
----------
- Better check access on closed files (closes :issue:`62`)
- Fix for :meth:`File.renameNode` where in certain cases
:meth:`File._g_updateLocation` was wrongly called (closes :issue:`208`).
Thanks to Michka Popoff.
- Fixed ptdump failure on data with nested columns (closes :issue:`213`).
Thanks to Alexander Ford.
- Fixed an error in :func:`open_file` when *filename* is a :class:`numpy.str_`
(closes :issue:`204`)
- Fixed :issue:`119`, :issue:`230` and :issue:`232`, where an index on
:class:`Time64Col` (only, :class:`Time32Col` was ok) hides the data on
selection from a Tables. Thanks to Jeff Reback.
- Fixed ``tables.tests.test_nestedtypes.ColsTestCase.test_00a_repr`` test
method. Now the ``repr`` of of cols on big-endian platforms is correctly
handled (closes :issue:`237`).
- Fixes bug with completely sorted indexes where *nrowsinbuf* must be equal
to or greater than the *chunksize* (thanks to Thadeus Burgess).
Closes :issue:`206` and :issue:`238`.
- Fixed an issue of the :meth:`Table.itersorted` with reverse iteration
(closes :issue:`252` and :issue:`253`).
Major changes in 2.4:
- Improved HDF5 error logging management.
- Added support for the float16 data type.
- Leaf nodes now have attributes for retrieving the size of data in memory
and on disk.
- Configurable maximum number of threads for Blosc and Numexpr.
- ndim (read-only) attribute added to Leaf, Atom and Col objects.
- Added read support for variable length string attributes.
Full changelog for 2.4.0, see:
http://pytables.github.io/release-notes/RELEASE_NOTES_v2.4.x.html
Major changes in 2.3:
- Integrated functionality from PyTablesPro (re-licensed under a BSD license).
- OPSI is a powerful and innovative indexing engine allowing PyTables
to perform fast queries on arbitrarily large tables.
- A fine-tuned LRU cache for both metadata (nodes) and regular data.
Full changelog for 2.3.x, see:
http://pytables.github.io/release-notes/RELEASE_NOTES_v2.3.x.html
Remove devel/py-ctypes (only needed by and supporting python24).
Remove PYTHON_VERSIONS_ACCEPTED and PYTHON_VERSIONS_INCOMPATIBLE
lines that just mirror defaults now.
Miscellaneous cleanup while editing all these files.
changes:
-new compressor called Blosc (speed-up)
-A new `tables.Expr` module (based on Numexpr) that allows to do
persistent, on-disk computations on many algebraic operations.
-Support for HDF5 hard links, soft links and automatic external links
-Suport for 'fancy' indexing
-bugfixes
- assume that Python 2.4 and 2.5 are compatible and allow checking for
fallout.
- remove PYTHON_VERSIONS_COMPATIBLE that are obsoleted by the 2.3+
default. Modify the others to deal with the removals.
developer is officially maintaining the package.
The rationale for changing this from "tech-pkg" to "pkgsrc-users" is
that it implies that any user can try to maintain the package (by
submitting patches to the mailing list). Since the folks most likely
to care about the package are the folks that want to use it or are
already using it, this would leverage the energy of users who aren't
developers.
This is a major version release.
changes include:
-Undo/Redo
-support for objects (and indexes!) with more than 2**31 rows
-new datatypes
-improved HDF5 support
-compression support
PyTables 0.9.1:
This release is mainly a maintenance version. In it, some bugs has been
fixed and a few improvements has been made. One important thing is that
chunk sizes in EArrays has been re-tuned to get much better performance
and compression rations. Besides, it has been tested against the latest
Python 2.4 and all test units seems to pass fine.
PyTables 0.9:
In this release you will find a series of quite exciting new features,
the most important being new indexing capabilities, in-kernel selections,
support for complex datatypes and the possibility to modify values in both
tables and arrays (yeah, finally :).
python*-pth packages into meta-packages which will install the non-pth
packages. Bump PKGREVISIONs on the non-pth versions to propagate the
thread change, but leave the *-pth versions untouched to not affect
existing installations.
Sync all PYTHON_VERSIONS_AFFECTED lines in package Makefiles.
in the process. (More information on tech-pkg.)
Bump PKGREVISION and BUILDLINK_DEPENDS of all packages using libtool and
installing .la files.
Bump PKGREVISION (only) of all packages depending directly on the above
via a buildlink3 include.
