Upstream changes:
HDF5 version 1.10.2 released on 2018-03-29
================================================================================
CONTENTS
- New Features
- Support for new platforms and languages
- Bug Fixes since HDF5-1.10.1
- Supported Platforms
- Tested Configuration Features Summary
- More Tested Platforms
- Known Problems
New Features
============
Configuration and Build Systems:
--------------------------------
- CMake builds
--------------
- Changed minimum CMake required version to 3.10.
This change removed the need to support a copy of the FindMPI.cmake module,
which has been removed, along with its subfolder in the config/cmake_ext_mod
location.
(ADB - 2018/03/09)
- Added pkg-config file generation
Added pkg-config file generation for the C, C++, HL, and HL C++ libraries.
In addition, builds on Linux will create h5cc, h5c++, h5hlcc, and h5hlc++ scripts in the bin
directory that use the pkg-config files. The scripts can be used to build HDF5 C and C++
applications (i.e, similar to the compiler scripts produced by the Autotools builds).
(ADB - 2018/03/08, HDFFV-4359)
- Refactored use of CMAKE_BUILD_TYPE for new variable, which understands
the type of generator in use.
Added new configuration macros to use new HDF_BUILD_TYPE variable. This
variable is set correctly for the type of generator being used for the build.
(ADB - 2018/01/08, HDFFV-10385, HDFFV-10296)
- Autotools builds
------------------
- Removed version-specific gcc/gfortran flags for version 4.0 (inclusive)
and earlier.
The config/gnu-flags file, which is sourced as a part of the configure
process, adds version-specific flags for use when building HDF5. Most of
these flags control warnings and do not affect the final product.
Flags for older versions of the compiler were consolidated into the
common flags section. Moving these flags simplifies maintenance of
the file.
The upshot of this is that building with ancient versions of gcc
(<= 4.0) will possibly no longer work without hand-hacking the file
to remove the flags not understood by that version of the compiler.
Nothing should change when building with gcc >= 4.1.
(DER - 2017/05/31, HDFFV-9937)
- -fno-omit-frame-pointer was added when building with debugging symbols
enabled.
Debugging symbols can be enabled independently of the overall build
mode in both the autotools and CMake. This allows (limited) debugging
of optimized code. Since many debuggers rely on the frame pointer,
we've disabled this optimization when debugging symbols are requested
(e.g.: via building with --enable-symbols).
(DER - 2017/05/31, HDFFV-10226)
Library:
--------
- Added an enumerated value to H5F_libver_t for H5Pset_libver_bounds().
Currently, the library defines two values for H5F_libver_t and supports
only two pairs of (low, high) combinations as derived from these values.
Thus the bounds setting via H5Pset_libver_bounds() is rather restricted.
Added an enumerated value (H5F_LIBVER_V18) to H5F_libver_t and
H5Pset_libver_bounds() now supports five pairs of (low, high) combinations
as derived from these values. This addition provides the user more
flexibility in setting bounds for object creation.
(VC - 2018/03/14)
- Added prefix option to VDS files.
Currently, VDS source files must be in the active directory to be
found by the virtual file. Adding the option of a prefix to be set
on the virtual file, using a data access property list (DAPL),
allows the source files to locate at an absolute or relative path
to the virtual file.
Private utility functions in H5D and H5L packages merged into single
function in H5F package.
New public APIs:
herr_t H5Pset_virtual_prefix(hid_t dapl_id, const char* prefix);
ssize_t H5Pget_virtual_prefix(hid_t dapl_id, char* prefix /*out*/, size_t size);
The prefix can also be set with an environment variable, HDF5_VDS_PREFIX.
(ADB - 2017/12/12, HDFFV-9724, HDFFV-10361)
- H5FDdriver_query() API call added to the C library.
This new library call allows the user to query a virtual file driver
(VFD) for the feature flags it supports (listed in H5FDpublic.h).
This can be useful to determine if a VFD supports SWMR, for example.
Note that some VFDs have feature flags that may only be present
after a file has been created or opened (e.g.: the core VFD will
have the H5FD_FEAT_POSIX_COMPAT_HANDLE flag set if the backing
store is switched on). Since the new API call queries a generic VFD
unassociated with a file, these flags will never be returned.
(DER - 2017/05/31, HDFFV-10215)
- H5FD_FEAT_DEFAULT_VFD_COMPATIBLE VFD feature flag added to the C library.
This new feature flag indicates that the VFD is compatible with the
default VFD. VFDs that set this flag create single files that follow
the canonical HDF5 file format.
(DER - 2017/05/31, HDFFV-10214)
- The H5I_REFERENCE value in the H5I_type_t enum (defined in H5Ipublic.h)
has been marked as deprectated.
This ID type value is not used in the C library. i.e.: There are no
hid_t values that are of ID type H5I_REFERENCE.
This enum value will be removed in a future major version of the library.
The code will remain unchanged in the HDF5 1.10.x releases and branches.
(DER - 2017/04/05, HDFFV-10252)
Parallel Library:
-----------------
- Enabled compression for parallel applications.
With this release parallel applications can create and write compressed
datasets (or the datasets with the filters such as Fletcher32 applied).
(EIP - 2018/03/29)
- Addressed slow file close on some Lustre file systems.
Slow file close has been reported on some Lustre file systems.
While the ultimate cause is not understood fully, the proximate
cause appears to be long delays in MPI_File_set_size() calls at
file close and flush.
To minimize this problem pending a definitive diagnosis and fix,
PHDF5 has been modified to avoid MPI_File_set_size() calls when
possible. This is done by comparing the library's EOA (End of
Allocation) with the file systems EOF, and skipping the
MPI_File_set_size() call if the two match.
(JRM - 2018/03/29)
- Optimized parallel open/location of the HDF5 super-block.
Previous releases of PHDF5 required all parallel ranks to
search for the HDF5 superblock signature when opening the
file. As this is accomplished more or less as a synchronous
operation, a large number of processes can experience a
slowdown in the file open due to filesystem contention.
As a first step in improving the startup/file-open performance,
we allow MPI rank 0 of the associated MPI communicator to locate
the base offset of the super-block and then broadcast that result
to the remaining ranks in the parallel group. Note that this
approach is utilized ONLY during file opens which employ the MPIO
file driver in HDF5 by previously having called H5Pset_fapl_mpio().
HDF5 parallel file operations which do not employ multiple ranks
e.g. specifiying MPI_COMM_SELF (whose MPI_Comm_size == 1)
as opposed to MPI_COMM_WORLD, will not be affected by this
optimization. Conversely, parallel file operations on subgroups
of MPI_COMM_WORLD are allowed to be run in parallel with each
subgroup operating as an independant collection of processes.
(RAW - 2017/10/10, HDFFV-10294)
- Added large (>2GB) MPI-IO transfers.
Previous releases of PHDF5 would fail when attempting to
read or write greater than 2GB of data in a single IO operation.
This issue stems principally from an MPI API whose definitions
utilize 32 bit integers to describe the number of data elements
and datatype that MPI should use to effect a data transfer.
Historically, HDF5 has invoked MPI-IO with the number of
elements in a contiguous buffer represented as the length
of that buffer in bytes.
Resolving the issue and thus enabling larger MPI-IO transfers
is accomplished first, by detecting when a user IO request would
exceed the 2GB limit as described above. Once a transfer request
is identified as requiring special handling, PHDF5 now creates a
derived datatype consisting of a vector of fixed sized blocks
which is in turn wrapped within a single MPI_Type_struct to
contain the vector and any remaining data. The newly created
datatype is then used in place of MPI_BYTE and can be used to
fulfill the original user request without encountering API
errors.
(RAW - 2017/09/10, HDFFV-8839)
C++ Library:
------------
- The following C++ API wrappers have been added to the C++ Library:
+ H5Lcreate_soft:
// Creates a soft link from link_name to target_name.
void link(const char *target_name, const char *link_name,...)
void link(const H5std_string& target_name,...)
+ H5Lcreate_hard:
// Creates a hard link from new_name to curr_name.
void link(const char *curr_name, const Group& new_loc,...)
void link(const H5std_string& curr_name, const Group& new_loc,...)
// Creates a hard link from new_name to curr_name in same location.
void link(const char *curr_name, const hid_t same_loc,...)
void link(const H5std_string& curr_name, const hid_t same_loc,...)
Note: previous version of H5Location::link will be deprecated.
+ H5Lcopy:
// Copy an object from a group of file to another.
void copyLink(const char *src_name, const Group& dst,...)
void copyLink(const H5std_string& src_name, const Group& dst,...)
// Copy an object from a group of file to the same location.
void copyLink(const char *src_name, const char *dst_name,...)
void copyLink(const H5std_string& src_name,...)
+ H5Lmove:
// Rename an object in a group or file to a new location.
void moveLink(const char* src_name, const Group& dst,...)
void moveLink(const H5std_string& src_name, const Group& dst,...)
// Rename an object in a group or file to the same location.
void moveLink(const char* src_name, const char* dst_name,...)
void moveLink(const H5std_string& src_name,...)
Note: previous version H5Location::move will be deprecated.
+ H5Ldelete:
// Removes the specified link from this location.
void unlink(const char *link_name,
const LinkAccPropList& lapl = LinkAccPropList::DEFAULT)
void unlink(const H5std_string& link_name,
const LinkAccPropList& lapl = LinkAccPropList::DEFAULT)
Note: additional parameter is added to previous H5Location::unlink.
+ H5Tencode and H5Tdecode:
// Creates a binary object description of this datatype.
void DataType::encode() - C API H5Tencode()
// Returns the decoded type from the binary object description.
DataType::decode() - C API H5Tdecode()
ArrayType::decode() - C API H5Tdecode()
CompType::decode() - C API H5Tdecode()
DataType::decode() - C API H5Tdecode()
EnumType::decode() - C API H5Tdecode()
FloatType::decode() - C API H5Tdecode()
IntType::decode() - C API H5Tdecode()
StrType::decode() - C API H5Tdecode()
VarLenType::decode() - C API H5Tdecode()
+ H5Lget_info:
// Returns the information of the named link.
H5L_info_t getLinkInfo(const H5std_string& link_name,...)
(BMR - 2018/03/11, HDFFV-10149)
- Added class LinkCreatPropList for link create property list.
(BMR - 2018/03/11, HDFFV-10149)
- Added overloaded functions H5Location::createGroup to take a link
creation property list.
Group createGroup(const char* name, const LinkCreatPropList& lcpl)
Group createGroup(const H5std_string& name, const LinkCreatPropList& lcpl)
(BMR - 2018/03/11, HDFFV-10149)
- A document is added to the HDF5 C++ API Reference Manual to show the
mapping from a C API to C++ wrappers. It can be found from the main
page of the C++ API Reference Manual.
(BMR - 2017/10/17, HDFFV-10151)
Java Library:
----------------
- Wrapper added for enabling the error stack.
H5error_off would disable the error stack reporting. In order
to re-enable the reporting, the error stack info needs to be
saved so that H5error_on can revert state.
(ADB - 2018/03/13, HDFFV-10412)
- Wrappers were added for the following C APIs:
H5Pset_evict_on_close
H5Pget_evict_on_close
H5Pset_chunk_opts
H5Pget_chunk_opts
H5Pset_efile_prefix
H5Pget_efile_prefix
H5Pset_virtual_prefix
H5Pget_virtual_prefix
(ADB - 2017/12/20)
- The H5I_REFERENCE value in the H5I_type_t enum (defined in H5Ipublic.h)
has been marked as deprectated.
JNI code which refers to this value will be removed in a future
major version of the library. The code will remain unchanged in the
1.10.x releases and branches.
See the C library section, above, for further information.
(HDFFV-10252, DER, 2017/04/05)
Tools:
------
- h5diff has a new option to display error stack.
Updated h5diff with the --enable-error-stack argument, which
enables the display of the hdf5 error stack. This completes the
improvement to the main tools: h5copy, h5diff, h5dump, h5ls and
h5repack.
(ADB - 2017/08/30, HDFFV-9774)
Support for new platforms, languages and compilers.
=======================================
- None
New Features
============
Configuration
-------------
- CMake: Added NAMESPACE hdf5:: to package configuration files to allow
projects using installed HDF5 binaries built with CMake to link with
them without specifying the HDF5 library location via IMPORTED_LOCATION.
(ADB, 2016/10/17, HDFFV-10003)
- CMake: Changed the CTEST_BUILD_CONFIGURATION option to
CTEST_CONFIGURATION_TYPE as recommended by the CMake documentation.
(ADB, 2016/10/17, HDFFV-9971)
- CMake: Added support for GIT
(ADB, 2016/07/12)
Bug Fixes since HDF5-1.8.17
===========================
Configuration
-------------
- Fixed a problem preventing HDF5 to be built on 32-bit CYGWIN by
condensing cygwin configuration files into a single file and
removing outdated compiler settings.
(ABD, 2016/07/12, HDFFV-9946)
- CMake: Fixed a command length overflow error by converting custom
commands inside CMakeTest.cmake files into regular dependencies and
targets.
(ABD, 2016/07/12, HDFFV-9939)
- CMake: Fixed a timeout error that would occasionally occur when running
the virtual file driver tests simultaneously due to test directory and file
name collisions.
(ABD, 2016/09/19, HDFFV-9431)
Library
-------
- Fixed a memory leak that would occur when the library allocated memory
for an external file prefix (H5Pset_efile_prefix) and failed to free it.
(DER, 2016/04/29)
- Fixed an error that would occur when calling H5Adelete on an attribute
which is attached to an externally linked object in the target file and
whose datatype is a committed datatype in the main file.
(VC, 2016-07-04, HDFFV-9940)
- Fixed a problem where a plugin compiled into a DLL in the default plugin
directory could not be found by the HDF5 library at runtime on Windows
when the HDF5_PLUGIN_PATH environment variable was not set.
(ABD, 2016/08/01, HDFFV-9706)
- Fixed an issue where H5Pset_alignment could result in misaligned blocks
with some input combinations, causing an assertion failure in debug mode.
(NAF, 2016/08/11, HDFFV-9948)
- A number of issues were fixed when reading/writing from/to corrupted
files to ensure that the library fails gracefully in these cases:
* Writing to a corrupted file that has an object message which is
incorrectly marked as sharable on disk results in a buffer overflow /
invalid write instead of a clean error message.
* Decoding data from a corrupted file with a dataset encoded with the
H5Z_NBIT decoding can result in a code execution vulnerability under
the context of the application using the HDF5 library.
* When decoding an array datatype from a corrupted file, the HDF5 library
fails to return an error in production if the number of dimensions
decoded is greater than the maximum rank.
* When decoding an "old style" array datatype from a corrupted file, the
HDF5 library fails to return an error in production if the number of
dimensions decoded is greater than the maximum rank.
(NAF, 2016/10/06, HDFFV-9950, HDFFV-9951, HDFFV-9992, HDFFV-9993)
- Fixed an error that would occur when copying an object with an attribute
which is a compound datatype consisting of a variable length string.
(VC, 2016-10-17, HDFFV-7991)
Parallel Library
----------------
- Fixed a bug that could occur when allocating a chunked dataset in parallel
with an alignment set and an alignment threshold greater than the chunk
size but less than or equal to the raw data aggregator size.
(NAF, 2016/08/11, HDFFV-9969)
Performance
-------------
- None
Tools
-----
- Fixed an error in the compiler wrapper scripts (h5cc, h5fc, et al.)
in which they would erroneously drop the file argument specified via
the -o flag when the -o flag was specified before the -c flag on the
command line, resulting in a failure to compile.
(LRK, 2016/06/08, HDFFV-9938, HDFFV-9530)
- h5repack User Defined (UD) filter parameters were not parsed correctly.
The UD filter parameters were not being parsed correctly. Reworked coding
section to parse the correct values and verify number of parameters.
(ADB, 2016/10/19, HDFFV-9996, HDFFV-9974, HDFFV-9515, HDFFV-9039)
Fortran API
-----------
- Fortran library fails to compile and fails tests with NAG compiler.
* Removed the non-standard assumption that KIND=SIZEOF, in the HDF5
configure programs.
* Removed Fortran 66 character/integer conversions from tests.
* Removed the use of C_SIZEOF in the test programs
* Changed to using STORAGE_SIZE in the test programs if available. Otherwise,
uses C_SIZEOF or SIZEOF.
(MSB, 2016/9/22, HDFFV-9973)
- Fortran segfaults for F03 tests with NAG compiler
* Removed INTENT(OUT) from 'fillvalue' in F2003 interface
for H5Pget_fill_value_f.
(MSB, 2016/9/22, HDFFV-9980)
C++ API
-------
- The macro H5_NO_NAMESPACE is deprecated from the HDF5 C++ API library.
In future releases, the macros H5_NO_STD and OLD_HEADER_FILENAME may
also be removed.
(BMR, 2016/10/27, HDFFV-9532)
High-Level APIs:
---------------
- The high-level API Packet Table (PT) did not write data correctly when
the datatype is a compound type that has string type as one of the
members. This problem started in 1.8.15, after the fix of HDFFV-9042
was applied, which caused the Packet Table to use native type to access
the data. It should be up to the application to specify whether the
buffer to be read into memory in the machine’s native architecture.
Thus, the PT is fixed to not use native type but to make a copy of the
user's provided datatype during creation or the packet table's datatype
during opening. If an application wishes to use native type to read the
data, then the application will request that. However, the Packet Table
doesn't provide a way to specify memory datatype in this release. This
feature will be available in future releases, HDFFV-10023.
(BMR, 2016/10/27, HDFFV-9758)
Fortran High-Level APIs:
------------------------
- None
Testing
-------
- None
Supported Platforms
===================
The following platforms are supported and have been tested for this release.
They are built with the configure process unless specified otherwise.
Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
#1 SMP x86_64 GNU/Linux compilers:
(platypus/mayll) Version 4.4.7 20120313
Versions 4.8.4, 4.9.3, 5.2.0
PGI C, Fortran, C++ for 64-bit target on
x86-64;
Version 15.7-0
Intel(R) C (icc), C++ (icpc), Fortran (icc)
compilers:
Version 15.0.3.187 Build 20150407
MPICH 3.1.4 compiled with GCC 4.9.3
Linux 2.6.32-573.18.1.el6 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16)
#1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16)
(ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16)
IBM XL C/C++ V13.1
IBM XL Fortran V15.1
Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
#1 SMP x86_64 GNU/Linux compilers:
(kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4)
Versions 4.9.3, 5.2.0
Intel(R) C (icc), C++ (icpc), Fortran (icc)
compilers:
Version 15.0.3.187 Build 20150407
MPICH 3.1.4 compiled with GCC 4.9.3
SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc
(emu) Sun Fortran 95 8.6 SunOS_sparc
Sun C++ 5.12 SunOS_sparc
Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
Cygwin(CYGWIN_NT-6.1 2.2.1(0.289/5/3) gcc(4.9.3) compiler and gfortran)
(cmake and autotools)
Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
Windows 10 Visual Studio 2015 w/ Intel Fortran 16 (cmake)
Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 16 (cmake)
Mac OS X Mt. Lion 10.8.5 Apple LLVM version 5.1 (clang-503.0.40)
64-bit gfortran GNU Fortran (GCC) 4.8.2
(swallow/kite) Intel icc/icpc/ifort version 15.0.3
Mac OS X Mavericks 10.9.5 Apple LLVM version 6.0 (clang-600.0.57)
64-bit gfortran GNU Fortran (GCC) 4.9.2
(wren/quail) Intel icc/icpc/ifort version 15.0.3
Mac OS X Yosemite 10.10.5 Apple LLVM version 6.1 (clang-602.0.53)
64-bit gfortran GNU Fortran (GCC) 4.9.2
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
Mac OS X El Capitan 10.11.4 Apple LLVM version 7.3.0 (clang-703.0.29)
64-bit gfortran GNU Fortran (GCC) 5.2.0
(VM osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2
Tested Configuration Features Summary
=====================================
In the tables below
y = tested
n = not tested in this release
C = Cluster
W = Workstation
x = not working in this release
dna = does not apply
( ) = footnote appears below second table
<blank> = testing incomplete on this feature or platform
Platform C F90/ F90 C++ zlib SZIP
parallel F2003 parallel
SunOS 5.11 32-bit n y/y n y y y
SunOS 5.11 64-bit n y/y n y y y
Windows 7 y y/y n y y y
Windows 7 x64 y y/y n y y y
Windows 7 Cygwin n y/y n y y n
Windows 10 n y/y n y y y
Windows 10 x64 n y/y n y y y
Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y
Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y
Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y
AIX 6.1 32- and 64-bit n y/n n y y y
CentOS 6.7 Linux 2.6.32 x86_64 GNU y y/y y y y y
CentOS 6.7 Linux 2.6.32 x86_64 Intel n y/y n y y y
CentOS 6.7 Linux 2.6.32 x86_64 PGI n y/y n y y y
CentOS 7.1 Linux 3.10.0 x86_64 GNU y y/y y y y y
CentOS 7.1 Linux 3.10.0 x86_64 Intel n y/y n y y y
Linux 2.6.32-431.11.2.el6.ppc64 n y/n n y y y
Platform Shared Shared Shared Thread-
C libs F90 libs C++ libs safe
SunOS 5.11 32-bit y y y y
SunOS 5.11 64-bit y y y y
Windows 7 y y y y
Windows 7 x64 y y y y
Windows 7 Cygwin n n n y
Windows 10 y y y y
Windows 10 x64 y y y y
Mac OS X Mountain Lion 10.8.5 64-bit y n y y
Mac OS X Mavericks 10.9.5 64-bit y n y y
Mac OS X Yosemite 10.10.5 64-bit y n y y
AIX 6.1 32- and 64-bit y n n y
CentOS 6.7 Linux 2.6.32 x86_64 GNU y y y y
CentOS 6.7 Linux 2.6.32 x86_64 Intel y y y y
CentOS 6.7 Linux 2.6.32 x86_64 PGI y y y y
CentOS 7.1 Linux 3.10.0 x86_64 GNU y y y y
CentOS 7.1 Linux 3.10.0 x86_64 Intel y y y y
Linux 2.6.32-431.11.2.el6.ppc64 y y y y
Compiler versions for each platform are listed in the preceding
"Supported Platforms" table.
More Tested Platforms
=====================
The following platforms are not supported but have been tested for this release.
Linux 2.6.32-573.22.1.el6 g95 (GCC 4.0.3 (g95 0.94!)
#1 SMP x86_64 GNU/Linux
(platypus)
Debian8.4.0 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1 x86_64 GNU/Linux
gcc (Debian 4.9.2-10) 4.9.2
GNU Fortran (Debian 4.9.2-10) 4.9.2
(cmake and autotools)
Fedora24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
gcc (GCC) 6.1.1 20160621 (Red Hat 6.1.1-3)
GNU Fortran (GCC) 6.1.1 20160621 (Red Hat 6.1.1-3)
(cmake and autotools)
CentOS 7.2 3.10.0-327.28.2.el7.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4)
GNU Fortran (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4)
(cmake and autotools)
Ubuntu 16.04 4.4.0-38-generic #62-Ubuntu SMP x86_64 GNU/Linux
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.2) 5.4.0
GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2) 5.4.0
(cmake and autotools)
Known Problems
==============
* On windows platforms in debug configurations, the VFD flush1 tests will fail
with the split and multi VFD drivers. These tests will display a modal debug
dialog which must be answered or wait for the test timeout to expire.
(ADB - 2014/06/23 - HDFFV-8851)
* CLANG compiler with the options -fcatch-undefined-behavior and -ftrapv
catches some undefined behavior in the alignment algorithm of the macro DETECT_I
in H5detect.c (Issue 8147). Since the algorithm is trying to detect the alignment
of integers, ideally the flag -fcatch-undefined-behavior shouldn't to be used for
H5detect.c. In the future, we can separate flags for H5detect.c from the rest of
the library. (SLU - 2013/10/16)
* Make provided by Solaris fails in "make check". Solaris users should use
gmake to build and install the HDF5 software. (AKC - 2013/10/08 - HDFFV-8534)
* The C++ and FORTRAN bindings are not currently working on FreeBSD with the
native release 8.2 compilers (4.2.1), but are working with gcc 4.6 from the
ports (and probably gcc releases after that).
(QAK - 2012/10/19)
* The following h5dump test case fails in BG/P machines (and potentially other
machines that use a command script to launch executables):
h5dump --no-compact-subset -d "AHFINDERDIRECT::ah_centroid_t[0] it=0 tl=0"
tno-subset.h5
This is due to the embedded spaces in the dataset name being interpreted
by the command script launcher as meta-characters, thus passing three
arguments to h5dump's -d flag. The command passes if run by hand, just
not via the test script.
(AKC - 2012/05/03)
* The STDIO VFD does not work on some architectures, possibly due to 32/64
bit or large file issues. The basic STDIO VFD test is known to fail on
64-bit SunOS 5.10 on SPARC when built with -m64 and 32-bit OS X/Darwin
10.7.0. The STDIO VFD test has been disabled while we investigate and
a fix should appear in a future release.
(DER - 2011/10/14 - HDFFV-8235)
* h5diff can report inconsistent results when comparing datasets of enum type
that contain invalid values. This is due to how enum types are handled in
the library and will be addressed in a future release.
(DER - 2011/10/14 - HDFFV-7527)
* The links test can fail under the stdio VFD due to some issues with external
links. This will be investigated and fixed in a future release.
(DER - 2011/10/14 - HDFFV-7768)
* After the shared library support was fixed for some bugs, it was discovered
that "make prefix=XXX install" no longer works for shared libraries. It
still works correctly for static libraries. Therefore, if you want to
install the HDF5 shared libraries in a location such as /usr/local/hdf5,
you need to specify the location via the --prefix option during configure
time. E.g, ./configure --prefix=/usr/local/hdf5 ...
(AKC - 2011/05/07 - HDFFV-7583)
* The parallel test, t_shapesame, in testpar/, may run for a long time and may
be terminated by the alarm signal. If that happens, one can increase the
alarm seconds (default is 1200 seconds = 20 minutes) by setting the
environment variable, $HDF5_ALARM_SECONDS, to a larger value such as 3600
(60 minutes). Note that the t_shapesame test may fail in some systems
(see the "While working on the 1.8.6 release..." problem below). If
it does, it will waste more time if $HDF5_ALARM_SECONDS is set
to a larger value.
(AKC - 2011/05/07)
* Shared Fortran libraries are not quite working on AIX. While they are
generated when --enable-shared is specified, the fortran and hl/fortran
tests fail. the issue. HL and C++ shared libraries should now be
working as intended, however.
(MAM - 2011/04/20)
* While working on the 1.8.6 release of HDF5, a bug was discovered that can
occur when reading from a dataset in parallel shortly after it has been
written to collectively. The issue was exposed by a new test in the parallel
HDF5 test suite, but had existed before that. We believe the problem lies with
certain MPI implementations and/or file systems.
We have provided a pure MPI test program, as well as a standalone HDF5
program, that can be used to determine if this is an issue on your system.
They should be run across multiple nodes with a varying number of processes.
These programs can be found at:
http://www.hdfgroup.org/ftp/HDF5/examples/known_problems/
(NAF - 2011/01/19)
* All the VFL drivers aren't backward compatible. In H5FDpublic.h, the
structure H5FD_class_t changed in 1.8. There is new parameter added to
get_eoa and set_eoa callback functions. A new callback function
get_type_map was added in. The public function H5FDrealloc was taken
out in 1.8. The problem only happens when users define their own driver
for 1.6 and try to plug in 1.8 library. Because there's only one user
complaining about it, we (Elena, Quincey, and I) decided to leave it as
it is (see bug report #1279). Quincey will make a plan for 1.10.
(SLU - 2010/02/02)
* The --enable-static-exec configure flag will only statically link libraries
if the static version of that library is present. If only the shared version
of a library exists (i.e., most system libraries on Solaris, AIX, and Mac,
for example, only have shared versions), the flag should still result in a
successful compilation, but note that the installed executables will not be
fully static. Thus, the only guarantee on these systems is that the
executable is statically linked with just the HDF5 library.
(MAM - 2009/11/04)
* A dataset created or rewritten with a v1.6.3 library or after cannot be read
with the v1.6.2 library or before when the Fletcher32 EDC filter is enabled.
There was a bug in the calculation of the Fletcher32 checksum in the
library before v1.6.3; the checksum value was not consistent between big-
endian and little-endian systems. This bug was fixed in Release 1.6.3.
However, after fixing the bug, the checksum value was no longer the same as
before on little-endian system. Library releases after 1.6.4 can still read
datasets created or rewritten with an HDF5 library of v1.6.2 or before.
(SLU - 2005/06/30)
New Features
============
Configuration
-------------
- Cmakehdf5: Added Ability to Run Multiple Make Commands
Added option --njobs to specify up to how many jobs to launch during
build (cmake) and testing (ctest).
(AKC - 2015/12/13, HDFFV-9612)
- Cmakehdf5: Added Szip Support and Verbose Option
Added --with-szlib to support the Szip library; and
--enable/disable-verbose to display all CMake process output.
(AKC - 2015/11/16, HDFFV-8932 and DAILYTEST-195)
- CMake minimum is now 3.1.0. (ADB - 2015/11/14)
- Large File System (LFS) Support has Changed in the Autotools
We assume that fseeko and ftello exist.
The *64 I/O functions and types are no longer explicitly used.
We now rely on a mapping provided by _FILE_OFFSET_BITS (or its
equivalent).
_LARGEFILE(64)_SOURCE is no longer exposed via AM_CPPFLAGS.
(DER - 2016/03/29, HDFFV-9626 and HDFFV-9541)
Library
-------
- New API Calls for Searching for External Dataset Storage
API calls that determine the search path for dataset external
storage were added. H5Pset/get_efile_prefix() API calls were added
to the library. These functions give control over the search path
for dataset external storage that has been configured with
H5Pset_external().
Additionally, the HDF5_EXTFILE_PREFIX environment variable can be
used to control the search path.
(DER - 2016/04/20, HDFFV-8740)
High-Level APIs
---------------
C Packet Table API
------------------
- Replacement of a Public Function with H5PTcreate
The existing function H5PTcreate_fl limits applications so they
can use the deflate compression only. The public function
H5PTcreate has been added to replace H5PTcreate_fl. H5PTcreate
takes a property list identifier to provide flexibility on
creation properties. This also removes the following warning:
"deprecated conversion from string constant to "char*"
[-Wwrite-strings]".
(BMR - 2016/04/25, HDFFV-9708, HDFFV-8615)
- New Public Functions: H5PTget_dataset and H5PTget_type
Two accessor functions have been added. H5PTget_dataset returns
the identifier of the dataset associated with the packet table,
and H5PTget_type returns the identifier of the datatype used by
the packet table.
(BMR - 2016/04/25, HDFFV-8623 patch 3)
- Regarding #ifdef VLPT_REMOVED
The #ifdef VLPT_REMOVED blocks have been removed from the packet
table (PT) library source except for the following functions:
+ H5PTis_varlen() has been made available again
+ H5PTfree_vlen_readbuff() is now H5PTfree_vlen_buff()
(BMR - 2016/04/25, HDFFV-442)
C++ Packet Table API
--------------------
- New Constructor in FL_PacketTable
An overloaded constructor has been added to FL_PacketTable and
takes a property list identifier to provide flexibility on
creation properties such as compression.
FL_PacketTable(hid_t fileID, const char* name, hid_t dtypeID,
hsize_t chunkSize = 0, hid_t plistID = H5P_DEFAULT)
(BMR - 2016/04/25, HDFFV-8623 patch 5)
- New Member Functions in PacketTable
Two accessor wrappers were added to class PacketTable.
PacketTable::GetDataset() returns the identifier of the dataset
associated with the packet table, and PacketTable::GetDatatype()
returns the identifier of the datatype that the packet table uses.
(BMR - 2016/04/25, HDFFV-8623 patch 4)
- New Member Functions with "char*" as an Argument
Overloaded functions were added to provide the "const char*"
argument; the existing version will be deprecated in future
releases. This also removes the following warning:
"deprecated conversion from string constant to "char*"
[-Wwrite-strings]".
(BMR - 2016/04/25, HDFFV-8623 patch 1, HDFFV-8615)
- Regarding #ifdef VLPT_REMOVED
The #ifdef VLPT_REMOVED blocks have been removed from the packet
table library source code except for the following functions:
+ VL_PacketTable::IsVariableLength() was moved to PacketTable
+ VL_PacketTable::FreeReadBuff() is now PacketTable::FreeBuff()
(BMR - 2016/04/25, HDFFV-442)
C++ API
-------
- New Member Function in DSetCreatPropList
DSetCreatPropList::setNbit() was added to setup N-bit compression for
a dataset.
(BMR - 2016/04/25, HDFFV-8623 patch 7)
- New Overloaded "const" Member Functions in ArrayType
The two following functions were added:
ArrayType::getArrayNDims() const
ArrayType::getArrayDims() const
to provide const version, and the non-const version was marked
deprecated. In-memory array information, ArrayType::rank and
ArrayType::dimensions, were removed. This is an implementation
detail and should not affect applications.
(BMR, 2016/04/25, HDFFV-9725)
- New member function added
The assignment operator ArrayType::operator= is added because ArrayType
has pointer data members.
(BMR, 2016/03/07, HDFFV-9562)
Support for New Platforms, Languages, and Compilers
===================================================
- Mac OS X El Capitan 10.11.4 with compilers Apple clang/clang++
version 7.3.0 from Xcode 7.3, gfortran GNU Fortran (GCC) 5.2.0
and Intel icc/icpc/ifort version 16.0.2
Bug Fixes since HDF5-1.8.16
===========================
Configuration
-------------
- Updated Linux Language Level Flags to Match the Autotools. Removed
Linux-specific Flags from OS X.
An addition to the flags simply being out of sync with the Autotools,
the Linux flags were used on OS X builds which led to symbols not being
found. Although this was non-fatal and compilation continued (implicit
definitions were used by the compiler and the symbols resolved at link
time), a large number of warnings were raised.
Linux changes:
* CHANGED: _POSIX_C_SOURCE (from 199605 to 200112L)
* ADDED: _GNU_SOURCE
* REMOVED: _BSD_SOURCE
* REMOVED: _DEFAULT_SOURCE
(DER - 2015/12/08, HDFFV-9627)
- The --enable-clear-file-buffers configure Option was Non-functional
so the Feature was Always Enabled (its default value).
Regardless of the configure flag, the setting was always enabled when
the Autotools were used to configure HDF5. This was due to the "no"
option being processed after the "*" option in configure.ac so "*"
matched first. CMake was unaffected.
The option now works correctly.
NOTE that builders are always advised to leave this option enabled.
When disabled, buffers that are written to disk may contain the
memory's previous contents, which may include secure information.
The performance overhead of the feature (a single memset call per
allocation) is minimal.
(DER - 2016/02/03, HDFFV-9676)
- Added a patch to remove '"'s from arguments for MPI compilers that
were causing errors compiling H5lib_settings.c with SGI MPT.
(LRK - 2016/04/20, HDFFV-9439)
Library
-------
- Fixed shared file pointer problem which caused a crash when running a
program provided by a user.
(VC - 2016/04/01, HDFFV-9469)
- Fixed some format string warnings that prevent compiling with
-Werror=format-security on gcc.
These only appeared in error messages and would not cause problems
under normal operation.
(DER - 2016/01/13, HDFFV-9640)
- Fixed a library segmentation fault when accessing a corrupted
file provided by a user.
(MSC - 2016/02/19, HDFFV-9670)
Tools
-----
- h5dump: Sub-setting Fixed for Dimensions Greater than Two
When a dataset has more than two dimensions, sub-setting would
incorrectly calculate the data that needed to be displayed.
Added in block and stride calculations that account for dimensions
greater than two. NOTE: lines that have line breaks inserted
because of display length calculations may have index info that
is incorrect until the next dimension break.
(ADB - 2016/03/07, HDFFV-9698)
- h5dump: Issue with Argument Segmentation Fault
When an argument with an optional value was at the end of the command
line with a value, h5dump would crash. Reworked check for remaining
arguments.
(ADB - 2016/03/07, HDFFV-9570, HDFFV-9684)
- h5dump: Issue with Default Fill Value
Added all default cases of fill value to the display of fill value.
(ADB -, 2016/03/07, HDFFV-9241)
- h5dump: Clarified Help
Clarified usage of -O F option in h5dump utility help.
(ADB - 2016/03/07, HDFFV-9066)
- h5dump: Issue with Double Free Fault
Added a check for filename not null before calling free().
(ADB - 2016/01/27, HDFFV-9639)
- VS2015 Release Changed how Timezone was Handled
Created a function, HDget_timezone, in H5system.c. Replaced
timezone variable usage with function call.
(ADB - 2015/11/02, HDFFV-9550)
C++ API
-------
- Removal of Obsolete Methods
The overloaded methods which had parameters that should be const
but were not have been removed.
(BMR - 2016/01/13, HDFFV-9789)
High-Level APIs:
---------------
- Fixed Memory Leak in Packet Table API
Applied user's patch to fix memory leak in the creation of a
packet table.
(BMR - 2016/04/25, HDFFV-9700)
Known Problems
==============
* On windows platforms in debug configurations, the VFD flush1 tests will fail
with the split and multi VFD drivers. These tests will display a modal debug
dialog which must be answered or wait for the test timeout to expire.
(ADB - 2014/06/23 - HDFFV-8851)
* CLANG compiler with the options -fcatch-undefined-behavior and -ftrapv
catches some undefined behavior in the alignment algorithm of the macro DETECT_I
in H5detect.c (Issue 8147). Since the algorithm is trying to detect the alignment
of integers, ideally the flag -fcatch-undefined-behavior shouldn't to be used for
H5detect.c. In the future, we can separate flags for H5detect.c from the rest of
the library. (SLU - 2013/10/16)
* Make provided by Solaris fails in "make check". Solaris users should use
gmake to build and install the HDF5 software. (AKC - 2013/10/08 - HDFFV-8534)
* The C++ and FORTRAN bindings are not currently working on FreeBSD with the
native release 8.2 compilers (4.2.1), but are working with gcc 4.6 from the
ports (and probably gcc releases after that).
(QAK - 2012/10/19)
* The following h5dump test case fails in BG/P machines (and potentially other
machines that use a command script to launch executables):
h5dump --no-compact-subset -d "AHFINDERDIRECT::ah_centroid_t[0] it=0 tl=0"
tno-subset.h5
This is due to the embedded spaces in the dataset name being interpreted
by the command script launcher as meta-characters, thus passing three
arguments to h5dump's -d flag. The command passes if run by hand, just
not via the test script.
(AKC - 2012/05/03)
* The STDIO VFD does not work on some architectures, possibly due to 32/64
bit or large file issues. The basic STDIO VFD test is known to fail on
64-bit SunOS 5.10 on SPARC when built with -m64 and 32-bit OS X/Darwin
10.7.0. The STDIO VFD test has been disabled while we investigate and
a fix should appear in a future release.
(DER - 2011/10/14 - HDFFV-8235)
* h5diff can report inconsistent results when comparing datasets of enum type
that contain invalid values. This is due to how enum types are handled in
the library and will be addressed in a future release.
(DER - 2011/10/14 - HDFFV-7527)
* The links test can fail under the stdio VFD due to some issues with external
links. This will be investigated and fixed in a future release.
(DER - 2011/10/14 - HDFFV-7768)
* After the shared library support was fixed for some bugs, it was discovered
that "make prefix=XXX install" no longer works for shared libraries. It
still works correctly for static libraries. Therefore, if you want to
install the HDF5 shared libraries in a location such as /usr/local/hdf5,
you need to specify the location via the --prefix option during configure
time. E.g, ./configure --prefix=/usr/local/hdf5 ...
(AKC - 2011/05/07 - HDFFV-7583)
* The parallel test, t_shapesame, in testpar/, may run for a long time and may
be terminated by the alarm signal. If that happens, one can increase the
alarm seconds (default is 1200 seconds = 20 minutes) by setting the
environment variable, $HDF5_ALARM_SECONDS, to a larger value such as 3600
(60 minutes). Note that the t_shapesame test may fail in some systems
(see the "While working on the 1.8.6 release..." problem below). If
it does, it will waste more time if $HDF5_ALARM_SECONDS is set
to a larger value.
(AKC - 2011/05/07)
* Shared Fortran libraries are not quite working on AIX. While they are
generated when --enable-shared is specified, the fortran and hl/fortran
tests fail. the issue. HL and C++ shared libraries should now be
working as intended, however.
(MAM - 2011/04/20)
* While working on the 1.8.6 release of HDF5, a bug was discovered that can
occur when reading from a dataset in parallel shortly after it has been
written to collectively. The issue was exposed by a new test in the parallel
HDF5 test suite, but had existed before that. We believe the problem lies with
certain MPI implementations and/or file systems.
We have provided a pure MPI test program, as well as a standalone HDF5
program, that can be used to determine if this is an issue on your system.
They should be run across multiple nodes with a varying number of processes.
These programs can be found at:
http://www.hdfgroup.org/ftp/HDF5/examples/known_problems/
(NAF - 2011/01/19)
* All the VFL drivers aren't backward compatible. In H5FDpublic.h, the
structure H5FD_class_t changed in 1.8. There is new parameter added to
get_eoa and set_eoa callback functions. A new callback function
get_type_map was added in. The public function H5FDrealloc was taken
out in 1.8. The problem only happens when users define their own driver
for 1.6 and try to plug in 1.8 library. Because there's only one user
complaining about it, we (Elena, Quincey, and I) decided to leave it as
it is (see bug report #1279). Quincey will make a plan for 1.10.
(SLU - 2010/02/02)
* The --enable-static-exec configure flag will only statically link libraries
if the static version of that library is present. If only the shared version
of a library exists (i.e., most system libraries on Solaris, AIX, and Mac,
for example, only have shared versions), the flag should still result in a
successful compilation, but note that the installed executables will not be
fully static. Thus, the only guarantee on these systems is that the
executable is statically linked with just the HDF5 library.
(MAM - 2009/11/04)
* A dataset created or rewritten with a v1.6.3 library or after cannot be read
with the v1.6.2 library or before when the Fletcher32 EDC filter is enabled.
There was a bug in the calculation of the Fletcher32 checksum in the
library before v1.6.3; the checksum value was not consistent between big-
endian and little-endian systems. This bug was fixed in Release 1.6.3.
However, after fixing the bug, the checksum value was no longer the same as
before on little-endian system. Library releases after 1.6.4 can still read
datasets created or rewritten with an HDF5 library of v1.6.2 or before.
(SLU - 2005/06/30)
* Improvements made to the CMake build system.
* Added Memory Allocation Functions that Use the Library's Allocator
* Added Functions to Control the Value of H5PL_no_plugin_g without
Using an Environment Variable
* MPI_Finalize and HDF5 Library Shutdown
* New Wrappers for C Functions H5P[s/g]et_libver_bounds
* more...
Issues found with existing distfiles:
distfiles/eclipse-sourceBuild-srcIncluded-3.0.1.zip
distfiles/fortran-utils-1.1.tar.gz
distfiles/ivykis-0.39.tar.gz
distfiles/enum-1.11.tar.gz
distfiles/pvs-3.2-libraries.tgz
distfiles/pvs-3.2-linux.tgz
distfiles/pvs-3.2-solaris.tgz
distfiles/pvs-3.2-system.tgz
No changes made to these distinfo files.
Otherwise, existing SHA1 digests verified and found to be the same on
the machine holding the existing distfiles (morden). All existing
SHA1 digests retained for now as an audit trail.
Upstream changes:
Release 1.8.14 of November 2014 (current release) versus Release 1.8.13
In the main library C Interface
No new C functions have been added.
The behavior of the following C function has been changed:
The function H5Pset_core_write_tracking has been modified to enforce the limitation that the value of the page_size parameter cannot equal 0 (zero):
H5Pset_core_write_tracking
The following performance optimization has been implemented in the Parallel HDF5 library:
The algoritm for writing chunk fill values for new datasets has been optimized to write out all chunk fill values collectively in a single MPI-IO call. This was previously done independently by rank 0 (zero) one block at a time. This optimization should produce a significant performance improvement when creating chunked datasets in parallel, most notably when chunk dimensions are relatively small. This optimization will be evident in use of the following function:
H5Dcreate
In the high-level library C Interface
High-level HDF5 interfaces now ensure that strings are null-terminated and will not overflow a buffer.
In several high-level functions, the type from the file was previously re-used as the memory type and no check is made to ensure that a string was actually null-terminated.
All high-level functions now check for NULL pointers; verify that a received a string is null-terminated; and verify that the string will not overflow the buffer. The minimum length of the buffers is now used in strncmp to avoid overflow.
In the main library C++ Interface
No new C++ methods have been added.
The following changes have been made in the C++ interface:
Changed defaults
Default value of class member id:
The data member id in several classes that represent HDF5 objects are now initialized to H5I_INVALID_ID.
Default value of a PropList member id:
The data member id in class PropList is now initialized to H5P_DEFAULT.
In HDF5 Release 1.8.13 and earlier, the above were initialized to 0 (zero).
Changed file access flags
The flag H5F_ACC_CREAT has been removed from the C++ interface.
In the main library Fortran Interface
No new Fortran subroutines have been added.
Changed Fortran INTEGER parameter types:
Types of the following Fortran parameter constants have been changed from INTEGER to INTEGER(HID_T):
H5FD_CORE H5FD_MULTI
H5FD_FAMILY H5FD_SEC2
H5FD_LOG H5FD_STDIO
H5FD_MPIO
Other internal int types have been changed to hid_t, but this change is expected to be transparent to the user
Release 1.8.13 of May 2014 versus Release 1.8.12
In the C Interface (main library)
The following new C functions have been added:
To free memory allocated by HDF5 function calls:
H5free_memory
This function should be used to free memory that has been allocated by HDF5 API calls such as H5Tget_member_name and H5Pget_class_name.
The primary value for this call is on Windows systems, where it is common for application code and the HDF5 Library to be using different C run-time libraries (CRT). Using this new call ensures that the same CRT handles both the allocation and free.
This function can also be useful in any case where the library uses a different memory manager than the application, such as when a debug memory manager is in use or when the HDF5 Library is wrapped for use in a managed language like Python or Java.
New configuration property for the CORE driver:
H5Pset_core_write_tracking
H5Pget_core_write_tracking
With this property, the CORE driver, H5Pset_fapl_core, can be configured to track dirty regions in the file and write out only the changed regions on flush or close operations. Additionally, a page aggregation size can be set that will aggregate small writes into larger writes. For example, setting a 1 mebibyte (1 MiB) page aggregation size will logically partition the the in-memory file into 1 MiB pages that will be written out in their entirety if even a single byte is dirtied.
The following C functions have been modified:
H5Rget_name
If the name parameter is passed as NULL, then the size parameter is ignored and the function returns the size of the buffer required to read the name (not including the NULL terminator).
H5Zfilter_avail
This function has been updated to correctly identify dynamically loaded filters. If a filter is not found among registered filters, the function now checks for dynamically loaded filters.
In the C++ Interface
The following new features have been added:
Wrappers to the class H5Object to get an object’s name:
ssize_t getObjName(char *obj_name, size_t buf_size = 0)
ssize_t getObjName(H5std_string& obj_name, size_t len = 0)
H5std_string getObjName()
Wrappers to the class H5CommonFG to get a child object’s type from a group or file:
H5O_type_t childObjType(const H5std_string& objname)
H5O_type_t childObjType(const char* objname)
H5O_type_t childObjType(hsize_t index,
H5_index_t index_type=H5_INDEX_NAME,
H5_iter_order_t order=H5_ITER_INC, const char* objname=".")
Wrappers to the class DSetMemXferPropList for setting or getting a transform property list:
DSetMemXferPropList(const char* expression);
void setDataTransform(const char* expression)
void setDataTransform(const H5std_string& expression)
ssize_t getDataTransform(char* exp, size_t buf_size=0)
H5std_string getDataTransform()
Wrapper to CompType for setting the size of a compound datatype:
void setSize(size_t size)
Overloaded functions to provide prototypes that declare constant arguments const:
DataSet::fillMemBuf
DataSet::getVlenBufSize
DataSpace::extentCopy
DataType::commit
FileAccPropList::setSplit
H5File::getVFDHandle
Additional overload to the class H5Location to get a comment as a char*:
ssize_t getComment(const char* name, size_t buf_size, char* comment)
Additional overloads to the class Attribute to get an attribute’s name for convenience:
ssize_t getName(char* attr_name, size_t buf_size=0)
ssize_t getName(H5std_string& attr_name, size_t buf_size=0)
In the Fortran Interface
The following new Fortran subroutines have been added:
Fortran2003 subroutines:
h5pset_file_image_f
h5pget_file_image_f
h5fget_file_image_f (Added in Release 1.8.11, but not documented at that time.)
The following Fortran subroutine has been changed:
h5pset_chunk_cache_f
The values of the constants H5D_CHUNK_CACHE_NSLOTS_DFLT_F and H5D_CHUNK_CACHE_NBYTES_DFLT_F have been changed to INTEGER(KIND=size_t).
Other New Features
Parallel I/O support:
Parallel I/O support has been enhanced to allow collective I/O on point selections.
Thread safety (Windows only)
The thread-safe HDF5 library no longer leaks thread-local storage resources on Windows with Win32 threads. This cleanup can only be performed when the library is built as a shared library, so a thread-safe HDF5 static library with Win32 threads is no longer supported.
This change affects only Windows environments.
Also see “Build and Configuration” below.
Command-line Tools with Changed Behavior
h5repack
h5repack has been updated to correctly handle dynamically loaded filters.
Build and Configuration
Thread safety
When configuring a thread-safe HDF5 Library with --enable-threadsafe, it is no longer necessary to specify --with-pthreads if the Pthreads library is in a standard location.
Removed Functions and Wrappers
The MPI-POSIX driver has been removed. The following C functions and the corresponding Fortran subroutines and C++ wrappers therefore are no longer included in the HDF5 distribution:
H5Pset_fapl_mpiposix
H5Pget_fapl_mpiposix
Applications performing parallel I/O should use the MPI-IO driver, H5Pset_fapl_mpio.
Configuration
-------------
- Added a configuration option to change the default plugin path.
The configure option is --with-default-plugindir=location.
The cmake option is -DH5_DEFAULT_PLUGINDIR:PATH=location.
HDFFV-8513. (ADB 2013/09/04)
- Renamed FFLAGS to FCFLAGS in configure. (ADB 2013/08/13)
- CMake can now package a compressed examples file, the default for
Windows binaries from HDF Group. (ADB - 2013/07/22)
Tools
-----
- h5repack: Added the ability to use plugin filters to read and write
files. The option uses the filter number. HDFFV-8345
(ADB - 2013/09/04).
- h5dump: Added the option -N --any_path, which searches the file for
paths that match the search path. HDFFV-7989 (ADB - 2013/08/12).
- h5dump: Added the optional arg 0 to -A, which excludes attributes
from display. HDFFV-8134 (ADB - 2013/08/01).
C++ API
-------
- Added tutorial examples to C++/examples. They can be installed by
"make install-examples" and, in the installed directory, they can be
executed by running the script file run-c++-ex.sh. (BMR - 2013/09/28)
- A new class, H5::H5Location, is added to represent the location concept
in the C library. It is a base class to H5::H5File and H5::H5Ojbect,
whose member functions are moved into H5::H5Location. H5::H5File can
now inherent those functions. As a result, an H5::H5File object can have
an attribute. (BMR - 2013/09/27)
- Added wrappers for H5Rget_obj_type2 to retrieve the type of the object
that an object reference points to. (BMR - 2013/09/27)
H5O_type_t H5Location::getRefObjType(void *ref, H5R_type_t ref_type)
- Added wrappers for H5Aexist to check whether an attribute exists given
a name. (BMR - 2013/09/27)
bool H5::H5Location::attrExists(const char* name)
bool H5::H5Location::attrExists(const H5std_string& name)
- Added a number of overloaded functions for convenience. (BMR - 2013/09/27)
Configuration
- Libtool version number is changed to 8.0.0 because there are API
changes. See below for details.
- Mac OS X 10.7 (Lion) and 10.8 (Mountain Lion) uses clang/clang++ as the
default C and C++ compilers.
- CMake minimum is now 2.8.10.
- A new tool, cmakehdf5, which is a build command script similar to
buildhdf5 is added and is available in the bin directory.
Library
- The library can load filter libraries dynamically during runtime. Users
can set the search path through environment variable HDF5_PLUGIN_PATH
and call H5Pset_filter to enable a dynamic filter.
- Added new API functions H5Dscatter and H5Dgather to scatter data to and
and gather data from a selection within a memory buffer.
- The library now supports the data conversion from enumeration to numeric
(integer and floating-point number) datatypes. See Issue HDFFV-8221.
Tools
- h5dump: added new option -O or -ddl to output the ddl text to a file. This
is a complement to the -o or --output option, which redirects the data to
a file.
Library
-------
- The library now behaves correctly when performing large I/O operations on
Mac OS-X. Previously, single I/O operations > 2 GB would fail since the
Darwin read/write calls cannot handle the number of bytes that their
parameter types imply.
Fixes HDFFV-7975 and HDFFV-8240 (DER - 07 JAN 2013)
- Fixed a bug in the core VFD that cause failures when opening files > 2 GB.
Fixes HDFFV-8124 and HDFFV-8158 (DER - 07 JAN 2013)
Tools
-----
- The following h5stat test case failed in BG/P machines (and potentially
other machines that display extra output if an MPI task returns with a
non-zero code.)
Testing h5stat notexist.h5
The test script was fixed to ignore the extra output.
HDFFV-8233 (AKC - 2012/12/17)
- h5diff: Fixed slowness when comparing HDF5 files with many attributes.
Much slower performance was identified with later release version
(from 1.8.7 to 1.8.10) compared to 1.8.6. The issue was introduced
from fixing an attribute related bug for 1.8.7 release in the past.
HDFFV-8145 (JKM 2012/12/13)
* The following new C function has been added:
H5Pget_mpio_no_collective_cause
* The following new Fortran subroutine has been added:
h5pget_mpio_actual_io_mode_f
* The syntax of the following high-level C function has changed:
H5TBAget_fill
This function’s return type has been changed to h5tri_t. A return value of 1 indicates that a fill value is present, 0 indicates that no fill value is present, and a negative value indicates an error. (The former return type was herr_t.)
Note that H5TBAget_fill and a companion function H5TBAget_title, both in the HDF5 Table (H5TB) high-level API, are undocumented. These functions will be added to the HDF5 Reference Manual shortly after HDF5 Release 1.8.10.
Changed Command-line Tool
* h5import has been updated to accept an input file created by h5dump.
This doesn't pass self-tests, but it isn't regression,
previous package didn't pass them as well.
(While here, make both packages share the same distinfo.)
Changes in HDF5 1.8.9
HDF5 1.8.9 is a minor release, but contains several important
new features and bug fixes. Changes in this release include the
following:
* Committed datatypes can now be merged when copying objects,
using the new flag H5O_COPY_MERGE_COMMITTED_DTYPE_FLAG.
Several new API routines have been included in support of
this feature.
* A file in memory can now be accessed in the same way as a
file on disk. New APIs were added to support this feature.
* Many changes to the tools have been made, including the following:
+ The h5repack performance has been improved dramatically
for large chunked datasets.
+ The -S (or --summary) option was added to h5stat to
display a summary of the file space usage for a file.
+ A wildcard can now be used with h5dump to specify multiple files.
* H5LTpath_valid was added to determine if a path is correct
and a link resolves to a valid object.
* The Fortran counterpart to H5Ocopy, h5ocopy_f, was added.
This release contains many other changes and bug fixes not listed here.
Please be sure to read the Release Notes for a comprehensive
list of new features and bug fixes.
Changes in HDF5 1.8.8
HDF5 1.8.8 is a minor release, but contains a few important new
features and bug fixes. Changes in this release include the
following:
* Changes to the Fortran interface:
+ Fortran wrappers for the Dimension Scales APIs were added.
For the new subroutine signatures, see:
http://www.hdfgroup.org/HDF5/doc/HL/RM_H5DS.html
+ The Fortran interface now uses the Fortran 2003 standard,
enabling a wider set of Fortran and HDF5 datatypes to be
supported, including:
o Any kind of INTEGER or REAL
o Fortran derived types
o Fortran and HDF5 enumeration
o HDF5 variable-length datatypes
o HDF5 compound datatypes of any complexity
It also contains new subroutines corresponding to those
C APIs which have callback functions as parameters.
For a general overview and information on how to enable
these new Fortran features, please see the following paper:
http://www.hdfgroup.org/HDF5/doc/fortran/NewFeatures_F2003.pdf
HDF5 examples that use Fortran 2003 (with the suffix "F03") can be found here:
http://www.hdfgroup.org/ftp/HDF5/examples/examples-by-api/api18-fortran.html
The --enable-fortran2003 configure flag was added to
enable Fortran 2003 support in the HDF5 Fortran library.
This flag should be used along with the --enable-fortran
flag and takes affect only when a Fortran compiler is
Fortran 2003 compliant.
* H5Tcreate now supports a string type (fixed-length and variable-length).
* New APIs, H5Pget_mpio_actual_chunk_opt_mode and H5Pget_mpio_actual_io_mode,
were added for querying whether/how a collective I/O operation completed.
* Numerous changes were made to the tools (h5diff, h5repack, h5jam, h5dump).
(yes I know there is the 1.8 branch, but I don't want to experiment
right now)
changes: many fixes, some C++ API improvements
pkgsrc change: DESTDIR support
changes:
-minor feature additions to the command line tools
-C++ API: Added support for Szip
-bugfixes
pkgsrc changes:
-fixes for the cxx option (should be a separate pkg)
-fixes for the threadsafe option -- replaced a pthread_equal() by a
simple "==" to make it usable with non-threaded apps
(not 100% portable, but should be fine for the platforms in question)
The shared libraries should not be linked against libpthread, but
that would require serious changes to the autoconf/make machinery
so I've left it alone for now.
If that is fixed, "threadsafe" could be made default.
-Don't try to strip scripts on install.
-make NetBSD ("netbsdelf") a recognized configuration, copied from
"freebsd"; this pulls in definitions for gcc, in particular it
disables some optimization
-work around a memory corruption problem in H5FDstream which appearently
is caused by use of free()d memory
The latter 2 changes make the library survive its selftests, and help
the "pytables" pkg as well.
bump PKGREVISION, raise BUILDLINK_RECOMMENDED
New Features
Configuration:
- Added some initial support for making purify (or similar memory
checking products) happier by initializing buffers to zero and
disabling the internal free list code. To take advantage of this,
define 'H5_USING_PURIFY' in your CFLAGS when building the library.
- WINDOWS building,testing and installing improvements
- On Windows, FORTRAN,C++ and C projects are merged into one zip file,
users can choose an option to build either FORTRAN or C++ or both
with basic C library.For detailed information,
please read INSTALL_Windows.txt.
- On Windows, szip compression library with or without encoder can be easily
turned off or on when building HDF5. For detailed information,
please read INSTALL_Windows.txt, especially section V.
- On Windows, an optional procedure for building,testing and installing
HDF5 from command line is provided. This procedure is supposed to be
convenient for experienced users, please read
INSTALL_windows_From_Command_Line.txt for details.
- On Windows, an alternative short instruction document for building,
testing and installing HDF5 is provided. This instruction is supposed to
be convenient for general users, please read
INSTALL_Windows_Short.txt for details.
- On Windows, h5repack,h5diff,h5ls and h5import tool tests have been added.
Library:
- Modified the way how HDF5 calculates 'pixels_per_scanline' parameter for
SZIP compression. Now there is no restriction on the size and shape of the
chunk except that the total number of elements in the chunk cannot be
bigger than 'pixels_per_block' parameter provided by the user.
- HDF5 can now link to SZIP with or without szip's encoder.
The new API function H5Zget_filter_info can be used to check
szip's status. Attempting to assign szip to a dataset property
list or attempting to write with szip will generate an error if
szip's encoder is disabled. JL/NF - 2004/6/30
- SZIP always uses K13 compression. This flag no longer needs to
be set when calling H5Pset_szip. If the flag for CHIP
compression is set, it will be ignored (since the two are mutually
exclusive). JL/NF - 2004/6/30
- A new API function H5Fget_name was added. It returns the name
of the file by object(file, group, data set, named data type,
attribute) ID. SLU - 2004/06/29
- A new API function H5Fget_filesize was added. It returns the
actual file size of the opened file. SLU - 2004/06/24
- Added option that if $HDF5_DISABLE_VERSION_CHECK is set to 2,
will suppress all library version mismatch warning messages.
Tools:
- h5repack was added to the tools suite. h5repack regenerates an HDF5 file
from another HDF5 file, optionally applying HDF5 filters (compression)
and/or chunking to the copied file. The filters options are read from
the command line. See /doc/html/Tools.html for more details.
PVN - 2004/9/13
- h5dump includes new features:
1) Printing of dataset filters, storage layout and fill value information.
2) Print a list of the file contents.
3) Escape non printing characters.
4) Print the content of the boot block.
5) Print array indices with the data (the default).
This version fixes some serious bugs in version 1.4.5 and includes the
following relevant changes:
* Fixed error in B-tree deletion routine which could cause groups to be
corrupted when objects are removed from them.
* Fixed error in file space freeing code which could cause metadata to
fail to be written to the file.
* Fixed error in library when configured with --enable-debug=all.
* Added a -force option to h5redeploy.
* When using gcc 3.x, we use -std=gnu99 instead of -ansi for compiling.
Changes:
* Allow scalar dataspaces to be used for parallel I/O. QAK - 2002/11/05
* Added environment variable "HDF5_DISABLE_VERSION_CHECK", which disables
the version checking between the header files and the library linked
into an application if set to '1'. This should be used with caution,
mis-matched headers and library binaries can cause _serious_ problems.
QAK - 2002/10/15
* Partially fixed space allocation inefficiencies in the file by
improving our algorithms for re-using freed space. QAK - 2002/08/27
* API tracing has been improved. Nested API calls don't screw up the
output format; function call and return event times can be logged;
total time spent in each function can be logged. The following
HDF5_DEBUG environment variable words affect tracing:
trace -- turn on/off basic tracing
ttimes -- turn on tracing and report event times and
time spent in each API function.
ttop -- turn on tracing but display only top-level
API calls.
* Several missing fortran APIs have been added to the library:
h5get_libversion_f h5tget_member_index_f h5dget_storage_size_f
h5check_version_f h5tvlen_create_f h5dvlen_get_max_len_f
h5garbage_collect_f h5dwrite_vl_f
h5dont_atexit_f h5dread_vl_f
Functions h5dvlen_get_max_len_f, h5dwrite_vl_f, and h5dread_vl_f support
VL Length C APIs functionality for integer, real and string datatypes.
See HDF5 Reference Manual and HDF5 FORTRAN90 User's Notes for more
information and for the functions description.
* H5Fopen without the H5F_ACC_CREAT flag should not succeed in creating
a new file with the 'core' VFL driver. QAK - 2003/01/24
* Corrected metadata caching bug in parallel I/O which could cause hangs
when chunked datasets were accessed with independent transfer mode.
QAK - 2003/01/23
* Allow opening objects with unknown object header messages.
QAK - 2003/01/21
* Added improved error assertion for nil VL strings. It return error
stack instead of a simple assertion. SLU - 2002/12/16
* Fixed h5dump bug(cannot dump data and datatype) for VL string.
SLU - 2002/11/18
* Fixed error condition where "none" selections were not being handled
correctly in serial & parallel. QAK - 2002/10/29
* Fixed problem where optimized hyperslab routines were incorrectly
invoked for parallel I/O operations in collective mode. QAK - 2002/07/22
* Fixed metadata corruption problem which could occur when many objects
are created in a file during parallel I/O. QAK - 2002/07/19
* Fixed minor problem with configuration when users specified /usr/include
and /usr/lib for the --with-* options that some compilers can't
handle. BW - 2003/01/23
* Changed internal error handling macros to reduce code size of library by
about 10%.
* API changes.
* Performance improvements
* ``h5cc'' script which helps compilation of HDF5 programs
* Bug fixes.
New Features
============
* XML output option for h5dump utility.
A new option --xml to output data in XML format has been added. The
XML output contains a complete description of the file, marked up in
XML.
The XML conforms to the HDF5 Document Type Definition (DTD), which
is available at:
http://hdf.ncsa.uiuc.edu/DTDs/HDF5-File.dtd
The XML output is suitable for use with other tools, including the
Java Tools:
http://hdf.ncsa.uiuc.edu/java-hdf5-html
Bug Fixes since HDF5-1.4.0 Release
==================================
* h4toh5 utility: conversion of images is fixed
Earlier releases of the h4toh5 utility produced images that did not
correctly conform to the HDF5 Image and Palette Specification.
http://hdf.ncsa.uiuc.edu/HDF5/doc/ImageSpec.html
Several required HDF5 attributes are omitted, and the dataspace
is reversed (i.e., the ht. and width of the image dataset is
incorrectly described.) For more information, please see:
http://hdf.ncsa.uiuc.edu/HDF5/H5Image/ImageDetails.htm
* Fixed bug with contiguous hyperslabs not being detected, causing
slower I/O than necessary.
* Fixed bug where non-aligned hyperslab I/O on chunked datasets was
causing errors during I/O
* The RCSID string in H5public.h was causing the C++ compiling problem
because when it was included multiple times, C++ did not like
multiple definitions of the same static variable. All occurance of
RCSID definition are removed since we have not used it consistently
before.
markd
wen
jperkin
minskim
alnsn
adam
agc
asau
drochner
dmcmahill
recht
jtb
wiz