New Features
============
Configuration
-------------
- Cmakehdf5: Added Ability to Run Multiple Make Commands
Added option --njobs to specify up to how many jobs to launch during
build (cmake) and testing (ctest).
(AKC - 2015/12/13, HDFFV-9612)
- Cmakehdf5: Added Szip Support and Verbose Option
Added --with-szlib to support the Szip library; and
--enable/disable-verbose to display all CMake process output.
(AKC - 2015/11/16, HDFFV-8932 and DAILYTEST-195)
- CMake minimum is now 3.1.0. (ADB - 2015/11/14)
- Large File System (LFS) Support has Changed in the Autotools
We assume that fseeko and ftello exist.
The *64 I/O functions and types are no longer explicitly used.
We now rely on a mapping provided by _FILE_OFFSET_BITS (or its
equivalent).
_LARGEFILE(64)_SOURCE is no longer exposed via AM_CPPFLAGS.
(DER - 2016/03/29, HDFFV-9626 and HDFFV-9541)
Library
-------
- New API Calls for Searching for External Dataset Storage
API calls that determine the search path for dataset external
storage were added. H5Pset/get_efile_prefix() API calls were added
to the library. These functions give control over the search path
for dataset external storage that has been configured with
H5Pset_external().
Additionally, the HDF5_EXTFILE_PREFIX environment variable can be
used to control the search path.
(DER - 2016/04/20, HDFFV-8740)
High-Level APIs
---------------
C Packet Table API
------------------
- Replacement of a Public Function with H5PTcreate
The existing function H5PTcreate_fl limits applications so they
can use the deflate compression only. The public function
H5PTcreate has been added to replace H5PTcreate_fl. H5PTcreate
takes a property list identifier to provide flexibility on
creation properties. This also removes the following warning:
"deprecated conversion from string constant to "char*"
[-Wwrite-strings]".
(BMR - 2016/04/25, HDFFV-9708, HDFFV-8615)
- New Public Functions: H5PTget_dataset and H5PTget_type
Two accessor functions have been added. H5PTget_dataset returns
the identifier of the dataset associated with the packet table,
and H5PTget_type returns the identifier of the datatype used by
the packet table.
(BMR - 2016/04/25, HDFFV-8623 patch 3)
- Regarding #ifdef VLPT_REMOVED
The #ifdef VLPT_REMOVED blocks have been removed from the packet
table (PT) library source except for the following functions:
+ H5PTis_varlen() has been made available again
+ H5PTfree_vlen_readbuff() is now H5PTfree_vlen_buff()
(BMR - 2016/04/25, HDFFV-442)
C++ Packet Table API
--------------------
- New Constructor in FL_PacketTable
An overloaded constructor has been added to FL_PacketTable and
takes a property list identifier to provide flexibility on
creation properties such as compression.
FL_PacketTable(hid_t fileID, const char* name, hid_t dtypeID,
hsize_t chunkSize = 0, hid_t plistID = H5P_DEFAULT)
(BMR - 2016/04/25, HDFFV-8623 patch 5)
- New Member Functions in PacketTable
Two accessor wrappers were added to class PacketTable.
PacketTable::GetDataset() returns the identifier of the dataset
associated with the packet table, and PacketTable::GetDatatype()
returns the identifier of the datatype that the packet table uses.
(BMR - 2016/04/25, HDFFV-8623 patch 4)
- New Member Functions with "char*" as an Argument
Overloaded functions were added to provide the "const char*"
argument; the existing version will be deprecated in future
releases. This also removes the following warning:
"deprecated conversion from string constant to "char*"
[-Wwrite-strings]".
(BMR - 2016/04/25, HDFFV-8623 patch 1, HDFFV-8615)
- Regarding #ifdef VLPT_REMOVED
The #ifdef VLPT_REMOVED blocks have been removed from the packet
table library source code except for the following functions:
+ VL_PacketTable::IsVariableLength() was moved to PacketTable
+ VL_PacketTable::FreeReadBuff() is now PacketTable::FreeBuff()
(BMR - 2016/04/25, HDFFV-442)
C++ API
-------
- New Member Function in DSetCreatPropList
DSetCreatPropList::setNbit() was added to setup N-bit compression for
a dataset.
(BMR - 2016/04/25, HDFFV-8623 patch 7)
- New Overloaded "const" Member Functions in ArrayType
The two following functions were added:
ArrayType::getArrayNDims() const
ArrayType::getArrayDims() const
to provide const version, and the non-const version was marked
deprecated. In-memory array information, ArrayType::rank and
ArrayType::dimensions, were removed. This is an implementation
detail and should not affect applications.
(BMR, 2016/04/25, HDFFV-9725)
- New member function added
The assignment operator ArrayType::operator= is added because ArrayType
has pointer data members.
(BMR, 2016/03/07, HDFFV-9562)
Support for New Platforms, Languages, and Compilers
===================================================
- Mac OS X El Capitan 10.11.4 with compilers Apple clang/clang++
version 7.3.0 from Xcode 7.3, gfortran GNU Fortran (GCC) 5.2.0
and Intel icc/icpc/ifort version 16.0.2
Bug Fixes since HDF5-1.8.16
===========================
Configuration
-------------
- Updated Linux Language Level Flags to Match the Autotools. Removed
Linux-specific Flags from OS X.
An addition to the flags simply being out of sync with the Autotools,
the Linux flags were used on OS X builds which led to symbols not being
found. Although this was non-fatal and compilation continued (implicit
definitions were used by the compiler and the symbols resolved at link
time), a large number of warnings were raised.
Linux changes:
* CHANGED: _POSIX_C_SOURCE (from 199605 to 200112L)
* ADDED: _GNU_SOURCE
* REMOVED: _BSD_SOURCE
* REMOVED: _DEFAULT_SOURCE
(DER - 2015/12/08, HDFFV-9627)
- The --enable-clear-file-buffers configure Option was Non-functional
so the Feature was Always Enabled (its default value).
Regardless of the configure flag, the setting was always enabled when
the Autotools were used to configure HDF5. This was due to the "no"
option being processed after the "*" option in configure.ac so "*"
matched first. CMake was unaffected.
The option now works correctly.
NOTE that builders are always advised to leave this option enabled.
When disabled, buffers that are written to disk may contain the
memory's previous contents, which may include secure information.
The performance overhead of the feature (a single memset call per
allocation) is minimal.
(DER - 2016/02/03, HDFFV-9676)
- Added a patch to remove '"'s from arguments for MPI compilers that
were causing errors compiling H5lib_settings.c with SGI MPT.
(LRK - 2016/04/20, HDFFV-9439)
Library
-------
- Fixed shared file pointer problem which caused a crash when running a
program provided by a user.
(VC - 2016/04/01, HDFFV-9469)
- Fixed some format string warnings that prevent compiling with
-Werror=format-security on gcc.
These only appeared in error messages and would not cause problems
under normal operation.
(DER - 2016/01/13, HDFFV-9640)
- Fixed a library segmentation fault when accessing a corrupted
file provided by a user.
(MSC - 2016/02/19, HDFFV-9670)
Tools
-----
- h5dump: Sub-setting Fixed for Dimensions Greater than Two
When a dataset has more than two dimensions, sub-setting would
incorrectly calculate the data that needed to be displayed.
Added in block and stride calculations that account for dimensions
greater than two. NOTE: lines that have line breaks inserted
because of display length calculations may have index info that
is incorrect until the next dimension break.
(ADB - 2016/03/07, HDFFV-9698)
- h5dump: Issue with Argument Segmentation Fault
When an argument with an optional value was at the end of the command
line with a value, h5dump would crash. Reworked check for remaining
arguments.
(ADB - 2016/03/07, HDFFV-9570, HDFFV-9684)
- h5dump: Issue with Default Fill Value
Added all default cases of fill value to the display of fill value.
(ADB -, 2016/03/07, HDFFV-9241)
- h5dump: Clarified Help
Clarified usage of -O F option in h5dump utility help.
(ADB - 2016/03/07, HDFFV-9066)
- h5dump: Issue with Double Free Fault
Added a check for filename not null before calling free().
(ADB - 2016/01/27, HDFFV-9639)
- VS2015 Release Changed how Timezone was Handled
Created a function, HDget_timezone, in H5system.c. Replaced
timezone variable usage with function call.
(ADB - 2015/11/02, HDFFV-9550)
C++ API
-------
- Removal of Obsolete Methods
The overloaded methods which had parameters that should be const
but were not have been removed.
(BMR - 2016/01/13, HDFFV-9789)
High-Level APIs:
---------------
- Fixed Memory Leak in Packet Table API
Applied user's patch to fix memory leak in the creation of a
packet table.
(BMR - 2016/04/25, HDFFV-9700)
Known Problems
==============
* On windows platforms in debug configurations, the VFD flush1 tests will fail
with the split and multi VFD drivers. These tests will display a modal debug
dialog which must be answered or wait for the test timeout to expire.
(ADB - 2014/06/23 - HDFFV-8851)
* CLANG compiler with the options -fcatch-undefined-behavior and -ftrapv
catches some undefined behavior in the alignment algorithm of the macro DETECT_I
in H5detect.c (Issue 8147). Since the algorithm is trying to detect the alignment
of integers, ideally the flag -fcatch-undefined-behavior shouldn't to be used for
H5detect.c. In the future, we can separate flags for H5detect.c from the rest of
the library. (SLU - 2013/10/16)
* Make provided by Solaris fails in "make check". Solaris users should use
gmake to build and install the HDF5 software. (AKC - 2013/10/08 - HDFFV-8534)
* The C++ and FORTRAN bindings are not currently working on FreeBSD with the
native release 8.2 compilers (4.2.1), but are working with gcc 4.6 from the
ports (and probably gcc releases after that).
(QAK - 2012/10/19)
* The following h5dump test case fails in BG/P machines (and potentially other
machines that use a command script to launch executables):
h5dump --no-compact-subset -d "AHFINDERDIRECT::ah_centroid_t[0] it=0 tl=0"
tno-subset.h5
This is due to the embedded spaces in the dataset name being interpreted
by the command script launcher as meta-characters, thus passing three
arguments to h5dump's -d flag. The command passes if run by hand, just
not via the test script.
(AKC - 2012/05/03)
* The STDIO VFD does not work on some architectures, possibly due to 32/64
bit or large file issues. The basic STDIO VFD test is known to fail on
64-bit SunOS 5.10 on SPARC when built with -m64 and 32-bit OS X/Darwin
10.7.0. The STDIO VFD test has been disabled while we investigate and
a fix should appear in a future release.
(DER - 2011/10/14 - HDFFV-8235)
* h5diff can report inconsistent results when comparing datasets of enum type
that contain invalid values. This is due to how enum types are handled in
the library and will be addressed in a future release.
(DER - 2011/10/14 - HDFFV-7527)
* The links test can fail under the stdio VFD due to some issues with external
links. This will be investigated and fixed in a future release.
(DER - 2011/10/14 - HDFFV-7768)
* After the shared library support was fixed for some bugs, it was discovered
that "make prefix=XXX install" no longer works for shared libraries. It
still works correctly for static libraries. Therefore, if you want to
install the HDF5 shared libraries in a location such as /usr/local/hdf5,
you need to specify the location via the --prefix option during configure
time. E.g, ./configure --prefix=/usr/local/hdf5 ...
(AKC - 2011/05/07 - HDFFV-7583)
* The parallel test, t_shapesame, in testpar/, may run for a long time and may
be terminated by the alarm signal. If that happens, one can increase the
alarm seconds (default is 1200 seconds = 20 minutes) by setting the
environment variable, $HDF5_ALARM_SECONDS, to a larger value such as 3600
(60 minutes). Note that the t_shapesame test may fail in some systems
(see the "While working on the 1.8.6 release..." problem below). If
it does, it will waste more time if $HDF5_ALARM_SECONDS is set
to a larger value.
(AKC - 2011/05/07)
* Shared Fortran libraries are not quite working on AIX. While they are
generated when --enable-shared is specified, the fortran and hl/fortran
tests fail. the issue. HL and C++ shared libraries should now be
working as intended, however.
(MAM - 2011/04/20)
* While working on the 1.8.6 release of HDF5, a bug was discovered that can
occur when reading from a dataset in parallel shortly after it has been
written to collectively. The issue was exposed by a new test in the parallel
HDF5 test suite, but had existed before that. We believe the problem lies with
certain MPI implementations and/or file systems.
We have provided a pure MPI test program, as well as a standalone HDF5
program, that can be used to determine if this is an issue on your system.
They should be run across multiple nodes with a varying number of processes.
These programs can be found at:
http://www.hdfgroup.org/ftp/HDF5/examples/known_problems/
(NAF - 2011/01/19)
* All the VFL drivers aren't backward compatible. In H5FDpublic.h, the
structure H5FD_class_t changed in 1.8. There is new parameter added to
get_eoa and set_eoa callback functions. A new callback function
get_type_map was added in. The public function H5FDrealloc was taken
out in 1.8. The problem only happens when users define their own driver
for 1.6 and try to plug in 1.8 library. Because there's only one user
complaining about it, we (Elena, Quincey, and I) decided to leave it as
it is (see bug report #1279). Quincey will make a plan for 1.10.
(SLU - 2010/02/02)
* The --enable-static-exec configure flag will only statically link libraries
if the static version of that library is present. If only the shared version
of a library exists (i.e., most system libraries on Solaris, AIX, and Mac,
for example, only have shared versions), the flag should still result in a
successful compilation, but note that the installed executables will not be
fully static. Thus, the only guarantee on these systems is that the
executable is statically linked with just the HDF5 library.
(MAM - 2009/11/04)
* A dataset created or rewritten with a v1.6.3 library or after cannot be read
with the v1.6.2 library or before when the Fletcher32 EDC filter is enabled.
There was a bug in the calculation of the Fletcher32 checksum in the
library before v1.6.3; the checksum value was not consistent between big-
endian and little-endian systems. This bug was fixed in Release 1.6.3.
However, after fixing the bug, the checksum value was no longer the same as
before on little-endian system. Library releases after 1.6.4 can still read
datasets created or rewritten with an HDF5 library of v1.6.2 or before.
(SLU - 2005/06/30)
1d8367eef9