* Improvements made to the CMake build system.
* Added Memory Allocation Functions that Use the Library's Allocator
* Added Functions to Control the Value of H5PL_no_plugin_g without
Using an Environment Variable
* MPI_Finalize and HDF5 Library Shutdown
* New Wrappers for C Functions H5P[s/g]et_libver_bounds
* more...
Issues found with existing distfiles:
distfiles/eclipse-sourceBuild-srcIncluded-3.0.1.zip
distfiles/fortran-utils-1.1.tar.gz
distfiles/ivykis-0.39.tar.gz
distfiles/enum-1.11.tar.gz
distfiles/pvs-3.2-libraries.tgz
distfiles/pvs-3.2-linux.tgz
distfiles/pvs-3.2-solaris.tgz
distfiles/pvs-3.2-system.tgz
No changes made to these distinfo files.
Otherwise, existing SHA1 digests verified and found to be the same on
the machine holding the existing distfiles (morden). All existing
SHA1 digests retained for now as an audit trail.
Upstream changes:
Release 1.8.14 of November 2014 (current release) versus Release 1.8.13
In the main library C Interface
No new C functions have been added.
The behavior of the following C function has been changed:
The function H5Pset_core_write_tracking has been modified to enforce the limitation that the value of the page_size parameter cannot equal 0 (zero):
H5Pset_core_write_tracking
The following performance optimization has been implemented in the Parallel HDF5 library:
The algoritm for writing chunk fill values for new datasets has been optimized to write out all chunk fill values collectively in a single MPI-IO call. This was previously done independently by rank 0 (zero) one block at a time. This optimization should produce a significant performance improvement when creating chunked datasets in parallel, most notably when chunk dimensions are relatively small. This optimization will be evident in use of the following function:
H5Dcreate
In the high-level library C Interface
High-level HDF5 interfaces now ensure that strings are null-terminated and will not overflow a buffer.
In several high-level functions, the type from the file was previously re-used as the memory type and no check is made to ensure that a string was actually null-terminated.
All high-level functions now check for NULL pointers; verify that a received a string is null-terminated; and verify that the string will not overflow the buffer. The minimum length of the buffers is now used in strncmp to avoid overflow.
In the main library C++ Interface
No new C++ methods have been added.
The following changes have been made in the C++ interface:
Changed defaults
Default value of class member id:
The data member id in several classes that represent HDF5 objects are now initialized to H5I_INVALID_ID.
Default value of a PropList member id:
The data member id in class PropList is now initialized to H5P_DEFAULT.
In HDF5 Release 1.8.13 and earlier, the above were initialized to 0 (zero).
Changed file access flags
The flag H5F_ACC_CREAT has been removed from the C++ interface.
In the main library Fortran Interface
No new Fortran subroutines have been added.
Changed Fortran INTEGER parameter types:
Types of the following Fortran parameter constants have been changed from INTEGER to INTEGER(HID_T):
H5FD_CORE H5FD_MULTI
H5FD_FAMILY H5FD_SEC2
H5FD_LOG H5FD_STDIO
H5FD_MPIO
Other internal int types have been changed to hid_t, but this change is expected to be transparent to the user
Release 1.8.13 of May 2014 versus Release 1.8.12
In the C Interface (main library)
The following new C functions have been added:
To free memory allocated by HDF5 function calls:
H5free_memory
This function should be used to free memory that has been allocated by HDF5 API calls such as H5Tget_member_name and H5Pget_class_name.
The primary value for this call is on Windows systems, where it is common for application code and the HDF5 Library to be using different C run-time libraries (CRT). Using this new call ensures that the same CRT handles both the allocation and free.
This function can also be useful in any case where the library uses a different memory manager than the application, such as when a debug memory manager is in use or when the HDF5 Library is wrapped for use in a managed language like Python or Java.
New configuration property for the CORE driver:
H5Pset_core_write_tracking
H5Pget_core_write_tracking
With this property, the CORE driver, H5Pset_fapl_core, can be configured to track dirty regions in the file and write out only the changed regions on flush or close operations. Additionally, a page aggregation size can be set that will aggregate small writes into larger writes. For example, setting a 1 mebibyte (1 MiB) page aggregation size will logically partition the the in-memory file into 1 MiB pages that will be written out in their entirety if even a single byte is dirtied.
The following C functions have been modified:
H5Rget_name
If the name parameter is passed as NULL, then the size parameter is ignored and the function returns the size of the buffer required to read the name (not including the NULL terminator).
H5Zfilter_avail
This function has been updated to correctly identify dynamically loaded filters. If a filter is not found among registered filters, the function now checks for dynamically loaded filters.
In the C++ Interface
The following new features have been added:
Wrappers to the class H5Object to get an object’s name:
ssize_t getObjName(char *obj_name, size_t buf_size = 0)
ssize_t getObjName(H5std_string& obj_name, size_t len = 0)
H5std_string getObjName()
Wrappers to the class H5CommonFG to get a child object’s type from a group or file:
H5O_type_t childObjType(const H5std_string& objname)
H5O_type_t childObjType(const char* objname)
H5O_type_t childObjType(hsize_t index,
H5_index_t index_type=H5_INDEX_NAME,
H5_iter_order_t order=H5_ITER_INC, const char* objname=".")
Wrappers to the class DSetMemXferPropList for setting or getting a transform property list:
DSetMemXferPropList(const char* expression);
void setDataTransform(const char* expression)
void setDataTransform(const H5std_string& expression)
ssize_t getDataTransform(char* exp, size_t buf_size=0)
H5std_string getDataTransform()
Wrapper to CompType for setting the size of a compound datatype:
void setSize(size_t size)
Overloaded functions to provide prototypes that declare constant arguments const:
DataSet::fillMemBuf
DataSet::getVlenBufSize
DataSpace::extentCopy
DataType::commit
FileAccPropList::setSplit
H5File::getVFDHandle
Additional overload to the class H5Location to get a comment as a char*:
ssize_t getComment(const char* name, size_t buf_size, char* comment)
Additional overloads to the class Attribute to get an attribute’s name for convenience:
ssize_t getName(char* attr_name, size_t buf_size=0)
ssize_t getName(H5std_string& attr_name, size_t buf_size=0)
In the Fortran Interface
The following new Fortran subroutines have been added:
Fortran2003 subroutines:
h5pset_file_image_f
h5pget_file_image_f
h5fget_file_image_f (Added in Release 1.8.11, but not documented at that time.)
The following Fortran subroutine has been changed:
h5pset_chunk_cache_f
The values of the constants H5D_CHUNK_CACHE_NSLOTS_DFLT_F and H5D_CHUNK_CACHE_NBYTES_DFLT_F have been changed to INTEGER(KIND=size_t).
Other New Features
Parallel I/O support:
Parallel I/O support has been enhanced to allow collective I/O on point selections.
Thread safety (Windows only)
The thread-safe HDF5 library no longer leaks thread-local storage resources on Windows with Win32 threads. This cleanup can only be performed when the library is built as a shared library, so a thread-safe HDF5 static library with Win32 threads is no longer supported.
This change affects only Windows environments.
Also see “Build and Configuration” below.
Command-line Tools with Changed Behavior
h5repack
h5repack has been updated to correctly handle dynamically loaded filters.
Build and Configuration
Thread safety
When configuring a thread-safe HDF5 Library with --enable-threadsafe, it is no longer necessary to specify --with-pthreads if the Pthreads library is in a standard location.
Removed Functions and Wrappers
The MPI-POSIX driver has been removed. The following C functions and the corresponding Fortran subroutines and C++ wrappers therefore are no longer included in the HDF5 distribution:
H5Pset_fapl_mpiposix
H5Pget_fapl_mpiposix
Applications performing parallel I/O should use the MPI-IO driver, H5Pset_fapl_mpio.
Configuration
-------------
- Added a configuration option to change the default plugin path.
The configure option is --with-default-plugindir=location.
The cmake option is -DH5_DEFAULT_PLUGINDIR:PATH=location.
HDFFV-8513. (ADB 2013/09/04)
- Renamed FFLAGS to FCFLAGS in configure. (ADB 2013/08/13)
- CMake can now package a compressed examples file, the default for
Windows binaries from HDF Group. (ADB - 2013/07/22)
Tools
-----
- h5repack: Added the ability to use plugin filters to read and write
files. The option uses the filter number. HDFFV-8345
(ADB - 2013/09/04).
- h5dump: Added the option -N --any_path, which searches the file for
paths that match the search path. HDFFV-7989 (ADB - 2013/08/12).
- h5dump: Added the optional arg 0 to -A, which excludes attributes
from display. HDFFV-8134 (ADB - 2013/08/01).
C++ API
-------
- Added tutorial examples to C++/examples. They can be installed by
"make install-examples" and, in the installed directory, they can be
executed by running the script file run-c++-ex.sh. (BMR - 2013/09/28)
- A new class, H5::H5Location, is added to represent the location concept
in the C library. It is a base class to H5::H5File and H5::H5Ojbect,
whose member functions are moved into H5::H5Location. H5::H5File can
now inherent those functions. As a result, an H5::H5File object can have
an attribute. (BMR - 2013/09/27)
- Added wrappers for H5Rget_obj_type2 to retrieve the type of the object
that an object reference points to. (BMR - 2013/09/27)
H5O_type_t H5Location::getRefObjType(void *ref, H5R_type_t ref_type)
- Added wrappers for H5Aexist to check whether an attribute exists given
a name. (BMR - 2013/09/27)
bool H5::H5Location::attrExists(const char* name)
bool H5::H5Location::attrExists(const H5std_string& name)
- Added a number of overloaded functions for convenience. (BMR - 2013/09/27)
Configuration
- Libtool version number is changed to 8.0.0 because there are API
changes. See below for details.
- Mac OS X 10.7 (Lion) and 10.8 (Mountain Lion) uses clang/clang++ as the
default C and C++ compilers.
- CMake minimum is now 2.8.10.
- A new tool, cmakehdf5, which is a build command script similar to
buildhdf5 is added and is available in the bin directory.
Library
- The library can load filter libraries dynamically during runtime. Users
can set the search path through environment variable HDF5_PLUGIN_PATH
and call H5Pset_filter to enable a dynamic filter.
- Added new API functions H5Dscatter and H5Dgather to scatter data to and
and gather data from a selection within a memory buffer.
- The library now supports the data conversion from enumeration to numeric
(integer and floating-point number) datatypes. See Issue HDFFV-8221.
Tools
- h5dump: added new option -O or -ddl to output the ddl text to a file. This
is a complement to the -o or --output option, which redirects the data to
a file.
Library
-------
- The library now behaves correctly when performing large I/O operations on
Mac OS-X. Previously, single I/O operations > 2 GB would fail since the
Darwin read/write calls cannot handle the number of bytes that their
parameter types imply.
Fixes HDFFV-7975 and HDFFV-8240 (DER - 07 JAN 2013)
- Fixed a bug in the core VFD that cause failures when opening files > 2 GB.
Fixes HDFFV-8124 and HDFFV-8158 (DER - 07 JAN 2013)
Tools
-----
- The following h5stat test case failed in BG/P machines (and potentially
other machines that display extra output if an MPI task returns with a
non-zero code.)
Testing h5stat notexist.h5
The test script was fixed to ignore the extra output.
HDFFV-8233 (AKC - 2012/12/17)
- h5diff: Fixed slowness when comparing HDF5 files with many attributes.
Much slower performance was identified with later release version
(from 1.8.7 to 1.8.10) compared to 1.8.6. The issue was introduced
from fixing an attribute related bug for 1.8.7 release in the past.
HDFFV-8145 (JKM 2012/12/13)
* The following new C function has been added:
H5Pget_mpio_no_collective_cause
* The following new Fortran subroutine has been added:
h5pget_mpio_actual_io_mode_f
* The syntax of the following high-level C function has changed:
H5TBAget_fill
This function’s return type has been changed to h5tri_t. A return value of 1 indicates that a fill value is present, 0 indicates that no fill value is present, and a negative value indicates an error. (The former return type was herr_t.)
Note that H5TBAget_fill and a companion function H5TBAget_title, both in the HDF5 Table (H5TB) high-level API, are undocumented. These functions will be added to the HDF5 Reference Manual shortly after HDF5 Release 1.8.10.
Changed Command-line Tool
* h5import has been updated to accept an input file created by h5dump.
This doesn't pass self-tests, but it isn't regression,
previous package didn't pass them as well.
(While here, make both packages share the same distinfo.)
Changes in HDF5 1.8.9
HDF5 1.8.9 is a minor release, but contains several important
new features and bug fixes. Changes in this release include the
following:
* Committed datatypes can now be merged when copying objects,
using the new flag H5O_COPY_MERGE_COMMITTED_DTYPE_FLAG.
Several new API routines have been included in support of
this feature.
* A file in memory can now be accessed in the same way as a
file on disk. New APIs were added to support this feature.
* Many changes to the tools have been made, including the following:
+ The h5repack performance has been improved dramatically
for large chunked datasets.
+ The -S (or --summary) option was added to h5stat to
display a summary of the file space usage for a file.
+ A wildcard can now be used with h5dump to specify multiple files.
* H5LTpath_valid was added to determine if a path is correct
and a link resolves to a valid object.
* The Fortran counterpart to H5Ocopy, h5ocopy_f, was added.
This release contains many other changes and bug fixes not listed here.
Please be sure to read the Release Notes for a comprehensive
list of new features and bug fixes.
Changes in HDF5 1.8.8
HDF5 1.8.8 is a minor release, but contains a few important new
features and bug fixes. Changes in this release include the
following:
* Changes to the Fortran interface:
+ Fortran wrappers for the Dimension Scales APIs were added.
For the new subroutine signatures, see:
http://www.hdfgroup.org/HDF5/doc/HL/RM_H5DS.html
+ The Fortran interface now uses the Fortran 2003 standard,
enabling a wider set of Fortran and HDF5 datatypes to be
supported, including:
o Any kind of INTEGER or REAL
o Fortran derived types
o Fortran and HDF5 enumeration
o HDF5 variable-length datatypes
o HDF5 compound datatypes of any complexity
It also contains new subroutines corresponding to those
C APIs which have callback functions as parameters.
For a general overview and information on how to enable
these new Fortran features, please see the following paper:
http://www.hdfgroup.org/HDF5/doc/fortran/NewFeatures_F2003.pdf
HDF5 examples that use Fortran 2003 (with the suffix "F03") can be found here:
http://www.hdfgroup.org/ftp/HDF5/examples/examples-by-api/api18-fortran.html
The --enable-fortran2003 configure flag was added to
enable Fortran 2003 support in the HDF5 Fortran library.
This flag should be used along with the --enable-fortran
flag and takes affect only when a Fortran compiler is
Fortran 2003 compliant.
* H5Tcreate now supports a string type (fixed-length and variable-length).
* New APIs, H5Pget_mpio_actual_chunk_opt_mode and H5Pget_mpio_actual_io_mode,
were added for querying whether/how a collective I/O operation completed.
* Numerous changes were made to the tools (h5diff, h5repack, h5jam, h5dump).
pkgsrc notion that free software should not depend on non-free
software (because that results in not having distributable binary
packages, and makes the build fail) unless skipping the dependency
renders the software non-useful.
(The hdf web page says that the default binary builds of hdf include
szip support, but then go on about how users that don't qualify for
the no-cost license have to install a different version.)
If someone who grasps hdf5 can say that hdf5 w/o szip is generally not
useful, we can turn this back on and disable by default the hdf5
option in octave.
(yes I know there is the 1.8 branch, but I don't want to experiment
right now)
changes: many fixes, some C++ API improvements
pkgsrc change: DESTDIR support
This changes the buildlink3.mk files to use an include guard for the
recursive include. The use of BUILDLINK_DEPTH, BUILDLINK_DEPENDS,
BUILDLINK_PACKAGES and BUILDLINK_ORDER is handled by a single new
variable BUILDLINK_TREE. Each buildlink3.mk file adds a pair of
enter/exit marker, which can be used to reconstruct the tree and
to determine first level includes. Avoiding := for large variables
(BUILDLINK_ORDER) speeds up parse time as += has linear complexity.
The include guard reduces system time by avoiding reading files over and
over again. For complex packages this reduces both %user and %sys time to
half of the former time.
changes:
-minor feature additions to the command line tools
-C++ API: Added support for Szip
-bugfixes
pkgsrc changes:
-fixes for the cxx option (should be a separate pkg)
-fixes for the threadsafe option -- replaced a pthread_equal() by a
simple "==" to make it usable with non-threaded apps
(not 100% portable, but should be fine for the platforms in question)
The shared libraries should not be linked against libpthread, but
that would require serious changes to the autoconf/make machinery
so I've left it alone for now.
If that is fixed, "threadsafe" could be made default.
-Don't try to strip scripts on install.
since they always need a C compiler, even when the source code is
completely in C++.
For some other packages, stated in the comment that a C compiler is
really not needed.
and add a new helper target and script, "show-buildlink3", that outputs
a listing of the buildlink3.mk files included as well as the depth at
which they are included.
For example, "make show-buildlink3" in fonts/Xft2 displays:
zlib
fontconfig
iconv
zlib
freetype2
expat
freetype2
Xrender
renderproto
RECOMMENDED is removed. It becomes ABI_DEPENDS.
BUILDLINK_RECOMMENDED.foo becomes BUILDLINK_ABI_DEPENDS.foo.
BUILDLINK_DEPENDS.foo becomes BUILDLINK_API_DEPENDS.foo.
BUILDLINK_DEPENDS does not change.
IGNORE_RECOMMENDED (which defaulted to "no") becomes USE_ABI_DEPENDS
which defaults to "yes".
Added to obsolete.mk checking for IGNORE_RECOMMENDED.
I did not manually go through and fix any aesthetic tab/spacing issues.
I have tested the above patch on DragonFly building and packaging
subversion and pkglint and their many dependencies.
I have also tested USE_ABI_DEPENDS=no on my NetBSD workstation (where I
have used IGNORE_RECOMMENDED for a long time). I have been an active user
of IGNORE_RECOMMENDED since it was available.
As suggested, I removed the documentation sentences suggesting bumping for
"security" issues.
As discussed on tech-pkg.
I will commit to revbump, pkglint, pkg_install, createbuildlink separately.
Note that if you use wip, it will fail! I will commit to pkgsrc-wip
later (within day).
developer is officially maintaining the package.
The rationale for changing this from "tech-pkg" to "pkgsrc-users" is
that it implies that any user can try to maintain the package (by
submitting patches to the mailing list). Since the folks most likely
to care about the package are the folks that want to use it or are
already using it, this would leverage the energy of users who aren't
developers.
-make NetBSD ("netbsdelf") a recognized configuration, copied from
"freebsd"; this pulls in definitions for gcc, in particular it
disables some optimization
-work around a memory corruption problem in H5FDstream which appearently
is caused by use of free()d memory
The latter 2 changes make the library survive its selftests, and help
the "pytables" pkg as well.
bump PKGREVISION, raise BUILDLINK_RECOMMENDED
