v3.5 (2021-09-29)
-----------------
* :issue:`555`: Add support for dumping statistics in JSON format using ``--json``.
* :issue:`541`: Add a "Read fate breakdown" section heading to the report, and also
add statistics for reads discarded because of ``--discard-untrimmed`` and
``--discard-trimmed``. With this, the numbers in that section should add up to 100%.
* Add option ``-Q``, which allows to specify a quality-trimming threshold for R2 that is
different from the one for R1.
* :issue:`567`: Add ``noindels`` adapter-trimming parameter. You can now write
``-a "ADAPTER;noindels"`` to disallow indels for a single adapter only.
* :issue:`570`: Fix ``--pair-adapters`` not finding some pairs when reads contain
more than one adapter.
* :issue:`524`: Fix a memory leak when using ``--info-file`` with multiple cores.
* :issue:`559`: Fix adjacent base statistics not being shown for linked adapters.
This flag should be set for packages that import pkg_resources
and thus need setuptools after the build step.
Set this flag for packages that need it and bump PKGREVISION.
Updates for evolving libxtend and biolibc APIs
Add --version flag
Filter out unused SAM fields on input
Numerous other minor fixes and enhancements
Changes: https://github.com/auerlab/ad2vcf/releases/tag/0.1.4
Version 2.2.0
Changes
-------
* Data elements with a VR of **AT** must now be set with values
acceptable to :func:`~pydicom.tag.Tag`, and are always stored as a
:class:`~pydicom.tag.BaseTag`. Previously, any Python type could be
set.
* :meth:`BaseTag.__eq__()<pydicom.tag.BaseTag.__eq__>` returns ``False`` rather
than raising an exception when the operand cannot be converted to
:class:`~pydicom.tag.BaseTag` (:pr:`1327`)
* :meth:`DA.__str__()<pydicom.valuerep.DA.__str__>`,
:meth:`DT.__str__()<pydicom.valuerep.DT.__str__>` and
:meth:`TM.__str__()<pydicom.valuerep.TM.__str__>` return valid DICOM
strings instead of the formatted date and time representations
(:issue:`1262`)
* If comparing :class:`~pydicom.dataset.FileDataset` instances, the file
metadata is now ignored. This makes it possible to compare a
:class:`~pydicom.dataset.FileDataset` object with a
:class:`~pydicom.dataset.Dataset` object.
* :func:`~pydicom.pixel_data_handlers.rle_handler.rle_encode_frame` is
deprecated and will be removed in v3.0, use
:meth:`~pydicom.dataset.Dataset.compress` or
:attr:`~pydicom.encoders.RLELosslessEncoder` instead.
* :func:`~pydicom.filereader.read_file` is deprecated and will be removed in
v3.0, use :func:`~pydicom.filereader.dcmread` instead.
* :func:`~pydicom.filewriter.write_file` is deprecated and will be removed in
v3.0, use :func:`~pydicom.filewriter.dcmwrite` instead.
* Data dictionaries updated to version 2021b of the DICOM Standard
* :class:`~pydicom.dataset.Dataset` no longer inherits from :class:`dict`
Enhancements
------------
* Added a command-line interface for pydicom. Current subcommands are:
* ``show``: display all or part of a DICOM file
* ``codify`` to produce Python code for writing files or sequence items
from scratch.
Please see the :ref:`cli_guide` for examples and details
of all the options for each command.
* A field containing an invalid number of bytes will result in a warning
instead of an exception when
:attr:`~pydicom.config.convert_wrong_length_to_UN` is set to ``True``.
* Private tags known via the private dictionary will now get the configured
VR if read from a dataset instead of **UN** (:issue:`1051`).
* While reading explicit VR, a switch to implicit VR will be silently attempted
if the VR bytes are not valid VR characters, and config option
:attr:`~pydicom.config.assume_implicit_vr_switch` is ``True`` (default)
* New functionality to help with correct formatting of decimal strings (**DS**)
* Added :func:`~pydicom.valuerep.is_valid_ds` to check whether a string is
valid as a DICOM decimal string and
:func:`~pydicom.valuerep.format_number_as_ds` to format a given ``float``
or ``Decimal`` as a DS while retaining the highest possible level of
precision
* If :attr:`~pydicom.config.enforce_valid_values` is set to ``True``, all
**DS** objects created will be checked for the validity of their string
representations.
* Added optional ``auto_format`` parameter to the init methods of
:class:`~pydicom.valuerep.DSfloat` and
:class:`~pydicom.valuerep.DSdecimal` and the :func:`~pydicom.valuerep.DS`
factory function to allow explicitly requesting automatic formatting of
the string representations of these objects when they are constructed.
* Added methods to construct :class:`~pydicom.valuerep.PersonName` objects
from individual components of names (``family_name``, ``given_name``, etc.).
See :meth:`~pydicom.valuerep.PersonName.from_named_components` and
:meth:`~pydicom.valuerep.PersonName.from_named_components_veterinary`.
* Added support for downloading the large test files with the `requests
<https://docs.python-requests.org/en/master/>`_ package in addition to
:mod:`urllib.request` (:pr:`1340`)
* Ensured :func:`~pydicom.pixel_data_handlers.util.convert_color_space` uses
32-bit floats for calculation, added `per_frame` flag to allow frame-by-frame
processing and improved the speed by ~20-60% (:issue:`1348`)
* Optimisations for RLE encoding using *pydicom* (~40% faster).
* Added support for faster decoding (~4-5x) and encoding (~20x) of *RLE Lossless*
*Pixel Data* via the `pylibjpeg-rle
<https://github.com/pydicom/pylibjpeg-rle>`_ plugin (:pr:`1361`, :pr:`1372`).
* Added :func:`Dataset.compress()<pydicom.dataset.Dataset.compress>` function for
compressing uncompressed pixel data using a given encoding format as specified
by a UID. Only *RLE Lossless* is currently supported (:pr:`1372`)
* Added :mod:`~pydicom.encoders` module and the following encoders:
* :attr:`~pydicom.encoders.RLELosslessEncoder` with 'pydicom', 'pylibjpeg'
and 'gdcm' plugins
* Added `read` parameter to :func:`~pydicom.data.get_testdata_file`
to allow reading and returning the corresponding dataset (:pr:`1372`)
* Handle decoded RLE segments with padding (:issue:`1438`)
* Add option to JSON functions to suppress exception and continue (:pr:`1332`)
* Allow searching :class:`~pydicom.fileset.FileSet` s for a list of elements (:pr:`1428`)
* Added hash function to SR :class:`~pydicom.sr.Code` (:pr:`1434`)
Fixes
-----
* Fixed pickling a :class:`~pydicom.dataset.Dataset` instance with sequences
after the sequence had been read (:issue:`1278`)
* Fixed JSON export of numeric values
* Fixed handling of sequences of unknown length that switch to implicit
encoding, and sequences with VR **UN** (:issue:`1312`)
* Do not load external data sources until needed - fixes problems with
standard workflow if `setuptools` are not installed (:issue:`1341`)
* Fixed empty **PN** elements read from file being :class:`str` rather than
:class:`~pydicom.valuerep.PersonName` (:issue:`1338`)
* Fixed handling of JPEG (10918-1) images compressed using RGB colourspace
rather than YBR with the Pillow pixel data handler (:pr:`878`)
* Allow to deepcopy a `~pydicom.dataset.FileDataset` object (:issue:`1147`)
* Fixed elements with a VR of **OL**, **OD** and **OV** not being set correctly
when an encoded backslash was part of the element value (:issue:`1412`)
* Fixed expansion of linear segments with floating point steps in
segmented LUTs (:issue:`1415`)
* Fixed handling of code extensions with person name component delimiter
(:pr:`1449`)
* Fixed bug decoding RBG jpg with APP14 marker due to change in Pillow (:pr:`1444`)
* Fixed decoding for `FloatPixelData` and `DoubleFloatPixelData` via
`pydicom.pixel_data_handlers.numpy_handler` (:issue:`1457`)
Biolibc-tools is a collection of simple, fast, and memory-efficient
programs for processing biological data. These programs built on
biolibc are not complex enough to warrant separate projects.
Version 2.2.0
Changes
Data elements with a VR of AT must now be set with values acceptable to Tag(), and are always stored as a BaseTag. Previously, any Python type could be set.
BaseTag.__eq__() returns False rather than raising an exception when the operand cannot be converted to BaseTag
DA.__str__(), DT.__str__() and TM.__str__() return valid DICOM strings instead of the formatted date and time representations
If comparing FileDataset instances, the file metadata is now ignored. This makes it possible to compare a FileDataset object with a Dataset object.
rle_encode_frame() is deprecated and will be removed in v3.0, use compress() or RLELosslessEncoder instead.
read_file() is deprecated and will be removed in v3.0, use dcmread() instead.
write_file() is deprecated and will be removed in v3.0, use dcmwrite() instead.
Data dictionaries updated to version 2021b of the DICOM Standard
Dataset no longer inherits from dict
Enhancements
Added a command-line interface for pydicom. Current subcommands are:
show: display all or part of a DICOM file
codify to produce Python code for writing files or sequence items from scratch.
Please see the Command-line Interface Guide for examples and details of all the options for each command.
A field containing an invalid number of bytes will result in a warning instead of an exception when convert_wrong_length_to_UN is set to True.
Private tags known via the private dictionary will now get the configured VR if read from a dataset instead of UN
While reading explicit VR, a switch to implicit VR will be silently attempted if the VR bytes are not valid VR characters, and config option assume_implicit_vr_switch is True (default)
New functionality to help with correct formatting of decimal strings (DS)
Added is_valid_ds() to check whether a string is valid as a DICOM decimal string and format_number_as_ds() to format a given float or Decimal as a DS while retaining the highest possible level of precision
If enforce_valid_values is set to True, all DS objects created will be checked for the validity of their string representations.
Added optional auto_format parameter to the init methods of DSfloat and DSdecimal and the DS() factory function to allow explicitly requesting automatic formatting of the string representations of these objects when they are constructed.
Added methods to construct PersonName objects from individual components of names (family_name, given_name, etc.). See from_named_components() and from_named_components_veterinary().
Added support for downloading the large test files with the requests package in addition to urllib.request
Ensured convert_color_space() uses 32-bit floats for calculation, added per_frame flag to allow frame-by-frame processing and improved the speed by ~20-60%
Optimisations for RLE encoding using pydicom (~40% faster).
Added support for faster decoding (~4-5x) and encoding (~20x) of RLE Lossless Pixel Data via the pylibjpeg-rle plugin
Added Dataset.compress() function for compressing uncompressed pixel data using a given encoding format as specified by a UID. Only RLE Lossless is currently supported
Added encoders module and the following encoders:
RLELosslessEncoder with ‘pydicom’, ‘pylibjpeg’ and ‘gdcm’ plugins
Added read parameter to get_testdata_file() to allow reading and returning the corresponding dataset
Handle decoded RLE segments with padding
Add option to JSON functions to suppress exception and continue
Allow searching FileSet s for a list of elements
Added hash function to SR Code
Fixes
Fixed pickling a Dataset instance with sequences after the sequence had been read
Fixed JSON export of numeric values
Fixed handling of sequences of unknown length that switch to implicit encoding, and sequences with VR UN
Do not load external data sources until needed - fixes problems with standard workflow if setuptools are not installed
Fixed empty PN elements read from file being str rather than PersonName
Fixed handling of JPEG (10918-1) images compressed using RGB colourspace rather than YBR with the Pillow pixel data handler
Allow to deepcopy a ~pydicom.dataset.FileDataset object
Fixed elements with a VR of OL, OD and OV not being set correctly when an encoded backslash was part of the element value
Fixed expansion of linear segments with floating point steps in segmented LUTs
Fixed handling of code extensions with person name component delimiter
Fixed bug decoding RBG jpg with APP14 marker due to change in Pillow
Fixed decoding for FloatPixelData and DoubleFloatPixelData via pydicom.pixel_data_handlers.numpy_handler
Version 2.1.1
Fixes
Remove py.typed
Fix ImportError with Python 3.6.0
Fix converting Sequences with Bulk Data when loading from JSON
Version 2.1.0
Changelog
Dropped support for Python 3.5 (only Python 3.6+ supported)
Enhancements
Large testing data is no longer distributed within the pydicom package with the aim to reduce the package download size. These test files will download on-the-fly whenever either the tests are run, or should the file(s) be requested via the data manager functions. For example:
To download all files and get their paths on disk you can run pydicom.data.get_testdata_files().
To download an individual file and get its path on disk you can use pydicom.data.get_testdata_file(), e.g. for RG1_UNCI.dcm use pydicom.data.get_testdata_file("RG1_UNCI.dcm")
Added a new pixel data handler based on pylibjpeg which supports all (non-retired) JPEG transfer syntaxes
Added apply_rescale() alias
Added apply_voi() and apply_windowing()
Added prefer_lut keyword parameter to apply_voi_lut() and handle empty VOI LUT module elements
Added ability to register external data sources for use with the functions in pydicom.data
__contains__, __next__ and __iter__ implementations added to PersonName
Added convenience constants for the MPEG transfer syntaxes to pydicom.uid
Added support for decoding Waveform Data:
Added pydicom.waveforms module and generate_multiplex() and multiplex_array() functions.
Added Dataset.waveform_array() which returns an ndarray for the multiplex group at index within a Waveform Sequence element.
When JPEG 2000 image data is unsigned and the Pixel Representation is 1 the image data is converted to signed
Added keyword property for the new UID keywords in version 2020d of the DICOM Standard
Added testing of the variable names used when setting Dataset attributes and INVALID_KEYWORD_BEHAVIOR config option to allow customizing the behavior when a camel case variable name is used that isn’t a known element keyword
Added INVALID_KEY_BEHAVIOR config option to allow customizing the behavior when an invalid key is used with the Dataset in operator
Implemented full support (loading, accessing, modifying, writing) of DICOM File-sets and their DICOMDIR files via the FileSet class
Added AllTransferSyntaxes
Added option to turn on pydicom future breaking behavior to allow user code to check itself against the next major version release. Set environment variable “PYDICOM_FUTURE” to “True” or call future_behavior()
Added another signature to the bulk_data_uri_handler in from_json to allow for the communication of not just the URI but also the tag and VR to the handler. Previous handlers will work as expected, new signature handlers will get the additional information.
pack_bits() can now be used with 2D or 3D input arrays and will pad the packed data to even length by default.
Elements with the IS VR accept float strings that are convertible to integers without loss, e.g. “1.0”
Added encapsulate_extended() function for use when an Extended Offset Table is required
Changes
Reading and adding unknown non-private tags now does not raise an exception per default, only when enforce_valid_values is set
Data dictionaries updated to version 2020d of the DICOM Standard
Updated a handful of the SOP Class variable names in _storage_sopclass_uids to use the new UID keywords. Variables with Multiframe in them become MultiFrame, those with and in them become And, and DICOSQuadrupoleResonanceQRStorage becomes DICOSQuadrupoleResonanceStorage.
The following UID constants are deprecated and will be removed in v2.2:
JPEGBaseline: use JPEGBaseline8Bit
JPEGExtended: use JPEGExtended12Bit
JPEGLossless: use JPEGLosslessSV1
JPEGLSLossy: use JPEGLSNearLossless
JPEG2000MultiComponentLossless: use JPEG2000MCLossless
JPEG2000MultiComponent: use JPEG2000MC
In v3.0 the value for JPEGLossless will change from 1.2.840.10008.1.2.4.70 to 1.2.840.10008.1.2.4.57 to match its UID keyword
The following lists of UIDs are deprecated and will be removed in v2.2:
JPEGLossyCompressedPixelTransferSyntaxes: use JPEGTransferSyntaxes
JPEGLSSupportedCompressedPixelTransferSyntaxes: use JPEGLSTransferSyntaxes
JPEG2000CompressedPixelTransferSyntaxes: use JPEG2000TransferSyntaxes
RLECompressedLosslessSyntaxes: use RLETransferSyntaxes
UncompressedPixelTransferSyntaxes: use UncompressedTransferSyntaxes
PILSupportedCompressedPixelTransferSyntaxes
DicomDir and the dicomdir module are deprecated and will be removed in v3.0. Use FileSet instead
pydicom.overlay_data_handlers is deprecated, use pydicom.overlays instead
Removed transfer syntax limitations when converting overlays to an ndarray
The overlay_data_handlers config option is deprecated, the default handler will always be used.
Fixes
Dataset.copy() now works as expected
Optimistically parse undefined length non-SQ data as if it’s encapsulated pixel data to avoid erroring out on embedded sequence delimiter
Fixed get_testdata_file() and get_testdata_files() raising an exception if no network connection is available
Fixed GDCM < v2.8.8 not returning the pixel array for datasets not read from a file-like
Raise TypeError if dcmread() or dcmwrite() is called with wrong argument
Gracefully handle empty Specific Character Set
Fixed empty ambiguous VR elements raising an exception
Allow apply_voi_lut() to apply VOI lookup to an input float array
Fixed Dataset.setdefault() not adding working correctly when the default value is None and not adding private elements when enforce_valid_values is True
Version 2.0.0
Changelog
Dropped support for Python 2 (only Python 3.5+ supported)
Changes to Dataset.file_meta
file_meta now shown by default in dataset str or repr output; pydicom.config.show_file_meta can be set False to restore previous behavior
new FileMetaDataset class that accepts only group 2 data elements
Deprecation warning given unless Dataset.file_meta set with a FileMetaDataset object (in pydicom 3, it will be required)
Old PersonName class removed; PersonName3 renamed to PersonName. Classes PersonNameUnicode and PersonName3 are aliased to PersonName but are deprecated and will be removed in version 2.1
dataelem.isMultiValue (previously deprecated) has been removed. Use dataelem.DataElement.VM instead.
Enhancements
Allow PathLike objects for filename argument in dcmread, dcmwrite and Dataset.save_as
Deflate post-file meta information data when writing a dataset with the Deflated Explicit VR Little Endian transfer syntax UID
Added config.replace_un_with_known_vr to be able to switch off automatic VR conversion for known tags with VR “UN”
Added config.use_DS_numpy and config.use_IS_numpy to have multi-valued data elements with VR of DS or IS return a numpy array
Fixes
Fixed reading of datasets with an empty Specific Character Set tag
Fixed failure to parse dataset with an empty LUT Descriptor or Red/Green/Blue Palette Color LUT Descriptor element.
Made Dataset.save_as a wrapper for dcmwrite
Removed 1.2.840.10008.1.2.4.70 - JPEG Lossless (Process 14, SV1) from the Pillow pixel data handler as Pillow doesn’t support JPEG Lossless.
Fixed error when writing elements with a VR of OF
Fixed improper conversion when reading elements with a VR of OF
Fixed apply_voi_lut() and apply_modality_lut() not handling (0028,3006) LUT Data with a VR of OW
Fixed access to private creator tag in raw datasets
Fixed description of newly added known private tag
Fixed update of private blocks after deleting private creator
Fixed bug in updating pydicom.config.use_DS_Decimal flag in DS_decimal()
Major API overhaul
New classes for FASTA and FASTQ
Generate accessor and mutator functions for all classes
Generate man pages for all functions and macros
Export delimiter-separated-value class to libxtend
Classify ChIP/ATAC-Seq peaks based on features provided in a GFF
Peaks are provided in a BED file sorted by chromosome and position. The GFF
must be sorted by chromosome and position, with gene-level features separated
by ### tags and each gene organized into subfeatures such as transcripts and
exons. This is the default for common data sources.
Import sam_buff_t class and VCF functions from ad2vcf
Add BED and GFF support
Isolate headers under include/biolibc
Numerous small enhancements and fixes
Upstream change log: https://github.com/auerlab/biolibc/releases
