kpriv/pkgsrc
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
pkgsrc/geography/gdal-lib/Makefile.common

37 lines
1.2 KiB
Text
Raw Normal View History

gdal-lib: Force the use of openjpeg over jasper for JPEG 2000 support Add pkg-config to USE_TOOLS to ensure openjpeg gets detected properly. See https://github.com/mdadams/jasper/issues/208 bump PKGREVISION
2019-07-16 23:24:47 +02:00
# $NetBSD: Makefile.common,v 1.6 2019/07/16 21:24:47 nia Exp $
Update to 2.2.3 * Add Makefile.common to prepare for geography/py-gdal Changelog: = GDAL/OGR 2.2.3 Release Notes = The 2.2.3 release is a bug fix release. == Build == * nmake.opt: Ensure PDB is included in release DLL if WITH_PDB requested (#7055) == Port == * /vsicurl/ and derived filesystems: redirect ReadDir() to ReadDirEx() (#7045) * /vsicurl/: enable redirection optimization on signed URLs of Google Cloud Storage. Helps for the PLScenes driver (#7067) * /vsicurl/: fix 2.2 regression regarding retrieval of file size of FTP file (#7088) * /vsis3/: fix to avoid invalid content to be sent if Write() writes more than 50 MB in a single call (#7058) * /vsis3/: fix Seek(Tell(), SEEK_SET) fails if current position is not 0 (#7062) * /vsis3/: fix support of non-ASCII characters in keys (#7143, #7146) * Fix CPLCopyTree() that doesn't properly on MSVC 2015 (and possibly other platforms) (#7070) == Algorithms == * RPC transformer: set output coordinates to HUGE_VAL when failure occurs, so that a following coordinate transformation can detect the error too (#7090) * GDALGrid() with linear algorithm: avoid assertions/segmentation fault when GDALTriangulationFindFacetDirected() fails (#7101) * GDALComputeProximity(): fix int32 overflow when computing distances on large input datasets (#7102) * GDALResampleChunk32R_Gauss: fix potential out of bounds access (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4056) == GDAL utilities == * gdalwarp: fix issue with GDALAdjustValueToDataType(Float32, +/- inf) that didn't preserve infinity, which affected gdalwarp -dstnodata inf (#7097) * gdalwarp -crop_to_cutline: reduce number of iterations to find the appropriate densification (#7119) * gdal_contour: return with non-0 code if field creation or contour generation failed (#7147) == GDAL drivers == GeoRaster driver: * fix int overflow (#6999) GPKG driver: * speed-up statistics retrieval on non-Byte datasets (#7096) GRIB driver: * if GRIB_ADJUST_LONGITUDE_RANGE config option is set to YES, adjust the longitude range to be close to [-180,180] when possible for products whose left origin is close to 180deg. (#7103) GSAG driver: * avoid assertion on some files with nul characters (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3726) GTiff driver: * make sure that -co PHOTOMETRIC=RGB overrides the color interpretation of the first 3 bands of the source datasets (#7064) * on reading use GeogTOWGS84GeoKey to override the defaults TOWGS84 values coming from EPSG code (#7144) * fix potential crash when reading in RGBA mode with internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4157) * fix potential crash with old-jpeg compression and internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4180) HTTP driver: * do not open the underlying dataset with GDAL_OF_SHARED, to avoid later assertion JP2KAK driver: * fix unix build with Kakadu 7.A and later (#7048, #7081) JPEG driver: * Add compatibility with libjpeg-turbo 1.5.2 that honours max_memory_to_use (cf https://github.com/libjpeg-turbo/libjpeg-turbo/issues/162) JP2OpenJPEG driver: * Add support for openjpeg 2.3 (#7074) netCDF driver: * fix raster read as nodata with Byte datatype, (valid_range={0,255} or _Unsigned = True) and negative _FillValue (#7069) * be more tolerant on the formatting of standard parallel (space separated instead of {x,y,...} syntax), and accept up to 2/1000 error on spacing to consider a regular grid, to be able to read files provided by the national weather institute of Netherlands (KNMI) (#7086) PDF driver: * round to upper integer when computing a DPI such that page size remains within limits accepted by Acrobat (#7083) Sentinel2 driver: * add support for direct opening of .zip files of new safe_compact L1C products (#7085) VRT driver: * avoid error being emitted when opening a VRTRawRasterBand in a .zip files (#7056) == OGR core == * Fix OGR[Curve]Polygon::Intersects(OGRPoint*) to return true when point is on polygon boundary (#7091) * importFromWkt(): fix import of GEOMETRYCOLLECTION ending with POINT EMPTY or LINESTRING EMPTY (#7128, 2.1 regression) == OGR utilities == * ogrtindex: fix crash when using -f SQLITE -t_srs XXXX (#7053) == OGR drivers == Amigocloud driver: * Fixed data field types (https://github.com/OSGeo/gdal/pull/246) * Output list of datasets if dataset id is not provided. * Implemented waiting for job to complete on the server. This will improve write to AmigoCloud reliability. * Updated documentation. FileGDB driver: * remove erroneous ODsCCreateGeomFieldAfterCreateLayer capability declaration (https://github.com/OSGeo/gdal/pull/247) GeoJSON driver: * ESRIJson: recognize documents that lack geometry fields (#7071) * ESRIJson: recognize documents starting with a very long fieldAliases list (#7107) GeoRSS driver: * fix detection of field type (#7108) GML driver: * fix FORCE_SRS_DETECTION=YES effect on feature count and SRS reporting on gml files with .gfs (#7046) * do not try to open kml files (#7061) * GML geometry parsing: fix robustness issue with gml:PolyhedralSurface * do not report gml:name / gml:description of features as layer metadata GPKG driver: * do not try to update extent on gpkg_contents after GetExtent() on a empty layer of a datasource opened in read-only mode GTM driver: * fix null pointer dereference in case of read error (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4047) KML / LIBKML drivers: MDB driver: * fix multi-thread support (https://issues.qgis.org/issues/16039) MITAB driver * do not emit error if the .ind file is missing, just a debug message (#7094) MySQL: * fix build with MariaDB 10.2 (#7079) OCI driver: * initialize in multi-threaded compatible mode to fix QGIS related crashes (https://issues.qgis.org/issues/17311), and fix a few memleaks ODS driver: * avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050) PLScenes driver: * backport data/plscenesconf.json from trunk to add SkySatScene fields Shapefile driver: * Fix GetFeatureCount() to properly take into account spatial filter when attribute filter also in effect (#7123) SQLite/Spatialite driver: * don't invalidate statistics when running a PRAGMA (https://issues.qgis.org/issues/17424) * SQLite dialect: support SQLite 3.21, and LIKE, <>, IS NOT, IS NOT NULL, IS NULL and IS operators (#7149) XLS driver: * workaround opening filenames with incompatible character set on Windows (https://issues.qgis.org/issues/9301) XLSX driver: * fix non working detection of Date/Time fields in some documents (#7073) * fix opening of documents with x: namespace in xl/workbook.xml (#7110) * avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050) == SWIG bindings == * map GRA_Max, GRA_Min, GRA_Med, GRA_Q1 and GRA_Q3 (#7153) == Python bindings == * remove 'from . import _gdal_array' line from gdal_array.py that is not necessary with normal execution of the bindings, and cause errors with PyInstaller (#7044) = GDAL/OGR 2.2.2 Release Notes = The 2.2.2 release is a bug fix release. == Build == * Windows build: always build the PDF driver, even when none of poppler/podofo/pdfium are available, in which case it is write-only (#6938) * Compilation fixes on Ubuntu 16.04 with explicit --std=c++03 (#6952) == Port == * /vsigzip/: make Eof() detect end of stream when receive a Z_BUF_ERROR error. Fixes #6944. * Fix memleak in VSIGSFSHandler::GetFileList() * /vsigzip/: avoid trying to write a .gz.properties file on a /vsicurl/ file (#7016) * Fix registration of /vsicrypt/ file system handler when compiled as a plugin (#7030) * CPLStrtod(): parse string like '-1.#IND0000000' as NaN instead of -1 (#7031) == Algorithms == * Warper: when operating on single-band, skip target pixels whose source center pixel is nodata (2.2 regression, #7001) * Warper: avoid blocking when number of rows is 1 and NUM_THREADS > 1 (#7041). Also limit the number of threads so that each one processes at least 65536 pixels == GDAL core == * GDALGCPsToGeoTransform(): add GDAL_GCPS_TO_GEOTRANSFORM_APPROX_OK=YES and GDAL_GCPS_TO_GEOTRANSFORM_APPROX_THRESHOLD=threshold_in_pixel configuration option (#6995) * Fix issue with GDALProxyRasterBand::FlushCache() not flushing everything required (#7040) * RawDataset::IRasterIO(): don't assume all bands are RawRasterBand == GDAL utilities == * gdal2tiles.py: fix GDAL 2.2 regression where tilemapresource.xml was no longer generated (#6966) * gdal_translate/DefaultCreateCopy(): do not destroy target file in case of failed copy wen using -co APPEND_SUBDATASET=YES (#7019) * gdal_translate: make -b mask[,xx] use the appropriate band data type (#7028) == GDAL drivers == GeoRaster driver: * add support for GCP (#6973) GPKG driver: * do not error out if extent in gpkg_contents is present but invalid (#6976) * fix opening subdatasets with absolute filenames on Windows (https://issues.qgis.org/issues/16997) * fix possible assertion / corruption when creating a raster GeoPackage (#7022) GSAG driver: * fix reading issue that could cause a spurious 0 value to be read and shift all following values (#6992) GTiff driver: * fix reading subsampled JPEG-in-TIFF when block height = 8 (#6988) * when reading a COMPD_CS (and GTIFF_REPORT_COMPD_CS=YES), set the name from the GTCitationGeoKey (#7011) GTX driver: * do not emit error when opening with GDAL_PAM_ENABLED=NO (#6996) HF2 driver: * fix reading tiles that are 1-pixel wide (2.1 regression, #6949) IDRISI driver: * Fix memleak in IDRISI Open() ISIS3 driver: * make sure that -co USE_SRC_HISTORY=NO -co ADD_GDAL_HISTORY=NO results in remove of History section (#6968) * fix logic to initialize underlying GeoTIFF file in IWriteBlock(), and implement Fill() (#7040) JPEG2000 driver: * Fix build failure in C++03 mode on Jasper inclusion in RHEL 6 (#6951) * Fix build failure with recent Jasper that no longer define uchar JP2OpenJPEG driver: * Add support for building against OpenJPEG 2.2 (#7002) * fix performance issues with small images with very small tile size, such as some Sentinel2 quicklooks (#7012) * properly use the opj_set_decode_area() API (#7018) netCDF driver: * avoid vector vs raster variable confusion that prevents reading Sentinel3 datasets, and avoid invalid geolocation array to be reported (#6974) PDF driver: * add support for Poppler 0.58 (#7033) PDS driver: * fix parsing of labels with nested list constructs (2.2 regression, #6970) SRTMHGT driver: * recognizes the .hgt.gz extension (#7016) VRT driver: * avoid stack buffer read overflow with complex data type and scale = 0. (oss-fuzz#2468) * fix uninitialized buffer in areas without sources when using non pixel packed spacing (#6965) * Warped VRT: correctly take into account cutline for implicit overviews; also avoid serializing a duplicate CUTLINE warping options in warped .vrt (#6954) * Warped VRT: fix implicit overview when output geotransform is not the same as the transformer dst geotransform (#6972) * fix IGetDataCoverageStatus() in the case of non-simple sources, which unbreaks gdalenhance -equalize (#6987) == OGR core == * OGRCompondCurve::addCurveDirectly(): try reversing non contiguous curves (for GML/NAS) * OGR API SPY: fix the way we map dataset handles to variable name, to avoid invalid reuses of variable still active * OGRParseDate(): avoid false-positive time detection, in particular for GeoJSON (#7014) * OGRCurve::get_isClosed(): do not take into account M component (#7017) * OGRLineString::setPoint() and addPoint() with a OGRPoint* argument. properly takes into account ZM, Z and M dimensions (#7017) == OGRSpatialReference == * Fix OGRSpatialReference::IsSame() to return FALSE when comparing EPSG:3857 (web_mercator) and EPSG:3395 (WGS84 Mercator) (#7029) * importFromProj4(): implement import of Hotine Oblique Mercator Two Points Natural Origin, and fix OGRSpatialReference::Validate() for that formulation (#7042) == OGR utilities == * ogr2ogr: fix small memory leak when using -limit switch * ogr2ogr: make -f GMT work again (#6993) == OGR drivers == AVCBin driver: * fix 2.1 regression regarding attributes fetching (#6950) DXF driver: * fix reading files where INSERT is the last entity and DXF_MERGE_BLOCK_GEOMETRIES is false (#7006) * avoid segfault when creating a DXF file with only the 'blocks' layer (#6998) * fix potential null pointer dereference in PrepareLineStyle (#7038) GeoJSON driver: * fix 2.2 regression regarding lack of setting the FeatureCollection CRS on feature geometries (fixes https://github.com/r-spatial/sf/issues/449#issuecomment-319369945) * prevent infinite recursion when reading geocouch spatiallist with properties.properties.properties (#7036) GML driver: * fix field type detection logic to avoid a field with xsi:nil=true to be identified as a string (#7027) * JPGIS FGD v4: fix logic so that coordinate order reported is lon/lat (https://github.com/OSGeo/gdal/pull/241) GMLAS driver: * fix memleak in swe:DataRecord handling * avoid false-positive warning about attribute found in document but ignored according to configuration, when the attribute is actually not present in the document but has a default value (#6956) * (quick-and-not-prefect) fix to avoid a <xs:any> to prevent all other fields from being set (#6957) * get the srsName when set on the srsName of the gml:pos of a gml:Point (#6962) * CityGML related fixes: better take into account .xsd whose namespace prefix is not in the document, fix discovery of substitution elements, handle gYear data type (#6975) GPKG driver: * fix feature count after SetFeature() with sqlite < 3.7.8 (#6953) * do not take into account gpkg_contents for the extent of vector layers when it is present but invalid (#6976) * fix logic to detect FID on SQL result layer when several FID fields are selected (#7026) KML / LIBKML drivers: * read documents with an explicit kml prefix (#6981) MSSQLSpatial driver: * fix issues on x64 (#6930) * properly format DELETE statement with schema (#7039) OCI driver: * check view with unquoted table name (#5552) OpenFileGDB driver: * properly read GeneralPolygon with M component whose all values are set to NaN (#7017) OSM driver: * increase string buffer for osm XML files (#6964) Shapefile driver: * use VSIStatL() in Create() to properly work with /vsimem/ directories (#6991) * fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (#7031) SQLite/Spatialite driver: * escape integer primary key column name on table creation (#7007) == C# bindings == * Implement RasterIO extensions (#6915) == Python bindings == * fix reference count issue on gdal.VSIStatL() when it returns None, that can cause Python crashes if None refcount drops to zero * add C-API to create driver instances for use in SWIG (https://trac.osgeo.org/osgeo4w/ticket/466) == Security oriented fixes == Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them. (oss-fuzz#XXXX is a shortcut to https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=XXXX) * NAS: avoid assertion / null ptr deref and possibly other failures on corrupted files. (oss-fuzz#2366, oss-fuzz#2580) * NTF: avoid null ptr deref. Related to https://oss-fuzz.com/v2/testcase-detail/6527758899347456 * VSIMemHandle::Read(): avoid unwanted unsigned int overflow that cause a later heap buffer overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5227283000328192. * MBTiles: fix use after free. (oss-fuzz#2388) * netCDF: fix stack buffer overflow when building vector layers from netCDF variables if they don't have the expected number of dimensions. (oss-fuzz#2400) * DXF: prevent infinite loop / excessive memory allocation on truncated file. (#oss-fuzz#2403) * DXF: avoid excessive memory allocation. (oss-fuzz#2444) * XYZ: fix write heap-buffer-overflow (oss-fuzz#2426) * GTiff: make IGetDataCoverageStatus() properly set the current TIFF directory, and only implement it for 'normal' bands. To avoid heap buffer overflows. (oss-fuzz#2438) * GTiff: avoid null pointer derefrence when requested implicit overviews on corrupted JPEG-compressed file. (oss-fuzz#2441) * GTiff: avoid potential issue on corrupted files (oss-fuzz#2481) * GTiff: don't override member nBlocksPerRow member of GTiffJPEGOverviewBand with unrelated value, in case of single-strip case. (oss-fuzz#3020) * PCIDSK: for band interleave channel, correctly use pixel_offset in case it is different from pixel_size. (oss-fuzz#2440) * MITAB: fix skipping of blank lines when reading MIF geometries, and avoid potential very loop loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4516730841858048 * CPLKeywordParser: avoid potential heap buffer overflow read. (oss-fuzz#2706) * CPLKeywordParser: avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/5733173726019584 * morphFromESRI(): fixes heap-after-free uses. (oss-fuzz#2864) * OGR_VRT: fix null pointer dereference on GetExtent() on an invalid layer. (oss-fuzz#3017) * OGR_VRT: avoid crash on cyclic VRT. (oss-fuzz#3123) * OSM: avoid potential write heap buffer overflow on corrupted PBF files. (oss-fuzz#3022) * FAST: fix potential read heap buffer overflow. (oss-fuzz#3025) * CPG: avoid null pointer dereference on failed open. (oss-fuzz#3136) = GDAL/OGR 2.2.1 Release Notes = The 2.2.1 release is a bug fix release. == Build == * fix compilation without BIGTIFF_SUPPORT (#6890) * configure: detect if std::isnan() is available. Helps compilation on some MacOSX setups, combined together --without-cpp11. Refs https://github.com/macports/macports-ports/pull/480 * fix compilation against ancient Linux kernel headers (#6909) * fix detection of 64bit file API with clang 5 (#6912) * configure: use .exe extension when building with mingw64* toolchains (fixes #6919) * mongoDB: compilation fix on Windows == Port == * /vsicurl/: fix occasional inappropriate failures in Read() with some combinations of initial offset, file size and read request size (#6901) * Add a VSICurlClearCache() function (bound to SWIG as gdal.VSICurlClearCache()) to be able to clear /vsicurl/ related caches (#6937) == Algorithms == * GDALRasterize(): avoid hang in some cases with all_touched option (#5580) * gdal_rasterize: fix segfault when rasterizing onto a raster with RPC (#6922) == GDAL utilities == * ogr_merge.py: fix '-single -o out.shp in.shp' case (#6888) == GDAL drivers == AIGRID driver: * fix handling on raw 32-bit AIG blocks ENVISAT driver: * fix 2.2 regression in initialization of members of MerisL2FlagBand. (#6929) GeoRaster driver: * Fix memory allocation failure (#6884) * add support for JP2-F in BLOB compression (corrections on geo-reference) (#6861) GPKG driver: * avoid corruption of gpkg_tile_matrix when building overviews, down to a level where they are smaller than the tile size (#6932) GTIFF driver: * Internal libtiff: fix libtiff 4.0.8 regression regarding creating of single strip uncompressed TIFF files (#6924) netCDF driver: * add support for radian and microradian units for geostationnary projection (https://github.com/OSGeo/gdal/pull/220) NWT_GRC driver: * Fix handling of alpha values in GRC color table (#6905) * Handle case of 0-len GRC class names (#6907) VRT driver: * speed-up SerializeToXML() in case of big number of bands XYZ driver: * fix 2.2 regression where the driver hangs on some dataset with missing samples (#6934) == OGR utilities == * ogr2ogr/GDALVectorTranslate(): fix crash when using -f PDF -a_srs (#6920) == OGR drivers == GeoJSON driver: * ESRIJson: avoid endless looping on servers that don't support resultOffset (#6895) * ESRIJson: use 'latestWkid' in priority over 'wkid' when reading 'spatialReference' (https://github.com/OSGeo/gdal/pull/218) * GeoJSON writer: accept writing ZM or M geometry by dropping the M component (#6935) GPKG driver: * make driver robust to difference of cases between table_name in gpkg_contents/gpkg_geometry_columns and name in sqlite_master (#6916) MITAB driver: * recognize Reseau_National_Belge_1972 / EPSG:31370 on writing (#6903) MySQL driver: * fix compilation issue with Arch Linux and mariadb 10.1.23 (fixes #6899) PG driver: * do not be confused by a 'geometry' table in a non-PostGIS enabled database (#6896) PLScenes: * remove support for V0. Deprecate V1 API. Only Data V1 is supported ( #6933) == Perl bindings == * Backport the fix to #6142 Install man page according to GDALmake.opt if INSTALL_BASE is set. * always return something from non-void functions (#6898) == Python bindings == * Accept callback = 0 since SWIG generates it as the default argument of BandRasterIONumPy(). Fixes https://github.com/OSGeo/gdal/pull/219 * Fix 2.2 regression preventing use of callback function in Band.ComputeStatistics() (#6927) == Security oriented fixes == Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them (some have been found locally, so no related ticket) * Fix CPLErrorSetState to ensure it does not write beyond DEFAULT_LAST_ERR_MSG_SIZE and correctly null-terminates last message. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1529. * Open() and Stat() methods of a number of virtual file systems: check that the filename starts with the full prefix. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1543. * VRT pixel functions: fix crash with 'complex' when source count is < 2. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1549 * OpenAIR: fix potential out-of-buffer read if we need to ingest 30000 bytes * Several fixes in importFromWkb() and importFromWkt() * GDALDataset and GDALRasterBand::ReportError(): fix crash if dataset name has a % character * NASAKeywordHandler::SkipWhite(): fix out of bounds read * MITAB: ParseTABFileFields(): fix out of bounds read. * MITAB: ParseMIFHeader(): fix memory leak and out-of-bounds read on corrupted file * MITAB: ParseMIFHeader(): fix memory leaks on corrupted files * MITAB: avoid potentially veryyyy long loop when stroking arcs. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1644 * MITAB: avoid heap-buffer-overflow in MITABCoordSys2TABProjInfo(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1653 * OSARDataset::Open(): fix crash if pOpenInfo->fpL == NULL. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1565 * OGRESRIJSONReadPolygon(): fix crash in error code path * DXF: prevent null ptr deref and out-of-bounds read on corrupted file * DXF: TranslateSPLINE(): sanitize integer values read to avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1843 * KML::unregisterLayerIfMatchingThisNode(): use memmove() instead of memcpy(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1580 * KML: fix crash on weird structure with recursively empty folders. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1591 * KML: fix null ptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1634 * OGRCurveCollection::importBodyFromWkb(): fix potential crash if the curve in the collection has not the same dimension has the collection dimension. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1584 * OGRCompoundCurve::importFromWkb(): avoid potential stack overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5192348843638784 * TIGER: fix potential stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1586 and 2020 * TIGER: avoid potential infinite looping. Fixes https://oss-fuzz.com/v2/testcase-detail/4724815883665408 * VFK: avoid out-of-bounds read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1596 and 2074 * CPLHexToBinary(): avoid reading outside of hex2char array on on-ASCII input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1606 * OGR PDS: avoid int32 overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=160 * GeoRSS: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1611. * VSIArchiveFilesystemHandler::SplitFilename(): improve performance. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1597 * OGRGeometryFactory::organizePolygons(): fix crash on empty polygons. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1620 * JML: fix null pointer dereference on invalid files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1622 * Shape: prevent null ptr deref on truncated MultiPointM geometry. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1638 * /vsisubfile/: avoid Tell() to return negative values. And make VSIIngestFile() more robust to unsigned overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1642 * GTM: avoid useless recursive opening of files when provided with a gzip-compressed input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1650 * GTiff: fix heap-buffer-overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1677 * GTiff: avoid heap-buffer-overfow on corrupted State Plane citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2050 * GTiff: avoid potential stack buffer overflow on corrupted Imagine citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2105 * GTiff: prevent heap overflow and fix reading of multi-band band-interleaved 16/24-bit float datasets. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2264 * GTiff: fix potential infinite loop when parsing some 24-bit float denormalized numbers. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2315 * Internal libjson-c: fix stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1682 * ILI1/ILI2: fix null pointer dereference when opening filename ','. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1689 * ILI1: fix various crashes on corrupted files (including, but not limited to, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1760, 1784, 1926) * ILI2: use proper cast operator. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1813 * ILI2: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1993 * ILI2: fix crash due to unhandled exception. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2191 * morphFromESRI(): fix heap-use-after-free issue. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1690 * morphFromESRI(): prevent potential null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1783 and 1867 * SEGUKOOA: fix inversion of leap year that caused index-out-of-bound reading on day 366 of leap years (2.2 regression). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1698 * CPLParseXMLString(): make it error out on invalid XML file even under CPLTurnErrorIntoWarning() mode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1703. * GML / NAS: fix memory leak in error code path, and potential heap-buffer-read-overflow * NTF: fix various issues: heap & stack buffer-overflow, null ptr derefs, memory leaks. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1819 , 1820 , 1823, 1833, 1854, 1862, 1910, 1931, 1961, 1990, 1995, 1996, 2003, 2033, 2052, 2077, 2084, 2103, 2130, 2135, 2146, 2166, 2185, 2187, https://oss-fuzz.com/v2/testcase-detail/4696417694121984 * OGRCreateFromMultiPatch(): avoid assertion on NaN coordinates. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1930 * GXF: validate nGType to avoid later out-of-bound read in GXFReadRawScanlineFrom(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1975 * GXF: fix int overflow and avoid excessive memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2207 * DGN: prevent heap-buffer-overflow read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1988 * COSAR: fix leak of file descriptor. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2019 * ISO8211: prevent stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2022 * WEBP: prevent int32 overflow and too large memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2059 * IRIS: fix heap-buffer-overflow in some cases of nDataTypeCode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2069 * E00GRID: avoid heap and stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2090 , 2182, 2237, 2327 * VICAR: fix null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2116 * VICAR: avoid use-after-free and heap-buffer-overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/4825577427042304 * VICAR: fix potential endless loop on broken files. Fixes https://oss-fuzz.com/v2/testcase-detail/6261508172414976 * REC: fix nullptr deref * REC: fix potential stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2165 * GDALGetJPEG2000Structure() / DumpGeoTIFFBox(): fix memory leak. * DumpGeoTIFFBox(): reject GeoJP2 boxes with a TIFF with band_count > 1 * DumpJPK2CodeStream(): avoid potentially very long loop * GDALGetJPEG2000Structure(): avoid bad performance on corrupted JP2 boxes. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2214 * GPKG: fix potential heap-buffer overflow in GPkgHeaderFromWKB(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2150 * GPKG: fix potential null ptr deref. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2240 * GPKG: avoid potential division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2253 * SEGUKOOA: prevent read beyond end of buffer. (#6921) * SRP: avoid potential stack buffer overflow and excessive memory allocation/processing time * CPLUnixTimeToYMDHMS(): avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4949567697059840 * Selafin: fix double frees. Fixes https://oss-fuzz.com/v2/testcase-detail/6429713822121984 * CEOS: fix heap buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2259 * CEOS: fix memleak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2279 * FAST: avoid null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2290 * netCDF: avoid stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2302 GDAL/OGR 2.2.0 Release Notes = == In a nutshell... == * New GDAL/raster drivers: - DERIVED driver: read-support. Expose subdatasets in a a new metadata domain, called DERIVED_SUBDATASETS - JP2Lura driver: read/create support for JPEG-2000 images using Luratech JP2 Library - PRF: add read-only support for PHOTOMOD PRF file format driver (github #173) - RRASTER driver: read-support .grd/.gri files handled by the R 'raster' package (#6249) * New OGR/vector drivers: - CAD driver: read support for DWG R2000 files (GSoC 2016 project) - DGNv8 driver: read-write support for DGN 8.0 format (using Teigha ODA libraries) - GMLAS driver: read-write support. XML/GML driver driven by Application Schemas. * New utility script: ogrmerge.py to merge several vector datasets into a single one * New /vsigs/ and /vsigs_streaming/ virtual file systems to read Google Cloud Storage non-public files * Significantly improved drivers: - NWT_GRD: write support (#6533) - FileGDB/OpenFileGDB: add support to read curve geometries (#5890) - VRT derived band: add the capability to define pixel functions in Python - Add read support for RasterLite2 coverages in SQLite driver - GPKG: implement tiled gridded elevation data extension - ISIS3: add write support and improve read support * RFC 63: Add GDALRasterBand::GetDataCoverageStatus() and implement it in GTiff and VRT drivers https://trac.osgeo.org/gdal/wiki/rfc63_sparse_datasets_improvements * RFC 64: Triangle, Polyhedral surface and TIN https://trac.osgeo.org/gdal/wiki/rfc64_triangle_polyhedralsurface_tin ==> this RFC introduces potential backward incompatible behaviour. Consult MIGRATION_GUIDE.txt * RFC 66: OGR random layer read/write capabilities https://trac.osgeo.org/gdal/wiki/rfc66_randomlayerreadwrite * RFC 67: add null field state for OGR features, in addition to unset fields https://trac.osgeo.org/gdal/wiki/rfc67_nullfieldvalues ==> this RFC introduces potential backward incompatible behaviour. Consult MIGRATION_GUIDE.txt * Upgrade to EPSG database v9.0 (#6772) * Python bindings: Global Interpreter Lock (GIL) released before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open()) * Continued major efforts on sanitization of code base * Remove bridge and vb6 bindings (#6640) * GNM built by default == Installed files == * Removed: data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv * Added plscenesconf.json, gmlasconf.xsd == Backward compatibility issues == See MIGRATION_GUIDE.TXT == GDAL/OGR 2.2.0 - General Changes == Build(Unix): * improve detection of packaged libfyba (SOSI) --with-sosi, as in Ubuntu 16.04 (#6488) * Sort files in static library to make the build reproducible (#6520) * fix libqhull include path when it is /usr/local/include/libqhull (#6522) * FileGDB: compilation fix on Linux in C++11 mode * configure: make pdfium detection not fail if there are just warnings. And make configure fail if --with-pdfium was required but failed (#6653) * Make ./configure --with-xerces fail if not found * Don't install script documentation in INST_BIN (github #157) * configure: auto-detect webp without requiring explicit --with-webp * configure: use pkg-config for HDF5 detection so that works out of the box on recent Ubuntu * auto-detect JDK 8 on Ubuntu * MDB: allow libjvm.so to be dlopen'd with --with-jvm-lib=dlopen (Unix only, github #177) * configure: delete temporary directories on the mac * configure: make sure --with-macosx-framework is correctly defined * configure: error out if --with-ld-shared is specified (#6769) * configure: remove bashism. * configure: fix --without-mrf (#6811) * configure: take into account CXXFLAGS and LDFLAGS in a few more cases (cryptopp, podofo, libdap) * Vagrant: all lxc and Hyper-V provider support; use vagrant-cachier for package caching * configure: update DWG support to work with Teigha libraries * Internal libgeotiff: hide symbols in --with-hide-internal-symbols mode * Shape: do not export Shapelib symbols for builds --with-hide-internal-symbols (#6860) Build(Windows): * Try to avoid confusion between libqhull io.h and mingw own io.h (#6590) * update script to generate most recent Visual C++ project files (#6635) * fix broken and missing dependencies in makefile.vc * add a way to use an external zlib (github #171) * Rename makegdal_gen.bat to generate_vcxproj.bat * generate_vcxproj.bat: Set correct value of PlatformToolset property based on specified Visual C++ version. Add NMAKE command line parameters: MSVC_VER based on specified Visual C++ version, DEBUG=1 WITH_PDB=1 to Debug build configuration. * generate_vcxproj.bat: generate project for autotest/cpp (Ticket #6815) * Add WIN64=1 to NMAKE command line options. * Add HDF4_INCLUDE option (#6805) * Add MSVC compiler option /MP to build with parallel processes. * Add ZLIB_LIB missing from EXTERNAL_LIBS Build(all): * make Xerces 3.1 the minimal version * drop support for PostgreSQL client library older than 7.4, or non security maintained releases older than 8.1.4, 8.0.8, 7.4.13, 7.3.15 == GDAL 2.2.0 - Overview of Changes == Port: * Export VSICreateCachedFile() as CPL_DLL so as to enable building JP2KAK as a plugin * Added possibility to find GDAL_DATA path using INST_DATA definition without execution GDALAllRegister if GDAL_DATA placed in version named directory on Linux (#6543) * Unix filesystem: make error message about failed open to report the filename (#6545) * File finder: Remove hardcoded find location (/usr/local/share/gdal) (#6543) * Win32 filesystem handler: make Truncate() turn on sparse files when doing file extension * Add VSIFGetRangeStatusL() and VSISupportsSparseFiles() * GDAL_NO_HARDCODED_FIND compilation option (#6543,#6531) to block file open calls (for sandboxed systems) * Add VSIMallocAligned(), VSIMallocAlignedAuto() and VSIFreeAligned() APIs * /vsizip / /vsitar: support alternate syntax /vsitar/{/path/to/the/archive}/path/inside/the/tar/file so as not to be dependent on file extension and enable chaining * Optimize opening of /vsitar/my.tar.gz/my_single_file * /vsizip/ : support creating non-ASCII filenames inside a ZIP (#6631) * VSI Unix file system: fix behaviour in a+ mode that made MRF caching not work on Mac and other BSD systems * Fix deadlock at CPLWorkerThreadPool destruction (#6646) * Windows: honour GDAL_FILENAME_IS_UTF8 setting to call LoadLibraryW() (#6650) * CPLFormFilename(): always use / path separator for /vsimem, even on Windows * /vsimem/: add trick to limit the file size, so as to be able to test how drivers handle write errors * /vsimem/: fix potential crash when closing -different- handles pointing to the same file from different threads (#6683) * CPLHTTPFetch(): add MAX_FILE_SIZE option * CPLHTTPFetch(): add a CAINFO option to set the path to the CA bundle file. As a fallback also honour the CURL_CA_BUNDLE and SSL_CERT_FILE environment variables used by the curl binary, which makes this setting also available for /vsicurl/, /vsicurl_streaming/, /vsis3/ and /vsis3_streaming/ file systems (#6732) * CPLHTTPFetch(): don't disable peer certificate verification when doing https (#6734) * CPLHTTPFetch(): cleanly deal with multiple headers passed with HEADERS and separated with newlines * CPLHTTPFetch(): add a CONNECTTIMEOUT option * CPLHTTPFetch(): add a GDAL_HTTP_HEADER_FILE / HEADER_FILE option. * CPLHTTPSetOptions(): make redirection of POST requests to still be POST requests after redirection (#6849) * /vsicurl/: take CPL_VSIL_CURL_ALLOWED_EXTENSIONS into account even if GDAL_DISABLE_READDIR_ON_OPEN is defined (#6681) * /vsicurl/: get modification time if available from GET or HEAD results * /vsis3/: add a AWS_REQUEST_PAYER=requester configuration option (github #186) * CPLParseXMLString(): do not reset error state * Windows: fix GetFreeDiskSpace() * Fix GetDiskFreeSpace() on 32bit Linux to avoid 32bit overflow when free disk space is above 4 GB (#6750) * Fix CPLPrintUIntBig() to really print a unsigned value and not a signed one * Add CPL_HAS_GINT64, GINT64_MIN/MAX, GUINT64_MAX macros (#6747) * Add CPLGetConfigOptions(), CPLSetConfigOptions(), CPLGetThreadLocalConfigOptions() and CPLSetThreadLocalConfigOptions() to help improving compatibility between osgeo.gdal python bindings and rasterio (related to https://github.com/mapbox/rasterio/pull/969) * MiniXML serializer: fix potential buffer overflow. Core: * Proxy dataset: add consistency checks in (the unlikely) case the proxy and underlying dataset/bands would not share compatible characteristics * GDALPamDataset::TryLoadXML(): do not reset error context when parsing .aux.xml * PAM/VRT: only take into account <Entry> elements when deserializing a <ColorTable> * GDALCopyWords(): add fast copy path when src data type == dst data type == Int16 or UInt16 * GetVirtualMemAuto(): allow USE_DEFAULT_IMPLEMENTATION=NO to prevent the default implementation from being used * Nodata comparison: fix test when nodata is FLT_MIN or DBL_MIN (#6578) * GetHistogram() / ComputeRasterMinMax() / ComputeStatistics(): better deal with precision issues of nodata comparison on Float32 data type * Fast implementation of GDALRasterBand::ComputeStatistics() for GDT_Byte and GDT_UInt16 (including use of SSE2/AVX2) * Driver manage: If INST_DATA is not requested, do not check the GDAL_DATA variable. * Make sure that GDALSetCacheMax() initialize the raster block mutex (#6611) * External overview: fix incorrect overview building when generating from greater overview factors to lower ones, in compressed and single-band case (#6617) * Speed-up SSE2 implementation of GDALCopy4Words from float to byte/uint16/int16 * Add SSE2 and SSSE3 implementations of GDALCopyWords from Byte with 2,3 or 4 byte stride to packed byte * GDALCopyWords(): SSE2-accelerated Byte->Int32 and Byte->Float32 packed conversions * Fix GDALRasterBand::IRasterIO() on a VRT dataset that has resampled sources, on requests such as nXSize == nBufXSize but nXSize != dfXSize * GDALRasterBand::IRasterIO(): add small epsilon to floating-point srcX and srcY to avoid some numeric precision issues when rounding. * Add GDALRasterBand::GetActualBlockSize() (#1233) * Fix potential deadlock in multithreaded writing scenarios (#6661) * Fix thread-unsafe behaviour when using GetLockedBlock()/MarkDirty()/DropLock() lower level interfaces (#6665) * Fix multi-threading issues in read/write scenarios (#6684) * Resampled RasterIO(): so as to get consistent results, use band datatype as intermediate type if it is different from the buffer type * Add GDALIdentifyDriverEx() function (github #152) * GDALOpenInfo: add a papszAllowedDrivers member and fill it in GDALOpenEx() * GDALDefaultOverviews::BuildOverviews(): improve progress report * Average and mode overview/rasterio resampling: correct source pixel computation due to numerical precision issues when downsampling by an integral factor, and also in oversampling use cases (github #156) * Overview building: add experimental GDAL_OVR_PROPAGATE_NODATA config option that can be set to YES so that a nodata value in source samples will cause the target pixel to be zeroed. Only implemented for AVERAGE resampling right now * GDALValidateOptions(): fix check of min/max values * GMLJP2 v2: update to 2.0.1 corrigendum and add capability to set gml:RectifiedGrid/gmlcov:rangeType content. Set SRSNAME_FORMAT=OGC_URL by default when converting to GML. Add gml:boundedBy in gmljp2:GMLJP2RectifiedGridCoverage * GMLJP2 v2: ensure KML root node id unicity when converting annotations on the fly. When generating GML features, make sure that PREFIX and TARGET_NAMESPACE are unique when specifying several documents. Algorithms: * RPC transformer: speed-up DEM extraction by requesting and caching a larger buffer, instead of doing many queries of just a few pixels that can be costly with VRT for example * GDALDeserializeRPCTransformer(): for consistency, use the same default value as in GDALCreateRPCTransformer() if <PixErrThreshold> is missing (so use 0.1 instead of 0.25 as before) * TPS solver: when Armadillo fails sometimes, fallback to old method * GDALCreateGenImgProjTransformer2(): add SRC_APPROX_ERROR_IN_SRS_UNIT, SRC_APPROX_ERROR_IN_PIXEL, DST_APPROX_ERROR_IN_SRS_UNIT, DST_APPROX_ERROR_IN_PIXEL, REPROJECTION_APPROX_ERROR_IN_SRC_SRS_UNIT and REPROJECTION_APPROX_ERROR_IN_DST_SRS_UNIT transformer options, so as to be able to have approximate sub-transformers * Fix GDAL_CG_Create() to call GDALContourGenerator::Init() (#6491) * GDALContourGenerate(): handle the case where the nodata value is NaN (#6519) * GDALGridCreate(): fix hang in multi-threaded case when pfnProgress is NULL or GDALDummyProgress (#6552) * GDAL contour: fix incorrect oriented contour lines in some rare cases (#6563) * Warper: multiple performance improvements for cubic interpolation and uint16 data type * Warper: add SRC_ALPHA_MAX and DST_ALPHA_MAX warp options to control the maximum value of the alpha channel. Set now to 65535 for UInt16 (and 32767 for Int16), or to 2^NBITS-1. 255 used for other cases as before * Warper: avoid undefined behaviour when doing floating point to int conversion, that may trigger exception with some compilers (LLVM 8) (#6753) * OpenCL warper: update cubicConvolution to use same formula as CPU case (#6664) * OpenCL warper: fix compliance to the spec. Fix issues with NVidia opencl (#6624, #6669) * OpenCL warper: use GPU based over CPU based implementation when possible, use non-Intel OpenCL implementation when possible. Add BLACKLISTED_OPENCL_VENDOR and PREFERRED_OPENCL_VENDOR to customize choice of implementation Utilities: * gdalinfo -json: fix order of points in wgs84Extent.coordinates (github #166) * gdalwarp: do not densify cutlines by default when CUTLINE_BLEND_DIST is used (#6507) * gdalwarp: when -to RPC_DEM is specified, make -et default to 0 as documented (#6608) * gdalwarp: improve detection of source alpha band and auto-setting of target alpha band. Automatically set PHOTOMETRIC=RGB on target GeoTIFF when input colors are RGB * gdalwarp: add a -nosrcalpha option to wrap the alpha band as a regular band and not as the alpha band * gdalwarp: avoid cutline densification when no transform at all is involved (related to #6648) * gdalwarp: fix failure with cutline on a layer of touching polygons (#6694) * gdalwarp: allow to set UNIFIED_SRC_NODATA=NO to override the default that set it to YES * gdalwarp: fix -to SRC_METHOD=NO_GEOTRANSFORM -to DST_METHOD=NO_GEOTRANSFORM mode (#6721) * gdalwarp: add support for shifting the values of input DEM when source and/or target SRS references a proj.4 vertical datum shift grid * gdalwarp: fix crash when -multi and -to RPC_DEM are used together (#6869) * gdal_translate: when using -projwin with default nearest neighbour resampling, align on integer source pixels (#6610) * gdal_translate & gdalwarp: lower the default value of GDAL_MAX_DATASET_POOL_SIZE to 100 on MacOSX (#6604) * gdal_translate: avoid useless directory scanning on GeoTIFF files * gdal_translate: make "-a_nodata inf -ot Float32" work without warning * gdal_translate: set nodata value on vrtsource on scale / unscale / expand cases (github #199) * GDALTranslate(): make it possible to create a anonymous target VRT from a (anonymous) memory source * gdaldem: speed-up computations for src type = Byte/Int16/UInt16 and particularly for hillshade * gdaldem hillshade: add a -multidirectional option * GDALDEMProcessing() API: fix -alt support (#6847) * gdal_polygonize.py: explicitly set output layer geometry type to be polygon (#6530) * gdal_polygonize.py: add support for -b mask[,band_number] option to polygonize a mask band * gdal_rasterize: make sure -3d, -burn and -a are exclusive * gdal_rasterize: fix segfaults when rasterizing into an ungeoreferenced raster, or when doing 'gdal_rasterize my.shp my.tif' with a non existing my.tif (#6738) * gdal_rasterize: fix crash when rasterizing empty polygon (#6844) * gdal_grid: add a smoothing parameter to invdistnn algorithm (github #196) * gdal_retile.py: add a -overlap switch * gdal2tiles.py: do not crash on empty tiles generation (#6057) * gdal2tiles.py: handle requested tile at too low zoom to get any data (#6795) * gdal2tiles: fix handling of UTF-8 filenames (#6794) * gdal2xyz: use %d formatting for Int32/UInt32 data types (#6644) * gdal_edit.py: add -scale and -offset switches (#6833) * gdaltindex: emit warning in -src_srs_format WKT when WKT is too large * gdalbuildvrt: add a -oo switch to specify dataset open options Python samples: * add validate_cloud_optimized_geotiff.py * add validate_gpkg.py Multi-driver: * Add GEOREF_SOURCES open option / GDAL_GEOREF_SOURCES config. option to all JPEG2000 drivers and GTiff to control which sources of georeferencing can be used and their respective priority AIGRID driver: * fix 2.1.0 regression when reading statistics (.sta) file with only 3 values, and fix <2.1 behaviour to read them in LSB order (#6633) AAIGRID driver: * auto-detect Float64 when the nodata value is not representable in the Float32 range ADRG driver: * handle north and south polar zones (ZNA 9 and 18) (#6783) ASRP driver: * fix georeferencing of polar arc zone images (#6560) BPG driver: * declare GDALRegister_BPG as C exported for building as a plugin (#6693) DIMAP driver: * DIMAP: for DIMAP 2, read RPC from RPC_xxxxx.XML file (#6539) * DIMAP/Pleiades metadata reader: take into tiling to properly shift RPC (#6293) * add support for tiled DIMAP 2 datasets (#6293) DODS driver: * fix crash on URL that are not DODS servers (#6718) DTED driver: * correctly create files at latitudes -80, -75, -70 and -50 (#6859) ECW driver: * Add option ECW_ALWAYS_UPWARD=TRUE/FALSE to work around issues with "Downward" oriented images (#6516). ENVI driver: * on closing, pad image file with trailing nul bytes if needed (#6662) * add read/write support for rotated geotransform (#1778) GeoRaster driver: * fix report of rotation (#6593) * support for JP2-F compression (#6861) * support direct loading of JPEG-F when blocking=no (#6861) * default blocking increased from 256x256 to 512x512 (#6861) GPKG driver: * implement tiled gridded elevation data extension * add VERSION creation option * check if transaction COMMIT is successful (#6667) * fix crash on overview building on big overview factor (#6668) * fix crash when opening an empty raster with USE_TILE_EXTENT=YES * fix gpkg_zoom_other registration GTiff driver: * support SPARSE_OK=YES in CreateCopy() mode (and in update mode with the SPARSE_OK=YES open option), by actively detecting blocks filled with 0/nodata about to be written * When writing missing blocks (i.e. non SPARSE case), use the nodata value when defined. Otherwise fallback to 0 as before. * in FillEmptyTiles() (i.e. in the TIFF non-sparse mode), avoid writing zeroes to file so as to speed up file creation when filesystem supports ... sparse files * add write support for half-precision floating point (Float32 with NBITS=16) * handle storing (and reading) band color interpretation in GDAL internal metadata when it doesn't match the capabilities of the TIFF format, such as B,G,R ordering (#6651) * Fix RasterIO() reported when downsampling a RGBA JPEG compressed TIFF file (#6943) * Switch search order in GTIFGetOGISDefn() - Look for gdal_datum.csv before datum.csv (#6531) * optimize IWriteBlock() to avoid reloading tile/strip from disk in multiband contig/pixel-interleave layouts when all blocks are dirty * fix race between empty block filling logic and background compression threads when using Create() interface and NUM_THREADS creation option (#6582) * use VSIFTruncateL() to do file extension * optimize reading and writing of 1-bit rasters * fix detection of blocks larger than 2GB on opening on 32-bit builds * fix saving and loading band description (#6592) * avoid reading external metadata file that could be related to the target filename when using Create() or CreateCopy() (#6594) * do not generate erroneous ExtraSamples tag when translating from a RGB UInt16, without explicit PHOTOMETRIC=RGB (#6647) * do not set a PCSCitationGeoKey = 'LUnits = ...' as the PROJCS citation on reading * fix creating an image with the Create() interface with BLOCKYSIZE > image height (#6743) * fix so that GDAL_DISABLE_READDIR_ON_OPEN = NO / EMPTY_DIR is properly honoured and doesn't cause a useless directory listing * make setting GCPs when geotransform is already set work (with warning about unsetting the geotransform), and vice-versa) (#6751) * correctly detect error return of TIFFReadRGBATile() and TIFFReadRGBAStrip() * in the YCBCR RGBA interface case, only expose RGB bands, as the alpha is always 255 * don't check free disk space when outputting to /vsistdout/ (#6768) * make GetUnitType() use VERT_CS unit as a fallback (#6675) * in COPY_SRC_OVERVIEWS=YES mode, set nodata value on overview bands * read GCPs in ESRI <GeodataXform> .aux.xml * explicitly write YCbCrSubsampling tag, so as to avoid (latest version of) libtiff to try reading the first strip to guess it. Helps performance for cloud optimized geotiffs * map D_North_American_1927 datum citation name to OGC North_American_Datum_1927 so that datum is properly recognized (#6863) * Internal libtiff. Resync with CVS (post 4.0.7) * Internal libtiff: fix 1.11 regression that prevents from reading one-strip files that have no StripByteCounts tag (#6490) GRASS driver: * plugin configure: add support for GRASS 7.2 (#6785) * plugin makefile: do not clone datum tables and drivers (#2953) * use Rast_get_window/Rast_set_window for GRASS 7 (#6853) GRIB driver: * Add (minimalistic) support for template 4.15 needed to read Wide Area Forecast System (WAFS) products (#5768) * **Partial** resynchronization with degrib-2.0.3, mostly to get updated tables (related to #5768) * adds MRMS grib2 decoder table (http://www.nssl.noaa.gov/projects/mrms/operational/tables.php) (github #160) * enable PNG decoding on Unix (#5661, github #160) * remove explicitly JPEG2000 decompression through Jasper and use generic GDAL code so that other drivers can be triggered * fix a few crashes on malformed files GTX driver: * add a SHIFT_ORIGIN_IN_MINUS_180_PLUS_180 open option HDF4 driver: * Fixed erroneous type casting in HDF4Dataset::AnyTypeToDouble() that breaks reading georeferencing and other metadata HDF5 driver: * correct number of GCPs to avoid dummy trailing (0,0)->(0,0,0) and remove +180 offset applied to GCP longitude. Add instead a heuristics to determine if the product is crossing the antimeridian, and a HDF5_SHIFT_GCPX_BY_180 config option to be able to override the heuristics (#6666) HFA driver: * fix reading and writing of TOWGS84 parameters (github #132) * export overview type from HFA files to GDAL metadata as OVERVIEWS_ALGORITHM (github #135) * make .ige initialization use VSIFTruncateL() to be faster on Windows * add support for TMSO and HOM Variant A projections (#6615) * Add elevation units read from HFA files metadata (github #169) * set binning type properly according to layerType being thematic or not (#6854) Idrisi driver: * use geotransform of source dataset even if it doesn't have a SRS (#6727) * make Create() zero-initialize the .rst file (#6873) ILWIS driver: * avoid IniFile::Load() to set the bChanged flag, so as to avoid a rewrite of files when just opening datasets ISCE driver: * fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL 2.1.0 (#6556) * fix misbehaviour on big endian hosts * add support for reading and writing georeferencing (#6630, #6634) * make parsing of properties case insensitive (#6637) ISIS3 driver: * add write support * add mask band support on read * get label in json:ISIS3 metadata domain JPEGLS driver: JP2ECW driver: * fix crash when translating a Float64 raster (at least with SDK 3.3) JP2KAK driver: * add support for Kakadu v7.9. v7.8 should not be used. It has a bug fixed in v7.9 * catch exceptions in jp2_out.write_header() JP2OpenJPEG driver: * add a USE_TILE_AS_BLOCK=YES open option that can help with whole image conversion * prevent endless looping in openjpeg in case of short write * for single-line organized images, such as found in some GRIB2 JPEG2000 images, use a Wx1 block size to avoid huge performance issues (#6719) * ignore warnings related to empty tag-trees. JPIPKAK driver: * fix random crashes JPIP in multi-tread environment (#6809) KEA driver: * Add support for Get/SetLinearBinning (#6855) KMLSuperOverlay driver: * recognize simple document made of GroundOverlay (#6712) * Add FORMAT=AUTO option. Uses PNG for semi-transparent areas, else JPG. (#4745) LAN driver: * remove wrong byte-swapping for big-endian hosts MAP driver: * change logic to detect image file when its path is not absolute MBTiles driver: * on opening if detecting 3 bands, expose 4 bands since there might be transparent border tiles (#6836) * fix setting of minzoom when computing overviews out of order * do not open .mbtiles that contain vector tiles, which are not supported by the driver MEM driver: * disable R/W mutex for tiny performance increase in resampled RasterIO * add support for overviews * add support for mask bands MRF driver: * bug fix in PNG and JPEG codecs * Fixing a problem with setting NoData for MRFs generated with Create * fix plugin building (#6498) * rename CS variable so as to avoid build failure on Solaris 11 (#6559) * Allow MRF to write the data file directly to an S3 bucket. * Allow relative paths when MRF is open via the metadata string. * Add support for spacing (unused space) between tiles. Defaults to none. * Read a single LERC block as an MRF file. MSG driver: * fix incorrect georeference calculation for msg datasets (github #129) NetCDF driver: * add support for reading SRS from srid attribute when it exists and has content like urn:ogc:def:crs:EPSG::XXXX (#6613) * fix crash on datasets with 1D variable with 0 record (#6645) * fix erroneous detection of a non-longitude X axis as a longitude axis that caused a shift of 360m on the georeferencing (#6759) * read/write US_survey_foot unit for linear units of projection * apply 'add_offset' and 'scale_factor' on x and y variables when present, such as in files produced by NOAA from the new GOES-16 (GOES-R) satellite (github #200) * add a HONOUR_VALID_RANGE=YES/NO open option to control whether pixel values outside of the validity range should be set to the nodata value (#6857) * fix crash on int64/uint64 dimensions and variables, and add support for them (#6870) NITF driver: * add support for writing JPEG2000 compressed images with JP2OpenJPEG driver * fix writing with JP2KAK driver (emit codestream only instead of JP2 format) * fix setting of NBPR/NBPC/NPPBH/NPPBV fields for JPEG2000 (fixes #4322); in JP2ECW case, make sure that the default PROFILE=NPJE implies 1024 block size at the NITF level * implement creation of RPC00B TRE for RPC metadata in CreateCopy() mode * add support for reading&writing _rpc.txt files * nitf_spec.xml: Add support for MTIRPB TRE in NITF image segment. Also makes minor change to BLOCKA to include default values (github #127) * nitf_spec.xml: add IMASDA and IMRFCA TREs * GetFileList(): Small optimization to avoid useless file probing. NWT_GRD: * detect short writes OpenJPEG driver: * support direct extracting of GeoRaster JP2-F BLOB (#6861) PCIDSK driver: * handle Exceptions returned from destructor and check access rights in setters (github #183) PDF driver: * implement loading/saving of metadata from/into PAM (#6600) * implement reading from/writing to PAM for geotransform and projection (#6603) * prevent crashes on dataset reopening in case of short write PLScenes driver: * add a METADATA open option PostgisRaster driver: * fix potential crash when one tile has a lower number of bands than the max of the table (#6267) R driver: * fix out-of-memory (oom) with corrupt R file Raw drivers: * prevent crashes on dataset closing in case of short write RMF driver: * fix wrong counter decrement that caused compressed RMF to be incorrectly decompressed (github #153) * fix load/store inversion of cm and dm units in MTW files (github #162) * fix reading nodata for non-double data type (github #174) ROIPAC driver: * add support for reading/writing .flg files (#6504) * fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL >= 2.0.0 (#6591) * fix for big endian hosts RS2 driver: * add support for reading RPC from product.xml SAFE driver: * fix handling of SLC Products by providing access to measurements as subdatasets (#6514) Sentinel2 driver: * add support for new "Safe Compact" encoding of L1C products (fixes #6745) SQLite driver: * Add read support for RasterLite2 coverages in SQLite driver SRTMHGT driver: * open directly .hgt.zip files * accept filenames like NXXEYYY.SRTMGL1.hgt (#6614) * handle files for latitude >= 50 (#6840) VRT driver: * add default pixel functions: real, imag, complex, mod, phase, conj, etc... for complex data types (github #141) * avoid useless floating point values in SrcRect / DstRect (#6568) * avoid buffer initialization in RasterIO() when possible (replace ancient and likely broken concept of bEqualAreas) * make CheckCompatibleForDatasetIO() return FALSE on VRTDerivedRasterBands (#6599) * VRT warp: fix issue with partial blocks at the right/bottom and dest nodata values that are different per band (#6581) * fix performance issue when nodata set at band level and non-nearest resampling used (#6628) * VRTComplexSource: do temp computations on double to avoid precision issues when band data type is Int32/UInt32/CInt32/Float64/CFloat64 (#6642) * VRT derived band: add the capability to define pixel functions in Python * CreateCopy(): detect short writes * Fix linking error on VRTComplexSource::RasterIOInternal<float>() (#6748) * avoid recursion in xml:VRT metadata (#6767) * prevent 'Destination buffer too small' error when calling GetMetadata('xml:VRT') on a in-memory VRT copied from a VRT * fix 2.1 regression that can cause crash in VRTSimpleSource::GetFileList() (#6802) WMS driver: * Added support for open options to WMS minidrivers * Refactored the multi-http code to make it possible to do range requests. * Added a minidriver_mrf, which reads from remote MRFs using range requests. * Made the minidriver_arcgis work with an ImageService, not only MapService. * Added static cache of server response. * Allow tiledWMS to work in off-line mode by including the server response in the .wms file itself. * honour GDAL_HTTP_USERAGENT config option when it is set and <UserAgent> is missing (#6825) * WMS/WMTS: better deal with tiles with different band count (grayscale, gray+alpha, palette, rgb, rgba) (github #208) * Make HTTPS options apply to tWMS minidriver init, better HTTP error reporting WMTS driver: * do not take into account WGS84BoundingBox/BoundingBox that would be the result of the densified reprojection of the bbox of the most precise tile matrix * add TILEMATRIX / ZOOM_LEVEL open options * accept tiles of small dimensions (github #210) XYZ driver: == OGR 2.2.0 - Overview of Changes == Core: * Layer algebra: Add KEEP_LOWER_DIMENSION_GEOMETRIES=YES/NO option to Intersection, Union and Identity. Default is yes, but it is set to no unless result layer is of geom type unknown. If set to no, result features which would have lower dim geoms are skipped if operating on two layers with same geom dim. * Fix crash/corrupted values when running importFromWkb() on POLYGON M/POLYGON ZM geometries (#6562) * Add OGR_WKT_PRECISION config option that defaults to 15 to select the number of decimals when outputting to WKT * Make OGRFeature::SetField(string) accept JSon serialized arrays for the String/Integer/Integer64/RealList types; add reciprocal OGRFeature::GetFieldAsSerializedJSon() for those types * OGRGeometryFactory::transformWithOptions(): better deal with reprojection from polar projection to WGS84, and projections crossing the antimeridian to WGS84, by splitting geometries prior to reprojection (#6705) * LinearRing transformTo(): force last point to be identical to first one in case it is not. * GML geometry parsing: avoid 'Cannot add a compound curve inside a compound curve' error (#6777) * OGR SQL: fix IN filtering on MapInfo indexed columns (#6798) * OGR SQL: add support for LIMIT and OFFSET keywords * OGR SQL: add comparisons on date / datetime (#6810) * OGR SQL: increase efficiency of DISTINCT operator * OGREnvelope: change initialization to fix issue when getting MULTIPOINT(0 0,1 1) envelope (#6841) * OGRParse: fix parsing logic to avoid false positive detection of string as datetime (#6867) OGRSpatialReference: * Upgrade to EPSG database v9.0 (#6772) * OGRCT: upgrade LIBNAME of mingw and cygwin to libproj-9.dll and cygproj-9.dll to be up-to-date with proj 4.9.X (recommended method is using ./configure --with-static-proj4 instead) (#6501) * importFromESRI(): fix import of multi line MERCATOR SRS (#6523) * morphToESRI(): correctly compute standard_parallel_1 of Mercator(2SP) projection from scale factor of Mercator(1SP) (#6456, #4861) * exportToProj4(): recognize explicit OSR_USE_ETMERC=NO to avoid using etmerc with proj >= 4.9.3 * importFromProj4(): do not set a AUTHORITY node for strings like '+init=epsg:XXXX +some_param=val' * importFromProj4(): be robust with missing proj.4 epsg dictionary when importing '+init=epsg:xxxx +other_parm=value' * AutoIdentifyEPSG(): add identification of EPSG:3995 (Arctic Polar Stereographic on WGS84) and EPSG:3031 (Antarctic Polar Stereographic on WGS84) * OGRCoordinateTransformation: avoid potential bugs in proj.4 on NaN input * importFromEPSG(): take into account DX,DY,DZ,RX,RY,RZ,DS columns in pcs.csv to add per-PCS TOWGS84 overrides (geotiff #52) * Coordinate transformation: prevent unnecessary coordinate transformations (github #184, #185) Utilities: * ogr2ogr: do not return error on ogr2ogr --utility_version * ogr2ogr: keep -append and -overwrite when -update follows * ogr2ogr: fix heuristics to detect likely absence of '-f' (#6561) * ogr2ogr: do not emit a warning when updating/overwriting a destination dataset that is not a Shapefile and if -f is not specified (#6561) * ogr2ogr: make overwriting of lots of PG tables less prone to PostgreSQL 'out of shared memory' errors, by committing transactions after each table recreation * ogr2ogr: prevent 'ogr2ogr same.shp same.shp' (#1465) * ogr2ogr: add a -limit option to limit the number of features read per layer * GDALVectorTranslate(): imply update mode if access mode not explicitly specified and hDstDS != NULL (#6612) * ogrlineref: Fix crash if no new layer name provided * ogrtindex: add -src_srs_name, -src_srs_format and -t_srs option to allow indexing files that have different projections Multidriver: * PGeo/FileGDB/OpenFileGDB: OGRCreateFromShapeBin(): do not error out on empty lines/polygons * GPKG/SQLite/PG/FileGDB/MEM: properly set/reset field subtype with AlterFieldDefn() (#6689) * DXF, GeoJSON, GML, KML, LIBKML, ODS, Shape, XLSX: report operating system error if file creation fails (#2253) AmigoCloud driver: * add option to receive an AmigoCloud API key in the connection string, fix page size (github #137) Carto: * renamed from CartoDB * fix CartoDB'fication() by removing manual creation of the_geom_webmercator mercator, and also attach created sequence to table.cartodb_id (#6565) CSV driver: * add read/write support for String/Integer/Integer64/RealList types as serialized JSon arrays DGN driver: * avoid crash with -ftrapv on 250K_NF44NE_Area_V7.dgn (relates to #6806) * use coordinate delta encoding from the attribute records array (6806) DXF driver: * sanitize layer name on export to avoid forbidden characters * reader: convert GeometryCollection to MultiPoint, MultiLineString or MultiPolygon when possible (QGIS #10485) * add font of TEXT and MTEXT to OGR style string (https://github.com/OSGeo/gdal/pull/198) DWG driver: * compilation fixes with Teigha 4.2.2 ElasticSearch driver: * add support for ElasticSearch 5.0 and fix 2.X support (#6724) * implement translation from SQL to E.S. query language FileGDB driver: * add support to read curve geometries (#5890) * support opening '.' directory GeoJSON driver: * writer: add a RFC7946=YES creation option (#6705) * read and write 'name' and 'description' members at FeatureCollection level * fix field type detection when first value of a field is null (#6517) * improve/fix field type promotion * fix wrong behaviour when there's a 'id' at Feature level and 'id' or 'ID' field in properties (#6538) * in case top level id is a negative integer, put the value in a 'id' attribute (#6538) * ESRI Json reader: support multilinestring from esriGeometryPolyline * ESRI Json reader: do not set field width of 2147483647 (#6529) * ESRI Json reader: support reading M and ZM geometries * Add CPL_json_object_object_get() and use it, to avoid deprecation warnings on json_object_object_get() * TopoJSON reader: sanitize invalid polygons (such as found in the 'TopoJSON' layer of http://bl.ocks.org/mbostock/raw/4090846/us.json) * writer: fix segfaults on NULL geometry with -lco WRITE_BBOX=YES (#6698) * writer: fix crash if NATIVE_MEDIA_TYPE creation option is specified alone * Add support of TopoJSON without 'transform' element (github #192) * don't set SRS if 'crs' set to null (github #206) GML driver: * remove hack for CityGML regarding forcing srsDimension to 3 if not specified. Instead add a GML_SRS_DIMENSION_IF_MISSING config option that can be set to 3 if needed (#6597) * consider srsName with URL like 'http://www.opengis.net/def/crs/EPSG/0/' as following EPSG axis order. Add SWAP_COORDINATES=YES/NO/AUTO (and GML_SWAP_COORDINATES configuration option) to override all auto guessing (#6678) * add a SRSNAME_FORMAT=SHORT/OGC_URN/OGC_URL dataset creation option * OGR_G_ExportToGMLEx(): add a COORD_SWAP option * Writer: make ogr:FeatureCollection be a substitutionGroup of AbstractFeature for GML 3.2 output, so as to be compatible of GMLJP2 v2 * GML and NAS: improve error reporting (mention feature id and gml_id) when parsing fails because of geometry decoding error * GML and NAS: add GML_SKIP_CORRUPTED_FEATURES and NAS_SKIP_CORRUPTED_FEATURES configuration options to avoid stopping reading a layer because of a corrupted geometry * Add support for Japanese GML FGD v4.1 format (github #204) GPKG driver: * list all tables/views by default (useful for non spatial tables not registered as aspatial). Add ASPATIAL_VARIANT layer creation option to select the way how to create a non-spatial table as aspatial. * Create non-spatial tables by default conforming to GPKG v1.2 'attributes' * use OGR_CURRENT_DATE config option if defined as the value of the 'last_change' of column of 'gpkg_contents' so as to be able to have binary identical output * update last_change column when content of raster or vector tables has changed * do not emit error when running ExecuteSQL() with a spatial filter on an empty layer (#6639) * update schema to reflect CURRENT_TIMESTAMP -> 'now' changes (github #155) * better table and column quoting. * Robustify layer deletion (make it accessible through DROP TABLE xxx and DELLAYER:xxx syntax) and renaming * implement DeleteField(), AlterFieldDefn(), ReorderFields() * make GetExtent() save extent if not already cached * add special SQL 'RECOMPUTE EXTENT ON layer_name' to force recomputation of extent * check identifier unicity on layer creation * add possibility to disable foreign key check by setting OGR_GPKG_FOREIGN_KEY_CHECK=NO config option * add HasSpatialIndex(tblname,geomcolname) SQL function * don't show Spatialite vgpkg_ virtual tables (#6707) * SQLite/GPKG: add explicit error message when trying to open a read-only WAL-enabled database (#6776) * SQLite/GPKG: make sure when closing a WAL-enabled database opened in read-only mode to reopen it in read-write mode so that the -wal and -shm files are removed (#6776) * make GetFeature() works on non conformant tables that have no integer primary key field (#6799), and be robust to non standard column types * remove triggers related to metadata tables that cause issues and have been removed by latest revisions of the spec. * declare feature id column of features tables and tile pyramid user data tables as NOT NULL (#6807) * add a gpkg_ogr_contents table to store feature count. * speed-up GetFeatureCount() with only a spatial filter set * improve column recognition for SQL result layer * add/override ST_Transform() and SridFromAuthCRS() from Spatialite to make them work with gpkg_spatial_ref_sys * add ImportFromEPSG() * make ST_Min/MaxX/Y(), ST_SRID(), ST_GeometryType() and ST_IsEmpty() work with Spatialite geometries * improve performance of spatial index creation and use on multi-gigabyte databases * better support of spatial views, by adding a special behaviour if a column is named OGC_FID * avoid potential denial of services by adding LIMIT clauses * slightly more efficient implementation of GetExtent() if extent in gpkg_contents is empty * create a dummy 'ogr_empty_table' features table in case we have no 'features' or 'tiles' table, so as to be conformant with Req 17 of the GeoPackage specification * add DEFAULT '' to metadata column of gpkg_metadata table * accept opening a .gpkg without vector content and without gpkg_geometry_columns table in vector mode if we also open in update mode; remove capability of opening a .gpkg without vector content but with gpkg_geometry_columns table in vector mode if we only open in read-only mode; fix creation of a vector layer in a database if it initially lacks a gpkg_geometry_columns table * fix appending a raster to a vector database without pre-existing raster support tables * add minimalistic support for definition_12_063 column in gpkg_spatial_ref_sys, so that insertion of new SRS doesn't fail * use GEOMETRYCOLLECTION instead of GEOMCOLLECTION for SQL and gpkg_geometry_columns.geometry_type_name * do not warn if gpkg_metadata extension declared GPX driver: * ignore wpt/rtept/trkpt with empty content for lat or lon ILI driver: * ILI1: fix crash in JoinSurfaceLayer() when the multicurve of the feature of the poSurfaceLineLayer layer is empty (#6688) * ILI1: make polygon reconstruction in Surface layers robust to curves not in natural order (#6728) * ILI2: assign FID to features (#6839) * ILI2: fix crashing bug in Create() if model file not specified KML driver: * add a DOCUMENT_ID datasource creation option to set the id of the root <Document> node LIBKML driver: * fix crash when reading <gx:TimeStamp> or <gx:TimeSpan> elements (#6518) * add a DOCUMENT_ID datasource creation option to set the id of the root <Document> node * emit style related errors as warnings to make datasets openable by SWIG bindings (#6850) MITAB driver: * limit (width, precision) of numeric fields on creation to (20,16) for compatibility with MapInfo (#6392) * add support for oblique stereographic (#6598) * Adds the authority code for Irish national grid (Ireland_1965) (github #149) * fix spelling for Euref_89 and add EPSG code (#6816) MSSQLSpatial driver: * Fix bulk insert with table names containing spaces (#6527) * Build optional mssql plugin with SQL Native Client support for MSSQL Bulk Copy * Fix MSSQL select layer to recognize geometry column with sqlncli (#6641) MySQL driver: * fix spatial filtering on recent mysql by adding a SRID in the rectangle geometry * do not force NOT NULL constraint on geometry field if no spatial index is used NAS driver: * support multiple 'anlass' in updates NWT_GRD: * add write support (#6533) OCI driver: * Add options for faster feature loading (#6606) * add WORKSPACE open option * correctly handle OFTInteger64 case in loader layer (bug found by cppcheck multiCondition) * support for long identifiers (up to 128 long) when running of 12.2 or + (#6866) * OCILOB VSIL driver: new driver to streams in and out of Oracle BLOB as a GDAL large virtual file system (#6861) ODS driver: * fix FID filtering (#6788) OGDI driver: * make GetNextRawFeature() report an error when it is not end of layer * better error reporting when the layer list cannot be established * catch non-fatal OGDI errors emitted by OGDI 3.2.0 if OGDI_STOP_ON_ERROR environment variable is set to NO, and emit them as CPLError()s * display OGDI error message if opening fails OpenFileGDB driver: * do not error out on geometries that have a declared M array but that is missing (#6528) * add support to read curve geometries (#5890) * transcode UTF-16 strings found in column names, alias, etc... to UTF-8 (instead of using only their ASCII byte) (#6544) * do not emit an error on a empty table whose declaration has M settings (#6564) * support opening '.' directory * improve detection of some form of TINs from MULTIPATCH, and for MultiPatch layers, try to select a better geometry type for those layers (#5888) * fix bug when field description offset is beyond 4GB (#6830) OSM driver: * fix 'too many tags in relation' error when parsing .osm files * allow key=value entries in closed_ways_are_polygons= configuration (#6476) * allow OSM_SQLITE_CACHE config option to be greater than 2047 PG driver: * fix insertion of binary/bytea content in non-copy mode (#6566) * fix errors caused by missing geometry_columns/spatial_ref_sys tables in non PostGIS databases, that prevent reading more than 500 features (QGIS #10904) * avoid errors with field default expressions like 'foo'::text (#6872) PLScenes driver: * add HTTP retry logic (#6655) * V0 API: workaround limitations on filtering on image_statistics.image_quality (#6657) * add support for Data V1 API S57 driver: * Fix ogr2ogr -f S57 (#6549) * fix crashes if the s57objectclasses.csv resource file contains invalid lines * remove data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv and move their content into main s57attributes.csv and s57objectclasses.csv files (#6673) * Update s57 attributes and object classes according to s-57 reference (github #202) * add POSACC and QUAPOS fields for geometric primitive layers (github #205) SDE driver: * rename driver to OGR_SDE. Fix build (#6714) SEGY driver: * Accept SEGY files with ASCII headers that have nul terminated strings (#6674) Shapefile driver: * auto-repack by default at dataset closing and FlushCache()/SyncToDisk() time. Controlled by AUTO_REPACK open and layer creation options (that default to YES) * generate .dbf end-of-file 0x1A character by default. Add DBF_EOF_CHAR layer creation options / open options to control that behaviour * writing: use strerrno() for better error messages (QGIS #13468) * change REPACK implementation on Windows to be robust to remaining file descriptors openend on the .shp/.shx/.dbf (#6672, QGIS #15570) * Fix issue in DBFCloneEmpty() one a one field DBF not yet written to disk * add call to AutoIdentifyEPSG() when reading a .prj * support reading .dbf with substantial padding after last field definition. * when rewriting the geometry of the last record in the .shp, do it at the file offset it previously used (#6787) SOSI driver: * make registration of driver work again (2.1.0 regression) (#6500) * update to latest version of https://github.com/kartverket/gdal. Make SOSI driver support more geometry types (including curved geometries) plus provides some improvements on attribute-type mapping. Fix some memory errors/leaks. Disable by default non working creation code (#6503) SQLite/Spatialite driver: * do not emit error when running ExecuteSQL() with a spatial filter on an empty layer (#6639) * add read/write support for String/Integer/Integer64/RealList types as serialized JSon arrays * Spatialite: avoid crash when creating layer with geom_type = wkbCurve (fixes #6660) * Spatialite: do not report some BLOB columns as geometry columns of tables/views (when found before the geometry column(s)) (#6695, #6659) * fix update of features with multiple geometry columns (#6696) * speed-up dataset closing when creating a database with many spatial layers * Spatialite: avoid spatial views to cause layers 'layer_name(geometry_name)' to be publicly listed (#6740) * Spatialite: speed-up creation of database with INIT_WITH_EPSG=NO that is slow without transaction (not sure why as the table is empty...) * use AUTOINCREMENT for feature id column * allow OGR_SQLITE_CACHE to be set to > 2047 (MB) without overflowing * SQLite/GPKG: use SQLITE_OPEN_NOMUTEX flag to open databases. * GPKG/SQLite: fix ExecuteSQL() to work with a statement with 2 SELECT and ORDER BY clause (#6832) * SQLite/GPKG: change default page_size to 4096 bytes. * Update layer statistics for Spatialite 4 DB (#6838) * Remove traces of support of SQLite < 3.6.0 * SQLite dialect: properly quote column names when needed (github #214) VFK driver: * allow reading multiple VFK files into single DB support amendment VFK files * recreate DB in the case that it's outdated (VFK DB created by previous versions of GDAL) * allow reading DB as valid datasource (#6509) * new tables in backend SQLite database (geometry_columns/spatial_ref_sys) to enable reading DB datasource by SQLite driver * new configuration option OGR_VFK_DB_READ to enable reading backend database by VFK driver VRT driver: * add support for 'm' attribute in PointFromColumns mode (#6505) WFS driver: * invalidate underlying layer when SetIgnoredFields() is called (QGIS #15112) * don't crash on empty <Keyword/> declaration (#6586) * fix potential nullptr dereference on dataset without layer (github #179) XLSX driver: * only list worksheets (and no charts) as layers (#6680) == SWIG Language Bindings == All bindings: * map osr.GetTargetLinearUnits() (#6627) * allow wkbCurve/wkbSurface as valid values for the geom type of GeomFieldDefn object * map GDALIdentifyDriverEx() Java bindings: * Fix SWIG Java bindings for GNM (#6434) * Fix crash on GetDefaultHistogram() if the C++ method returns an error (#6812) Perl bindings: * return value always from GetGeomFieldIndex (#6506) * the Warp method requires a list of datasets (#6521) * when 'use bigint' is in effect, int var is a ref. * Separate the module building in the CPAN distribution and in the GDAL source tree. The CPAN distribution will be developed at https://github.com/ajolma * Fix the Extension method in Driver per RFC 46 * The Inv method of GeoTransform did not return a new object. Allow a single point in Apply method of GeoTransform. * Test for existence of PDL and require it if available. * allow decimation/replication in Piddle i/o. * support resampling in ReadTile. * Polygonize: Require explicit 8 for Connectedness to set 8-connectedness and allow 8CONNECTED as an option. * use Safefree to free memory allocated in Perl (#6796) Python bindings: * release the GIL before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open()) * add outputType option to gdal.Rasterize() * fix build issues when CXX is defined in the environment * gdal.VectorTranslate(): add spatSRS option * when enabling ogr.UseExceptions(), use the GDAL error message in the exception text (if available), when the exception is linked to an error value in the OGRErr return code * gdal.VectorTranslate(): accept a single string as value of the layers option (instead of iterating over each of its characters) * Regenerate Python bindings with SWIG 3.0.8 to avoid issue with Python 3.5. Add backward compatibility in Band.ComputeStatistics() to accept 0/1 as input instead of the expected bool value (#6749) * fix gdal.DEMProcessingOptions(zeroForFlat=True) (#6775) * fix 'import osgeo.gdal_array' with python3 and SWIG 3.0.10 (#6801) * allow gdal.FileFromMemBuffer() to use buffer > 2GB (#6828) * accept unicode strings as field name argument in Feature (like SetField, GetField, etc...) and FeatureDefn methods = GDAL/OGR 2.1.0 Release Notes = == In a nutshell... == * New GDAL/raster drivers: - CALS: read/write driver for CALS Type I rasters - DB2 driver: read/write support for DB2 database (Windows only) - ISCE: read/write driver (#5991) - MRF: read/write driver (#6342) - SAFE: read driver for ESA SENTINEL-1 SAR products (#6054) - SENTINEL2: read driver for ESA SENTINEL-2 L1B/LC1/L2A products - WMTS: read driver for OGC WMTS services * New OGR/vector drivers: - AmigoCloud: read/write support for AmigoCloud mapping platform - DB2 driver: read/write support for DB2 database (Windows only) - MongoDB: read/write driver - netCDF: read/write driver - VDV: read/write VDV-451/VDV-452 driver, with specialization for the Austrian official open government street graph format * Significantly improved drivers: - CSV: new options, editing capabilities of existing file - ElasticSearch: read support and support writing any geometry type - GeoJSON: editing capabilities of existing file, "native data" (RFC 60) support - MBTiles: add raster write support. fixes in open support - PDF: add PDFium library as a possible back-end. - PLScenes: add support for V1 API - VRT: on-the-fly pan-sharpening - GTiff: multi-threaded compression for some compression methods * Port library: add /vsis3/, /vsis3_streaming/, /vsicrypt/ virtual file systems * RFC 26: Add hash-set band block cache implementation for very larger rasters (WMS, WMTS, ...) http://trac.osgeo.org/gdal/wiki/rfc26_blockcache * RFC 48: Geographical networks support (GNM) https://trac.osgeo.org/gdal/wiki/rfc48_geographical_networks_support * RFC 58: Add DeleteNoDataValue(): https://trac.osgeo.org/gdal/wiki/rfc58_removing_dataset_nodata_value * RFC 59.1: Make GDAL/OGR utilities available as library functions: https://trac.osgeo.org/gdal/wiki/rfc59.1_utilities_as_a_library For gdalinfo, gdal_translate, gdalwarp, ogr2ogr, gdaldem, nearblack, gdalgrid, gdal_rasterize, gdalbuildvrt Available in C, Python, Perl and Java bindings. * RFC 60: Improved round-tripping in OGR https://trac.osgeo.org/gdal/wiki/rfc60_improved_roundtripping_in_ogr Implemented in GeoJSON driver * RFC 61: Support for measured geometries. https://trac.osgeo.org/gdal/wiki/rfc61_support_for_measured_geometries Implemented in Shapefile, PostgreSQL/PostGIS, PGDump, MEM, SQLite, GeoPackage, FileGDB, OpenFileGDB, CSV, VRT * Upgrade to EPSG database v8.8 * General sanitization pass to clean-up code, fix a lot of compiler warnings, as well as issues pointed by static code analyzers, such as Coverity Scan (credits to Kurt Schwehr for tackling a large part of them) or CLang Static Analyzer. * Fixes in a number of drivers to be more robust against corrupted files (most found with American Fuzzy Lop): RIK, INGR, Northwood, HF2, CEOS, GTiff, GXF, BMP, NITF, HFA, VRT, FIT, CEOS2, NWT_GRD/NWT_GRC, MITAB, RPFTOC, DBF/Shape, XYZ, VFK, DXF, NAS, GSAG, GS7BG, OpenFileGDB, RMF, AIGRID, OpenAIR, EHDR, ISO8211, FAST, USGSDEM, DGN, SGI, OpenJPEG, PCRaster, BSB, ADRG, SRP, JPEG, Leveller, VICAR, PCIDSK, XPM as well as in portability library (CPL), algorithms and raster core. * Driver removed: - Google Maps Engine (GME) (#6261) == New installed files == * gdal_pansharpen.py * data/vdv452.xml * data/vdv452.xsd * data/netcdf_config.xsd * /path/where/bash-completion/scripts/are/installed/gdal-bash-completion.sh == Backward compatibility issues == See MIGRATION_GUIDE.TXT == GDAL/OGR 2.1.0 - General Changes == Build(Unix): * Allow plugin for HDF4, HDF5, GeoRaster, FileGDB, OCI and PG * Add a --with-gnm option to enable GNM * Add --enable-lto switch to turn on link time optimization (GCC >= 5) * Add --with-spatialite=dlopen --with-spatialite-soname=libspatialite.so[.X] syntax to allow linking against spatialite through dlopen() mechanism (#6386) * Add a lot of warning options when available in the compiler. * Change detection test of podofo to work with podofo 0.9.3 * Fix gdalserver compilation with recent GNU libc (such as in Arch Linux) (#6073) * Add pkg-config support for libkml fork (#6077) * Update config.sub and config.guess to their latest upstream versions so as to be able to detect new architectures (android, ppcle64, etc..) (#6101) * Fix python package installation in custom prefixes (#4563) * configure: check that CXX is really a working compiler (#4436) * Build support for Kakadu 7.7 * thinplatespline.cpp: avoid using optimized version of VizGeorefSpline2DBase_func4() with ICC versions that fail on it (#6350) * Add bash completion for GDAL/OGR utilities and scripts (#6381) * Add support for ODBC autodetection with mingw64 (#6000) * Remove macos 9 and older support (#6133). * Remove dist_docs, burnpath and pszUpdableINST_DATA in gcore/gdaldrivermanager.cpp as they are have not been used for a while (#6140). * Removed --without-ogr configure flag (#6117). Always build with OGR. Build(Windows): * Add support for Visual Studio 2015 (a.k.a MSVC_VER=1900 or VC 14) * Add KAKFLAGS to nmake.opt with KDU_{MAJOR,MINOR,PATCH}_VERSION define-s. * nmake.opt: make it less error prone to define SETARGV when paths include spaces (#6032) * nmake.opt: add CL.EXE compiler option /FC to display full path of source code file in diagnostics * Remove WinCE support (#6133) * nmake.opt: add /I flag to the INSTALL command so that xcopy will be smart enough to create a directory when copying files and avoid prompting for user input (https://github.com/OSGeo/gdal/pull/95) * Enable mssql spatial blugin build, use BCP as the default option for the sqlncli enabled builds * Add support to find MrSID 9.5 dll * Fix Windows build with recent MySQL versions and Visual Studio 2015 (#6457) Build(all): * Compatibility with C++11 and C++14 * Optional crypto++ dependency for /vsicrypt/ support * Optional mongocxx dependency for MongoDB support Other: * Disable copy constructor and assignment operators in classes OGRFieldDefn, OGRGeomFieldDefn, OGRFeature, GDALMultiDomainMetadata, GDALDefaultOverviews, GDALOpenInfoGDALDataset, GDALRasterBlock, GDALRasterBand and GDALDriver (#6100) == GDAL 2.1.0 - Overview of Changes == Port: * Add /vsicrypt/ virtual file system for reading/creating/update encrypted files on the fly, with random access capabilities * Add /vsis3/ and /vsis3_streaming/ virtual file systems to read/write objects from AWS S3 buckets * /vsizip/: avoid returning the previous file list of an already opened .zip if it has changed afterwards (#6005) * /vsizip/: use CP437 on Windows when ICONV support is available (#6410) * /vsimem/: implement append mode (#6049) * /vsistdin/: fix caching of first 1MB / VRT driver: read XML content from /vsistdin/ in a streaming compatible way (#6061) * /vsistdout/: flush when closing the handle (#6149) * Add VSIReadDirEx() with a limit on the number of files before giving up and corresponding VSIFilesystemHandler::ReadDirEx() virtual function * cpl_vsil_cache: rework to be able to work on very large files on 32bit systems * Add CPLThreadLocaleC class to use thread-specific locale settings (when available) * VSIWin32Handle::Flush(): add VSI_FLUSH config option that can be set to TRUE to force FlushFileBuffers(). (hack related to #5556) * Fix VSIL append mode in Windows (#6065) * Make CPLCreateMutexEx() support CPL_MUTEX_REGULAR; and fix CPL_MUTEX_ADAPTIVE to avoid continuing in code path for creation of recursive mutex * Add CPLWorkerThreadPool API * Add CPLGetThreadLocalConfigOption() * Fix CPL_LSBINT16PTR() and CPL_LSBINT32PTR() to work with non-byte pointer (#6090) * CPLRecodeStub(): add special case for CP437 -> UTF-8 when input is only printable ASCII * CPLHTTPFetch(): add LOW_SPEED_TIME and LOW_SPEED_LIMIT options * CPLGetValueType(): improve heuristics to avoid detecting some WKB strings as real numbers (#6128) * Add CPL_SHA256* and CPL_HMAC_SHA256 functions * Make CPLVirtualMemFileMapNew() work on all systems having mmap(), and thus GeoTIFF GTIFF_VIRTUAL_MEM_IO optimization too * Add VSI_MALLOC_VERBOSE() and similar macros to output an error message in case of failed alloc * CPLParseXMLString(): limit depth of elements to 10000 * Win32 file management: handle files whose total file path length is greater than 255 characters in Open() and Stat() * Make CPLFormFilename(absolute_path, "..", NULL) truncate when possible * Add VSIGetDiskFreeSpace() * Implement CPLIsInf() for Solaris 11 * Add a CPL_FINAL macro that expands to C++11 final keyword when C++11 is enabled, and use it in a few places * Avoid CPLEscapeString (CPLES_URL) encoding some characters unnecessarily (#5526) * Add CPLTestBool(), CPLTestBoolean(), CPLFetchBool() * Fix CPLGetValueType() to recognize D1 as a string and not a real number (#6305) * CPLFormFilename(): use '/' directory separator on Windows on /vsicurl_streaming/ files (#6310) * Add hack enabled by -DDEBUG_BOOL to detect implicit int->bool conversions that dislike MSVC (#6325) * Add hack to '#define NULL nullptr' when NULL_AS_NULLPTR is defined. Must be used together with -std=c++11 -Wzero-as-null-pointer-constant with GCC to detect misuses of NULL/nullptr (#6327) * Add VSIError mechanism to store errors related to filesystem calls, and use it for /vsis3/. Add new CPLE_ error numbers. (https://github.com/OSGeo/gdal/pull/98) * Fix CPLsscanf() to be conformant regarding how space/tab/... separators are handled, so as to fix OGR GMT to be able to read coordinates separated by tabulations (#6453) Core: * Change default value of GDAL_CACHEMAX to 5% of usable physical RAM * Allow open options name to be prefixed by @ to be silently ignored when not existing in driver options (RFC 60) * Honour NBITS metadata item when doing RasterIO() with non-nearest resampling (#6024) * GDALClientServer: pass open options to INSTR_Open call * Improve performance of GDALCopyWords() float->byte/int16/uint16 by using SSE2 * Decorate prototypes of RasterIO() related operations with CPL_WARN_UNUSED_RESULT * Avoid deadlock when writing 2 datasets in 2 threads (#6163) * Remove obsolete symbols __pure_virtual(), GDALCreateProjDef(), GDALReprojectToLongLat(), GDALReprojectFromLongLat() and GDALDestroyProjDef() * Remove obsolete non-template-based implementation of GDALCopyWords(). We don't support anymore such compilers * GDALJP2Box::ReadBoxData(): do not abort if memory allocation fails * Add GDALAdjustValueToDataType() in C API, and use it in GDALTranslate() and GDALWarp() * GDALDefaultOverviews::CreateMaskBand(): fix logic error related to writing per-band mask * Block cache: fix excessive memory consumption when dealing with datasets with different block sizes (#6226) * EXIFExtractMetadata(): fix potential 1-byte write buffer stack overflow * MDReader: do no attempt reading side-car files on /vsisubfile/ (#6241) * GDALCheckBandCount(): allow by default a maximum number of bands to 65536 * GDALOpenInfo: add StealSiblingFiles() and AreSiblingFilesLoaded() methods * GDALOpenInfo::GetSiblingFiles(): give up after GDAL_READDIR_LIMIT_ON_OPEN (default=1000) files in the directory * GDALDefaultOverviews: add TransferSiblingFiles() method * GDALDriver::DefaultCreateCopy(): fix logic related to propagation of NBITS / PIXELTYPE metadata items as creation options * Reorder driver registration a bit so that formats with efficient identification are probed first * Add GDALIdentifyEnum (GDAL_IDENTIFY_UNKNOWN, GDAL_IDENTIFY_FALSE or GDAL_IDENTIFY_TRUE) for return values of Identify() * GDALLoadRPCFile(): load .rpc files from Ikonos products that have extra end-of-line character (#6341) * Export GDALRegenerateOverviewsMultiBand() symbol, but mostly for plugins (#6342) * Add GDAL_NO_AUTOLOAD to compile out the body of AutoLoadDriver (#6380) * Avoid ComputeStatistics(), GetHistogram() and ComputeRasterMinMax() to use only the first column of blocks in approximation mode for a raster whose shape of blocks is a square (#6378) * Add GDALGetDataTypeSizeBits() and GDALGetDataTypeSizeBytes(). * GDALReadOziMapFile(): fix reading Ozi external files from virtual file systems (https://github.com/OSGeo/gdal/pull/114) * Add CPLSetCurrentErrorHandlerCatchDebug() to enable custom error handlers not to intercept debug messages Algorithms: * RPC transformer: provide SSE2 accelerated transformer * RPC: fix off-by-half pixel computation of (pixel, line), and in bilinear and bicubic RPC DEM interpolation; fix off-by-one pixel registration for Pleiades RPC (#5993) * RPC: make RPCInverseTransformPoint() check convergence has been reached (#6162) * RPC DEM: optimize when DEM is in NAD83 or any other geodetic CS that transform as a no-op to WGS 84 * RPC DEM: take into account vertical datum of the DEM when present to convert DEM elevations to ellipsoidal heights (#6084) * RPC DEM: do fallback cubic -> bilinear and bilinear -> near on DEM edges * RPC DEM: improve inverse transformer to validate error threshold and improve convergence (#6162, #6377) * RPC: fix issues with dateline (#6472) * TPS solver: discard duplicated GCP to avoid avoidable error, warning when 2 GCPs have same (pixel,line) but different (X,Y) or the reverse * Warper: rework multithreaded computations to use a thread pool rather than forking threads each time * Warper: avoid really excessive processing time for some warping with target areas completely off the source raster (especially when involving RPC) (#6182) * Warper: CreateKernelMask(): fix potential 32 bit integer overflow when using warp memory value > 2GB (#6448) * gdal_grid: add linear interpolation algorithm * gdal_grid: add invdistnn algorithm, variation on the existing inverse distance weighting algorithm with quadtree to search for points only in the neighborhood (#6038) * gdal_grid: fix crash in optimized mode with GCC 4.4 on 64bit (#5987) * gdal_grid: compile gdalgrid AVX optimization for Windows when supported by compiler * Add GDALTriangulationXXXX() API through libqhull * Sieve filter: fix crash on nodata polygons (#6096) * Sieve filter: improvement to walk through the biggest neighbour chain until we find a polygon larger than the threshold (#6296) * GDALFPolygonize(): factor implementation with integer case * GDALComputeMedianCutPCT(): fix to make it work with rasters with more than 2 billion pixels (#6146) * Overview: Make average and gauss methods aware of transparent color table entries (#6371) Utilities: * gdalinfo: workaround bug in proj <= 4.9.1 on datasets with a SRS with a vertical shift grid (#6083) * gdal_translate: detect more reliably if specified bands are not in default order * gdal_translate: fix -a_nodata with negative values on rasters that have PIXELTYPE=SIGNEDBYTE; fix preserving PIXELTYPE=SIGNEDBYTE with VRT * gdal_translate: fix random behaviour when -scale is used without source bounds (#6455) * gdal_rasterize: do on-the-fly reprojection of input vector onto output raster * gdal_rasterize: rasterize: always create output with 1/2 cell buffer of input geometry envelope (#6058) * gdal_rasterize: add the -dialect option * gdal_rasterize: accept NaN as a valid value for -init / -burn (#6467) * gdalwarp: add -doo option to specify open options of (existing) output dataset * gdalwarp: if RPC_DEM warping option is specified, use exact transformer by default (#5993) * gdalwarp: make it honour nodata value of existing dataset (if -dstNoData isn't explicitly specified) * gdalwarp: do not propagate STATISTICS_ of second or following source datasets * gdalwarp: do not emit warning when using -cutline with a SRS and the source raster has RPC or GEOLOCATION * gdalwarp: check that the cutline is valid after transformation/reprojection * gdalwarp: better deal when dealing with a mix of RGB and RGBA datasets as input * gdalwarp: fix -srcnodata to not put garbage values as target dstnodata (#6315) * gdalwarp: densify cutline to avoid invalid geometry after reprojection to source raster, especially in the RPC case (#6375) * gdalserver: add a -nofork mode (Unix only for now), so that multiple clients can connect to the same dataset. Useful for safe 'concurrent' updates * add gdal_pansharpen.py script * gdal2tiles.py: fix error on a raster with less than 3 bands that resulted in a 'IndexError: list index out of range' * gdal2tiles.py: Change EPSG:3785 / EPSG:900913 into EPSG:3857 (#5622) * gdal2tiles.py: add Leaflet template (https://github.com/OSGeo/gdal/pull/71) * gdal2tiles.py: add -q switch for quiet mode * gdaldem: correctly deal with NaN as nodata value (#6066) * gdaldem color-relief: deal with the case of repeated entries with the same value and the input raster has pixels that match that value exactly (#6422) * gdaladdo: emit error message if passed an invalid dataset name (#6240) * gdaladdo: do not silence warnings when opening in update mode, if the open is successful * gdalbuildvrt: fix potential crash when using -b switch (#6095) * gdalbuildvrt: accept nan as value for -srcnodata and -vrtnodata * gdalbuildvrt: return non zero return code if the flush of the VRT to disk failed * gdal_merge.py: takes again into account -n flag (#6067) * gdalbuildvrt / gdal_translate / VRT: use floating point values for source and destination offsets and sizes (#6127) * gdalmove.py: fix to run with GDAL 2.0 gdal.InvGeoTransform() signature * gdal_calc.py: Add * from gdalnumeric to gdal_calc.py eval namespace again, to fix 2.0 regression that made for example 'log10(A)' to no longer work (https://github.com/OSGeo/gdal/pull/121) Python samples: * gdalpythonserver.py: update to protocol 3 AAIGRID: * when writing with floating-point values, ensure at least one value has a decimal point to avoid issues with some readers (#6060) BMP driver: * BMP: avoid too big color table allocation in case of wrong iClrUsed value BLX driver: * converted to support VirtualIO ECRG driver: * change subdataset definition to make sure that they only consist of frames of same scale (#6043) * fix base34 decoding and Windows filename handling (#6271) ECW driver: * use wide char Open API on Windows when GDAL_FILENAME_IS_UTF8=YES (https://github.com/OSGeo/gdal/pull/70) * fix reading of ECW in /vsi file systems (with SDK >= 4) (#6482) GeoRaster driver: * fix deflate read error depending on endianness (#6252) GIF driver: * libgif: partial resync with giflib master (but keep ABI of 4.1.6) * fix crash when CreateCopy a source with one color in the color table GMT driver: * validate raster dimensions on opening, and acquire mutex in dataset destructor GPKG driver: * write support: fix various issues in update scenarios when interacting with the GDAL block cache that could result in lost/corrupted band data to be written in tiles (#6309) * write support: fix potential use of freed sqlite temporary DB handle when generating overviews with partial tiles (#6335) * write support: fix potential crash in scenarios when block cache is full (#6365) * write support: fix inversion of row/column in one of the SQL request involved in partial tiles management (#6339) * fix generation of files with 1-band paletted input dataset. Also improve the logic to retrieve color palette when forcing BAND_COUNT=1 on opening (#6450) GTiff driver: * add NUM_THREADS creation and open options to enable multi-threaded compression * fix GTiffDataset::IsBlockAvailable() wrong behaviour when compiling against internal libtiff, when a BigTIFF file has a TileByteCounts with LONG/4-byte counts and not LONG8 (#6001) * Correctly take into account overridden linear units for a geotiff with a EPSG PCS code (#6210, #4954) * make VirtualMemIO() work with non native endianness * improve VirtualMemIO() performance in tiled Contig read to single band copy * improve single band tiled VirtualMemIO reading * improve DirectIO() to work on tiled uncompressed geotiff, for non-resampling and nearest resampling cases. Also improve performance of resampling cases on un-tiled files * fix DirectIO() mode with complex types and inverted endianness (#6198) * optimize writing of 12-bit values * implement lazy loading of .aux.xml and .tfw/.wld * Internal libtiff: update to CVS HEAD post libtiff 4.0.6 * make SetColorInterpretation(GCI_AlphaBand) work on a 5 band or more GeoTIFF (#6102) * automatically set PHOTOMETRIC=RGB if manually assigning color interpretation Red,Green,Blue to band 1,2,3 before directory crystallization (#6272) * add GEOTIFF_KEYS_FLAVOR=ESRI_PE creation option to write EPSG:3857 in a ESRI compatible way (#5924) * call XTIFFInitialize() in LibgeotiffOneTimeInit() as the former isn't thread-safe, so better call it from the later which is thread-safe (#6163) * fix reading and writing angular units different from degree (namely arc-second, arc-minute, grad, gon and radian) (#6171) * do not use VirtualMemIO optimization on compressed /vsimem/ files (#6195) * correctly set GTRasterTypeGeoKey=RasterPixelIsPoint if AREA_OR_POINT=Point but there is no SRS set (#6225) * improve a bit error detection when writing * check free space before writing (only for big, non sparse, uncompressed) * do not read large 'one row' JBIG compressed files with the scanline API (#6264) * Fix SetMetadata() to properly clear existing PAM metadata (complement to #5807) * prevent potential out of bounds read/write to TIFFTAG_EXTRASAMPLES (#6282) * do not use first directory as potential mask, to avoid assertion in GTiffDataset::SetDirectory() (#6287) * reject files with strips/tiles/scanlines bigger than 2 GB to avoid 32 bit integer overflow. Also in case of files with Contig PlanarConfiguration do not make reading one block for band 2 OK when reading for band 1 issued an error (#6288) * GTIFFWriteDirectory(): avoid memory leak of codec related memory (#2055) * Make ALPHA=NO in CreateCopy() cancel alpha color interpretation even if present in source raster * fix problem with implicit overviews of JPEG-compressed files (#6308) * fix compilation problem with internal libtiff if DEFER_STRILE_LOAD isn't defined (which is not the default configuration) (https://github.com/OSGeo/gdal/pull/90) * use more appropriate error message when 4GB threshold is reached with external overviews, and try to make doc of BIGTIFF=IF_NEEDED/IF_SAFER clearer (#6353) HDF4 driver: * Skip quotation mark when parsing HDF-EOS metadata. HDF5 driver: * add Komsat Mission ID to possible value for HDF5 SAR product (https://github.com/OSGeo/gdal/pull/103) HF2 driver: * fix reading side of the driver to work on architectures where char is unsigned, like PPC or ARM (#6082) HFA driver: * when reading projection, preserve EPSG code if AutoIdentifyEPSG() identified the SRS, even if a PE string is present (#6079) * keep TOWGS84 even when using ESRI PE string (#6158) * fix crashes on corrupted files (#6208, #6286) INGR driver: * check that RLE decoding produced the expected number of bytes and error out otherwise; test that 'random' line seeking actually works JPEGLS driver: * fix build error (#6430) JP2ECW driver: * honour psExtraArg->eResampleAlg when upsampling (#6022) JP2KAK driver: * honour psExtraArg->eResampleAlg when upsampling (#6022) * try rounded dimensions to decide if the data is being requested exactly at a sub-resolution * support v7.7 on Unix (additional changes potentially needed on Windows) JP2OpenJPEG driver: * Safer multi-threaded use JPEG driver: * add USE_INTERNAL_OVERVIEWS open option (default to TRUE as in recent versions, can be set to FALSE to hide internal overviews KMLSuperOverlay driver: * fix 2.0 regression with some RasterIO() requests involving resampling (#6311) * recognize datasets that have a intermediate <Folder> that forms a <Document><Folder><Region/><GroundOverlay/> structure (#6343) LCP driver: * fix wrong use of endianness macros to fix behaviour on big-endian hots Leveller driver: * update to v9 read support (#5632,) MBTiles driver: * add raster write support * fix so that datasets whose lowest min zoom level is 16 or above are recognized * be robust to invalid 'bounds' at dataset opening (#6458) MEM driver: * avoid 32 bit overflows NetCDF driver: * add support for reading NC4 unsigned short attributes and variables (#6337) * read correctly nodata values in [128,255] range for (unsigned) Byte data type (#6175) * implement Get/SetUnitType() using the standard units attribute (https://github.com/OSGeo/gdal/pull/96) * optimize IReadBlock() and CheckData() handling of partial blocks in the x axis by re-using the GDAL block buffer instead of allocating a new temporary buffer for each block (#5950) * full read/write support for new NetCDF4 types NC_UBYTE, NC_USHORT, NC_UINT and NC_STRING for variables (except for NC_STRING) and attributes (https://github.com/OSGeo/gdal/pull/99) * add support for the geostationary projection (#6030) * fix one byte heap write overflow in NCDFTokenizeArray() (#6231) * fix potential buffer overflows with uses of nc_inq_varname(), nc_inq_attname() and nc_get_att_text() (#6227) * validate that gridmapping:GeoTransform has 6 values (#6244) * fix wrong use of deallocator when writing a GEOLOCATION array, and other issues * limit number of bands reported to 32768 by default * validate raster dimensions * validate content of NC_GLOBAL#GDAL variable (#6245) NGSGEOID driver: * make it work on > 2GB file NITF driver: * data/nitf_spec.xml: Add CSCCGA, MENSRB, SENSRB, STREOB, ENGRDA, EXPLTB and PATCHB TREs (https://github.com/OSGeo/gdal/pull/81, #6285, https://github.com/OSGeo/gdal/pull/86) * fix parser to properly deal with variable length items not in first nesting level Northwoord driver: * fix computation of intermediate color table values on non-Intel platforms (#6091) NTv2 driver: * support reading/writing/appending to files with big-endian order (cf https://github.com/OSGeo/proj.4/issues/345) OpenJPEG driver: * do not expose block dimensions larger than dataset dimensions to avoid wasting memory (#6233) PCIDSK driver: * Remove the old driver (#6172) PDF driver: * add PDFium library as a possible back-end. Initial support contributed by Klokan Technologies GmbH (http://www.maptiler.com/) * workaround a bug of PoDoFo 0.9.0 by avoiding loading of vector content in raster-only mode (doesn't prevent the crash if reading the vector layers) (#6069) * make OGC BP registration work with media box where bottom_y is negative and top_y = 0 (in non rotated case) * make OGC BP registration work with media box where min_x != 0 (in non-rotated case) * correctly take into account non-meter linear units with OGC BP encoding, and add support for US FOOT (#6292) PDS driver: * change default values of PDS_SampleProjOffset_Shift and PDS_LineProjOffset_Shift to 0.5 (#5941) * fix nodata value for UInt16 to be 0 (#6064) * accept 'ODL_VERSION_ID = ODL3' in header (#6279) PGChip driver: * driver removed from sources PLScenes driver: * PLScenes V0: avoid opening raster to generate dummy .aux.xml PNG driver: * Support writing 1, 2 or 4 bit single band * Add NBITS creation option * fix XML of creation option list * Update internal libpng to 1.2.56 PostgisRaster driver: * avoid Identify() to recognize OGR PostgreSQL connection strings with schemas option and cause loud 'QuietDelete' (#6034) Raw drivers: * better support for direct read of more than 2GB in single gulp (untested though) RIK driver: * fix Identify() method to recognize again non-RIK3 RIK datasets (#6078) RMF driver: * implement GetNoDataValue() * add read/write access to new RMF format for files larger than 4 Gb (version=0x201) (https://github.com/OSGeo/gdal/pull/11) ROIPAC driver: * Support offset and scale band (#6189) VICAR driver: * change PDS_SampleProjOffset_Shift and PDS_LineProjOffset_Shift default values to 0.5 (#5941) * fix loss of precision in scale and offset VRT driver: * expose implicit 'virtual' overviews for VRT whose bands are made of a single SimpleSource/ComplexSource * gdalvrt.xsd: Add capitalized versions of true and false (#6014) * GetSingleSimpleSource(): check there's a single source (#6025) * honour VRTRasterBand NBITS metadata with SimpleSource and ComplexSource * properly take into account nodata value declared at VRT band level when doing resampling with non-nearest * honour relativeToVRT when using AddBand() to add a VRTRawRasterBand (https://github.com/OSGeo/gdal/pull/67) * VRT warp: fix crash with implicit overviews and destination alpha band (#6081) * make GetDefaultHistogram() on a sourced raster band save the result in the VRT (#6088) * serialize NODATA and NoDataValue items with %.16g, e.g. so as to be able to hold large int32 nodata values (#6151) * VRTSourcedRasterBand: make ComputeRasterMinMax() and ComputeStatistics() forward bApproxOK to overview band (useful for implicit overviews) * make CreateCopy() preserve NBITS metadata item * avoid loading sibling file list if not available * VRT raw: don't truncate last figure of ImageOffset if there are left space padding (#6290) * VRTWarpedDataset::SetMetadataItem(): fix crash when calling with name=SrcOvrLevel and value=NULL (#6397) * Warped VRT: fix deadlock in situation where warped VRT datasets are read in multiple threads and the block cache reaches saturation. Also add a GDAL_ENABLE_READ_WRITE_MUTEX config option that can be set to NO to disable the read/write mutex mechanism in cases where it would deadlock (#6400) WMS driver: * add a IIP (Internet Imaging Protocol) minidriver * limit number of zoom levels for ArcGIS MapServer JSon (#6186) * determine a resolution that will not result in a number that is larger than the maximum size of an integer. Any value that exceeds the maximum size of an integer will raise an invalid dataset dimensions error. (https://github.com/OSGeo/gdal/pull/89) XYZ driver: * be more robust to not exactly equal X and Y spacing (#6461) == OGR 2.1.0 - Overview of Changes == Core: * Add OGREditableLayer class to add editing capabilities to drivers with none or limited editing capabilities * OGRGeometry: add DelaunayTriangulation() method (GEOS >= 3.4) * OGRGeometry and derived classes: implement copy constructor and assignment operator (#5990) * OGRGeometry: Fix result of Equals on POINT EMPTY with POINT(0 0) * OGRFeature SetField(): more type conversions allowed, particularly with array types * OGRFeature::SetGeometry()/SetGeometryDirectly(): make it work when passed geometry is the currently installed geometry (#6312) * OGR SQL: do not silently skip NULL values in the first records when evaluating a SELECT DISTINCT (#6020) * OGR SQL: correctly sort NULL values in first positions (#6155) * OGR SQL: fix CAST(x AS bigint) to return an evaluated int64 node, and not int32 (#6479) * OGR SQL: handle 'fid' as Integer64 in where clause, and allow CAST(fid AS bigint) in selected columns (#6484) * Add OGRUpdateFieldType() * Decorate a few functions/methods of the OGR layer API with CPL_WARN_UNUSED_RESULT * WKT export: use 15 significant figures, instead of 15 figures after decimal point (#6145) * WKT export: do not append .0 after non-finite values (#6319) * Fix typo in definition of name of OGR_FD_ReorderFieldDefns (final s was missing) * OGRLayer::SetIgnoredFields(): properly reset state of non first geometry fields (#6283) * Make OGRLayer::SetSpatialFilter(GetSpatialFilter()) work with non empty spatial filter (#6284) * OGRLayerDecorator: add missing CreateGeomField() * OGRLayer::Erase(): do not discard input geometries that have no intersection with method layer (#6322) * OGRLayer::Erase(): Speedup = ~70% * Add OGRPreparedGeometryContains() * Use prepared geometry intersects as pretest in layer Intersection, Union, and Identity methods if requested. Use prepared geometry containment as pretest in layer Intersection method if requested. * Bail out from layer algebra methods if GEOS calls fail and not SKIP_FAILURES. * OGR_G_SetPoints(): error out if padfX or padfY == NULL, do not change coordinate dimension to 3D when pabyZ == NULL, fix optimization on linestring to call setPoints() only if the strides are the ones of a double, not 0 as incorrectly done before (#6344) * OGRParseDate(): more strict validation to reject invalide dates (#6452) OGRSpatialReference: * Upgrade to EPSG database v8.8 * Add support for SCH (Spherical Cross-track Height) projection * Optimize reprojection typically between WGS84 based SRS and WebMercator * Correctly transform Mercator_2SP and _1SP to ESRI Mercator, and back from ESRI Mercator to Mercator_2SP (#4861) * No longer enforce C locale if running against latest proj that is locale safe (4.9.2 or later) * EPSGGetPCSInfo(): use pcs.override.csv in priority over pcs.csv to read projection name, UOM, UOMAngle, GeogCS, etc... (#6026) * morphToESRI(): use GCS_WGS_1972 as GCS name for EPSG:4322 (#6027) * morphToESRI(): use Mercator_Auxiliary_Sphere projection for EPSG:3857. morphFromESRI(): map Mercator_Auxiliary_Sphere to EPSG:3857 (#5924) * Align hard-coded WKT of well known GCS definitions of WGS84, WGS72, NAD27 and NAD83 with the WKT of their EPSG def (#6080) * morphFromESRI(): special case with PROJCS name 'WGS_84_Pseudo_Mercator' (#6134) * OSR C API: fix declarations of OSRSetAxes() and OSRSetWagner(), and add missing OSRSetHOMAC(), OSRSetMercator2SP() and OSRSetTPED() (#6183) * Recognize EPSG 9835 method (Lambert Cylindrical Equal Area (Ellipsoidal)), needed for EPSG:6933 PCS for example * importFromProj4/exportToProj4(): rework linear unit conversion between WKT name/values and proj4 unit name, and extend its scope in WKT to proj4 conversions * OSR ESRI .prj: add support for reading custom ellipsoid in Parameters line * on import of +proj=geos, if +sweep=x is used then store it as a proj4 extension node (#6030) Utilities: * ogrinfo / ogr2ogr: implement @filename syntax for -sql and -where * ogr2ogr: prevent the -gt setting from overriding transaction group size of 1 set by skipfailures earlier (#2409) * ogr2ogr: warn if -zfield field does not exist in source layer * ogr2ogr -skip: rollback dataset transaction in case of failure (#6328) * ogr2ogr: fix -append with a source dataset with a mix of existing and non existing layers in the target datasource (#6345) * ogr2ogr: imply quiet mode if /vsistdout/ is used as destination filename * ogr2ogr: make -dim and -nlt support measure geometry types CartoDB: * fix GetNextFeature() on a newly create table (#6109) * defer 'CartoDBfycation' at layer closing * optimize feature insertion with multiple rows INSERT CSV driver: * add editing capabilities of existing files * add X_POSSIBLE_NAMES, Y_POSSIBLE_NAMES, Z_POSSIBLE_NAMES, GEOM_POSSIBLE_NAMES and KEEP_GEOM_COLUMNS open options * add HEADERS open option to force OGR to handle numeric column names. (PR #63) * add EMPTY_STRING_AS_NULL=YES/NO open option * implement compatibility enhancements for GeoCSV specification (#5989) * fix detection of TAB delimiter in allCountries.csv when the first line has a comma (#6086) * fix issues with leading single quote, and missing first line after ResetReading(), when parsing allCountries.txt (#6087) * speed-up GetFeatureCount() on allCountries.txt * on CreateDataSource() with a .csv name, do not try to open other existing .csv files in the directory * make CreateGeomField() returns OGRERR_NONE in case of success instead of OGRERR_FAILURE (#6280) * avoid adding trailing comma in header line when writing 'WKT,a_single_field' DGN driver: * add partial 3D transformation support for cell headers DXF driver: * detect files without .dxf extension (#5994) * fix handling of ELLIPSE with Z extrusion axis = -1 (#5705) * take into account full definition of spline entity (degree, control points, weights and knots) when stroking splines (#6436) * better handling of various object coordinate systems found in dxf files for point, line, polyline, spline and ellipse entities. Add anchor position to text styles. Remove polygon/polyface mesh parsing from polyline entity (#6459) ElasticSearch driver: * use get /_stats instead of /_status for ElasticSearch 2.0 compatibility (#6346) FileGDB driver: * make CreateFeature() honour user set FID, and implement more fine grained transaction for Linux/Unix * give a hint of using FileGDB SDK 1.4 is FileGDB compression is used GeoJSON driver: * Add editing capabilities of existing files * Add ARRAY_AS_STRING=YES open option * Use '%.17g' formatting by default for floating-point numbers and add SIGNIFICANT_FIGURES layer creation option (#6291) * add a json_ex_get_object_by_path() function * fix crash on null / non-json object features (#6166) * serialize string values that are valid JSon dictionary or array as it (ie do not quote them) * make sure there's enough space to write the FeatureCollection bbox (#6262). Also avoid duplicating FeatureCollection bbox if source has one (trunk only) * Export POINT EMPTY as having a null geometry, instead as of being POINT(0 0) (#6349) * Do not 'promote' a null field to OFTString type if it had another type before (#6351) GME driver: * Driver removed. Maps Engine being shut down at the end of January 2016. GML driver: * VFR: add new attribute DatumVzniku (v1.6) * VFR: fix ST_UVOH type handling * VFR: fix ZpusobyOchrany attributes (data types and names) * VFR: fix CisloDomovni attributes (Integer->IntegerList) * VFR: fix TEA attributes of StavebniObjekty * add NAMESPACE_DECL=YES option to OGR_G_ExportToGMLEx() to add xmlns:gml=http://www.opengis.net/gml or http://www.opengis.net/gml/3.2 declaration; Also accept GML2 or GML32 as valid valiues for FORMAT option (#6214) * serialize in .gfs file the name of the geometry element when it is 'geometry' since this is a particular case (#6247) * fix logic error in BuildJointClassFromScannedSchema() (#6302) GPKG driver: * make it accept files with non standard extension if they still have the correct application_id (#6396); also accept the .gpkx extension that may be used for extended geopackages * emit warning when generating a database without .gpkg/.gpkx extension (#6396) * as GPKG 1.1 uses a different application_id, emit a more specific warning if the application id starts with GPxx (but is not GP10). Add GPKG_WARN_UNRECOGNIZED_APPLICATION_ID config option to avoid the warning * correct scope of gpkg_geom_XXXXX extensions to be read-write, and allow reading geometry types CURVE or SURFACE * avoid trying to insert a gpkg_geom_XXXX extension if already done (#6402) * writer: implement strategy to flush partial_tiles temporary database when it becomes too big (#6462) * writer: when writing to GoogleMapsCompatible tiling scheme, better deal with source rasters in EPSG:4326 with latitude = +/-90 (#6463) * fix generation of files with 1-band paletted input dataset. Also improve the logic to retrieve color palette when forcing BAND_COUNT=1 on opening (#6450) GPX driver: * fix crash when parsing a 'time' extension element at route/track level (2.0 regression, #6237) ILI driver: * ILI1: Support for Surface polygon rings spread over multiple geometry records * ILI1: add string TID support (https://github.com/OSGeo/gdal/pull/91) * Fix crash with models using types derived from INTERLIS * Fix memory leaks (#6178) JML driver: * remove arbitrary limitation preventing from reading geometries with <gml:coordinates> larger than 10 MB (#6338) KML driver: * fix crash on KML files without content but with nested folders (#6486) LIBKML driver: * for documents without folder, use document name when available as name of layer (#6409) Memory driver: * add support for sparse feature IDs * add ADVERTIZE_UTF8 layer creation option MITAB driver: * Add support for block sizes other than 512 bytes in .map files, for MapInfo 15.2 compatibility (#6298) * write correct datum id for EPSG:3857 * read MID files with TAB delimiter and empty first field (#5405) * use projection code 29 when exporting non-Polar Lambert Azimuthal Equal Area (#5220) * fix crashes when parsing invalid MIF geometries (#6273) MSSQLSpatial driver: * Implement MSSQL bulk insert (#4792) * do not treat a primary key that is not of integer type as the FID (#6235) NTF driver: * fix potential buffer overflows when reading too short lines (#6277) ODBC driver: * remove limitations to 500 columns ODS driver: * fix loss of precision in formula computation OpenFileGDB driver: * do not emit warning if SDC/CDF table detected and that FileGDB driver is present * fix min/max on columns without indices (#6150) * build correct geometry for a multi-part wkbMultiLineStringZ (#6332) * add support for reading SHPT_GENERALPOINT (#6478) OSM driver: * correct fields ids for the (non frequently used) Node message * do not override 'our' osm_id (the node, way or relation id) with a tag named 'osm_id' (#6347) * properly deal with polygons in other_relations geometrycollection (#6475) PG driver: * Add PRELUDE_STATEMENTS and CLOSING_STATEMENTS open option to be for example able to specify options, like statement_timeout, with pg_bouncer * Fix 2.0 regression when overwriting several existing PostGIS layers with ogr2ogr (#6018) * Update PG, PGDump and CartoDB drivers to correctly export POINT EMPTY for PostGIS 2.2 * avoid resetting error potentially emitted by ExecuteSQL() (#6194) * sanitize management of quoting for FID column at layer creation * fix to get SRID on result layer with PostGIS 2.2 * in copy mode (the default on layer creation), do not truncate the concatenated string list to the field width (#6356) * make such that GEOMETRY_NAME layer creation option is honoured in ogr2ogr when the source geometry field has a not-null constraint (#6366) * read and set DESCRIPTION metadata item from/into pg_description system table; add DESCRIPTION layer creation option * support int2[] and numeric[] types, better map float4[] type * remove code that was intended to handled binary cursors as it cannot be triggered * fix append of several layers in PG_USE_COPY mode and within transaction (ogr2ogr -append use case) (#6411) PGDump driver: * fix issue with case of ogc_fid field in case the FID layer creation option is not set by user or by ogr2ogr (related to #6232) * in copy mode (the default on layer creation), do not truncate the concatenated string list to the field width (#6356) * make such that GEOMETRY_NAME layer creation option is honoured in ogr2ogr when the source geometry field has a not-null constraint (#6366) * set DESCRIPTION metadata item from/into pg_description system table; add DESCRIPTION layer creation option Shapefile driver: * accept opening standalone .dbf files whose header length is not a multiple of 32 bytes (#6035) * fix REPACK crash on shapefile without .dbf (#6274) * add capability to restore/build a missing .shx file when defining SHAPE_RESTORE_SHX to TRUE (#5035) * avoid CreateLayer() to error out when passed wkbUnknown | wkb25D (#6473) SQLite/Spatialite driver: * support file:xxx URI syntax (derived from patch by joker99, #6150) * fix heuristics in OGRSQLiteSelectLayer::GetExtent() to not be used when there's a sub SELECT (#6062) * fix crash on GetLayerByName('non_existing_table(geom_column)') (#6103) * fix OGRSQLiteSelectLayerCommonBehaviour::GetBaseLayer() to no longer 'eat' consecutive characters in layer name (#6107) * Spatialite: turn debug messages warning about update not being supported because of missing or too old spatialite version as errors, and return NULL to the caller (#6199) * fix memleak in OGRSQLiteTableLayer destructor when updating geometry_columns_time * VFS: increase mxPathname to 2048 by default, and provide OGR_SQLITE_VFS_MAXPATHNAME config option to be able to configure that higher if that would be needed. Useful when dealing with very long names like /vsicurl/.... with AWS S3 security tokens * VFS: do not probe -wal files on /vsicurl/ SXF driver: * fix wrong use of endianness macros to fix behaviour on big-endian hots * add recoding from CP1251 for TEXT attribute that is now decoded * fix various issues (#6357) VRT driver: * implement CloseDependentDatasets() * fix editing with 'direct' geometry mode which could cause attribute column to be empty (#6289) * fix crash with a OGRVRTWarpedLayer using a source layer that would have non geometry column (unlikely to happen currently as this would require explicit disabling it, but more likey with following commit that createe VRT non-spatial layer implictly when the source is non-spatial, linked to #6336) * avoid creating an implicit wkbUnknown geometry field when the source has no geometry column and there's no XML elements related to geometry fields (#6336) XLSX driver: * fix reading sheets with more than > 26 columns and 'holes' (#6363) XPlane driver: * extend ICAO identifiers to 5 digits (#6003) == SWIG Language Bindings == All bindings: * add a options parameter to gdal.ReprojectImage() to pass warp options * Change ReadRaster and WriteRaster to use GIntBig and the *IOEx-methods * prevent NULL file pointer from being passed to VSIF*L functions * make gdal.Rename() accept Unicode strings * add SpatialReference.GetAxisName() and SpatialReference.GetAxisOrientation() (#6441) * add SpatialReference.GetAngularUnitsName() (#6445) Java bindings: * Fix typemap for input parameter of type GIntBig (fixes GetFeature(long), DeleteFeature(long), etc...) (#6464) * Bump minimal java version to 1.5 in case SWIG generates anotations (#6433, patch by Bas Couwenberg) * GNUmakefile: add -f in rm commands * GNUmakefile: add support for all hardening buildflags Perl bindings: * Fix #6050: string formatting in croak. * Perl Makefile.PL: add support for all hardening buildflags (#5998) * use strict and warnings in overridden constructors. * add $VERSION to ogr_perl.i (OGR.pm), which is required by pause.perl.org. * Add some basic module info for CPAN. * Bugfix for Geo::OGR::Feature->new(). * Add many utility level algorithms as methods to various classes. * New class for XML stuff * New Makefile.PL, which can download and build GDAL. This allows automatic testing of the CPAN module. * Wrap VSIStdoutSetRedirection and allow creating datasets via an object, which can write and close. * Geo::OGR::Driver and Geo::OGR::DataSource are now Perl wrappers for respective GDAL classes. * Add to the error stack also errors from the bindings * Fix sending utf8 from Perl to GDAL. Should also remove some "uninitialized value" warnings. * ReadTile and WriteTile methods for Dataset, ReadTile accepts now tile size and scaling algorithm. * Improved Parent - Child management. * Improved support for 64bit ints. * Measures support in Geometry class. * Many new tests Python bindings: * make Feature.ExportToJson() output boolean value for a boolean field * support floating point coordinates for the source windows of Band.ReadRaster() and Band.ReadAsArray() * fix build with SWIG 3.0.6 (#6045) * make gdal.OpenEx() throw a Python exception in case of failed open when exceptions are enables with gdal.UseExceptions() (#6075) * Disable opening a NumPy dataset with a filename returned by gdal_array.GetArrayFilename(() unless GDAL_ARRAY_OPEN_BY_FILENAME is set to TRUE * disable the warning about using deprecated wkb25DBit constant as it uses a trick that prevents the bindings from being used by py2exe / pyinstaller (#6364)
2018-02-19 13:05:17 +01:00
#
# used by geography/gdal-lib/Makefile
# used by geography/py-gdal/Makefile
geography/gdal-lib: Update to 2.4.1 pkgsrc changes: add patch to fix or work around std::fabs upstream changes: bug fixes
2019-03-22 15:00:49 +01:00
VERSION= 2.4.1
geography/gdal-lib: Update to 2.3.3 2.3.3: bug fixes and minor improvements 2.3.2: bug fixes and minor improvements 2.3.1: bug fixes and minor improvements = GDAL/OGR 2.3.0 Release Notes = Note: due to the change of SCM during the development, #XXXX still refers to Trac tickets. GitHub tickets are explicitly indicated with github #XXXX == In a nutshell... == * New GDAL drivers: - PDS4: read/write - RDA: DigitalGlobe Raster Data Access (read-only) * New OGR drivers: - MVT: add read/write driver for Mapbox Vector Tile standalone files or tilesets - ESRIJson and TopoJSON: read-only, splitted from existing code of the GeoJSON driver - WFS3: *experimental* read-only driver * RFC 68: Make C++11 a build requirement https://trac.osgeo.org/gdal/wiki/rfc68_cplusplus11 * RFC 70: Guess output format from extension of output filename, if no explicit format passed to C++ or Python utilities https://trac.osgeo.org/gdal/wiki/rfc70_output_format_guess * RFC 71: Move to GitHub for source code repository and issue tracker. https://trac.osgeo.org/gdal/wiki/rfc71_github_migration * Significantly improved drivers: - MBTiles (vector tiles support) - DXF - GRIB (GRIB2 write support) - WCS (support WCS 2.0) * Improvements in network-based /vsi handlers: /vsicurl, /vsis3, /vsigs. Add: - /vsiaz for Microsoft Azure Blobs - /vsioss for Alibaba Cloud Object Storage Service - /vsiswift/ for OpenStack Swift object storage * Command line utilities: use Unicode main on Windows to avoid issues with non-ASCII characters (#7065) * Update to EPSG v9.2 (#7125) * Update data/esri_extra.wkt and add data/esri_epsg.wkt, taken from https://github.com/Esri/projection-engine-db-doc (Apache v2 license) (#2163) * Add support for PROJ.5 new API (requires proj 5.0.1 or later). PROJ 4.X is still supported. * More than 1000 fixes for issues/vulnerabilities found by OSS-Fuzz * Remove raster OGDI driver (vector OGDI driver still there) (#7087) == New installed files == * data/pds4_template.xml * data/esri_epsg.wkt == Backward compatibility issues == See MIGRATION_GUIDE.TXT Note that changes are extensive; see NEWS in the distfile.
2019-03-09 02:54:59 +01:00
DISTNAME= gdal-${VERSION}
Update to 2.2.3 * Add Makefile.common to prepare for geography/py-gdal Changelog: = GDAL/OGR 2.2.3 Release Notes = The 2.2.3 release is a bug fix release. == Build == * nmake.opt: Ensure PDB is included in release DLL if WITH_PDB requested (#7055) == Port == * /vsicurl/ and derived filesystems: redirect ReadDir() to ReadDirEx() (#7045) * /vsicurl/: enable redirection optimization on signed URLs of Google Cloud Storage. Helps for the PLScenes driver (#7067) * /vsicurl/: fix 2.2 regression regarding retrieval of file size of FTP file (#7088) * /vsis3/: fix to avoid invalid content to be sent if Write() writes more than 50 MB in a single call (#7058) * /vsis3/: fix Seek(Tell(), SEEK_SET) fails if current position is not 0 (#7062) * /vsis3/: fix support of non-ASCII characters in keys (#7143, #7146) * Fix CPLCopyTree() that doesn't properly on MSVC 2015 (and possibly other platforms) (#7070) == Algorithms == * RPC transformer: set output coordinates to HUGE_VAL when failure occurs, so that a following coordinate transformation can detect the error too (#7090) * GDALGrid() with linear algorithm: avoid assertions/segmentation fault when GDALTriangulationFindFacetDirected() fails (#7101) * GDALComputeProximity(): fix int32 overflow when computing distances on large input datasets (#7102) * GDALResampleChunk32R_Gauss: fix potential out of bounds access (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4056) == GDAL utilities == * gdalwarp: fix issue with GDALAdjustValueToDataType(Float32, +/- inf) that didn't preserve infinity, which affected gdalwarp -dstnodata inf (#7097) * gdalwarp -crop_to_cutline: reduce number of iterations to find the appropriate densification (#7119) * gdal_contour: return with non-0 code if field creation or contour generation failed (#7147) == GDAL drivers == GeoRaster driver: * fix int overflow (#6999) GPKG driver: * speed-up statistics retrieval on non-Byte datasets (#7096) GRIB driver: * if GRIB_ADJUST_LONGITUDE_RANGE config option is set to YES, adjust the longitude range to be close to [-180,180] when possible for products whose left origin is close to 180deg. (#7103) GSAG driver: * avoid assertion on some files with nul characters (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3726) GTiff driver: * make sure that -co PHOTOMETRIC=RGB overrides the color interpretation of the first 3 bands of the source datasets (#7064) * on reading use GeogTOWGS84GeoKey to override the defaults TOWGS84 values coming from EPSG code (#7144) * fix potential crash when reading in RGBA mode with internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4157) * fix potential crash with old-jpeg compression and internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4180) HTTP driver: * do not open the underlying dataset with GDAL_OF_SHARED, to avoid later assertion JP2KAK driver: * fix unix build with Kakadu 7.A and later (#7048, #7081) JPEG driver: * Add compatibility with libjpeg-turbo 1.5.2 that honours max_memory_to_use (cf https://github.com/libjpeg-turbo/libjpeg-turbo/issues/162) JP2OpenJPEG driver: * Add support for openjpeg 2.3 (#7074) netCDF driver: * fix raster read as nodata with Byte datatype, (valid_range={0,255} or _Unsigned = True) and negative _FillValue (#7069) * be more tolerant on the formatting of standard parallel (space separated instead of {x,y,...} syntax), and accept up to 2/1000 error on spacing to consider a regular grid, to be able to read files provided by the national weather institute of Netherlands (KNMI) (#7086) PDF driver: * round to upper integer when computing a DPI such that page size remains within limits accepted by Acrobat (#7083) Sentinel2 driver: * add support for direct opening of .zip files of new safe_compact L1C products (#7085) VRT driver: * avoid error being emitted when opening a VRTRawRasterBand in a .zip files (#7056) == OGR core == * Fix OGR[Curve]Polygon::Intersects(OGRPoint*) to return true when point is on polygon boundary (#7091) * importFromWkt(): fix import of GEOMETRYCOLLECTION ending with POINT EMPTY or LINESTRING EMPTY (#7128, 2.1 regression) == OGR utilities == * ogrtindex: fix crash when using -f SQLITE -t_srs XXXX (#7053) == OGR drivers == Amigocloud driver: * Fixed data field types (https://github.com/OSGeo/gdal/pull/246) * Output list of datasets if dataset id is not provided. * Implemented waiting for job to complete on the server. This will improve write to AmigoCloud reliability. * Updated documentation. FileGDB driver: * remove erroneous ODsCCreateGeomFieldAfterCreateLayer capability declaration (https://github.com/OSGeo/gdal/pull/247) GeoJSON driver: * ESRIJson: recognize documents that lack geometry fields (#7071) * ESRIJson: recognize documents starting with a very long fieldAliases list (#7107) GeoRSS driver: * fix detection of field type (#7108) GML driver: * fix FORCE_SRS_DETECTION=YES effect on feature count and SRS reporting on gml files with .gfs (#7046) * do not try to open kml files (#7061) * GML geometry parsing: fix robustness issue with gml:PolyhedralSurface * do not report gml:name / gml:description of features as layer metadata GPKG driver: * do not try to update extent on gpkg_contents after GetExtent() on a empty layer of a datasource opened in read-only mode GTM driver: * fix null pointer dereference in case of read error (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4047) KML / LIBKML drivers: MDB driver: * fix multi-thread support (https://issues.qgis.org/issues/16039) MITAB driver * do not emit error if the .ind file is missing, just a debug message (#7094) MySQL: * fix build with MariaDB 10.2 (#7079) OCI driver: * initialize in multi-threaded compatible mode to fix QGIS related crashes (https://issues.qgis.org/issues/17311), and fix a few memleaks ODS driver: * avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050) PLScenes driver: * backport data/plscenesconf.json from trunk to add SkySatScene fields Shapefile driver: * Fix GetFeatureCount() to properly take into account spatial filter when attribute filter also in effect (#7123) SQLite/Spatialite driver: * don't invalidate statistics when running a PRAGMA (https://issues.qgis.org/issues/17424) * SQLite dialect: support SQLite 3.21, and LIKE, <>, IS NOT, IS NOT NULL, IS NULL and IS operators (#7149) XLS driver: * workaround opening filenames with incompatible character set on Windows (https://issues.qgis.org/issues/9301) XLSX driver: * fix non working detection of Date/Time fields in some documents (#7073) * fix opening of documents with x: namespace in xl/workbook.xml (#7110) * avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050) == SWIG bindings == * map GRA_Max, GRA_Min, GRA_Med, GRA_Q1 and GRA_Q3 (#7153) == Python bindings == * remove 'from . import _gdal_array' line from gdal_array.py that is not necessary with normal execution of the bindings, and cause errors with PyInstaller (#7044) = GDAL/OGR 2.2.2 Release Notes = The 2.2.2 release is a bug fix release. == Build == * Windows build: always build the PDF driver, even when none of poppler/podofo/pdfium are available, in which case it is write-only (#6938) * Compilation fixes on Ubuntu 16.04 with explicit --std=c++03 (#6952) == Port == * /vsigzip/: make Eof() detect end of stream when receive a Z_BUF_ERROR error. Fixes #6944. * Fix memleak in VSIGSFSHandler::GetFileList() * /vsigzip/: avoid trying to write a .gz.properties file on a /vsicurl/ file (#7016) * Fix registration of /vsicrypt/ file system handler when compiled as a plugin (#7030) * CPLStrtod(): parse string like '-1.#IND0000000' as NaN instead of -1 (#7031) == Algorithms == * Warper: when operating on single-band, skip target pixels whose source center pixel is nodata (2.2 regression, #7001) * Warper: avoid blocking when number of rows is 1 and NUM_THREADS > 1 (#7041). Also limit the number of threads so that each one processes at least 65536 pixels == GDAL core == * GDALGCPsToGeoTransform(): add GDAL_GCPS_TO_GEOTRANSFORM_APPROX_OK=YES and GDAL_GCPS_TO_GEOTRANSFORM_APPROX_THRESHOLD=threshold_in_pixel configuration option (#6995) * Fix issue with GDALProxyRasterBand::FlushCache() not flushing everything required (#7040) * RawDataset::IRasterIO(): don't assume all bands are RawRasterBand == GDAL utilities == * gdal2tiles.py: fix GDAL 2.2 regression where tilemapresource.xml was no longer generated (#6966) * gdal_translate/DefaultCreateCopy(): do not destroy target file in case of failed copy wen using -co APPEND_SUBDATASET=YES (#7019) * gdal_translate: make -b mask[,xx] use the appropriate band data type (#7028) == GDAL drivers == GeoRaster driver: * add support for GCP (#6973) GPKG driver: * do not error out if extent in gpkg_contents is present but invalid (#6976) * fix opening subdatasets with absolute filenames on Windows (https://issues.qgis.org/issues/16997) * fix possible assertion / corruption when creating a raster GeoPackage (#7022) GSAG driver: * fix reading issue that could cause a spurious 0 value to be read and shift all following values (#6992) GTiff driver: * fix reading subsampled JPEG-in-TIFF when block height = 8 (#6988) * when reading a COMPD_CS (and GTIFF_REPORT_COMPD_CS=YES), set the name from the GTCitationGeoKey (#7011) GTX driver: * do not emit error when opening with GDAL_PAM_ENABLED=NO (#6996) HF2 driver: * fix reading tiles that are 1-pixel wide (2.1 regression, #6949) IDRISI driver: * Fix memleak in IDRISI Open() ISIS3 driver: * make sure that -co USE_SRC_HISTORY=NO -co ADD_GDAL_HISTORY=NO results in remove of History section (#6968) * fix logic to initialize underlying GeoTIFF file in IWriteBlock(), and implement Fill() (#7040) JPEG2000 driver: * Fix build failure in C++03 mode on Jasper inclusion in RHEL 6 (#6951) * Fix build failure with recent Jasper that no longer define uchar JP2OpenJPEG driver: * Add support for building against OpenJPEG 2.2 (#7002) * fix performance issues with small images with very small tile size, such as some Sentinel2 quicklooks (#7012) * properly use the opj_set_decode_area() API (#7018) netCDF driver: * avoid vector vs raster variable confusion that prevents reading Sentinel3 datasets, and avoid invalid geolocation array to be reported (#6974) PDF driver: * add support for Poppler 0.58 (#7033) PDS driver: * fix parsing of labels with nested list constructs (2.2 regression, #6970) SRTMHGT driver: * recognizes the .hgt.gz extension (#7016) VRT driver: * avoid stack buffer read overflow with complex data type and scale = 0. (oss-fuzz#2468) * fix uninitialized buffer in areas without sources when using non pixel packed spacing (#6965) * Warped VRT: correctly take into account cutline for implicit overviews; also avoid serializing a duplicate CUTLINE warping options in warped .vrt (#6954) * Warped VRT: fix implicit overview when output geotransform is not the same as the transformer dst geotransform (#6972) * fix IGetDataCoverageStatus() in the case of non-simple sources, which unbreaks gdalenhance -equalize (#6987) == OGR core == * OGRCompondCurve::addCurveDirectly(): try reversing non contiguous curves (for GML/NAS) * OGR API SPY: fix the way we map dataset handles to variable name, to avoid invalid reuses of variable still active * OGRParseDate(): avoid false-positive time detection, in particular for GeoJSON (#7014) * OGRCurve::get_isClosed(): do not take into account M component (#7017) * OGRLineString::setPoint() and addPoint() with a OGRPoint* argument. properly takes into account ZM, Z and M dimensions (#7017) == OGRSpatialReference == * Fix OGRSpatialReference::IsSame() to return FALSE when comparing EPSG:3857 (web_mercator) and EPSG:3395 (WGS84 Mercator) (#7029) * importFromProj4(): implement import of Hotine Oblique Mercator Two Points Natural Origin, and fix OGRSpatialReference::Validate() for that formulation (#7042) == OGR utilities == * ogr2ogr: fix small memory leak when using -limit switch * ogr2ogr: make -f GMT work again (#6993) == OGR drivers == AVCBin driver: * fix 2.1 regression regarding attributes fetching (#6950) DXF driver: * fix reading files where INSERT is the last entity and DXF_MERGE_BLOCK_GEOMETRIES is false (#7006) * avoid segfault when creating a DXF file with only the 'blocks' layer (#6998) * fix potential null pointer dereference in PrepareLineStyle (#7038) GeoJSON driver: * fix 2.2 regression regarding lack of setting the FeatureCollection CRS on feature geometries (fixes https://github.com/r-spatial/sf/issues/449#issuecomment-319369945) * prevent infinite recursion when reading geocouch spatiallist with properties.properties.properties (#7036) GML driver: * fix field type detection logic to avoid a field with xsi:nil=true to be identified as a string (#7027) * JPGIS FGD v4: fix logic so that coordinate order reported is lon/lat (https://github.com/OSGeo/gdal/pull/241) GMLAS driver: * fix memleak in swe:DataRecord handling * avoid false-positive warning about attribute found in document but ignored according to configuration, when the attribute is actually not present in the document but has a default value (#6956) * (quick-and-not-prefect) fix to avoid a <xs:any> to prevent all other fields from being set (#6957) * get the srsName when set on the srsName of the gml:pos of a gml:Point (#6962) * CityGML related fixes: better take into account .xsd whose namespace prefix is not in the document, fix discovery of substitution elements, handle gYear data type (#6975) GPKG driver: * fix feature count after SetFeature() with sqlite < 3.7.8 (#6953) * do not take into account gpkg_contents for the extent of vector layers when it is present but invalid (#6976) * fix logic to detect FID on SQL result layer when several FID fields are selected (#7026) KML / LIBKML drivers: * read documents with an explicit kml prefix (#6981) MSSQLSpatial driver: * fix issues on x64 (#6930) * properly format DELETE statement with schema (#7039) OCI driver: * check view with unquoted table name (#5552) OpenFileGDB driver: * properly read GeneralPolygon with M component whose all values are set to NaN (#7017) OSM driver: * increase string buffer for osm XML files (#6964) Shapefile driver: * use VSIStatL() in Create() to properly work with /vsimem/ directories (#6991) * fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (#7031) SQLite/Spatialite driver: * escape integer primary key column name on table creation (#7007) == C# bindings == * Implement RasterIO extensions (#6915) == Python bindings == * fix reference count issue on gdal.VSIStatL() when it returns None, that can cause Python crashes if None refcount drops to zero * add C-API to create driver instances for use in SWIG (https://trac.osgeo.org/osgeo4w/ticket/466) == Security oriented fixes == Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them. (oss-fuzz#XXXX is a shortcut to https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=XXXX) * NAS: avoid assertion / null ptr deref and possibly other failures on corrupted files. (oss-fuzz#2366, oss-fuzz#2580) * NTF: avoid null ptr deref. Related to https://oss-fuzz.com/v2/testcase-detail/6527758899347456 * VSIMemHandle::Read(): avoid unwanted unsigned int overflow that cause a later heap buffer overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5227283000328192. * MBTiles: fix use after free. (oss-fuzz#2388) * netCDF: fix stack buffer overflow when building vector layers from netCDF variables if they don't have the expected number of dimensions. (oss-fuzz#2400) * DXF: prevent infinite loop / excessive memory allocation on truncated file. (#oss-fuzz#2403) * DXF: avoid excessive memory allocation. (oss-fuzz#2444) * XYZ: fix write heap-buffer-overflow (oss-fuzz#2426) * GTiff: make IGetDataCoverageStatus() properly set the current TIFF directory, and only implement it for 'normal' bands. To avoid heap buffer overflows. (oss-fuzz#2438) * GTiff: avoid null pointer derefrence when requested implicit overviews on corrupted JPEG-compressed file. (oss-fuzz#2441) * GTiff: avoid potential issue on corrupted files (oss-fuzz#2481) * GTiff: don't override member nBlocksPerRow member of GTiffJPEGOverviewBand with unrelated value, in case of single-strip case. (oss-fuzz#3020) * PCIDSK: for band interleave channel, correctly use pixel_offset in case it is different from pixel_size. (oss-fuzz#2440) * MITAB: fix skipping of blank lines when reading MIF geometries, and avoid potential very loop loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4516730841858048 * CPLKeywordParser: avoid potential heap buffer overflow read. (oss-fuzz#2706) * CPLKeywordParser: avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/5733173726019584 * morphFromESRI(): fixes heap-after-free uses. (oss-fuzz#2864) * OGR_VRT: fix null pointer dereference on GetExtent() on an invalid layer. (oss-fuzz#3017) * OGR_VRT: avoid crash on cyclic VRT. (oss-fuzz#3123) * OSM: avoid potential write heap buffer overflow on corrupted PBF files. (oss-fuzz#3022) * FAST: fix potential read heap buffer overflow. (oss-fuzz#3025) * CPG: avoid null pointer dereference on failed open. (oss-fuzz#3136) = GDAL/OGR 2.2.1 Release Notes = The 2.2.1 release is a bug fix release. == Build == * fix compilation without BIGTIFF_SUPPORT (#6890) * configure: detect if std::isnan() is available. Helps compilation on some MacOSX setups, combined together --without-cpp11. Refs https://github.com/macports/macports-ports/pull/480 * fix compilation against ancient Linux kernel headers (#6909) * fix detection of 64bit file API with clang 5 (#6912) * configure: use .exe extension when building with mingw64* toolchains (fixes #6919) * mongoDB: compilation fix on Windows == Port == * /vsicurl/: fix occasional inappropriate failures in Read() with some combinations of initial offset, file size and read request size (#6901) * Add a VSICurlClearCache() function (bound to SWIG as gdal.VSICurlClearCache()) to be able to clear /vsicurl/ related caches (#6937) == Algorithms == * GDALRasterize(): avoid hang in some cases with all_touched option (#5580) * gdal_rasterize: fix segfault when rasterizing onto a raster with RPC (#6922) == GDAL utilities == * ogr_merge.py: fix '-single -o out.shp in.shp' case (#6888) == GDAL drivers == AIGRID driver: * fix handling on raw 32-bit AIG blocks ENVISAT driver: * fix 2.2 regression in initialization of members of MerisL2FlagBand. (#6929) GeoRaster driver: * Fix memory allocation failure (#6884) * add support for JP2-F in BLOB compression (corrections on geo-reference) (#6861) GPKG driver: * avoid corruption of gpkg_tile_matrix when building overviews, down to a level where they are smaller than the tile size (#6932) GTIFF driver: * Internal libtiff: fix libtiff 4.0.8 regression regarding creating of single strip uncompressed TIFF files (#6924) netCDF driver: * add support for radian and microradian units for geostationnary projection (https://github.com/OSGeo/gdal/pull/220) NWT_GRC driver: * Fix handling of alpha values in GRC color table (#6905) * Handle case of 0-len GRC class names (#6907) VRT driver: * speed-up SerializeToXML() in case of big number of bands XYZ driver: * fix 2.2 regression where the driver hangs on some dataset with missing samples (#6934) == OGR utilities == * ogr2ogr/GDALVectorTranslate(): fix crash when using -f PDF -a_srs (#6920) == OGR drivers == GeoJSON driver: * ESRIJson: avoid endless looping on servers that don't support resultOffset (#6895) * ESRIJson: use 'latestWkid' in priority over 'wkid' when reading 'spatialReference' (https://github.com/OSGeo/gdal/pull/218) * GeoJSON writer: accept writing ZM or M geometry by dropping the M component (#6935) GPKG driver: * make driver robust to difference of cases between table_name in gpkg_contents/gpkg_geometry_columns and name in sqlite_master (#6916) MITAB driver: * recognize Reseau_National_Belge_1972 / EPSG:31370 on writing (#6903) MySQL driver: * fix compilation issue with Arch Linux and mariadb 10.1.23 (fixes #6899) PG driver: * do not be confused by a 'geometry' table in a non-PostGIS enabled database (#6896) PLScenes: * remove support for V0. Deprecate V1 API. Only Data V1 is supported ( #6933) == Perl bindings == * Backport the fix to #6142 Install man page according to GDALmake.opt if INSTALL_BASE is set. * always return something from non-void functions (#6898) == Python bindings == * Accept callback = 0 since SWIG generates it as the default argument of BandRasterIONumPy(). Fixes https://github.com/OSGeo/gdal/pull/219 * Fix 2.2 regression preventing use of callback function in Band.ComputeStatistics() (#6927) == Security oriented fixes == Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them (some have been found locally, so no related ticket) * Fix CPLErrorSetState to ensure it does not write beyond DEFAULT_LAST_ERR_MSG_SIZE and correctly null-terminates last message. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1529. * Open() and Stat() methods of a number of virtual file systems: check that the filename starts with the full prefix. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1543. * VRT pixel functions: fix crash with 'complex' when source count is < 2. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1549 * OpenAIR: fix potential out-of-buffer read if we need to ingest 30000 bytes * Several fixes in importFromWkb() and importFromWkt() * GDALDataset and GDALRasterBand::ReportError(): fix crash if dataset name has a % character * NASAKeywordHandler::SkipWhite(): fix out of bounds read * MITAB: ParseTABFileFields(): fix out of bounds read. * MITAB: ParseMIFHeader(): fix memory leak and out-of-bounds read on corrupted file * MITAB: ParseMIFHeader(): fix memory leaks on corrupted files * MITAB: avoid potentially veryyyy long loop when stroking arcs. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1644 * MITAB: avoid heap-buffer-overflow in MITABCoordSys2TABProjInfo(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1653 * OSARDataset::Open(): fix crash if pOpenInfo->fpL == NULL. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1565 * OGRESRIJSONReadPolygon(): fix crash in error code path * DXF: prevent null ptr deref and out-of-bounds read on corrupted file * DXF: TranslateSPLINE(): sanitize integer values read to avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1843 * KML::unregisterLayerIfMatchingThisNode(): use memmove() instead of memcpy(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1580 * KML: fix crash on weird structure with recursively empty folders. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1591 * KML: fix null ptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1634 * OGRCurveCollection::importBodyFromWkb(): fix potential crash if the curve in the collection has not the same dimension has the collection dimension. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1584 * OGRCompoundCurve::importFromWkb(): avoid potential stack overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5192348843638784 * TIGER: fix potential stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1586 and 2020 * TIGER: avoid potential infinite looping. Fixes https://oss-fuzz.com/v2/testcase-detail/4724815883665408 * VFK: avoid out-of-bounds read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1596 and 2074 * CPLHexToBinary(): avoid reading outside of hex2char array on on-ASCII input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1606 * OGR PDS: avoid int32 overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=160 * GeoRSS: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1611. * VSIArchiveFilesystemHandler::SplitFilename(): improve performance. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1597 * OGRGeometryFactory::organizePolygons(): fix crash on empty polygons. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1620 * JML: fix null pointer dereference on invalid files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1622 * Shape: prevent null ptr deref on truncated MultiPointM geometry. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1638 * /vsisubfile/: avoid Tell() to return negative values. And make VSIIngestFile() more robust to unsigned overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1642 * GTM: avoid useless recursive opening of files when provided with a gzip-compressed input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1650 * GTiff: fix heap-buffer-overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1677 * GTiff: avoid heap-buffer-overfow on corrupted State Plane citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2050 * GTiff: avoid potential stack buffer overflow on corrupted Imagine citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2105 * GTiff: prevent heap overflow and fix reading of multi-band band-interleaved 16/24-bit float datasets. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2264 * GTiff: fix potential infinite loop when parsing some 24-bit float denormalized numbers. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2315 * Internal libjson-c: fix stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1682 * ILI1/ILI2: fix null pointer dereference when opening filename ','. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1689 * ILI1: fix various crashes on corrupted files (including, but not limited to, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1760, 1784, 1926) * ILI2: use proper cast operator. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1813 * ILI2: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1993 * ILI2: fix crash due to unhandled exception. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2191 * morphFromESRI(): fix heap-use-after-free issue. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1690 * morphFromESRI(): prevent potential null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1783 and 1867 * SEGUKOOA: fix inversion of leap year that caused index-out-of-bound reading on day 366 of leap years (2.2 regression). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1698 * CPLParseXMLString(): make it error out on invalid XML file even under CPLTurnErrorIntoWarning() mode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1703. * GML / NAS: fix memory leak in error code path, and potential heap-buffer-read-overflow * NTF: fix various issues: heap & stack buffer-overflow, null ptr derefs, memory leaks. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1819 , 1820 , 1823, 1833, 1854, 1862, 1910, 1931, 1961, 1990, 1995, 1996, 2003, 2033, 2052, 2077, 2084, 2103, 2130, 2135, 2146, 2166, 2185, 2187, https://oss-fuzz.com/v2/testcase-detail/4696417694121984 * OGRCreateFromMultiPatch(): avoid assertion on NaN coordinates. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1930 * GXF: validate nGType to avoid later out-of-bound read in GXFReadRawScanlineFrom(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1975 * GXF: fix int overflow and avoid excessive memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2207 * DGN: prevent heap-buffer-overflow read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1988 * COSAR: fix leak of file descriptor. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2019 * ISO8211: prevent stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2022 * WEBP: prevent int32 overflow and too large memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2059 * IRIS: fix heap-buffer-overflow in some cases of nDataTypeCode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2069 * E00GRID: avoid heap and stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2090 , 2182, 2237, 2327 * VICAR: fix null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2116 * VICAR: avoid use-after-free and heap-buffer-overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/4825577427042304 * VICAR: fix potential endless loop on broken files. Fixes https://oss-fuzz.com/v2/testcase-detail/6261508172414976 * REC: fix nullptr deref * REC: fix potential stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2165 * GDALGetJPEG2000Structure() / DumpGeoTIFFBox(): fix memory leak. * DumpGeoTIFFBox(): reject GeoJP2 boxes with a TIFF with band_count > 1 * DumpJPK2CodeStream(): avoid potentially very long loop * GDALGetJPEG2000Structure(): avoid bad performance on corrupted JP2 boxes. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2214 * GPKG: fix potential heap-buffer overflow in GPkgHeaderFromWKB(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2150 * GPKG: fix potential null ptr deref. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2240 * GPKG: avoid potential division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2253 * SEGUKOOA: prevent read beyond end of buffer. (#6921) * SRP: avoid potential stack buffer overflow and excessive memory allocation/processing time * CPLUnixTimeToYMDHMS(): avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4949567697059840 * Selafin: fix double frees. Fixes https://oss-fuzz.com/v2/testcase-detail/6429713822121984 * CEOS: fix heap buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2259 * CEOS: fix memleak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2279 * FAST: avoid null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2290 * netCDF: avoid stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2302 GDAL/OGR 2.2.0 Release Notes = == In a nutshell... == * New GDAL/raster drivers: - DERIVED driver: read-support. Expose subdatasets in a a new metadata domain, called DERIVED_SUBDATASETS - JP2Lura driver: read/create support for JPEG-2000 images using Luratech JP2 Library - PRF: add read-only support for PHOTOMOD PRF file format driver (github #173) - RRASTER driver: read-support .grd/.gri files handled by the R 'raster' package (#6249) * New OGR/vector drivers: - CAD driver: read support for DWG R2000 files (GSoC 2016 project) - DGNv8 driver: read-write support for DGN 8.0 format (using Teigha ODA libraries) - GMLAS driver: read-write support. XML/GML driver driven by Application Schemas. * New utility script: ogrmerge.py to merge several vector datasets into a single one * New /vsigs/ and /vsigs_streaming/ virtual file systems to read Google Cloud Storage non-public files * Significantly improved drivers: - NWT_GRD: write support (#6533) - FileGDB/OpenFileGDB: add support to read curve geometries (#5890) - VRT derived band: add the capability to define pixel functions in Python - Add read support for RasterLite2 coverages in SQLite driver - GPKG: implement tiled gridded elevation data extension - ISIS3: add write support and improve read support * RFC 63: Add GDALRasterBand::GetDataCoverageStatus() and implement it in GTiff and VRT drivers https://trac.osgeo.org/gdal/wiki/rfc63_sparse_datasets_improvements * RFC 64: Triangle, Polyhedral surface and TIN https://trac.osgeo.org/gdal/wiki/rfc64_triangle_polyhedralsurface_tin ==> this RFC introduces potential backward incompatible behaviour. Consult MIGRATION_GUIDE.txt * RFC 66: OGR random layer read/write capabilities https://trac.osgeo.org/gdal/wiki/rfc66_randomlayerreadwrite * RFC 67: add null field state for OGR features, in addition to unset fields https://trac.osgeo.org/gdal/wiki/rfc67_nullfieldvalues ==> this RFC introduces potential backward incompatible behaviour. Consult MIGRATION_GUIDE.txt * Upgrade to EPSG database v9.0 (#6772) * Python bindings: Global Interpreter Lock (GIL) released before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open()) * Continued major efforts on sanitization of code base * Remove bridge and vb6 bindings (#6640) * GNM built by default == Installed files == * Removed: data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv * Added plscenesconf.json, gmlasconf.xsd == Backward compatibility issues == See MIGRATION_GUIDE.TXT == GDAL/OGR 2.2.0 - General Changes == Build(Unix): * improve detection of packaged libfyba (SOSI) --with-sosi, as in Ubuntu 16.04 (#6488) * Sort files in static library to make the build reproducible (#6520) * fix libqhull include path when it is /usr/local/include/libqhull (#6522) * FileGDB: compilation fix on Linux in C++11 mode * configure: make pdfium detection not fail if there are just warnings. And make configure fail if --with-pdfium was required but failed (#6653) * Make ./configure --with-xerces fail if not found * Don't install script documentation in INST_BIN (github #157) * configure: auto-detect webp without requiring explicit --with-webp * configure: use pkg-config for HDF5 detection so that works out of the box on recent Ubuntu * auto-detect JDK 8 on Ubuntu * MDB: allow libjvm.so to be dlopen'd with --with-jvm-lib=dlopen (Unix only, github #177) * configure: delete temporary directories on the mac * configure: make sure --with-macosx-framework is correctly defined * configure: error out if --with-ld-shared is specified (#6769) * configure: remove bashism. * configure: fix --without-mrf (#6811) * configure: take into account CXXFLAGS and LDFLAGS in a few more cases (cryptopp, podofo, libdap) * Vagrant: all lxc and Hyper-V provider support; use vagrant-cachier for package caching * configure: update DWG support to work with Teigha libraries * Internal libgeotiff: hide symbols in --with-hide-internal-symbols mode * Shape: do not export Shapelib symbols for builds --with-hide-internal-symbols (#6860) Build(Windows): * Try to avoid confusion between libqhull io.h and mingw own io.h (#6590) * update script to generate most recent Visual C++ project files (#6635) * fix broken and missing dependencies in makefile.vc * add a way to use an external zlib (github #171) * Rename makegdal_gen.bat to generate_vcxproj.bat * generate_vcxproj.bat: Set correct value of PlatformToolset property based on specified Visual C++ version. Add NMAKE command line parameters: MSVC_VER based on specified Visual C++ version, DEBUG=1 WITH_PDB=1 to Debug build configuration. * generate_vcxproj.bat: generate project for autotest/cpp (Ticket #6815) * Add WIN64=1 to NMAKE command line options. * Add HDF4_INCLUDE option (#6805) * Add MSVC compiler option /MP to build with parallel processes. * Add ZLIB_LIB missing from EXTERNAL_LIBS Build(all): * make Xerces 3.1 the minimal version * drop support for PostgreSQL client library older than 7.4, or non security maintained releases older than 8.1.4, 8.0.8, 7.4.13, 7.3.15 == GDAL 2.2.0 - Overview of Changes == Port: * Export VSICreateCachedFile() as CPL_DLL so as to enable building JP2KAK as a plugin * Added possibility to find GDAL_DATA path using INST_DATA definition without execution GDALAllRegister if GDAL_DATA placed in version named directory on Linux (#6543) * Unix filesystem: make error message about failed open to report the filename (#6545) * File finder: Remove hardcoded find location (/usr/local/share/gdal) (#6543) * Win32 filesystem handler: make Truncate() turn on sparse files when doing file extension * Add VSIFGetRangeStatusL() and VSISupportsSparseFiles() * GDAL_NO_HARDCODED_FIND compilation option (#6543,#6531) to block file open calls (for sandboxed systems) * Add VSIMallocAligned(), VSIMallocAlignedAuto() and VSIFreeAligned() APIs * /vsizip / /vsitar: support alternate syntax /vsitar/{/path/to/the/archive}/path/inside/the/tar/file so as not to be dependent on file extension and enable chaining * Optimize opening of /vsitar/my.tar.gz/my_single_file * /vsizip/ : support creating non-ASCII filenames inside a ZIP (#6631) * VSI Unix file system: fix behaviour in a+ mode that made MRF caching not work on Mac and other BSD systems * Fix deadlock at CPLWorkerThreadPool destruction (#6646) * Windows: honour GDAL_FILENAME_IS_UTF8 setting to call LoadLibraryW() (#6650) * CPLFormFilename(): always use / path separator for /vsimem, even on Windows * /vsimem/: add trick to limit the file size, so as to be able to test how drivers handle write errors * /vsimem/: fix potential crash when closing -different- handles pointing to the same file from different threads (#6683) * CPLHTTPFetch(): add MAX_FILE_SIZE option * CPLHTTPFetch(): add a CAINFO option to set the path to the CA bundle file. As a fallback also honour the CURL_CA_BUNDLE and SSL_CERT_FILE environment variables used by the curl binary, which makes this setting also available for /vsicurl/, /vsicurl_streaming/, /vsis3/ and /vsis3_streaming/ file systems (#6732) * CPLHTTPFetch(): don't disable peer certificate verification when doing https (#6734) * CPLHTTPFetch(): cleanly deal with multiple headers passed with HEADERS and separated with newlines * CPLHTTPFetch(): add a CONNECTTIMEOUT option * CPLHTTPFetch(): add a GDAL_HTTP_HEADER_FILE / HEADER_FILE option. * CPLHTTPSetOptions(): make redirection of POST requests to still be POST requests after redirection (#6849) * /vsicurl/: take CPL_VSIL_CURL_ALLOWED_EXTENSIONS into account even if GDAL_DISABLE_READDIR_ON_OPEN is defined (#6681) * /vsicurl/: get modification time if available from GET or HEAD results * /vsis3/: add a AWS_REQUEST_PAYER=requester configuration option (github #186) * CPLParseXMLString(): do not reset error state * Windows: fix GetFreeDiskSpace() * Fix GetDiskFreeSpace() on 32bit Linux to avoid 32bit overflow when free disk space is above 4 GB (#6750) * Fix CPLPrintUIntBig() to really print a unsigned value and not a signed one * Add CPL_HAS_GINT64, GINT64_MIN/MAX, GUINT64_MAX macros (#6747) * Add CPLGetConfigOptions(), CPLSetConfigOptions(), CPLGetThreadLocalConfigOptions() and CPLSetThreadLocalConfigOptions() to help improving compatibility between osgeo.gdal python bindings and rasterio (related to https://github.com/mapbox/rasterio/pull/969) * MiniXML serializer: fix potential buffer overflow. Core: * Proxy dataset: add consistency checks in (the unlikely) case the proxy and underlying dataset/bands would not share compatible characteristics * GDALPamDataset::TryLoadXML(): do not reset error context when parsing .aux.xml * PAM/VRT: only take into account <Entry> elements when deserializing a <ColorTable> * GDALCopyWords(): add fast copy path when src data type == dst data type == Int16 or UInt16 * GetVirtualMemAuto(): allow USE_DEFAULT_IMPLEMENTATION=NO to prevent the default implementation from being used * Nodata comparison: fix test when nodata is FLT_MIN or DBL_MIN (#6578) * GetHistogram() / ComputeRasterMinMax() / ComputeStatistics(): better deal with precision issues of nodata comparison on Float32 data type * Fast implementation of GDALRasterBand::ComputeStatistics() for GDT_Byte and GDT_UInt16 (including use of SSE2/AVX2) * Driver manage: If INST_DATA is not requested, do not check the GDAL_DATA variable. * Make sure that GDALSetCacheMax() initialize the raster block mutex (#6611) * External overview: fix incorrect overview building when generating from greater overview factors to lower ones, in compressed and single-band case (#6617) * Speed-up SSE2 implementation of GDALCopy4Words from float to byte/uint16/int16 * Add SSE2 and SSSE3 implementations of GDALCopyWords from Byte with 2,3 or 4 byte stride to packed byte * GDALCopyWords(): SSE2-accelerated Byte->Int32 and Byte->Float32 packed conversions * Fix GDALRasterBand::IRasterIO() on a VRT dataset that has resampled sources, on requests such as nXSize == nBufXSize but nXSize != dfXSize * GDALRasterBand::IRasterIO(): add small epsilon to floating-point srcX and srcY to avoid some numeric precision issues when rounding. * Add GDALRasterBand::GetActualBlockSize() (#1233) * Fix potential deadlock in multithreaded writing scenarios (#6661) * Fix thread-unsafe behaviour when using GetLockedBlock()/MarkDirty()/DropLock() lower level interfaces (#6665) * Fix multi-threading issues in read/write scenarios (#6684) * Resampled RasterIO(): so as to get consistent results, use band datatype as intermediate type if it is different from the buffer type * Add GDALIdentifyDriverEx() function (github #152) * GDALOpenInfo: add a papszAllowedDrivers member and fill it in GDALOpenEx() * GDALDefaultOverviews::BuildOverviews(): improve progress report * Average and mode overview/rasterio resampling: correct source pixel computation due to numerical precision issues when downsampling by an integral factor, and also in oversampling use cases (github #156) * Overview building: add experimental GDAL_OVR_PROPAGATE_NODATA config option that can be set to YES so that a nodata value in source samples will cause the target pixel to be zeroed. Only implemented for AVERAGE resampling right now * GDALValidateOptions(): fix check of min/max values * GMLJP2 v2: update to 2.0.1 corrigendum and add capability to set gml:RectifiedGrid/gmlcov:rangeType content. Set SRSNAME_FORMAT=OGC_URL by default when converting to GML. Add gml:boundedBy in gmljp2:GMLJP2RectifiedGridCoverage * GMLJP2 v2: ensure KML root node id unicity when converting annotations on the fly. When generating GML features, make sure that PREFIX and TARGET_NAMESPACE are unique when specifying several documents. Algorithms: * RPC transformer: speed-up DEM extraction by requesting and caching a larger buffer, instead of doing many queries of just a few pixels that can be costly with VRT for example * GDALDeserializeRPCTransformer(): for consistency, use the same default value as in GDALCreateRPCTransformer() if <PixErrThreshold> is missing (so use 0.1 instead of 0.25 as before) * TPS solver: when Armadillo fails sometimes, fallback to old method * GDALCreateGenImgProjTransformer2(): add SRC_APPROX_ERROR_IN_SRS_UNIT, SRC_APPROX_ERROR_IN_PIXEL, DST_APPROX_ERROR_IN_SRS_UNIT, DST_APPROX_ERROR_IN_PIXEL, REPROJECTION_APPROX_ERROR_IN_SRC_SRS_UNIT and REPROJECTION_APPROX_ERROR_IN_DST_SRS_UNIT transformer options, so as to be able to have approximate sub-transformers * Fix GDAL_CG_Create() to call GDALContourGenerator::Init() (#6491) * GDALContourGenerate(): handle the case where the nodata value is NaN (#6519) * GDALGridCreate(): fix hang in multi-threaded case when pfnProgress is NULL or GDALDummyProgress (#6552) * GDAL contour: fix incorrect oriented contour lines in some rare cases (#6563) * Warper: multiple performance improvements for cubic interpolation and uint16 data type * Warper: add SRC_ALPHA_MAX and DST_ALPHA_MAX warp options to control the maximum value of the alpha channel. Set now to 65535 for UInt16 (and 32767 for Int16), or to 2^NBITS-1. 255 used for other cases as before * Warper: avoid undefined behaviour when doing floating point to int conversion, that may trigger exception with some compilers (LLVM 8) (#6753) * OpenCL warper: update cubicConvolution to use same formula as CPU case (#6664) * OpenCL warper: fix compliance to the spec. Fix issues with NVidia opencl (#6624, #6669) * OpenCL warper: use GPU based over CPU based implementation when possible, use non-Intel OpenCL implementation when possible. Add BLACKLISTED_OPENCL_VENDOR and PREFERRED_OPENCL_VENDOR to customize choice of implementation Utilities: * gdalinfo -json: fix order of points in wgs84Extent.coordinates (github #166) * gdalwarp: do not densify cutlines by default when CUTLINE_BLEND_DIST is used (#6507) * gdalwarp: when -to RPC_DEM is specified, make -et default to 0 as documented (#6608) * gdalwarp: improve detection of source alpha band and auto-setting of target alpha band. Automatically set PHOTOMETRIC=RGB on target GeoTIFF when input colors are RGB * gdalwarp: add a -nosrcalpha option to wrap the alpha band as a regular band and not as the alpha band * gdalwarp: avoid cutline densification when no transform at all is involved (related to #6648) * gdalwarp: fix failure with cutline on a layer of touching polygons (#6694) * gdalwarp: allow to set UNIFIED_SRC_NODATA=NO to override the default that set it to YES * gdalwarp: fix -to SRC_METHOD=NO_GEOTRANSFORM -to DST_METHOD=NO_GEOTRANSFORM mode (#6721) * gdalwarp: add support for shifting the values of input DEM when source and/or target SRS references a proj.4 vertical datum shift grid * gdalwarp: fix crash when -multi and -to RPC_DEM are used together (#6869) * gdal_translate: when using -projwin with default nearest neighbour resampling, align on integer source pixels (#6610) * gdal_translate & gdalwarp: lower the default value of GDAL_MAX_DATASET_POOL_SIZE to 100 on MacOSX (#6604) * gdal_translate: avoid useless directory scanning on GeoTIFF files * gdal_translate: make "-a_nodata inf -ot Float32" work without warning * gdal_translate: set nodata value on vrtsource on scale / unscale / expand cases (github #199) * GDALTranslate(): make it possible to create a anonymous target VRT from a (anonymous) memory source * gdaldem: speed-up computations for src type = Byte/Int16/UInt16 and particularly for hillshade * gdaldem hillshade: add a -multidirectional option * GDALDEMProcessing() API: fix -alt support (#6847) * gdal_polygonize.py: explicitly set output layer geometry type to be polygon (#6530) * gdal_polygonize.py: add support for -b mask[,band_number] option to polygonize a mask band * gdal_rasterize: make sure -3d, -burn and -a are exclusive * gdal_rasterize: fix segfaults when rasterizing into an ungeoreferenced raster, or when doing 'gdal_rasterize my.shp my.tif' with a non existing my.tif (#6738) * gdal_rasterize: fix crash when rasterizing empty polygon (#6844) * gdal_grid: add a smoothing parameter to invdistnn algorithm (github #196) * gdal_retile.py: add a -overlap switch * gdal2tiles.py: do not crash on empty tiles generation (#6057) * gdal2tiles.py: handle requested tile at too low zoom to get any data (#6795) * gdal2tiles: fix handling of UTF-8 filenames (#6794) * gdal2xyz: use %d formatting for Int32/UInt32 data types (#6644) * gdal_edit.py: add -scale and -offset switches (#6833) * gdaltindex: emit warning in -src_srs_format WKT when WKT is too large * gdalbuildvrt: add a -oo switch to specify dataset open options Python samples: * add validate_cloud_optimized_geotiff.py * add validate_gpkg.py Multi-driver: * Add GEOREF_SOURCES open option / GDAL_GEOREF_SOURCES config. option to all JPEG2000 drivers and GTiff to control which sources of georeferencing can be used and their respective priority AIGRID driver: * fix 2.1.0 regression when reading statistics (.sta) file with only 3 values, and fix <2.1 behaviour to read them in LSB order (#6633) AAIGRID driver: * auto-detect Float64 when the nodata value is not representable in the Float32 range ADRG driver: * handle north and south polar zones (ZNA 9 and 18) (#6783) ASRP driver: * fix georeferencing of polar arc zone images (#6560) BPG driver: * declare GDALRegister_BPG as C exported for building as a plugin (#6693) DIMAP driver: * DIMAP: for DIMAP 2, read RPC from RPC_xxxxx.XML file (#6539) * DIMAP/Pleiades metadata reader: take into tiling to properly shift RPC (#6293) * add support for tiled DIMAP 2 datasets (#6293) DODS driver: * fix crash on URL that are not DODS servers (#6718) DTED driver: * correctly create files at latitudes -80, -75, -70 and -50 (#6859) ECW driver: * Add option ECW_ALWAYS_UPWARD=TRUE/FALSE to work around issues with "Downward" oriented images (#6516). ENVI driver: * on closing, pad image file with trailing nul bytes if needed (#6662) * add read/write support for rotated geotransform (#1778) GeoRaster driver: * fix report of rotation (#6593) * support for JP2-F compression (#6861) * support direct loading of JPEG-F when blocking=no (#6861) * default blocking increased from 256x256 to 512x512 (#6861) GPKG driver: * implement tiled gridded elevation data extension * add VERSION creation option * check if transaction COMMIT is successful (#6667) * fix crash on overview building on big overview factor (#6668) * fix crash when opening an empty raster with USE_TILE_EXTENT=YES * fix gpkg_zoom_other registration GTiff driver: * support SPARSE_OK=YES in CreateCopy() mode (and in update mode with the SPARSE_OK=YES open option), by actively detecting blocks filled with 0/nodata about to be written * When writing missing blocks (i.e. non SPARSE case), use the nodata value when defined. Otherwise fallback to 0 as before. * in FillEmptyTiles() (i.e. in the TIFF non-sparse mode), avoid writing zeroes to file so as to speed up file creation when filesystem supports ... sparse files * add write support for half-precision floating point (Float32 with NBITS=16) * handle storing (and reading) band color interpretation in GDAL internal metadata when it doesn't match the capabilities of the TIFF format, such as B,G,R ordering (#6651) * Fix RasterIO() reported when downsampling a RGBA JPEG compressed TIFF file (#6943) * Switch search order in GTIFGetOGISDefn() - Look for gdal_datum.csv before datum.csv (#6531) * optimize IWriteBlock() to avoid reloading tile/strip from disk in multiband contig/pixel-interleave layouts when all blocks are dirty * fix race between empty block filling logic and background compression threads when using Create() interface and NUM_THREADS creation option (#6582) * use VSIFTruncateL() to do file extension * optimize reading and writing of 1-bit rasters * fix detection of blocks larger than 2GB on opening on 32-bit builds * fix saving and loading band description (#6592) * avoid reading external metadata file that could be related to the target filename when using Create() or CreateCopy() (#6594) * do not generate erroneous ExtraSamples tag when translating from a RGB UInt16, without explicit PHOTOMETRIC=RGB (#6647) * do not set a PCSCitationGeoKey = 'LUnits = ...' as the PROJCS citation on reading * fix creating an image with the Create() interface with BLOCKYSIZE > image height (#6743) * fix so that GDAL_DISABLE_READDIR_ON_OPEN = NO / EMPTY_DIR is properly honoured and doesn't cause a useless directory listing * make setting GCPs when geotransform is already set work (with warning about unsetting the geotransform), and vice-versa) (#6751) * correctly detect error return of TIFFReadRGBATile() and TIFFReadRGBAStrip() * in the YCBCR RGBA interface case, only expose RGB bands, as the alpha is always 255 * don't check free disk space when outputting to /vsistdout/ (#6768) * make GetUnitType() use VERT_CS unit as a fallback (#6675) * in COPY_SRC_OVERVIEWS=YES mode, set nodata value on overview bands * read GCPs in ESRI <GeodataXform> .aux.xml * explicitly write YCbCrSubsampling tag, so as to avoid (latest version of) libtiff to try reading the first strip to guess it. Helps performance for cloud optimized geotiffs * map D_North_American_1927 datum citation name to OGC North_American_Datum_1927 so that datum is properly recognized (#6863) * Internal libtiff. Resync with CVS (post 4.0.7) * Internal libtiff: fix 1.11 regression that prevents from reading one-strip files that have no StripByteCounts tag (#6490) GRASS driver: * plugin configure: add support for GRASS 7.2 (#6785) * plugin makefile: do not clone datum tables and drivers (#2953) * use Rast_get_window/Rast_set_window for GRASS 7 (#6853) GRIB driver: * Add (minimalistic) support for template 4.15 needed to read Wide Area Forecast System (WAFS) products (#5768) * **Partial** resynchronization with degrib-2.0.3, mostly to get updated tables (related to #5768) * adds MRMS grib2 decoder table (http://www.nssl.noaa.gov/projects/mrms/operational/tables.php) (github #160) * enable PNG decoding on Unix (#5661, github #160) * remove explicitly JPEG2000 decompression through Jasper and use generic GDAL code so that other drivers can be triggered * fix a few crashes on malformed files GTX driver: * add a SHIFT_ORIGIN_IN_MINUS_180_PLUS_180 open option HDF4 driver: * Fixed erroneous type casting in HDF4Dataset::AnyTypeToDouble() that breaks reading georeferencing and other metadata HDF5 driver: * correct number of GCPs to avoid dummy trailing (0,0)->(0,0,0) and remove +180 offset applied to GCP longitude. Add instead a heuristics to determine if the product is crossing the antimeridian, and a HDF5_SHIFT_GCPX_BY_180 config option to be able to override the heuristics (#6666) HFA driver: * fix reading and writing of TOWGS84 parameters (github #132) * export overview type from HFA files to GDAL metadata as OVERVIEWS_ALGORITHM (github #135) * make .ige initialization use VSIFTruncateL() to be faster on Windows * add support for TMSO and HOM Variant A projections (#6615) * Add elevation units read from HFA files metadata (github #169) * set binning type properly according to layerType being thematic or not (#6854) Idrisi driver: * use geotransform of source dataset even if it doesn't have a SRS (#6727) * make Create() zero-initialize the .rst file (#6873) ILWIS driver: * avoid IniFile::Load() to set the bChanged flag, so as to avoid a rewrite of files when just opening datasets ISCE driver: * fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL 2.1.0 (#6556) * fix misbehaviour on big endian hosts * add support for reading and writing georeferencing (#6630, #6634) * make parsing of properties case insensitive (#6637) ISIS3 driver: * add write support * add mask band support on read * get label in json:ISIS3 metadata domain JPEGLS driver: JP2ECW driver: * fix crash when translating a Float64 raster (at least with SDK 3.3) JP2KAK driver: * add support for Kakadu v7.9. v7.8 should not be used. It has a bug fixed in v7.9 * catch exceptions in jp2_out.write_header() JP2OpenJPEG driver: * add a USE_TILE_AS_BLOCK=YES open option that can help with whole image conversion * prevent endless looping in openjpeg in case of short write * for single-line organized images, such as found in some GRIB2 JPEG2000 images, use a Wx1 block size to avoid huge performance issues (#6719) * ignore warnings related to empty tag-trees. JPIPKAK driver: * fix random crashes JPIP in multi-tread environment (#6809) KEA driver: * Add support for Get/SetLinearBinning (#6855) KMLSuperOverlay driver: * recognize simple document made of GroundOverlay (#6712) * Add FORMAT=AUTO option. Uses PNG for semi-transparent areas, else JPG. (#4745) LAN driver: * remove wrong byte-swapping for big-endian hosts MAP driver: * change logic to detect image file when its path is not absolute MBTiles driver: * on opening if detecting 3 bands, expose 4 bands since there might be transparent border tiles (#6836) * fix setting of minzoom when computing overviews out of order * do not open .mbtiles that contain vector tiles, which are not supported by the driver MEM driver: * disable R/W mutex for tiny performance increase in resampled RasterIO * add support for overviews * add support for mask bands MRF driver: * bug fix in PNG and JPEG codecs * Fixing a problem with setting NoData for MRFs generated with Create * fix plugin building (#6498) * rename CS variable so as to avoid build failure on Solaris 11 (#6559) * Allow MRF to write the data file directly to an S3 bucket. * Allow relative paths when MRF is open via the metadata string. * Add support for spacing (unused space) between tiles. Defaults to none. * Read a single LERC block as an MRF file. MSG driver: * fix incorrect georeference calculation for msg datasets (github #129) NetCDF driver: * add support for reading SRS from srid attribute when it exists and has content like urn:ogc:def:crs:EPSG::XXXX (#6613) * fix crash on datasets with 1D variable with 0 record (#6645) * fix erroneous detection of a non-longitude X axis as a longitude axis that caused a shift of 360m on the georeferencing (#6759) * read/write US_survey_foot unit for linear units of projection * apply 'add_offset' and 'scale_factor' on x and y variables when present, such as in files produced by NOAA from the new GOES-16 (GOES-R) satellite (github #200) * add a HONOUR_VALID_RANGE=YES/NO open option to control whether pixel values outside of the validity range should be set to the nodata value (#6857) * fix crash on int64/uint64 dimensions and variables, and add support for them (#6870) NITF driver: * add support for writing JPEG2000 compressed images with JP2OpenJPEG driver * fix writing with JP2KAK driver (emit codestream only instead of JP2 format) * fix setting of NBPR/NBPC/NPPBH/NPPBV fields for JPEG2000 (fixes #4322); in JP2ECW case, make sure that the default PROFILE=NPJE implies 1024 block size at the NITF level * implement creation of RPC00B TRE for RPC metadata in CreateCopy() mode * add support for reading&writing _rpc.txt files * nitf_spec.xml: Add support for MTIRPB TRE in NITF image segment. Also makes minor change to BLOCKA to include default values (github #127) * nitf_spec.xml: add IMASDA and IMRFCA TREs * GetFileList(): Small optimization to avoid useless file probing. NWT_GRD: * detect short writes OpenJPEG driver: * support direct extracting of GeoRaster JP2-F BLOB (#6861) PCIDSK driver: * handle Exceptions returned from destructor and check access rights in setters (github #183) PDF driver: * implement loading/saving of metadata from/into PAM (#6600) * implement reading from/writing to PAM for geotransform and projection (#6603) * prevent crashes on dataset reopening in case of short write PLScenes driver: * add a METADATA open option PostgisRaster driver: * fix potential crash when one tile has a lower number of bands than the max of the table (#6267) R driver: * fix out-of-memory (oom) with corrupt R file Raw drivers: * prevent crashes on dataset closing in case of short write RMF driver: * fix wrong counter decrement that caused compressed RMF to be incorrectly decompressed (github #153) * fix load/store inversion of cm and dm units in MTW files (github #162) * fix reading nodata for non-double data type (github #174) ROIPAC driver: * add support for reading/writing .flg files (#6504) * fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL >= 2.0.0 (#6591) * fix for big endian hosts RS2 driver: * add support for reading RPC from product.xml SAFE driver: * fix handling of SLC Products by providing access to measurements as subdatasets (#6514) Sentinel2 driver: * add support for new "Safe Compact" encoding of L1C products (fixes #6745) SQLite driver: * Add read support for RasterLite2 coverages in SQLite driver SRTMHGT driver: * open directly .hgt.zip files * accept filenames like NXXEYYY.SRTMGL1.hgt (#6614) * handle files for latitude >= 50 (#6840) VRT driver: * add default pixel functions: real, imag, complex, mod, phase, conj, etc... for complex data types (github #141) * avoid useless floating point values in SrcRect / DstRect (#6568) * avoid buffer initialization in RasterIO() when possible (replace ancient and likely broken concept of bEqualAreas) * make CheckCompatibleForDatasetIO() return FALSE on VRTDerivedRasterBands (#6599) * VRT warp: fix issue with partial blocks at the right/bottom and dest nodata values that are different per band (#6581) * fix performance issue when nodata set at band level and non-nearest resampling used (#6628) * VRTComplexSource: do temp computations on double to avoid precision issues when band data type is Int32/UInt32/CInt32/Float64/CFloat64 (#6642) * VRT derived band: add the capability to define pixel functions in Python * CreateCopy(): detect short writes * Fix linking error on VRTComplexSource::RasterIOInternal<float>() (#6748) * avoid recursion in xml:VRT metadata (#6767) * prevent 'Destination buffer too small' error when calling GetMetadata('xml:VRT') on a in-memory VRT copied from a VRT * fix 2.1 regression that can cause crash in VRTSimpleSource::GetFileList() (#6802) WMS driver: * Added support for open options to WMS minidrivers * Refactored the multi-http code to make it possible to do range requests. * Added a minidriver_mrf, which reads from remote MRFs using range requests. * Made the minidriver_arcgis work with an ImageService, not only MapService. * Added static cache of server response. * Allow tiledWMS to work in off-line mode by including the server response in the .wms file itself. * honour GDAL_HTTP_USERAGENT config option when it is set and <UserAgent> is missing (#6825) * WMS/WMTS: better deal with tiles with different band count (grayscale, gray+alpha, palette, rgb, rgba) (github #208) * Make HTTPS options apply to tWMS minidriver init, better HTTP error reporting WMTS driver: * do not take into account WGS84BoundingBox/BoundingBox that would be the result of the densified reprojection of the bbox of the most precise tile matrix * add TILEMATRIX / ZOOM_LEVEL open options * accept tiles of small dimensions (github #210) XYZ driver: == OGR 2.2.0 - Overview of Changes == Core: * Layer algebra: Add KEEP_LOWER_DIMENSION_GEOMETRIES=YES/NO option to Intersection, Union and Identity. Default is yes, but it is set to no unless result layer is of geom type unknown. If set to no, result features which would have lower dim geoms are skipped if operating on two layers with same geom dim. * Fix crash/corrupted values when running importFromWkb() on POLYGON M/POLYGON ZM geometries (#6562) * Add OGR_WKT_PRECISION config option that defaults to 15 to select the number of decimals when outputting to WKT * Make OGRFeature::SetField(string) accept JSon serialized arrays for the String/Integer/Integer64/RealList types; add reciprocal OGRFeature::GetFieldAsSerializedJSon() for those types * OGRGeometryFactory::transformWithOptions(): better deal with reprojection from polar projection to WGS84, and projections crossing the antimeridian to WGS84, by splitting geometries prior to reprojection (#6705) * LinearRing transformTo(): force last point to be identical to first one in case it is not. * GML geometry parsing: avoid 'Cannot add a compound curve inside a compound curve' error (#6777) * OGR SQL: fix IN filtering on MapInfo indexed columns (#6798) * OGR SQL: add support for LIMIT and OFFSET keywords * OGR SQL: add comparisons on date / datetime (#6810) * OGR SQL: increase efficiency of DISTINCT operator * OGREnvelope: change initialization to fix issue when getting MULTIPOINT(0 0,1 1) envelope (#6841) * OGRParse: fix parsing logic to avoid false positive detection of string as datetime (#6867) OGRSpatialReference: * Upgrade to EPSG database v9.0 (#6772) * OGRCT: upgrade LIBNAME of mingw and cygwin to libproj-9.dll and cygproj-9.dll to be up-to-date with proj 4.9.X (recommended method is using ./configure --with-static-proj4 instead) (#6501) * importFromESRI(): fix import of multi line MERCATOR SRS (#6523) * morphToESRI(): correctly compute standard_parallel_1 of Mercator(2SP) projection from scale factor of Mercator(1SP) (#6456, #4861) * exportToProj4(): recognize explicit OSR_USE_ETMERC=NO to avoid using etmerc with proj >= 4.9.3 * importFromProj4(): do not set a AUTHORITY node for strings like '+init=epsg:XXXX +some_param=val' * importFromProj4(): be robust with missing proj.4 epsg dictionary when importing '+init=epsg:xxxx +other_parm=value' * AutoIdentifyEPSG(): add identification of EPSG:3995 (Arctic Polar Stereographic on WGS84) and EPSG:3031 (Antarctic Polar Stereographic on WGS84) * OGRCoordinateTransformation: avoid potential bugs in proj.4 on NaN input * importFromEPSG(): take into account DX,DY,DZ,RX,RY,RZ,DS columns in pcs.csv to add per-PCS TOWGS84 overrides (geotiff #52) * Coordinate transformation: prevent unnecessary coordinate transformations (github #184, #185) Utilities: * ogr2ogr: do not return error on ogr2ogr --utility_version * ogr2ogr: keep -append and -overwrite when -update follows * ogr2ogr: fix heuristics to detect likely absence of '-f' (#6561) * ogr2ogr: do not emit a warning when updating/overwriting a destination dataset that is not a Shapefile and if -f is not specified (#6561) * ogr2ogr: make overwriting of lots of PG tables less prone to PostgreSQL 'out of shared memory' errors, by committing transactions after each table recreation * ogr2ogr: prevent 'ogr2ogr same.shp same.shp' (#1465) * ogr2ogr: add a -limit option to limit the number of features read per layer * GDALVectorTranslate(): imply update mode if access mode not explicitly specified and hDstDS != NULL (#6612) * ogrlineref: Fix crash if no new layer name provided * ogrtindex: add -src_srs_name, -src_srs_format and -t_srs option to allow indexing files that have different projections Multidriver: * PGeo/FileGDB/OpenFileGDB: OGRCreateFromShapeBin(): do not error out on empty lines/polygons * GPKG/SQLite/PG/FileGDB/MEM: properly set/reset field subtype with AlterFieldDefn() (#6689) * DXF, GeoJSON, GML, KML, LIBKML, ODS, Shape, XLSX: report operating system error if file creation fails (#2253) AmigoCloud driver: * add option to receive an AmigoCloud API key in the connection string, fix page size (github #137) Carto: * renamed from CartoDB * fix CartoDB'fication() by removing manual creation of the_geom_webmercator mercator, and also attach created sequence to table.cartodb_id (#6565) CSV driver: * add read/write support for String/Integer/Integer64/RealList types as serialized JSon arrays DGN driver: * avoid crash with -ftrapv on 250K_NF44NE_Area_V7.dgn (relates to #6806) * use coordinate delta encoding from the attribute records array (6806) DXF driver: * sanitize layer name on export to avoid forbidden characters * reader: convert GeometryCollection to MultiPoint, MultiLineString or MultiPolygon when possible (QGIS #10485) * add font of TEXT and MTEXT to OGR style string (https://github.com/OSGeo/gdal/pull/198) DWG driver: * compilation fixes with Teigha 4.2.2 ElasticSearch driver: * add support for ElasticSearch 5.0 and fix 2.X support (#6724) * implement translation from SQL to E.S. query language FileGDB driver: * add support to read curve geometries (#5890) * support opening '.' directory GeoJSON driver: * writer: add a RFC7946=YES creation option (#6705) * read and write 'name' and 'description' members at FeatureCollection level * fix field type detection when first value of a field is null (#6517) * improve/fix field type promotion * fix wrong behaviour when there's a 'id' at Feature level and 'id' or 'ID' field in properties (#6538) * in case top level id is a negative integer, put the value in a 'id' attribute (#6538) * ESRI Json reader: support multilinestring from esriGeometryPolyline * ESRI Json reader: do not set field width of 2147483647 (#6529) * ESRI Json reader: support reading M and ZM geometries * Add CPL_json_object_object_get() and use it, to avoid deprecation warnings on json_object_object_get() * TopoJSON reader: sanitize invalid polygons (such as found in the 'TopoJSON' layer of http://bl.ocks.org/mbostock/raw/4090846/us.json) * writer: fix segfaults on NULL geometry with -lco WRITE_BBOX=YES (#6698) * writer: fix crash if NATIVE_MEDIA_TYPE creation option is specified alone * Add support of TopoJSON without 'transform' element (github #192) * don't set SRS if 'crs' set to null (github #206) GML driver: * remove hack for CityGML regarding forcing srsDimension to 3 if not specified. Instead add a GML_SRS_DIMENSION_IF_MISSING config option that can be set to 3 if needed (#6597) * consider srsName with URL like 'http://www.opengis.net/def/crs/EPSG/0/' as following EPSG axis order. Add SWAP_COORDINATES=YES/NO/AUTO (and GML_SWAP_COORDINATES configuration option) to override all auto guessing (#6678) * add a SRSNAME_FORMAT=SHORT/OGC_URN/OGC_URL dataset creation option * OGR_G_ExportToGMLEx(): add a COORD_SWAP option * Writer: make ogr:FeatureCollection be a substitutionGroup of AbstractFeature for GML 3.2 output, so as to be compatible of GMLJP2 v2 * GML and NAS: improve error reporting (mention feature id and gml_id) when parsing fails because of geometry decoding error * GML and NAS: add GML_SKIP_CORRUPTED_FEATURES and NAS_SKIP_CORRUPTED_FEATURES configuration options to avoid stopping reading a layer because of a corrupted geometry * Add support for Japanese GML FGD v4.1 format (github #204) GPKG driver: * list all tables/views by default (useful for non spatial tables not registered as aspatial). Add ASPATIAL_VARIANT layer creation option to select the way how to create a non-spatial table as aspatial. * Create non-spatial tables by default conforming to GPKG v1.2 'attributes' * use OGR_CURRENT_DATE config option if defined as the value of the 'last_change' of column of 'gpkg_contents' so as to be able to have binary identical output * update last_change column when content of raster or vector tables has changed * do not emit error when running ExecuteSQL() with a spatial filter on an empty layer (#6639) * update schema to reflect CURRENT_TIMESTAMP -> 'now' changes (github #155) * better table and column quoting. * Robustify layer deletion (make it accessible through DROP TABLE xxx and DELLAYER:xxx syntax) and renaming * implement DeleteField(), AlterFieldDefn(), ReorderFields() * make GetExtent() save extent if not already cached * add special SQL 'RECOMPUTE EXTENT ON layer_name' to force recomputation of extent * check identifier unicity on layer creation * add possibility to disable foreign key check by setting OGR_GPKG_FOREIGN_KEY_CHECK=NO config option * add HasSpatialIndex(tblname,geomcolname) SQL function * don't show Spatialite vgpkg_ virtual tables (#6707) * SQLite/GPKG: add explicit error message when trying to open a read-only WAL-enabled database (#6776) * SQLite/GPKG: make sure when closing a WAL-enabled database opened in read-only mode to reopen it in read-write mode so that the -wal and -shm files are removed (#6776) * make GetFeature() works on non conformant tables that have no integer primary key field (#6799), and be robust to non standard column types * remove triggers related to metadata tables that cause issues and have been removed by latest revisions of the spec. * declare feature id column of features tables and tile pyramid user data tables as NOT NULL (#6807) * add a gpkg_ogr_contents table to store feature count. * speed-up GetFeatureCount() with only a spatial filter set * improve column recognition for SQL result layer * add/override ST_Transform() and SridFromAuthCRS() from Spatialite to make them work with gpkg_spatial_ref_sys * add ImportFromEPSG() * make ST_Min/MaxX/Y(), ST_SRID(), ST_GeometryType() and ST_IsEmpty() work with Spatialite geometries * improve performance of spatial index creation and use on multi-gigabyte databases * better support of spatial views, by adding a special behaviour if a column is named OGC_FID * avoid potential denial of services by adding LIMIT clauses * slightly more efficient implementation of GetExtent() if extent in gpkg_contents is empty * create a dummy 'ogr_empty_table' features table in case we have no 'features' or 'tiles' table, so as to be conformant with Req 17 of the GeoPackage specification * add DEFAULT '' to metadata column of gpkg_metadata table * accept opening a .gpkg without vector content and without gpkg_geometry_columns table in vector mode if we also open in update mode; remove capability of opening a .gpkg without vector content but with gpkg_geometry_columns table in vector mode if we only open in read-only mode; fix creation of a vector layer in a database if it initially lacks a gpkg_geometry_columns table * fix appending a raster to a vector database without pre-existing raster support tables * add minimalistic support for definition_12_063 column in gpkg_spatial_ref_sys, so that insertion of new SRS doesn't fail * use GEOMETRYCOLLECTION instead of GEOMCOLLECTION for SQL and gpkg_geometry_columns.geometry_type_name * do not warn if gpkg_metadata extension declared GPX driver: * ignore wpt/rtept/trkpt with empty content for lat or lon ILI driver: * ILI1: fix crash in JoinSurfaceLayer() when the multicurve of the feature of the poSurfaceLineLayer layer is empty (#6688) * ILI1: make polygon reconstruction in Surface layers robust to curves not in natural order (#6728) * ILI2: assign FID to features (#6839) * ILI2: fix crashing bug in Create() if model file not specified KML driver: * add a DOCUMENT_ID datasource creation option to set the id of the root <Document> node LIBKML driver: * fix crash when reading <gx:TimeStamp> or <gx:TimeSpan> elements (#6518) * add a DOCUMENT_ID datasource creation option to set the id of the root <Document> node * emit style related errors as warnings to make datasets openable by SWIG bindings (#6850) MITAB driver: * limit (width, precision) of numeric fields on creation to (20,16) for compatibility with MapInfo (#6392) * add support for oblique stereographic (#6598) * Adds the authority code for Irish national grid (Ireland_1965) (github #149) * fix spelling for Euref_89 and add EPSG code (#6816) MSSQLSpatial driver: * Fix bulk insert with table names containing spaces (#6527) * Build optional mssql plugin with SQL Native Client support for MSSQL Bulk Copy * Fix MSSQL select layer to recognize geometry column with sqlncli (#6641) MySQL driver: * fix spatial filtering on recent mysql by adding a SRID in the rectangle geometry * do not force NOT NULL constraint on geometry field if no spatial index is used NAS driver: * support multiple 'anlass' in updates NWT_GRD: * add write support (#6533) OCI driver: * Add options for faster feature loading (#6606) * add WORKSPACE open option * correctly handle OFTInteger64 case in loader layer (bug found by cppcheck multiCondition) * support for long identifiers (up to 128 long) when running of 12.2 or + (#6866) * OCILOB VSIL driver: new driver to streams in and out of Oracle BLOB as a GDAL large virtual file system (#6861) ODS driver: * fix FID filtering (#6788) OGDI driver: * make GetNextRawFeature() report an error when it is not end of layer * better error reporting when the layer list cannot be established * catch non-fatal OGDI errors emitted by OGDI 3.2.0 if OGDI_STOP_ON_ERROR environment variable is set to NO, and emit them as CPLError()s * display OGDI error message if opening fails OpenFileGDB driver: * do not error out on geometries that have a declared M array but that is missing (#6528) * add support to read curve geometries (#5890) * transcode UTF-16 strings found in column names, alias, etc... to UTF-8 (instead of using only their ASCII byte) (#6544) * do not emit an error on a empty table whose declaration has M settings (#6564) * support opening '.' directory * improve detection of some form of TINs from MULTIPATCH, and for MultiPatch layers, try to select a better geometry type for those layers (#5888) * fix bug when field description offset is beyond 4GB (#6830) OSM driver: * fix 'too many tags in relation' error when parsing .osm files * allow key=value entries in closed_ways_are_polygons= configuration (#6476) * allow OSM_SQLITE_CACHE config option to be greater than 2047 PG driver: * fix insertion of binary/bytea content in non-copy mode (#6566) * fix errors caused by missing geometry_columns/spatial_ref_sys tables in non PostGIS databases, that prevent reading more than 500 features (QGIS #10904) * avoid errors with field default expressions like 'foo'::text (#6872) PLScenes driver: * add HTTP retry logic (#6655) * V0 API: workaround limitations on filtering on image_statistics.image_quality (#6657) * add support for Data V1 API S57 driver: * Fix ogr2ogr -f S57 (#6549) * fix crashes if the s57objectclasses.csv resource file contains invalid lines * remove data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv and move their content into main s57attributes.csv and s57objectclasses.csv files (#6673) * Update s57 attributes and object classes according to s-57 reference (github #202) * add POSACC and QUAPOS fields for geometric primitive layers (github #205) SDE driver: * rename driver to OGR_SDE. Fix build (#6714) SEGY driver: * Accept SEGY files with ASCII headers that have nul terminated strings (#6674) Shapefile driver: * auto-repack by default at dataset closing and FlushCache()/SyncToDisk() time. Controlled by AUTO_REPACK open and layer creation options (that default to YES) * generate .dbf end-of-file 0x1A character by default. Add DBF_EOF_CHAR layer creation options / open options to control that behaviour * writing: use strerrno() for better error messages (QGIS #13468) * change REPACK implementation on Windows to be robust to remaining file descriptors openend on the .shp/.shx/.dbf (#6672, QGIS #15570) * Fix issue in DBFCloneEmpty() one a one field DBF not yet written to disk * add call to AutoIdentifyEPSG() when reading a .prj * support reading .dbf with substantial padding after last field definition. * when rewriting the geometry of the last record in the .shp, do it at the file offset it previously used (#6787) SOSI driver: * make registration of driver work again (2.1.0 regression) (#6500) * update to latest version of https://github.com/kartverket/gdal. Make SOSI driver support more geometry types (including curved geometries) plus provides some improvements on attribute-type mapping. Fix some memory errors/leaks. Disable by default non working creation code (#6503) SQLite/Spatialite driver: * do not emit error when running ExecuteSQL() with a spatial filter on an empty layer (#6639) * add read/write support for String/Integer/Integer64/RealList types as serialized JSon arrays * Spatialite: avoid crash when creating layer with geom_type = wkbCurve (fixes #6660) * Spatialite: do not report some BLOB columns as geometry columns of tables/views (when found before the geometry column(s)) (#6695, #6659) * fix update of features with multiple geometry columns (#6696) * speed-up dataset closing when creating a database with many spatial layers * Spatialite: avoid spatial views to cause layers 'layer_name(geometry_name)' to be publicly listed (#6740) * Spatialite: speed-up creation of database with INIT_WITH_EPSG=NO that is slow without transaction (not sure why as the table is empty...) * use AUTOINCREMENT for feature id column * allow OGR_SQLITE_CACHE to be set to > 2047 (MB) without overflowing * SQLite/GPKG: use SQLITE_OPEN_NOMUTEX flag to open databases. * GPKG/SQLite: fix ExecuteSQL() to work with a statement with 2 SELECT and ORDER BY clause (#6832) * SQLite/GPKG: change default page_size to 4096 bytes. * Update layer statistics for Spatialite 4 DB (#6838) * Remove traces of support of SQLite < 3.6.0 * SQLite dialect: properly quote column names when needed (github #214) VFK driver: * allow reading multiple VFK files into single DB support amendment VFK files * recreate DB in the case that it's outdated (VFK DB created by previous versions of GDAL) * allow reading DB as valid datasource (#6509) * new tables in backend SQLite database (geometry_columns/spatial_ref_sys) to enable reading DB datasource by SQLite driver * new configuration option OGR_VFK_DB_READ to enable reading backend database by VFK driver VRT driver: * add support for 'm' attribute in PointFromColumns mode (#6505) WFS driver: * invalidate underlying layer when SetIgnoredFields() is called (QGIS #15112) * don't crash on empty <Keyword/> declaration (#6586) * fix potential nullptr dereference on dataset without layer (github #179) XLSX driver: * only list worksheets (and no charts) as layers (#6680) == SWIG Language Bindings == All bindings: * map osr.GetTargetLinearUnits() (#6627) * allow wkbCurve/wkbSurface as valid values for the geom type of GeomFieldDefn object * map GDALIdentifyDriverEx() Java bindings: * Fix SWIG Java bindings for GNM (#6434) * Fix crash on GetDefaultHistogram() if the C++ method returns an error (#6812) Perl bindings: * return value always from GetGeomFieldIndex (#6506) * the Warp method requires a list of datasets (#6521) * when 'use bigint' is in effect, int var is a ref. * Separate the module building in the CPAN distribution and in the GDAL source tree. The CPAN distribution will be developed at https://github.com/ajolma * Fix the Extension method in Driver per RFC 46 * The Inv method of GeoTransform did not return a new object. Allow a single point in Apply method of GeoTransform. * Test for existence of PDL and require it if available. * allow decimation/replication in Piddle i/o. * support resampling in ReadTile. * Polygonize: Require explicit 8 for Connectedness to set 8-connectedness and allow 8CONNECTED as an option. * use Safefree to free memory allocated in Perl (#6796) Python bindings: * release the GIL before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open()) * add outputType option to gdal.Rasterize() * fix build issues when CXX is defined in the environment * gdal.VectorTranslate(): add spatSRS option * when enabling ogr.UseExceptions(), use the GDAL error message in the exception text (if available), when the exception is linked to an error value in the OGRErr return code * gdal.VectorTranslate(): accept a single string as value of the layers option (instead of iterating over each of its characters) * Regenerate Python bindings with SWIG 3.0.8 to avoid issue with Python 3.5. Add backward compatibility in Band.ComputeStatistics() to accept 0/1 as input instead of the expected bool value (#6749) * fix gdal.DEMProcessingOptions(zeroForFlat=True) (#6775) * fix 'import osgeo.gdal_array' with python3 and SWIG 3.0.10 (#6801) * allow gdal.FileFromMemBuffer() to use buffer > 2GB (#6828) * accept unicode strings as field name argument in Feature (like SetField, GetField, etc...) and FeatureDefn methods = GDAL/OGR 2.1.0 Release Notes = == In a nutshell... == * New GDAL/raster drivers: - CALS: read/write driver for CALS Type I rasters - DB2 driver: read/write support for DB2 database (Windows only) - ISCE: read/write driver (#5991) - MRF: read/write driver (#6342) - SAFE: read driver for ESA SENTINEL-1 SAR products (#6054) - SENTINEL2: read driver for ESA SENTINEL-2 L1B/LC1/L2A products - WMTS: read driver for OGC WMTS services * New OGR/vector drivers: - AmigoCloud: read/write support for AmigoCloud mapping platform - DB2 driver: read/write support for DB2 database (Windows only) - MongoDB: read/write driver - netCDF: read/write driver - VDV: read/write VDV-451/VDV-452 driver, with specialization for the Austrian official open government street graph format * Significantly improved drivers: - CSV: new options, editing capabilities of existing file - ElasticSearch: read support and support writing any geometry type - GeoJSON: editing capabilities of existing file, "native data" (RFC 60) support - MBTiles: add raster write support. fixes in open support - PDF: add PDFium library as a possible back-end. - PLScenes: add support for V1 API - VRT: on-the-fly pan-sharpening - GTiff: multi-threaded compression for some compression methods * Port library: add /vsis3/, /vsis3_streaming/, /vsicrypt/ virtual file systems * RFC 26: Add hash-set band block cache implementation for very larger rasters (WMS, WMTS, ...) http://trac.osgeo.org/gdal/wiki/rfc26_blockcache * RFC 48: Geographical networks support (GNM) https://trac.osgeo.org/gdal/wiki/rfc48_geographical_networks_support * RFC 58: Add DeleteNoDataValue(): https://trac.osgeo.org/gdal/wiki/rfc58_removing_dataset_nodata_value * RFC 59.1: Make GDAL/OGR utilities available as library functions: https://trac.osgeo.org/gdal/wiki/rfc59.1_utilities_as_a_library For gdalinfo, gdal_translate, gdalwarp, ogr2ogr, gdaldem, nearblack, gdalgrid, gdal_rasterize, gdalbuildvrt Available in C, Python, Perl and Java bindings. * RFC 60: Improved round-tripping in OGR https://trac.osgeo.org/gdal/wiki/rfc60_improved_roundtripping_in_ogr Implemented in GeoJSON driver * RFC 61: Support for measured geometries. https://trac.osgeo.org/gdal/wiki/rfc61_support_for_measured_geometries Implemented in Shapefile, PostgreSQL/PostGIS, PGDump, MEM, SQLite, GeoPackage, FileGDB, OpenFileGDB, CSV, VRT * Upgrade to EPSG database v8.8 * General sanitization pass to clean-up code, fix a lot of compiler warnings, as well as issues pointed by static code analyzers, such as Coverity Scan (credits to Kurt Schwehr for tackling a large part of them) or CLang Static Analyzer. * Fixes in a number of drivers to be more robust against corrupted files (most found with American Fuzzy Lop): RIK, INGR, Northwood, HF2, CEOS, GTiff, GXF, BMP, NITF, HFA, VRT, FIT, CEOS2, NWT_GRD/NWT_GRC, MITAB, RPFTOC, DBF/Shape, XYZ, VFK, DXF, NAS, GSAG, GS7BG, OpenFileGDB, RMF, AIGRID, OpenAIR, EHDR, ISO8211, FAST, USGSDEM, DGN, SGI, OpenJPEG, PCRaster, BSB, ADRG, SRP, JPEG, Leveller, VICAR, PCIDSK, XPM as well as in portability library (CPL), algorithms and raster core. * Driver removed: - Google Maps Engine (GME) (#6261) == New installed files == * gdal_pansharpen.py * data/vdv452.xml * data/vdv452.xsd * data/netcdf_config.xsd * /path/where/bash-completion/scripts/are/installed/gdal-bash-completion.sh == Backward compatibility issues == See MIGRATION_GUIDE.TXT == GDAL/OGR 2.1.0 - General Changes == Build(Unix): * Allow plugin for HDF4, HDF5, GeoRaster, FileGDB, OCI and PG * Add a --with-gnm option to enable GNM * Add --enable-lto switch to turn on link time optimization (GCC >= 5) * Add --with-spatialite=dlopen --with-spatialite-soname=libspatialite.so[.X] syntax to allow linking against spatialite through dlopen() mechanism (#6386) * Add a lot of warning options when available in the compiler. * Change detection test of podofo to work with podofo 0.9.3 * Fix gdalserver compilation with recent GNU libc (such as in Arch Linux) (#6073) * Add pkg-config support for libkml fork (#6077) * Update config.sub and config.guess to their latest upstream versions so as to be able to detect new architectures (android, ppcle64, etc..) (#6101) * Fix python package installation in custom prefixes (#4563) * configure: check that CXX is really a working compiler (#4436) * Build support for Kakadu 7.7 * thinplatespline.cpp: avoid using optimized version of VizGeorefSpline2DBase_func4() with ICC versions that fail on it (#6350) * Add bash completion for GDAL/OGR utilities and scripts (#6381) * Add support for ODBC autodetection with mingw64 (#6000) * Remove macos 9 and older support (#6133). * Remove dist_docs, burnpath and pszUpdableINST_DATA in gcore/gdaldrivermanager.cpp as they are have not been used for a while (#6140). * Removed --without-ogr configure flag (#6117). Always build with OGR. Build(Windows): * Add support for Visual Studio 2015 (a.k.a MSVC_VER=1900 or VC 14) * Add KAKFLAGS to nmake.opt with KDU_{MAJOR,MINOR,PATCH}_VERSION define-s. * nmake.opt: make it less error prone to define SETARGV when paths include spaces (#6032) * nmake.opt: add CL.EXE compiler option /FC to display full path of source code file in diagnostics * Remove WinCE support (#6133) * nmake.opt: add /I flag to the INSTALL command so that xcopy will be smart enough to create a directory when copying files and avoid prompting for user input (https://github.com/OSGeo/gdal/pull/95) * Enable mssql spatial blugin build, use BCP as the default option for the sqlncli enabled builds * Add support to find MrSID 9.5 dll * Fix Windows build with recent MySQL versions and Visual Studio 2015 (#6457) Build(all): * Compatibility with C++11 and C++14 * Optional crypto++ dependency for /vsicrypt/ support * Optional mongocxx dependency for MongoDB support Other: * Disable copy constructor and assignment operators in classes OGRFieldDefn, OGRGeomFieldDefn, OGRFeature, GDALMultiDomainMetadata, GDALDefaultOverviews, GDALOpenInfoGDALDataset, GDALRasterBlock, GDALRasterBand and GDALDriver (#6100) == GDAL 2.1.0 - Overview of Changes == Port: * Add /vsicrypt/ virtual file system for reading/creating/update encrypted files on the fly, with random access capabilities * Add /vsis3/ and /vsis3_streaming/ virtual file systems to read/write objects from AWS S3 buckets * /vsizip/: avoid returning the previous file list of an already opened .zip if it has changed afterwards (#6005) * /vsizip/: use CP437 on Windows when ICONV support is available (#6410) * /vsimem/: implement append mode (#6049) * /vsistdin/: fix caching of first 1MB / VRT driver: read XML content from /vsistdin/ in a streaming compatible way (#6061) * /vsistdout/: flush when closing the handle (#6149) * Add VSIReadDirEx() with a limit on the number of files before giving up and corresponding VSIFilesystemHandler::ReadDirEx() virtual function * cpl_vsil_cache: rework to be able to work on very large files on 32bit systems * Add CPLThreadLocaleC class to use thread-specific locale settings (when available) * VSIWin32Handle::Flush(): add VSI_FLUSH config option that can be set to TRUE to force FlushFileBuffers(). (hack related to #5556) * Fix VSIL append mode in Windows (#6065) * Make CPLCreateMutexEx() support CPL_MUTEX_REGULAR; and fix CPL_MUTEX_ADAPTIVE to avoid continuing in code path for creation of recursive mutex * Add CPLWorkerThreadPool API * Add CPLGetThreadLocalConfigOption() * Fix CPL_LSBINT16PTR() and CPL_LSBINT32PTR() to work with non-byte pointer (#6090) * CPLRecodeStub(): add special case for CP437 -> UTF-8 when input is only printable ASCII * CPLHTTPFetch(): add LOW_SPEED_TIME and LOW_SPEED_LIMIT options * CPLGetValueType(): improve heuristics to avoid detecting some WKB strings as real numbers (#6128) * Add CPL_SHA256* and CPL_HMAC_SHA256 functions * Make CPLVirtualMemFileMapNew() work on all systems having mmap(), and thus GeoTIFF GTIFF_VIRTUAL_MEM_IO optimization too * Add VSI_MALLOC_VERBOSE() and similar macros to output an error message in case of failed alloc * CPLParseXMLString(): limit depth of elements to 10000 * Win32 file management: handle files whose total file path length is greater than 255 characters in Open() and Stat() * Make CPLFormFilename(absolute_path, "..", NULL) truncate when possible * Add VSIGetDiskFreeSpace() * Implement CPLIsInf() for Solaris 11 * Add a CPL_FINAL macro that expands to C++11 final keyword when C++11 is enabled, and use it in a few places * Avoid CPLEscapeString (CPLES_URL) encoding some characters unnecessarily (#5526) * Add CPLTestBool(), CPLTestBoolean(), CPLFetchBool() * Fix CPLGetValueType() to recognize D1 as a string and not a real number (#6305) * CPLFormFilename(): use '/' directory separator on Windows on /vsicurl_streaming/ files (#6310) * Add hack enabled by -DDEBUG_BOOL to detect implicit int->bool conversions that dislike MSVC (#6325) * Add hack to '#define NULL nullptr' when NULL_AS_NULLPTR is defined. Must be used together with -std=c++11 -Wzero-as-null-pointer-constant with GCC to detect misuses of NULL/nullptr (#6327) * Add VSIError mechanism to store errors related to filesystem calls, and use it for /vsis3/. Add new CPLE_ error numbers. (https://github.com/OSGeo/gdal/pull/98) * Fix CPLsscanf() to be conformant regarding how space/tab/... separators are handled, so as to fix OGR GMT to be able to read coordinates separated by tabulations (#6453) Core: * Change default value of GDAL_CACHEMAX to 5% of usable physical RAM * Allow open options name to be prefixed by @ to be silently ignored when not existing in driver options (RFC 60) * Honour NBITS metadata item when doing RasterIO() with non-nearest resampling (#6024) * GDALClientServer: pass open options to INSTR_Open call * Improve performance of GDALCopyWords() float->byte/int16/uint16 by using SSE2 * Decorate prototypes of RasterIO() related operations with CPL_WARN_UNUSED_RESULT * Avoid deadlock when writing 2 datasets in 2 threads (#6163) * Remove obsolete symbols __pure_virtual(), GDALCreateProjDef(), GDALReprojectToLongLat(), GDALReprojectFromLongLat() and GDALDestroyProjDef() * Remove obsolete non-template-based implementation of GDALCopyWords(). We don't support anymore such compilers * GDALJP2Box::ReadBoxData(): do not abort if memory allocation fails * Add GDALAdjustValueToDataType() in C API, and use it in GDALTranslate() and GDALWarp() * GDALDefaultOverviews::CreateMaskBand(): fix logic error related to writing per-band mask * Block cache: fix excessive memory consumption when dealing with datasets with different block sizes (#6226) * EXIFExtractMetadata(): fix potential 1-byte write buffer stack overflow * MDReader: do no attempt reading side-car files on /vsisubfile/ (#6241) * GDALCheckBandCount(): allow by default a maximum number of bands to 65536 * GDALOpenInfo: add StealSiblingFiles() and AreSiblingFilesLoaded() methods * GDALOpenInfo::GetSiblingFiles(): give up after GDAL_READDIR_LIMIT_ON_OPEN (default=1000) files in the directory * GDALDefaultOverviews: add TransferSiblingFiles() method * GDALDriver::DefaultCreateCopy(): fix logic related to propagation of NBITS / PIXELTYPE metadata items as creation options * Reorder driver registration a bit so that formats with efficient identification are probed first * Add GDALIdentifyEnum (GDAL_IDENTIFY_UNKNOWN, GDAL_IDENTIFY_FALSE or GDAL_IDENTIFY_TRUE) for return values of Identify() * GDALLoadRPCFile(): load .rpc files from Ikonos products that have extra end-of-line character (#6341) * Export GDALRegenerateOverviewsMultiBand() symbol, but mostly for plugins (#6342) * Add GDAL_NO_AUTOLOAD to compile out the body of AutoLoadDriver (#6380) * Avoid ComputeStatistics(), GetHistogram() and ComputeRasterMinMax() to use only the first column of blocks in approximation mode for a raster whose shape of blocks is a square (#6378) * Add GDALGetDataTypeSizeBits() and GDALGetDataTypeSizeBytes(). * GDALReadOziMapFile(): fix reading Ozi external files from virtual file systems (https://github.com/OSGeo/gdal/pull/114) * Add CPLSetCurrentErrorHandlerCatchDebug() to enable custom error handlers not to intercept debug messages Algorithms: * RPC transformer: provide SSE2 accelerated transformer * RPC: fix off-by-half pixel computation of (pixel, line), and in bilinear and bicubic RPC DEM interpolation; fix off-by-one pixel registration for Pleiades RPC (#5993) * RPC: make RPCInverseTransformPoint() check convergence has been reached (#6162) * RPC DEM: optimize when DEM is in NAD83 or any other geodetic CS that transform as a no-op to WGS 84 * RPC DEM: take into account vertical datum of the DEM when present to convert DEM elevations to ellipsoidal heights (#6084) * RPC DEM: do fallback cubic -> bilinear and bilinear -> near on DEM edges * RPC DEM: improve inverse transformer to validate error threshold and improve convergence (#6162, #6377) * RPC: fix issues with dateline (#6472) * TPS solver: discard duplicated GCP to avoid avoidable error, warning when 2 GCPs have same (pixel,line) but different (X,Y) or the reverse * Warper: rework multithreaded computations to use a thread pool rather than forking threads each time * Warper: avoid really excessive processing time for some warping with target areas completely off the source raster (especially when involving RPC) (#6182) * Warper: CreateKernelMask(): fix potential 32 bit integer overflow when using warp memory value > 2GB (#6448) * gdal_grid: add linear interpolation algorithm * gdal_grid: add invdistnn algorithm, variation on the existing inverse distance weighting algorithm with quadtree to search for points only in the neighborhood (#6038) * gdal_grid: fix crash in optimized mode with GCC 4.4 on 64bit (#5987) * gdal_grid: compile gdalgrid AVX optimization for Windows when supported by compiler * Add GDALTriangulationXXXX() API through libqhull * Sieve filter: fix crash on nodata polygons (#6096) * Sieve filter: improvement to walk through the biggest neighbour chain until we find a polygon larger than the threshold (#6296) * GDALFPolygonize(): factor implementation with integer case * GDALComputeMedianCutPCT(): fix to make it work with rasters with more than 2 billion pixels (#6146) * Overview: Make average and gauss methods aware of transparent color table entries (#6371) Utilities: * gdalinfo: workaround bug in proj <= 4.9.1 on datasets with a SRS with a vertical shift grid (#6083) * gdal_translate: detect more reliably if specified bands are not in default order * gdal_translate: fix -a_nodata with negative values on rasters that have PIXELTYPE=SIGNEDBYTE; fix preserving PIXELTYPE=SIGNEDBYTE with VRT * gdal_translate: fix random behaviour when -scale is used without source bounds (#6455) * gdal_rasterize: do on-the-fly reprojection of input vector onto output raster * gdal_rasterize: rasterize: always create output with 1/2 cell buffer of input geometry envelope (#6058) * gdal_rasterize: add the -dialect option * gdal_rasterize: accept NaN as a valid value for -init / -burn (#6467) * gdalwarp: add -doo option to specify open options of (existing) output dataset * gdalwarp: if RPC_DEM warping option is specified, use exact transformer by default (#5993) * gdalwarp: make it honour nodata value of existing dataset (if -dstNoData isn't explicitly specified) * gdalwarp: do not propagate STATISTICS_ of second or following source datasets * gdalwarp: do not emit warning when using -cutline with a SRS and the source raster has RPC or GEOLOCATION * gdalwarp: check that the cutline is valid after transformation/reprojection * gdalwarp: better deal when dealing with a mix of RGB and RGBA datasets as input * gdalwarp: fix -srcnodata to not put garbage values as target dstnodata (#6315) * gdalwarp: densify cutline to avoid invalid geometry after reprojection to source raster, especially in the RPC case (#6375) * gdalserver: add a -nofork mode (Unix only for now), so that multiple clients can connect to the same dataset. Useful for safe 'concurrent' updates * add gdal_pansharpen.py script * gdal2tiles.py: fix error on a raster with less than 3 bands that resulted in a 'IndexError: list index out of range' * gdal2tiles.py: Change EPSG:3785 / EPSG:900913 into EPSG:3857 (#5622) * gdal2tiles.py: add Leaflet template (https://github.com/OSGeo/gdal/pull/71) * gdal2tiles.py: add -q switch for quiet mode * gdaldem: correctly deal with NaN as nodata value (#6066) * gdaldem color-relief: deal with the case of repeated entries with the same value and the input raster has pixels that match that value exactly (#6422) * gdaladdo: emit error message if passed an invalid dataset name (#6240) * gdaladdo: do not silence warnings when opening in update mode, if the open is successful * gdalbuildvrt: fix potential crash when using -b switch (#6095) * gdalbuildvrt: accept nan as value for -srcnodata and -vrtnodata * gdalbuildvrt: return non zero return code if the flush of the VRT to disk failed * gdal_merge.py: takes again into account -n flag (#6067) * gdalbuildvrt / gdal_translate / VRT: use floating point values for source and destination offsets and sizes (#6127) * gdalmove.py: fix to run with GDAL 2.0 gdal.InvGeoTransform() signature * gdal_calc.py: Add * from gdalnumeric to gdal_calc.py eval namespace again, to fix 2.0 regression that made for example 'log10(A)' to no longer work (https://github.com/OSGeo/gdal/pull/121) Python samples: * gdalpythonserver.py: update to protocol 3 AAIGRID: * when writing with floating-point values, ensure at least one value has a decimal point to avoid issues with some readers (#6060) BMP driver: * BMP: avoid too big color table allocation in case of wrong iClrUsed value BLX driver: * converted to support VirtualIO ECRG driver: * change subdataset definition to make sure that they only consist of frames of same scale (#6043) * fix base34 decoding and Windows filename handling (#6271) ECW driver: * use wide char Open API on Windows when GDAL_FILENAME_IS_UTF8=YES (https://github.com/OSGeo/gdal/pull/70) * fix reading of ECW in /vsi file systems (with SDK >= 4) (#6482) GeoRaster driver: * fix deflate read error depending on endianness (#6252) GIF driver: * libgif: partial resync with giflib master (but keep ABI of 4.1.6) * fix crash when CreateCopy a source with one color in the color table GMT driver: * validate raster dimensions on opening, and acquire mutex in dataset destructor GPKG driver: * write support: fix various issues in update scenarios when interacting with the GDAL block cache that could result in lost/corrupted band data to be written in tiles (#6309) * write support: fix potential use of freed sqlite temporary DB handle when generating overviews with partial tiles (#6335) * write support: fix potential crash in scenarios when block cache is full (#6365) * write support: fix inversion of row/column in one of the SQL request involved in partial tiles management (#6339) * fix generation of files with 1-band paletted input dataset. Also improve the logic to retrieve color palette when forcing BAND_COUNT=1 on opening (#6450) GTiff driver: * add NUM_THREADS creation and open options to enable multi-threaded compression * fix GTiffDataset::IsBlockAvailable() wrong behaviour when compiling against internal libtiff, when a BigTIFF file has a TileByteCounts with LONG/4-byte counts and not LONG8 (#6001) * Correctly take into account overridden linear units for a geotiff with a EPSG PCS code (#6210, #4954) * make VirtualMemIO() work with non native endianness * improve VirtualMemIO() performance in tiled Contig read to single band copy * improve single band tiled VirtualMemIO reading * improve DirectIO() to work on tiled uncompressed geotiff, for non-resampling and nearest resampling cases. Also improve performance of resampling cases on un-tiled files * fix DirectIO() mode with complex types and inverted endianness (#6198) * optimize writing of 12-bit values * implement lazy loading of .aux.xml and .tfw/.wld * Internal libtiff: update to CVS HEAD post libtiff 4.0.6 * make SetColorInterpretation(GCI_AlphaBand) work on a 5 band or more GeoTIFF (#6102) * automatically set PHOTOMETRIC=RGB if manually assigning color interpretation Red,Green,Blue to band 1,2,3 before directory crystallization (#6272) * add GEOTIFF_KEYS_FLAVOR=ESRI_PE creation option to write EPSG:3857 in a ESRI compatible way (#5924) * call XTIFFInitialize() in LibgeotiffOneTimeInit() as the former isn't thread-safe, so better call it from the later which is thread-safe (#6163) * fix reading and writing angular units different from degree (namely arc-second, arc-minute, grad, gon and radian) (#6171) * do not use VirtualMemIO optimization on compressed /vsimem/ files (#6195) * correctly set GTRasterTypeGeoKey=RasterPixelIsPoint if AREA_OR_POINT=Point but there is no SRS set (#6225) * improve a bit error detection when writing * check free space before writing (only for big, non sparse, uncompressed) * do not read large 'one row' JBIG compressed files with the scanline API (#6264) * Fix SetMetadata() to properly clear existing PAM metadata (complement to #5807) * prevent potential out of bounds read/write to TIFFTAG_EXTRASAMPLES (#6282) * do not use first directory as potential mask, to avoid assertion in GTiffDataset::SetDirectory() (#6287) * reject files with strips/tiles/scanlines bigger than 2 GB to avoid 32 bit integer overflow. Also in case of files with Contig PlanarConfiguration do not make reading one block for band 2 OK when reading for band 1 issued an error (#6288) * GTIFFWriteDirectory(): avoid memory leak of codec related memory (#2055) * Make ALPHA=NO in CreateCopy() cancel alpha color interpretation even if present in source raster * fix problem with implicit overviews of JPEG-compressed files (#6308) * fix compilation problem with internal libtiff if DEFER_STRILE_LOAD isn't defined (which is not the default configuration) (https://github.com/OSGeo/gdal/pull/90) * use more appropriate error message when 4GB threshold is reached with external overviews, and try to make doc of BIGTIFF=IF_NEEDED/IF_SAFER clearer (#6353) HDF4 driver: * Skip quotation mark when parsing HDF-EOS metadata. HDF5 driver: * add Komsat Mission ID to possible value for HDF5 SAR product (https://github.com/OSGeo/gdal/pull/103) HF2 driver: * fix reading side of the driver to work on architectures where char is unsigned, like PPC or ARM (#6082) HFA driver: * when reading projection, preserve EPSG code if AutoIdentifyEPSG() identified the SRS, even if a PE string is present (#6079) * keep TOWGS84 even when using ESRI PE string (#6158) * fix crashes on corrupted files (#6208, #6286) INGR driver: * check that RLE decoding produced the expected number of bytes and error out otherwise; test that 'random' line seeking actually works JPEGLS driver: * fix build error (#6430) JP2ECW driver: * honour psExtraArg->eResampleAlg when upsampling (#6022) JP2KAK driver: * honour psExtraArg->eResampleAlg when upsampling (#6022) * try rounded dimensions to decide if the data is being requested exactly at a sub-resolution * support v7.7 on Unix (additional changes potentially needed on Windows) JP2OpenJPEG driver: * Safer multi-threaded use JPEG driver: * add USE_INTERNAL_OVERVIEWS open option (default to TRUE as in recent versions, can be set to FALSE to hide internal overviews KMLSuperOverlay driver: * fix 2.0 regression with some RasterIO() requests involving resampling (#6311) * recognize datasets that have a intermediate <Folder> that forms a <Document><Folder><Region/><GroundOverlay/> structure (#6343) LCP driver: * fix wrong use of endianness macros to fix behaviour on big-endian hots Leveller driver: * update to v9 read support (#5632,) MBTiles driver: * add raster write support * fix so that datasets whose lowest min zoom level is 16 or above are recognized * be robust to invalid 'bounds' at dataset opening (#6458) MEM driver: * avoid 32 bit overflows NetCDF driver: * add support for reading NC4 unsigned short attributes and variables (#6337) * read correctly nodata values in [128,255] range for (unsigned) Byte data type (#6175) * implement Get/SetUnitType() using the standard units attribute (https://github.com/OSGeo/gdal/pull/96) * optimize IReadBlock() and CheckData() handling of partial blocks in the x axis by re-using the GDAL block buffer instead of allocating a new temporary buffer for each block (#5950) * full read/write support for new NetCDF4 types NC_UBYTE, NC_USHORT, NC_UINT and NC_STRING for variables (except for NC_STRING) and attributes (https://github.com/OSGeo/gdal/pull/99) * add support for the geostationary projection (#6030) * fix one byte heap write overflow in NCDFTokenizeArray() (#6231) * fix potential buffer overflows with uses of nc_inq_varname(), nc_inq_attname() and nc_get_att_text() (#6227) * validate that gridmapping:GeoTransform has 6 values (#6244) * fix wrong use of deallocator when writing a GEOLOCATION array, and other issues * limit number of bands reported to 32768 by default * validate raster dimensions * validate content of NC_GLOBAL#GDAL variable (#6245) NGSGEOID driver: * make it work on > 2GB file NITF driver: * data/nitf_spec.xml: Add CSCCGA, MENSRB, SENSRB, STREOB, ENGRDA, EXPLTB and PATCHB TREs (https://github.com/OSGeo/gdal/pull/81, #6285, https://github.com/OSGeo/gdal/pull/86) * fix parser to properly deal with variable length items not in first nesting level Northwoord driver: * fix computation of intermediate color table values on non-Intel platforms (#6091) NTv2 driver: * support reading/writing/appending to files with big-endian order (cf https://github.com/OSGeo/proj.4/issues/345) OpenJPEG driver: * do not expose block dimensions larger than dataset dimensions to avoid wasting memory (#6233) PCIDSK driver: * Remove the old driver (#6172) PDF driver: * add PDFium library as a possible back-end. Initial support contributed by Klokan Technologies GmbH (http://www.maptiler.com/) * workaround a bug of PoDoFo 0.9.0 by avoiding loading of vector content in raster-only mode (doesn't prevent the crash if reading the vector layers) (#6069) * make OGC BP registration work with media box where bottom_y is negative and top_y = 0 (in non rotated case) * make OGC BP registration work with media box where min_x != 0 (in non-rotated case) * correctly take into account non-meter linear units with OGC BP encoding, and add support for US FOOT (#6292) PDS driver: * change default values of PDS_SampleProjOffset_Shift and PDS_LineProjOffset_Shift to 0.5 (#5941) * fix nodata value for UInt16 to be 0 (#6064) * accept 'ODL_VERSION_ID = ODL3' in header (#6279) PGChip driver: * driver removed from sources PLScenes driver: * PLScenes V0: avoid opening raster to generate dummy .aux.xml PNG driver: * Support writing 1, 2 or 4 bit single band * Add NBITS creation option * fix XML of creation option list * Update internal libpng to 1.2.56 PostgisRaster driver: * avoid Identify() to recognize OGR PostgreSQL connection strings with schemas option and cause loud 'QuietDelete' (#6034) Raw drivers: * better support for direct read of more than 2GB in single gulp (untested though) RIK driver: * fix Identify() method to recognize again non-RIK3 RIK datasets (#6078) RMF driver: * implement GetNoDataValue() * add read/write access to new RMF format for files larger than 4 Gb (version=0x201) (https://github.com/OSGeo/gdal/pull/11) ROIPAC driver: * Support offset and scale band (#6189) VICAR driver: * change PDS_SampleProjOffset_Shift and PDS_LineProjOffset_Shift default values to 0.5 (#5941) * fix loss of precision in scale and offset VRT driver: * expose implicit 'virtual' overviews for VRT whose bands are made of a single SimpleSource/ComplexSource * gdalvrt.xsd: Add capitalized versions of true and false (#6014) * GetSingleSimpleSource(): check there's a single source (#6025) * honour VRTRasterBand NBITS metadata with SimpleSource and ComplexSource * properly take into account nodata value declared at VRT band level when doing resampling with non-nearest * honour relativeToVRT when using AddBand() to add a VRTRawRasterBand (https://github.com/OSGeo/gdal/pull/67) * VRT warp: fix crash with implicit overviews and destination alpha band (#6081) * make GetDefaultHistogram() on a sourced raster band save the result in the VRT (#6088) * serialize NODATA and NoDataValue items with %.16g, e.g. so as to be able to hold large int32 nodata values (#6151) * VRTSourcedRasterBand: make ComputeRasterMinMax() and ComputeStatistics() forward bApproxOK to overview band (useful for implicit overviews) * make CreateCopy() preserve NBITS metadata item * avoid loading sibling file list if not available * VRT raw: don't truncate last figure of ImageOffset if there are left space padding (#6290) * VRTWarpedDataset::SetMetadataItem(): fix crash when calling with name=SrcOvrLevel and value=NULL (#6397) * Warped VRT: fix deadlock in situation where warped VRT datasets are read in multiple threads and the block cache reaches saturation. Also add a GDAL_ENABLE_READ_WRITE_MUTEX config option that can be set to NO to disable the read/write mutex mechanism in cases where it would deadlock (#6400) WMS driver: * add a IIP (Internet Imaging Protocol) minidriver * limit number of zoom levels for ArcGIS MapServer JSon (#6186) * determine a resolution that will not result in a number that is larger than the maximum size of an integer. Any value that exceeds the maximum size of an integer will raise an invalid dataset dimensions error. (https://github.com/OSGeo/gdal/pull/89) XYZ driver: * be more robust to not exactly equal X and Y spacing (#6461) == OGR 2.1.0 - Overview of Changes == Core: * Add OGREditableLayer class to add editing capabilities to drivers with none or limited editing capabilities * OGRGeometry: add DelaunayTriangulation() method (GEOS >= 3.4) * OGRGeometry and derived classes: implement copy constructor and assignment operator (#5990) * OGRGeometry: Fix result of Equals on POINT EMPTY with POINT(0 0) * OGRFeature SetField(): more type conversions allowed, particularly with array types * OGRFeature::SetGeometry()/SetGeometryDirectly(): make it work when passed geometry is the currently installed geometry (#6312) * OGR SQL: do not silently skip NULL values in the first records when evaluating a SELECT DISTINCT (#6020) * OGR SQL: correctly sort NULL values in first positions (#6155) * OGR SQL: fix CAST(x AS bigint) to return an evaluated int64 node, and not int32 (#6479) * OGR SQL: handle 'fid' as Integer64 in where clause, and allow CAST(fid AS bigint) in selected columns (#6484) * Add OGRUpdateFieldType() * Decorate a few functions/methods of the OGR layer API with CPL_WARN_UNUSED_RESULT * WKT export: use 15 significant figures, instead of 15 figures after decimal point (#6145) * WKT export: do not append .0 after non-finite values (#6319) * Fix typo in definition of name of OGR_FD_ReorderFieldDefns (final s was missing) * OGRLayer::SetIgnoredFields(): properly reset state of non first geometry fields (#6283) * Make OGRLayer::SetSpatialFilter(GetSpatialFilter()) work with non empty spatial filter (#6284) * OGRLayerDecorator: add missing CreateGeomField() * OGRLayer::Erase(): do not discard input geometries that have no intersection with method layer (#6322) * OGRLayer::Erase(): Speedup = ~70% * Add OGRPreparedGeometryContains() * Use prepared geometry intersects as pretest in layer Intersection, Union, and Identity methods if requested. Use prepared geometry containment as pretest in layer Intersection method if requested. * Bail out from layer algebra methods if GEOS calls fail and not SKIP_FAILURES. * OGR_G_SetPoints(): error out if padfX or padfY == NULL, do not change coordinate dimension to 3D when pabyZ == NULL, fix optimization on linestring to call setPoints() only if the strides are the ones of a double, not 0 as incorrectly done before (#6344) * OGRParseDate(): more strict validation to reject invalide dates (#6452) OGRSpatialReference: * Upgrade to EPSG database v8.8 * Add support for SCH (Spherical Cross-track Height) projection * Optimize reprojection typically between WGS84 based SRS and WebMercator * Correctly transform Mercator_2SP and _1SP to ESRI Mercator, and back from ESRI Mercator to Mercator_2SP (#4861) * No longer enforce C locale if running against latest proj that is locale safe (4.9.2 or later) * EPSGGetPCSInfo(): use pcs.override.csv in priority over pcs.csv to read projection name, UOM, UOMAngle, GeogCS, etc... (#6026) * morphToESRI(): use GCS_WGS_1972 as GCS name for EPSG:4322 (#6027) * morphToESRI(): use Mercator_Auxiliary_Sphere projection for EPSG:3857. morphFromESRI(): map Mercator_Auxiliary_Sphere to EPSG:3857 (#5924) * Align hard-coded WKT of well known GCS definitions of WGS84, WGS72, NAD27 and NAD83 with the WKT of their EPSG def (#6080) * morphFromESRI(): special case with PROJCS name 'WGS_84_Pseudo_Mercator' (#6134) * OSR C API: fix declarations of OSRSetAxes() and OSRSetWagner(), and add missing OSRSetHOMAC(), OSRSetMercator2SP() and OSRSetTPED() (#6183) * Recognize EPSG 9835 method (Lambert Cylindrical Equal Area (Ellipsoidal)), needed for EPSG:6933 PCS for example * importFromProj4/exportToProj4(): rework linear unit conversion between WKT name/values and proj4 unit name, and extend its scope in WKT to proj4 conversions * OSR ESRI .prj: add support for reading custom ellipsoid in Parameters line * on import of +proj=geos, if +sweep=x is used then store it as a proj4 extension node (#6030) Utilities: * ogrinfo / ogr2ogr: implement @filename syntax for -sql and -where * ogr2ogr: prevent the -gt setting from overriding transaction group size of 1 set by skipfailures earlier (#2409) * ogr2ogr: warn if -zfield field does not exist in source layer * ogr2ogr -skip: rollback dataset transaction in case of failure (#6328) * ogr2ogr: fix -append with a source dataset with a mix of existing and non existing layers in the target datasource (#6345) * ogr2ogr: imply quiet mode if /vsistdout/ is used as destination filename * ogr2ogr: make -dim and -nlt support measure geometry types CartoDB: * fix GetNextFeature() on a newly create table (#6109) * defer 'CartoDBfycation' at layer closing * optimize feature insertion with multiple rows INSERT CSV driver: * add editing capabilities of existing files * add X_POSSIBLE_NAMES, Y_POSSIBLE_NAMES, Z_POSSIBLE_NAMES, GEOM_POSSIBLE_NAMES and KEEP_GEOM_COLUMNS open options * add HEADERS open option to force OGR to handle numeric column names. (PR #63) * add EMPTY_STRING_AS_NULL=YES/NO open option * implement compatibility enhancements for GeoCSV specification (#5989) * fix detection of TAB delimiter in allCountries.csv when the first line has a comma (#6086) * fix issues with leading single quote, and missing first line after ResetReading(), when parsing allCountries.txt (#6087) * speed-up GetFeatureCount() on allCountries.txt * on CreateDataSource() with a .csv name, do not try to open other existing .csv files in the directory * make CreateGeomField() returns OGRERR_NONE in case of success instead of OGRERR_FAILURE (#6280) * avoid adding trailing comma in header line when writing 'WKT,a_single_field' DGN driver: * add partial 3D transformation support for cell headers DXF driver: * detect files without .dxf extension (#5994) * fix handling of ELLIPSE with Z extrusion axis = -1 (#5705) * take into account full definition of spline entity (degree, control points, weights and knots) when stroking splines (#6436) * better handling of various object coordinate systems found in dxf files for point, line, polyline, spline and ellipse entities. Add anchor position to text styles. Remove polygon/polyface mesh parsing from polyline entity (#6459) ElasticSearch driver: * use get /_stats instead of /_status for ElasticSearch 2.0 compatibility (#6346) FileGDB driver: * make CreateFeature() honour user set FID, and implement more fine grained transaction for Linux/Unix * give a hint of using FileGDB SDK 1.4 is FileGDB compression is used GeoJSON driver: * Add editing capabilities of existing files * Add ARRAY_AS_STRING=YES open option * Use '%.17g' formatting by default for floating-point numbers and add SIGNIFICANT_FIGURES layer creation option (#6291) * add a json_ex_get_object_by_path() function * fix crash on null / non-json object features (#6166) * serialize string values that are valid JSon dictionary or array as it (ie do not quote them) * make sure there's enough space to write the FeatureCollection bbox (#6262). Also avoid duplicating FeatureCollection bbox if source has one (trunk only) * Export POINT EMPTY as having a null geometry, instead as of being POINT(0 0) (#6349) * Do not 'promote' a null field to OFTString type if it had another type before (#6351) GME driver: * Driver removed. Maps Engine being shut down at the end of January 2016. GML driver: * VFR: add new attribute DatumVzniku (v1.6) * VFR: fix ST_UVOH type handling * VFR: fix ZpusobyOchrany attributes (data types and names) * VFR: fix CisloDomovni attributes (Integer->IntegerList) * VFR: fix TEA attributes of StavebniObjekty * add NAMESPACE_DECL=YES option to OGR_G_ExportToGMLEx() to add xmlns:gml=http://www.opengis.net/gml or http://www.opengis.net/gml/3.2 declaration; Also accept GML2 or GML32 as valid valiues for FORMAT option (#6214) * serialize in .gfs file the name of the geometry element when it is 'geometry' since this is a particular case (#6247) * fix logic error in BuildJointClassFromScannedSchema() (#6302) GPKG driver: * make it accept files with non standard extension if they still have the correct application_id (#6396); also accept the .gpkx extension that may be used for extended geopackages * emit warning when generating a database without .gpkg/.gpkx extension (#6396) * as GPKG 1.1 uses a different application_id, emit a more specific warning if the application id starts with GPxx (but is not GP10). Add GPKG_WARN_UNRECOGNIZED_APPLICATION_ID config option to avoid the warning * correct scope of gpkg_geom_XXXXX extensions to be read-write, and allow reading geometry types CURVE or SURFACE * avoid trying to insert a gpkg_geom_XXXX extension if already done (#6402) * writer: implement strategy to flush partial_tiles temporary database when it becomes too big (#6462) * writer: when writing to GoogleMapsCompatible tiling scheme, better deal with source rasters in EPSG:4326 with latitude = +/-90 (#6463) * fix generation of files with 1-band paletted input dataset. Also improve the logic to retrieve color palette when forcing BAND_COUNT=1 on opening (#6450) GPX driver: * fix crash when parsing a 'time' extension element at route/track level (2.0 regression, #6237) ILI driver: * ILI1: Support for Surface polygon rings spread over multiple geometry records * ILI1: add string TID support (https://github.com/OSGeo/gdal/pull/91) * Fix crash with models using types derived from INTERLIS * Fix memory leaks (#6178) JML driver: * remove arbitrary limitation preventing from reading geometries with <gml:coordinates> larger than 10 MB (#6338) KML driver: * fix crash on KML files without content but with nested folders (#6486) LIBKML driver: * for documents without folder, use document name when available as name of layer (#6409) Memory driver: * add support for sparse feature IDs * add ADVERTIZE_UTF8 layer creation option MITAB driver: * Add support for block sizes other than 512 bytes in .map files, for MapInfo 15.2 compatibility (#6298) * write correct datum id for EPSG:3857 * read MID files with TAB delimiter and empty first field (#5405) * use projection code 29 when exporting non-Polar Lambert Azimuthal Equal Area (#5220) * fix crashes when parsing invalid MIF geometries (#6273) MSSQLSpatial driver: * Implement MSSQL bulk insert (#4792) * do not treat a primary key that is not of integer type as the FID (#6235) NTF driver: * fix potential buffer overflows when reading too short lines (#6277) ODBC driver: * remove limitations to 500 columns ODS driver: * fix loss of precision in formula computation OpenFileGDB driver: * do not emit warning if SDC/CDF table detected and that FileGDB driver is present * fix min/max on columns without indices (#6150) * build correct geometry for a multi-part wkbMultiLineStringZ (#6332) * add support for reading SHPT_GENERALPOINT (#6478) OSM driver: * correct fields ids for the (non frequently used) Node message * do not override 'our' osm_id (the node, way or relation id) with a tag named 'osm_id' (#6347) * properly deal with polygons in other_relations geometrycollection (#6475) PG driver: * Add PRELUDE_STATEMENTS and CLOSING_STATEMENTS open option to be for example able to specify options, like statement_timeout, with pg_bouncer * Fix 2.0 regression when overwriting several existing PostGIS layers with ogr2ogr (#6018) * Update PG, PGDump and CartoDB drivers to correctly export POINT EMPTY for PostGIS 2.2 * avoid resetting error potentially emitted by ExecuteSQL() (#6194) * sanitize management of quoting for FID column at layer creation * fix to get SRID on result layer with PostGIS 2.2 * in copy mode (the default on layer creation), do not truncate the concatenated string list to the field width (#6356) * make such that GEOMETRY_NAME layer creation option is honoured in ogr2ogr when the source geometry field has a not-null constraint (#6366) * read and set DESCRIPTION metadata item from/into pg_description system table; add DESCRIPTION layer creation option * support int2[] and numeric[] types, better map float4[] type * remove code that was intended to handled binary cursors as it cannot be triggered * fix append of several layers in PG_USE_COPY mode and within transaction (ogr2ogr -append use case) (#6411) PGDump driver: * fix issue with case of ogc_fid field in case the FID layer creation option is not set by user or by ogr2ogr (related to #6232) * in copy mode (the default on layer creation), do not truncate the concatenated string list to the field width (#6356) * make such that GEOMETRY_NAME layer creation option is honoured in ogr2ogr when the source geometry field has a not-null constraint (#6366) * set DESCRIPTION metadata item from/into pg_description system table; add DESCRIPTION layer creation option Shapefile driver: * accept opening standalone .dbf files whose header length is not a multiple of 32 bytes (#6035) * fix REPACK crash on shapefile without .dbf (#6274) * add capability to restore/build a missing .shx file when defining SHAPE_RESTORE_SHX to TRUE (#5035) * avoid CreateLayer() to error out when passed wkbUnknown | wkb25D (#6473) SQLite/Spatialite driver: * support file:xxx URI syntax (derived from patch by joker99, #6150) * fix heuristics in OGRSQLiteSelectLayer::GetExtent() to not be used when there's a sub SELECT (#6062) * fix crash on GetLayerByName('non_existing_table(geom_column)') (#6103) * fix OGRSQLiteSelectLayerCommonBehaviour::GetBaseLayer() to no longer 'eat' consecutive characters in layer name (#6107) * Spatialite: turn debug messages warning about update not being supported because of missing or too old spatialite version as errors, and return NULL to the caller (#6199) * fix memleak in OGRSQLiteTableLayer destructor when updating geometry_columns_time * VFS: increase mxPathname to 2048 by default, and provide OGR_SQLITE_VFS_MAXPATHNAME config option to be able to configure that higher if that would be needed. Useful when dealing with very long names like /vsicurl/.... with AWS S3 security tokens * VFS: do not probe -wal files on /vsicurl/ SXF driver: * fix wrong use of endianness macros to fix behaviour on big-endian hots * add recoding from CP1251 for TEXT attribute that is now decoded * fix various issues (#6357) VRT driver: * implement CloseDependentDatasets() * fix editing with 'direct' geometry mode which could cause attribute column to be empty (#6289) * fix crash with a OGRVRTWarpedLayer using a source layer that would have non geometry column (unlikely to happen currently as this would require explicit disabling it, but more likey with following commit that createe VRT non-spatial layer implictly when the source is non-spatial, linked to #6336) * avoid creating an implicit wkbUnknown geometry field when the source has no geometry column and there's no XML elements related to geometry fields (#6336) XLSX driver: * fix reading sheets with more than > 26 columns and 'holes' (#6363) XPlane driver: * extend ICAO identifiers to 5 digits (#6003) == SWIG Language Bindings == All bindings: * add a options parameter to gdal.ReprojectImage() to pass warp options * Change ReadRaster and WriteRaster to use GIntBig and the *IOEx-methods * prevent NULL file pointer from being passed to VSIF*L functions * make gdal.Rename() accept Unicode strings * add SpatialReference.GetAxisName() and SpatialReference.GetAxisOrientation() (#6441) * add SpatialReference.GetAngularUnitsName() (#6445) Java bindings: * Fix typemap for input parameter of type GIntBig (fixes GetFeature(long), DeleteFeature(long), etc...) (#6464) * Bump minimal java version to 1.5 in case SWIG generates anotations (#6433, patch by Bas Couwenberg) * GNUmakefile: add -f in rm commands * GNUmakefile: add support for all hardening buildflags Perl bindings: * Fix #6050: string formatting in croak. * Perl Makefile.PL: add support for all hardening buildflags (#5998) * use strict and warnings in overridden constructors. * add $VERSION to ogr_perl.i (OGR.pm), which is required by pause.perl.org. * Add some basic module info for CPAN. * Bugfix for Geo::OGR::Feature->new(). * Add many utility level algorithms as methods to various classes. * New class for XML stuff * New Makefile.PL, which can download and build GDAL. This allows automatic testing of the CPAN module. * Wrap VSIStdoutSetRedirection and allow creating datasets via an object, which can write and close. * Geo::OGR::Driver and Geo::OGR::DataSource are now Perl wrappers for respective GDAL classes. * Add to the error stack also errors from the bindings * Fix sending utf8 from Perl to GDAL. Should also remove some "uninitialized value" warnings. * ReadTile and WriteTile methods for Dataset, ReadTile accepts now tile size and scaling algorithm. * Improved Parent - Child management. * Improved support for 64bit ints. * Measures support in Geometry class. * Many new tests Python bindings: * make Feature.ExportToJson() output boolean value for a boolean field * support floating point coordinates for the source windows of Band.ReadRaster() and Band.ReadAsArray() * fix build with SWIG 3.0.6 (#6045) * make gdal.OpenEx() throw a Python exception in case of failed open when exceptions are enables with gdal.UseExceptions() (#6075) * Disable opening a NumPy dataset with a filename returned by gdal_array.GetArrayFilename(() unless GDAL_ARRAY_OPEN_BY_FILENAME is set to TRUE * disable the warning about using deprecated wkb25DBit constant as it uses a trick that prevents the bindings from being used by py2exe / pyinstaller (#6364)
2018-02-19 13:05:17 +01:00
CATEGORIES= geography
MASTER_SITES= http://download.osgeo.org/gdal/${PKGVERSION_NOREV}/
geography/gdal-lib: Update to 2.4.0 = GDAL/OGR 2.4.0 Release Notes = == In a nutshell... == * New GDAL drivers: - BYN: read/write support for Natural Resources Canada's Geoid binary format - EEDAI: read-only driver for Google Earth Engine Data API - IGNFHeightASCIIGrid: read-only driver to read IGN-France height correction ASCII grids - NGW: NextGIS Web read-only driver - NTv1: read-only driver for NTv1 datum shift grids * New OGR drivers: - EEDA: read-only driver for Google Earth Engine Data API - GeoJSONSeq: read/creation support of new-line or record-separator separated GeoJSON features (#378) - NGW: NextGIS Web read-write driver * Improved drivers: - BAG: add read support for variable-resolution grids, and write support for single-resolution grids - GTiff driver: add Lerc and WebP codecs - PostgisRaster: add support for out-db rasters - RMF - MSSQLSpatial * RFC 72: Make GDAL Python autotest suite use pytest framework * Add /vsihdfs/ virtual file system handler for Hadoop File System (via libhdfs) * Add /vsiwebhdfs/ read-write virtual file system for Web Hadoop File System REST API * gdal_contour rewriting: speed optimizations and capability to compute polygon isosurfaces. * Remove PHP and Ruby bindings. * Continued code linting in C++, Python scripts, Shell scripts and autotest == Backward compatibility issues == * The value of COMPRESSION_ZSTD used for ZStd-in-TIFF compression has been changed. ZStd-compressed TIFF files produced by GDAL 2.3.0 will not be readable
2019-03-09 18:39:01 +01:00
MAINTAINER= gdt@NetBSD.org
#MAINTAINER+= brook@nmsu.edu
Update to 2.2.3 * Add Makefile.common to prepare for geography/py-gdal Changelog: = GDAL/OGR 2.2.3 Release Notes = The 2.2.3 release is a bug fix release. == Build == * nmake.opt: Ensure PDB is included in release DLL if WITH_PDB requested (#7055) == Port == * /vsicurl/ and derived filesystems: redirect ReadDir() to ReadDirEx() (#7045) * /vsicurl/: enable redirection optimization on signed URLs of Google Cloud Storage. Helps for the PLScenes driver (#7067) * /vsicurl/: fix 2.2 regression regarding retrieval of file size of FTP file (#7088) * /vsis3/: fix to avoid invalid content to be sent if Write() writes more than 50 MB in a single call (#7058) * /vsis3/: fix Seek(Tell(), SEEK_SET) fails if current position is not 0 (#7062) * /vsis3/: fix support of non-ASCII characters in keys (#7143, #7146) * Fix CPLCopyTree() that doesn't properly on MSVC 2015 (and possibly other platforms) (#7070) == Algorithms == * RPC transformer: set output coordinates to HUGE_VAL when failure occurs, so that a following coordinate transformation can detect the error too (#7090) * GDALGrid() with linear algorithm: avoid assertions/segmentation fault when GDALTriangulationFindFacetDirected() fails (#7101) * GDALComputeProximity(): fix int32 overflow when computing distances on large input datasets (#7102) * GDALResampleChunk32R_Gauss: fix potential out of bounds access (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4056) == GDAL utilities == * gdalwarp: fix issue with GDALAdjustValueToDataType(Float32, +/- inf) that didn't preserve infinity, which affected gdalwarp -dstnodata inf (#7097) * gdalwarp -crop_to_cutline: reduce number of iterations to find the appropriate densification (#7119) * gdal_contour: return with non-0 code if field creation or contour generation failed (#7147) == GDAL drivers == GeoRaster driver: * fix int overflow (#6999) GPKG driver: * speed-up statistics retrieval on non-Byte datasets (#7096) GRIB driver: * if GRIB_ADJUST_LONGITUDE_RANGE config option is set to YES, adjust the longitude range to be close to [-180,180] when possible for products whose left origin is close to 180deg. (#7103) GSAG driver: * avoid assertion on some files with nul characters (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3726) GTiff driver: * make sure that -co PHOTOMETRIC=RGB overrides the color interpretation of the first 3 bands of the source datasets (#7064) * on reading use GeogTOWGS84GeoKey to override the defaults TOWGS84 values coming from EPSG code (#7144) * fix potential crash when reading in RGBA mode with internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4157) * fix potential crash with old-jpeg compression and internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4180) HTTP driver: * do not open the underlying dataset with GDAL_OF_SHARED, to avoid later assertion JP2KAK driver: * fix unix build with Kakadu 7.A and later (#7048, #7081) JPEG driver: * Add compatibility with libjpeg-turbo 1.5.2 that honours max_memory_to_use (cf https://github.com/libjpeg-turbo/libjpeg-turbo/issues/162) JP2OpenJPEG driver: * Add support for openjpeg 2.3 (#7074) netCDF driver: * fix raster read as nodata with Byte datatype, (valid_range={0,255} or _Unsigned = True) and negative _FillValue (#7069) * be more tolerant on the formatting of standard parallel (space separated instead of {x,y,...} syntax), and accept up to 2/1000 error on spacing to consider a regular grid, to be able to read files provided by the national weather institute of Netherlands (KNMI) (#7086) PDF driver: * round to upper integer when computing a DPI such that page size remains within limits accepted by Acrobat (#7083) Sentinel2 driver: * add support for direct opening of .zip files of new safe_compact L1C products (#7085) VRT driver: * avoid error being emitted when opening a VRTRawRasterBand in a .zip files (#7056) == OGR core == * Fix OGR[Curve]Polygon::Intersects(OGRPoint*) to return true when point is on polygon boundary (#7091) * importFromWkt(): fix import of GEOMETRYCOLLECTION ending with POINT EMPTY or LINESTRING EMPTY (#7128, 2.1 regression) == OGR utilities == * ogrtindex: fix crash when using -f SQLITE -t_srs XXXX (#7053) == OGR drivers == Amigocloud driver: * Fixed data field types (https://github.com/OSGeo/gdal/pull/246) * Output list of datasets if dataset id is not provided. * Implemented waiting for job to complete on the server. This will improve write to AmigoCloud reliability. * Updated documentation. FileGDB driver: * remove erroneous ODsCCreateGeomFieldAfterCreateLayer capability declaration (https://github.com/OSGeo/gdal/pull/247) GeoJSON driver: * ESRIJson: recognize documents that lack geometry fields (#7071) * ESRIJson: recognize documents starting with a very long fieldAliases list (#7107) GeoRSS driver: * fix detection of field type (#7108) GML driver: * fix FORCE_SRS_DETECTION=YES effect on feature count and SRS reporting on gml files with .gfs (#7046) * do not try to open kml files (#7061) * GML geometry parsing: fix robustness issue with gml:PolyhedralSurface * do not report gml:name / gml:description of features as layer metadata GPKG driver: * do not try to update extent on gpkg_contents after GetExtent() on a empty layer of a datasource opened in read-only mode GTM driver: * fix null pointer dereference in case of read error (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4047) KML / LIBKML drivers: MDB driver: * fix multi-thread support (https://issues.qgis.org/issues/16039) MITAB driver * do not emit error if the .ind file is missing, just a debug message (#7094) MySQL: * fix build with MariaDB 10.2 (#7079) OCI driver: * initialize in multi-threaded compatible mode to fix QGIS related crashes (https://issues.qgis.org/issues/17311), and fix a few memleaks ODS driver: * avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050) PLScenes driver: * backport data/plscenesconf.json from trunk to add SkySatScene fields Shapefile driver: * Fix GetFeatureCount() to properly take into account spatial filter when attribute filter also in effect (#7123) SQLite/Spatialite driver: * don't invalidate statistics when running a PRAGMA (https://issues.qgis.org/issues/17424) * SQLite dialect: support SQLite 3.21, and LIKE, <>, IS NOT, IS NOT NULL, IS NULL and IS operators (#7149) XLS driver: * workaround opening filenames with incompatible character set on Windows (https://issues.qgis.org/issues/9301) XLSX driver: * fix non working detection of Date/Time fields in some documents (#7073) * fix opening of documents with x: namespace in xl/workbook.xml (#7110) * avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050) == SWIG bindings == * map GRA_Max, GRA_Min, GRA_Med, GRA_Q1 and GRA_Q3 (#7153) == Python bindings == * remove 'from . import _gdal_array' line from gdal_array.py that is not necessary with normal execution of the bindings, and cause errors with PyInstaller (#7044) = GDAL/OGR 2.2.2 Release Notes = The 2.2.2 release is a bug fix release. == Build == * Windows build: always build the PDF driver, even when none of poppler/podofo/pdfium are available, in which case it is write-only (#6938) * Compilation fixes on Ubuntu 16.04 with explicit --std=c++03 (#6952) == Port == * /vsigzip/: make Eof() detect end of stream when receive a Z_BUF_ERROR error. Fixes #6944. * Fix memleak in VSIGSFSHandler::GetFileList() * /vsigzip/: avoid trying to write a .gz.properties file on a /vsicurl/ file (#7016) * Fix registration of /vsicrypt/ file system handler when compiled as a plugin (#7030) * CPLStrtod(): parse string like '-1.#IND0000000' as NaN instead of -1 (#7031) == Algorithms == * Warper: when operating on single-band, skip target pixels whose source center pixel is nodata (2.2 regression, #7001) * Warper: avoid blocking when number of rows is 1 and NUM_THREADS > 1 (#7041). Also limit the number of threads so that each one processes at least 65536 pixels == GDAL core == * GDALGCPsToGeoTransform(): add GDAL_GCPS_TO_GEOTRANSFORM_APPROX_OK=YES and GDAL_GCPS_TO_GEOTRANSFORM_APPROX_THRESHOLD=threshold_in_pixel configuration option (#6995) * Fix issue with GDALProxyRasterBand::FlushCache() not flushing everything required (#7040) * RawDataset::IRasterIO(): don't assume all bands are RawRasterBand == GDAL utilities == * gdal2tiles.py: fix GDAL 2.2 regression where tilemapresource.xml was no longer generated (#6966) * gdal_translate/DefaultCreateCopy(): do not destroy target file in case of failed copy wen using -co APPEND_SUBDATASET=YES (#7019) * gdal_translate: make -b mask[,xx] use the appropriate band data type (#7028) == GDAL drivers == GeoRaster driver: * add support for GCP (#6973) GPKG driver: * do not error out if extent in gpkg_contents is present but invalid (#6976) * fix opening subdatasets with absolute filenames on Windows (https://issues.qgis.org/issues/16997) * fix possible assertion / corruption when creating a raster GeoPackage (#7022) GSAG driver: * fix reading issue that could cause a spurious 0 value to be read and shift all following values (#6992) GTiff driver: * fix reading subsampled JPEG-in-TIFF when block height = 8 (#6988) * when reading a COMPD_CS (and GTIFF_REPORT_COMPD_CS=YES), set the name from the GTCitationGeoKey (#7011) GTX driver: * do not emit error when opening with GDAL_PAM_ENABLED=NO (#6996) HF2 driver: * fix reading tiles that are 1-pixel wide (2.1 regression, #6949) IDRISI driver: * Fix memleak in IDRISI Open() ISIS3 driver: * make sure that -co USE_SRC_HISTORY=NO -co ADD_GDAL_HISTORY=NO results in remove of History section (#6968) * fix logic to initialize underlying GeoTIFF file in IWriteBlock(), and implement Fill() (#7040) JPEG2000 driver: * Fix build failure in C++03 mode on Jasper inclusion in RHEL 6 (#6951) * Fix build failure with recent Jasper that no longer define uchar JP2OpenJPEG driver: * Add support for building against OpenJPEG 2.2 (#7002) * fix performance issues with small images with very small tile size, such as some Sentinel2 quicklooks (#7012) * properly use the opj_set_decode_area() API (#7018) netCDF driver: * avoid vector vs raster variable confusion that prevents reading Sentinel3 datasets, and avoid invalid geolocation array to be reported (#6974) PDF driver: * add support for Poppler 0.58 (#7033) PDS driver: * fix parsing of labels with nested list constructs (2.2 regression, #6970) SRTMHGT driver: * recognizes the .hgt.gz extension (#7016) VRT driver: * avoid stack buffer read overflow with complex data type and scale = 0. (oss-fuzz#2468) * fix uninitialized buffer in areas without sources when using non pixel packed spacing (#6965) * Warped VRT: correctly take into account cutline for implicit overviews; also avoid serializing a duplicate CUTLINE warping options in warped .vrt (#6954) * Warped VRT: fix implicit overview when output geotransform is not the same as the transformer dst geotransform (#6972) * fix IGetDataCoverageStatus() in the case of non-simple sources, which unbreaks gdalenhance -equalize (#6987) == OGR core == * OGRCompondCurve::addCurveDirectly(): try reversing non contiguous curves (for GML/NAS) * OGR API SPY: fix the way we map dataset handles to variable name, to avoid invalid reuses of variable still active * OGRParseDate(): avoid false-positive time detection, in particular for GeoJSON (#7014) * OGRCurve::get_isClosed(): do not take into account M component (#7017) * OGRLineString::setPoint() and addPoint() with a OGRPoint* argument. properly takes into account ZM, Z and M dimensions (#7017) == OGRSpatialReference == * Fix OGRSpatialReference::IsSame() to return FALSE when comparing EPSG:3857 (web_mercator) and EPSG:3395 (WGS84 Mercator) (#7029) * importFromProj4(): implement import of Hotine Oblique Mercator Two Points Natural Origin, and fix OGRSpatialReference::Validate() for that formulation (#7042) == OGR utilities == * ogr2ogr: fix small memory leak when using -limit switch * ogr2ogr: make -f GMT work again (#6993) == OGR drivers == AVCBin driver: * fix 2.1 regression regarding attributes fetching (#6950) DXF driver: * fix reading files where INSERT is the last entity and DXF_MERGE_BLOCK_GEOMETRIES is false (#7006) * avoid segfault when creating a DXF file with only the 'blocks' layer (#6998) * fix potential null pointer dereference in PrepareLineStyle (#7038) GeoJSON driver: * fix 2.2 regression regarding lack of setting the FeatureCollection CRS on feature geometries (fixes https://github.com/r-spatial/sf/issues/449#issuecomment-319369945) * prevent infinite recursion when reading geocouch spatiallist with properties.properties.properties (#7036) GML driver: * fix field type detection logic to avoid a field with xsi:nil=true to be identified as a string (#7027) * JPGIS FGD v4: fix logic so that coordinate order reported is lon/lat (https://github.com/OSGeo/gdal/pull/241) GMLAS driver: * fix memleak in swe:DataRecord handling * avoid false-positive warning about attribute found in document but ignored according to configuration, when the attribute is actually not present in the document but has a default value (#6956) * (quick-and-not-prefect) fix to avoid a <xs:any> to prevent all other fields from being set (#6957) * get the srsName when set on the srsName of the gml:pos of a gml:Point (#6962) * CityGML related fixes: better take into account .xsd whose namespace prefix is not in the document, fix discovery of substitution elements, handle gYear data type (#6975) GPKG driver: * fix feature count after SetFeature() with sqlite < 3.7.8 (#6953) * do not take into account gpkg_contents for the extent of vector layers when it is present but invalid (#6976) * fix logic to detect FID on SQL result layer when several FID fields are selected (#7026) KML / LIBKML drivers: * read documents with an explicit kml prefix (#6981) MSSQLSpatial driver: * fix issues on x64 (#6930) * properly format DELETE statement with schema (#7039) OCI driver: * check view with unquoted table name (#5552) OpenFileGDB driver: * properly read GeneralPolygon with M component whose all values are set to NaN (#7017) OSM driver: * increase string buffer for osm XML files (#6964) Shapefile driver: * use VSIStatL() in Create() to properly work with /vsimem/ directories (#6991) * fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (#7031) SQLite/Spatialite driver: * escape integer primary key column name on table creation (#7007) == C# bindings == * Implement RasterIO extensions (#6915) == Python bindings == * fix reference count issue on gdal.VSIStatL() when it returns None, that can cause Python crashes if None refcount drops to zero * add C-API to create driver instances for use in SWIG (https://trac.osgeo.org/osgeo4w/ticket/466) == Security oriented fixes == Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them. (oss-fuzz#XXXX is a shortcut to https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=XXXX) * NAS: avoid assertion / null ptr deref and possibly other failures on corrupted files. (oss-fuzz#2366, oss-fuzz#2580) * NTF: avoid null ptr deref. Related to https://oss-fuzz.com/v2/testcase-detail/6527758899347456 * VSIMemHandle::Read(): avoid unwanted unsigned int overflow that cause a later heap buffer overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5227283000328192. * MBTiles: fix use after free. (oss-fuzz#2388) * netCDF: fix stack buffer overflow when building vector layers from netCDF variables if they don't have the expected number of dimensions. (oss-fuzz#2400) * DXF: prevent infinite loop / excessive memory allocation on truncated file. (#oss-fuzz#2403) * DXF: avoid excessive memory allocation. (oss-fuzz#2444) * XYZ: fix write heap-buffer-overflow (oss-fuzz#2426) * GTiff: make IGetDataCoverageStatus() properly set the current TIFF directory, and only implement it for 'normal' bands. To avoid heap buffer overflows. (oss-fuzz#2438) * GTiff: avoid null pointer derefrence when requested implicit overviews on corrupted JPEG-compressed file. (oss-fuzz#2441) * GTiff: avoid potential issue on corrupted files (oss-fuzz#2481) * GTiff: don't override member nBlocksPerRow member of GTiffJPEGOverviewBand with unrelated value, in case of single-strip case. (oss-fuzz#3020) * PCIDSK: for band interleave channel, correctly use pixel_offset in case it is different from pixel_size. (oss-fuzz#2440) * MITAB: fix skipping of blank lines when reading MIF geometries, and avoid potential very loop loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4516730841858048 * CPLKeywordParser: avoid potential heap buffer overflow read. (oss-fuzz#2706) * CPLKeywordParser: avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/5733173726019584 * morphFromESRI(): fixes heap-after-free uses. (oss-fuzz#2864) * OGR_VRT: fix null pointer dereference on GetExtent() on an invalid layer. (oss-fuzz#3017) * OGR_VRT: avoid crash on cyclic VRT. (oss-fuzz#3123) * OSM: avoid potential write heap buffer overflow on corrupted PBF files. (oss-fuzz#3022) * FAST: fix potential read heap buffer overflow. (oss-fuzz#3025) * CPG: avoid null pointer dereference on failed open. (oss-fuzz#3136) = GDAL/OGR 2.2.1 Release Notes = The 2.2.1 release is a bug fix release. == Build == * fix compilation without BIGTIFF_SUPPORT (#6890) * configure: detect if std::isnan() is available. Helps compilation on some MacOSX setups, combined together --without-cpp11. Refs https://github.com/macports/macports-ports/pull/480 * fix compilation against ancient Linux kernel headers (#6909) * fix detection of 64bit file API with clang 5 (#6912) * configure: use .exe extension when building with mingw64* toolchains (fixes #6919) * mongoDB: compilation fix on Windows == Port == * /vsicurl/: fix occasional inappropriate failures in Read() with some combinations of initial offset, file size and read request size (#6901) * Add a VSICurlClearCache() function (bound to SWIG as gdal.VSICurlClearCache()) to be able to clear /vsicurl/ related caches (#6937) == Algorithms == * GDALRasterize(): avoid hang in some cases with all_touched option (#5580) * gdal_rasterize: fix segfault when rasterizing onto a raster with RPC (#6922) == GDAL utilities == * ogr_merge.py: fix '-single -o out.shp in.shp' case (#6888) == GDAL drivers == AIGRID driver: * fix handling on raw 32-bit AIG blocks ENVISAT driver: * fix 2.2 regression in initialization of members of MerisL2FlagBand. (#6929) GeoRaster driver: * Fix memory allocation failure (#6884) * add support for JP2-F in BLOB compression (corrections on geo-reference) (#6861) GPKG driver: * avoid corruption of gpkg_tile_matrix when building overviews, down to a level where they are smaller than the tile size (#6932) GTIFF driver: * Internal libtiff: fix libtiff 4.0.8 regression regarding creating of single strip uncompressed TIFF files (#6924) netCDF driver: * add support for radian and microradian units for geostationnary projection (https://github.com/OSGeo/gdal/pull/220) NWT_GRC driver: * Fix handling of alpha values in GRC color table (#6905) * Handle case of 0-len GRC class names (#6907) VRT driver: * speed-up SerializeToXML() in case of big number of bands XYZ driver: * fix 2.2 regression where the driver hangs on some dataset with missing samples (#6934) == OGR utilities == * ogr2ogr/GDALVectorTranslate(): fix crash when using -f PDF -a_srs (#6920) == OGR drivers == GeoJSON driver: * ESRIJson: avoid endless looping on servers that don't support resultOffset (#6895) * ESRIJson: use 'latestWkid' in priority over 'wkid' when reading 'spatialReference' (https://github.com/OSGeo/gdal/pull/218) * GeoJSON writer: accept writing ZM or M geometry by dropping the M component (#6935) GPKG driver: * make driver robust to difference of cases between table_name in gpkg_contents/gpkg_geometry_columns and name in sqlite_master (#6916) MITAB driver: * recognize Reseau_National_Belge_1972 / EPSG:31370 on writing (#6903) MySQL driver: * fix compilation issue with Arch Linux and mariadb 10.1.23 (fixes #6899) PG driver: * do not be confused by a 'geometry' table in a non-PostGIS enabled database (#6896) PLScenes: * remove support for V0. Deprecate V1 API. Only Data V1 is supported ( #6933) == Perl bindings == * Backport the fix to #6142 Install man page according to GDALmake.opt if INSTALL_BASE is set. * always return something from non-void functions (#6898) == Python bindings == * Accept callback = 0 since SWIG generates it as the default argument of BandRasterIONumPy(). Fixes https://github.com/OSGeo/gdal/pull/219 * Fix 2.2 regression preventing use of callback function in Band.ComputeStatistics() (#6927) == Security oriented fixes == Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them (some have been found locally, so no related ticket) * Fix CPLErrorSetState to ensure it does not write beyond DEFAULT_LAST_ERR_MSG_SIZE and correctly null-terminates last message. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1529. * Open() and Stat() methods of a number of virtual file systems: check that the filename starts with the full prefix. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1543. * VRT pixel functions: fix crash with 'complex' when source count is < 2. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1549 * OpenAIR: fix potential out-of-buffer read if we need to ingest 30000 bytes * Several fixes in importFromWkb() and importFromWkt() * GDALDataset and GDALRasterBand::ReportError(): fix crash if dataset name has a % character * NASAKeywordHandler::SkipWhite(): fix out of bounds read * MITAB: ParseTABFileFields(): fix out of bounds read. * MITAB: ParseMIFHeader(): fix memory leak and out-of-bounds read on corrupted file * MITAB: ParseMIFHeader(): fix memory leaks on corrupted files * MITAB: avoid potentially veryyyy long loop when stroking arcs. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1644 * MITAB: avoid heap-buffer-overflow in MITABCoordSys2TABProjInfo(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1653 * OSARDataset::Open(): fix crash if pOpenInfo->fpL == NULL. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1565 * OGRESRIJSONReadPolygon(): fix crash in error code path * DXF: prevent null ptr deref and out-of-bounds read on corrupted file * DXF: TranslateSPLINE(): sanitize integer values read to avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1843 * KML::unregisterLayerIfMatchingThisNode(): use memmove() instead of memcpy(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1580 * KML: fix crash on weird structure with recursively empty folders. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1591 * KML: fix null ptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1634 * OGRCurveCollection::importBodyFromWkb(): fix potential crash if the curve in the collection has not the same dimension has the collection dimension. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1584 * OGRCompoundCurve::importFromWkb(): avoid potential stack overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5192348843638784 * TIGER: fix potential stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1586 and 2020 * TIGER: avoid potential infinite looping. Fixes https://oss-fuzz.com/v2/testcase-detail/4724815883665408 * VFK: avoid out-of-bounds read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1596 and 2074 * CPLHexToBinary(): avoid reading outside of hex2char array on on-ASCII input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1606 * OGR PDS: avoid int32 overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=160 * GeoRSS: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1611. * VSIArchiveFilesystemHandler::SplitFilename(): improve performance. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1597 * OGRGeometryFactory::organizePolygons(): fix crash on empty polygons. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1620 * JML: fix null pointer dereference on invalid files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1622 * Shape: prevent null ptr deref on truncated MultiPointM geometry. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1638 * /vsisubfile/: avoid Tell() to return negative values. And make VSIIngestFile() more robust to unsigned overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1642 * GTM: avoid useless recursive opening of files when provided with a gzip-compressed input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1650 * GTiff: fix heap-buffer-overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1677 * GTiff: avoid heap-buffer-overfow on corrupted State Plane citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2050 * GTiff: avoid potential stack buffer overflow on corrupted Imagine citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2105 * GTiff: prevent heap overflow and fix reading of multi-band band-interleaved 16/24-bit float datasets. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2264 * GTiff: fix potential infinite loop when parsing some 24-bit float denormalized numbers. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2315 * Internal libjson-c: fix stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1682 * ILI1/ILI2: fix null pointer dereference when opening filename ','. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1689 * ILI1: fix various crashes on corrupted files (including, but not limited to, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1760, 1784, 1926) * ILI2: use proper cast operator. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1813 * ILI2: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1993 * ILI2: fix crash due to unhandled exception. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2191 * morphFromESRI(): fix heap-use-after-free issue. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1690 * morphFromESRI(): prevent potential null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1783 and 1867 * SEGUKOOA: fix inversion of leap year that caused index-out-of-bound reading on day 366 of leap years (2.2 regression). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1698 * CPLParseXMLString(): make it error out on invalid XML file even under CPLTurnErrorIntoWarning() mode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1703. * GML / NAS: fix memory leak in error code path, and potential heap-buffer-read-overflow * NTF: fix various issues: heap & stack buffer-overflow, null ptr derefs, memory leaks. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1819 , 1820 , 1823, 1833, 1854, 1862, 1910, 1931, 1961, 1990, 1995, 1996, 2003, 2033, 2052, 2077, 2084, 2103, 2130, 2135, 2146, 2166, 2185, 2187, https://oss-fuzz.com/v2/testcase-detail/4696417694121984 * OGRCreateFromMultiPatch(): avoid assertion on NaN coordinates. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1930 * GXF: validate nGType to avoid later out-of-bound read in GXFReadRawScanlineFrom(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1975 * GXF: fix int overflow and avoid excessive memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2207 * DGN: prevent heap-buffer-overflow read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1988 * COSAR: fix leak of file descriptor. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2019 * ISO8211: prevent stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2022 * WEBP: prevent int32 overflow and too large memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2059 * IRIS: fix heap-buffer-overflow in some cases of nDataTypeCode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2069 * E00GRID: avoid heap and stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2090 , 2182, 2237, 2327 * VICAR: fix null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2116 * VICAR: avoid use-after-free and heap-buffer-overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/4825577427042304 * VICAR: fix potential endless loop on broken files. Fixes https://oss-fuzz.com/v2/testcase-detail/6261508172414976 * REC: fix nullptr deref * REC: fix potential stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2165 * GDALGetJPEG2000Structure() / DumpGeoTIFFBox(): fix memory leak. * DumpGeoTIFFBox(): reject GeoJP2 boxes with a TIFF with band_count > 1 * DumpJPK2CodeStream(): avoid potentially very long loop * GDALGetJPEG2000Structure(): avoid bad performance on corrupted JP2 boxes. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2214 * GPKG: fix potential heap-buffer overflow in GPkgHeaderFromWKB(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2150 * GPKG: fix potential null ptr deref. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2240 * GPKG: avoid potential division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2253 * SEGUKOOA: prevent read beyond end of buffer. (#6921) * SRP: avoid potential stack buffer overflow and excessive memory allocation/processing time * CPLUnixTimeToYMDHMS(): avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4949567697059840 * Selafin: fix double frees. Fixes https://oss-fuzz.com/v2/testcase-detail/6429713822121984 * CEOS: fix heap buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2259 * CEOS: fix memleak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2279 * FAST: avoid null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2290 * netCDF: avoid stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2302 GDAL/OGR 2.2.0 Release Notes = == In a nutshell... == * New GDAL/raster drivers: - DERIVED driver: read-support. Expose subdatasets in a a new metadata domain, called DERIVED_SUBDATASETS - JP2Lura driver: read/create support for JPEG-2000 images using Luratech JP2 Library - PRF: add read-only support for PHOTOMOD PRF file format driver (github #173) - RRASTER driver: read-support .grd/.gri files handled by the R 'raster' package (#6249) * New OGR/vector drivers: - CAD driver: read support for DWG R2000 files (GSoC 2016 project) - DGNv8 driver: read-write support for DGN 8.0 format (using Teigha ODA libraries) - GMLAS driver: read-write support. XML/GML driver driven by Application Schemas. * New utility script: ogrmerge.py to merge several vector datasets into a single one * New /vsigs/ and /vsigs_streaming/ virtual file systems to read Google Cloud Storage non-public files * Significantly improved drivers: - NWT_GRD: write support (#6533) - FileGDB/OpenFileGDB: add support to read curve geometries (#5890) - VRT derived band: add the capability to define pixel functions in Python - Add read support for RasterLite2 coverages in SQLite driver - GPKG: implement tiled gridded elevation data extension - ISIS3: add write support and improve read support * RFC 63: Add GDALRasterBand::GetDataCoverageStatus() and implement it in GTiff and VRT drivers https://trac.osgeo.org/gdal/wiki/rfc63_sparse_datasets_improvements * RFC 64: Triangle, Polyhedral surface and TIN https://trac.osgeo.org/gdal/wiki/rfc64_triangle_polyhedralsurface_tin ==> this RFC introduces potential backward incompatible behaviour. Consult MIGRATION_GUIDE.txt * RFC 66: OGR random layer read/write capabilities https://trac.osgeo.org/gdal/wiki/rfc66_randomlayerreadwrite * RFC 67: add null field state for OGR features, in addition to unset fields https://trac.osgeo.org/gdal/wiki/rfc67_nullfieldvalues ==> this RFC introduces potential backward incompatible behaviour. Consult MIGRATION_GUIDE.txt * Upgrade to EPSG database v9.0 (#6772) * Python bindings: Global Interpreter Lock (GIL) released before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open()) * Continued major efforts on sanitization of code base * Remove bridge and vb6 bindings (#6640) * GNM built by default == Installed files == * Removed: data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv * Added plscenesconf.json, gmlasconf.xsd == Backward compatibility issues == See MIGRATION_GUIDE.TXT == GDAL/OGR 2.2.0 - General Changes == Build(Unix): * improve detection of packaged libfyba (SOSI) --with-sosi, as in Ubuntu 16.04 (#6488) * Sort files in static library to make the build reproducible (#6520) * fix libqhull include path when it is /usr/local/include/libqhull (#6522) * FileGDB: compilation fix on Linux in C++11 mode * configure: make pdfium detection not fail if there are just warnings. And make configure fail if --with-pdfium was required but failed (#6653) * Make ./configure --with-xerces fail if not found * Don't install script documentation in INST_BIN (github #157) * configure: auto-detect webp without requiring explicit --with-webp * configure: use pkg-config for HDF5 detection so that works out of the box on recent Ubuntu * auto-detect JDK 8 on Ubuntu * MDB: allow libjvm.so to be dlopen'd with --with-jvm-lib=dlopen (Unix only, github #177) * configure: delete temporary directories on the mac * configure: make sure --with-macosx-framework is correctly defined * configure: error out if --with-ld-shared is specified (#6769) * configure: remove bashism. * configure: fix --without-mrf (#6811) * configure: take into account CXXFLAGS and LDFLAGS in a few more cases (cryptopp, podofo, libdap) * Vagrant: all lxc and Hyper-V provider support; use vagrant-cachier for package caching * configure: update DWG support to work with Teigha libraries * Internal libgeotiff: hide symbols in --with-hide-internal-symbols mode * Shape: do not export Shapelib symbols for builds --with-hide-internal-symbols (#6860) Build(Windows): * Try to avoid confusion between libqhull io.h and mingw own io.h (#6590) * update script to generate most recent Visual C++ project files (#6635) * fix broken and missing dependencies in makefile.vc * add a way to use an external zlib (github #171) * Rename makegdal_gen.bat to generate_vcxproj.bat * generate_vcxproj.bat: Set correct value of PlatformToolset property based on specified Visual C++ version. Add NMAKE command line parameters: MSVC_VER based on specified Visual C++ version, DEBUG=1 WITH_PDB=1 to Debug build configuration. * generate_vcxproj.bat: generate project for autotest/cpp (Ticket #6815) * Add WIN64=1 to NMAKE command line options. * Add HDF4_INCLUDE option (#6805) * Add MSVC compiler option /MP to build with parallel processes. * Add ZLIB_LIB missing from EXTERNAL_LIBS Build(all): * make Xerces 3.1 the minimal version * drop support for PostgreSQL client library older than 7.4, or non security maintained releases older than 8.1.4, 8.0.8, 7.4.13, 7.3.15 == GDAL 2.2.0 - Overview of Changes == Port: * Export VSICreateCachedFile() as CPL_DLL so as to enable building JP2KAK as a plugin * Added possibility to find GDAL_DATA path using INST_DATA definition without execution GDALAllRegister if GDAL_DATA placed in version named directory on Linux (#6543) * Unix filesystem: make error message about failed open to report the filename (#6545) * File finder: Remove hardcoded find location (/usr/local/share/gdal) (#6543) * Win32 filesystem handler: make Truncate() turn on sparse files when doing file extension * Add VSIFGetRangeStatusL() and VSISupportsSparseFiles() * GDAL_NO_HARDCODED_FIND compilation option (#6543,#6531) to block file open calls (for sandboxed systems) * Add VSIMallocAligned(), VSIMallocAlignedAuto() and VSIFreeAligned() APIs * /vsizip / /vsitar: support alternate syntax /vsitar/{/path/to/the/archive}/path/inside/the/tar/file so as not to be dependent on file extension and enable chaining * Optimize opening of /vsitar/my.tar.gz/my_single_file * /vsizip/ : support creating non-ASCII filenames inside a ZIP (#6631) * VSI Unix file system: fix behaviour in a+ mode that made MRF caching not work on Mac and other BSD systems * Fix deadlock at CPLWorkerThreadPool destruction (#6646) * Windows: honour GDAL_FILENAME_IS_UTF8 setting to call LoadLibraryW() (#6650) * CPLFormFilename(): always use / path separator for /vsimem, even on Windows * /vsimem/: add trick to limit the file size, so as to be able to test how drivers handle write errors * /vsimem/: fix potential crash when closing -different- handles pointing to the same file from different threads (#6683) * CPLHTTPFetch(): add MAX_FILE_SIZE option * CPLHTTPFetch(): add a CAINFO option to set the path to the CA bundle file. As a fallback also honour the CURL_CA_BUNDLE and SSL_CERT_FILE environment variables used by the curl binary, which makes this setting also available for /vsicurl/, /vsicurl_streaming/, /vsis3/ and /vsis3_streaming/ file systems (#6732) * CPLHTTPFetch(): don't disable peer certificate verification when doing https (#6734) * CPLHTTPFetch(): cleanly deal with multiple headers passed with HEADERS and separated with newlines * CPLHTTPFetch(): add a CONNECTTIMEOUT option * CPLHTTPFetch(): add a GDAL_HTTP_HEADER_FILE / HEADER_FILE option. * CPLHTTPSetOptions(): make redirection of POST requests to still be POST requests after redirection (#6849) * /vsicurl/: take CPL_VSIL_CURL_ALLOWED_EXTENSIONS into account even if GDAL_DISABLE_READDIR_ON_OPEN is defined (#6681) * /vsicurl/: get modification time if available from GET or HEAD results * /vsis3/: add a AWS_REQUEST_PAYER=requester configuration option (github #186) * CPLParseXMLString(): do not reset error state * Windows: fix GetFreeDiskSpace() * Fix GetDiskFreeSpace() on 32bit Linux to avoid 32bit overflow when free disk space is above 4 GB (#6750) * Fix CPLPrintUIntBig() to really print a unsigned value and not a signed one * Add CPL_HAS_GINT64, GINT64_MIN/MAX, GUINT64_MAX macros (#6747) * Add CPLGetConfigOptions(), CPLSetConfigOptions(), CPLGetThreadLocalConfigOptions() and CPLSetThreadLocalConfigOptions() to help improving compatibility between osgeo.gdal python bindings and rasterio (related to https://github.com/mapbox/rasterio/pull/969) * MiniXML serializer: fix potential buffer overflow. Core: * Proxy dataset: add consistency checks in (the unlikely) case the proxy and underlying dataset/bands would not share compatible characteristics * GDALPamDataset::TryLoadXML(): do not reset error context when parsing .aux.xml * PAM/VRT: only take into account <Entry> elements when deserializing a <ColorTable> * GDALCopyWords(): add fast copy path when src data type == dst data type == Int16 or UInt16 * GetVirtualMemAuto(): allow USE_DEFAULT_IMPLEMENTATION=NO to prevent the default implementation from being used * Nodata comparison: fix test when nodata is FLT_MIN or DBL_MIN (#6578) * GetHistogram() / ComputeRasterMinMax() / ComputeStatistics(): better deal with precision issues of nodata comparison on Float32 data type * Fast implementation of GDALRasterBand::ComputeStatistics() for GDT_Byte and GDT_UInt16 (including use of SSE2/AVX2) * Driver manage: If INST_DATA is not requested, do not check the GDAL_DATA variable. * Make sure that GDALSetCacheMax() initialize the raster block mutex (#6611) * External overview: fix incorrect overview building when generating from greater overview factors to lower ones, in compressed and single-band case (#6617) * Speed-up SSE2 implementation of GDALCopy4Words from float to byte/uint16/int16 * Add SSE2 and SSSE3 implementations of GDALCopyWords from Byte with 2,3 or 4 byte stride to packed byte * GDALCopyWords(): SSE2-accelerated Byte->Int32 and Byte->Float32 packed conversions * Fix GDALRasterBand::IRasterIO() on a VRT dataset that has resampled sources, on requests such as nXSize == nBufXSize but nXSize != dfXSize * GDALRasterBand::IRasterIO(): add small epsilon to floating-point srcX and srcY to avoid some numeric precision issues when rounding. * Add GDALRasterBand::GetActualBlockSize() (#1233) * Fix potential deadlock in multithreaded writing scenarios (#6661) * Fix thread-unsafe behaviour when using GetLockedBlock()/MarkDirty()/DropLock() lower level interfaces (#6665) * Fix multi-threading issues in read/write scenarios (#6684) * Resampled RasterIO(): so as to get consistent results, use band datatype as intermediate type if it is different from the buffer type * Add GDALIdentifyDriverEx() function (github #152) * GDALOpenInfo: add a papszAllowedDrivers member and fill it in GDALOpenEx() * GDALDefaultOverviews::BuildOverviews(): improve progress report * Average and mode overview/rasterio resampling: correct source pixel computation due to numerical precision issues when downsampling by an integral factor, and also in oversampling use cases (github #156) * Overview building: add experimental GDAL_OVR_PROPAGATE_NODATA config option that can be set to YES so that a nodata value in source samples will cause the target pixel to be zeroed. Only implemented for AVERAGE resampling right now * GDALValidateOptions(): fix check of min/max values * GMLJP2 v2: update to 2.0.1 corrigendum and add capability to set gml:RectifiedGrid/gmlcov:rangeType content. Set SRSNAME_FORMAT=OGC_URL by default when converting to GML. Add gml:boundedBy in gmljp2:GMLJP2RectifiedGridCoverage * GMLJP2 v2: ensure KML root node id unicity when converting annotations on the fly. When generating GML features, make sure that PREFIX and TARGET_NAMESPACE are unique when specifying several documents. Algorithms: * RPC transformer: speed-up DEM extraction by requesting and caching a larger buffer, instead of doing many queries of just a few pixels that can be costly with VRT for example * GDALDeserializeRPCTransformer(): for consistency, use the same default value as in GDALCreateRPCTransformer() if <PixErrThreshold> is missing (so use 0.1 instead of 0.25 as before) * TPS solver: when Armadillo fails sometimes, fallback to old method * GDALCreateGenImgProjTransformer2(): add SRC_APPROX_ERROR_IN_SRS_UNIT, SRC_APPROX_ERROR_IN_PIXEL, DST_APPROX_ERROR_IN_SRS_UNIT, DST_APPROX_ERROR_IN_PIXEL, REPROJECTION_APPROX_ERROR_IN_SRC_SRS_UNIT and REPROJECTION_APPROX_ERROR_IN_DST_SRS_UNIT transformer options, so as to be able to have approximate sub-transformers * Fix GDAL_CG_Create() to call GDALContourGenerator::Init() (#6491) * GDALContourGenerate(): handle the case where the nodata value is NaN (#6519) * GDALGridCreate(): fix hang in multi-threaded case when pfnProgress is NULL or GDALDummyProgress (#6552) * GDAL contour: fix incorrect oriented contour lines in some rare cases (#6563) * Warper: multiple performance improvements for cubic interpolation and uint16 data type * Warper: add SRC_ALPHA_MAX and DST_ALPHA_MAX warp options to control the maximum value of the alpha channel. Set now to 65535 for UInt16 (and 32767 for Int16), or to 2^NBITS-1. 255 used for other cases as before * Warper: avoid undefined behaviour when doing floating point to int conversion, that may trigger exception with some compilers (LLVM 8) (#6753) * OpenCL warper: update cubicConvolution to use same formula as CPU case (#6664) * OpenCL warper: fix compliance to the spec. Fix issues with NVidia opencl (#6624, #6669) * OpenCL warper: use GPU based over CPU based implementation when possible, use non-Intel OpenCL implementation when possible. Add BLACKLISTED_OPENCL_VENDOR and PREFERRED_OPENCL_VENDOR to customize choice of implementation Utilities: * gdalinfo -json: fix order of points in wgs84Extent.coordinates (github #166) * gdalwarp: do not densify cutlines by default when CUTLINE_BLEND_DIST is used (#6507) * gdalwarp: when -to RPC_DEM is specified, make -et default to 0 as documented (#6608) * gdalwarp: improve detection of source alpha band and auto-setting of target alpha band. Automatically set PHOTOMETRIC=RGB on target GeoTIFF when input colors are RGB * gdalwarp: add a -nosrcalpha option to wrap the alpha band as a regular band and not as the alpha band * gdalwarp: avoid cutline densification when no transform at all is involved (related to #6648) * gdalwarp: fix failure with cutline on a layer of touching polygons (#6694) * gdalwarp: allow to set UNIFIED_SRC_NODATA=NO to override the default that set it to YES * gdalwarp: fix -to SRC_METHOD=NO_GEOTRANSFORM -to DST_METHOD=NO_GEOTRANSFORM mode (#6721) * gdalwarp: add support for shifting the values of input DEM when source and/or target SRS references a proj.4 vertical datum shift grid * gdalwarp: fix crash when -multi and -to RPC_DEM are used together (#6869) * gdal_translate: when using -projwin with default nearest neighbour resampling, align on integer source pixels (#6610) * gdal_translate & gdalwarp: lower the default value of GDAL_MAX_DATASET_POOL_SIZE to 100 on MacOSX (#6604) * gdal_translate: avoid useless directory scanning on GeoTIFF files * gdal_translate: make "-a_nodata inf -ot Float32" work without warning * gdal_translate: set nodata value on vrtsource on scale / unscale / expand cases (github #199) * GDALTranslate(): make it possible to create a anonymous target VRT from a (anonymous) memory source * gdaldem: speed-up computations for src type = Byte/Int16/UInt16 and particularly for hillshade * gdaldem hillshade: add a -multidirectional option * GDALDEMProcessing() API: fix -alt support (#6847) * gdal_polygonize.py: explicitly set output layer geometry type to be polygon (#6530) * gdal_polygonize.py: add support for -b mask[,band_number] option to polygonize a mask band * gdal_rasterize: make sure -3d, -burn and -a are exclusive * gdal_rasterize: fix segfaults when rasterizing into an ungeoreferenced raster, or when doing 'gdal_rasterize my.shp my.tif' with a non existing my.tif (#6738) * gdal_rasterize: fix crash when rasterizing empty polygon (#6844) * gdal_grid: add a smoothing parameter to invdistnn algorithm (github #196) * gdal_retile.py: add a -overlap switch * gdal2tiles.py: do not crash on empty tiles generation (#6057) * gdal2tiles.py: handle requested tile at too low zoom to get any data (#6795) * gdal2tiles: fix handling of UTF-8 filenames (#6794) * gdal2xyz: use %d formatting for Int32/UInt32 data types (#6644) * gdal_edit.py: add -scale and -offset switches (#6833) * gdaltindex: emit warning in -src_srs_format WKT when WKT is too large * gdalbuildvrt: add a -oo switch to specify dataset open options Python samples: * add validate_cloud_optimized_geotiff.py * add validate_gpkg.py Multi-driver: * Add GEOREF_SOURCES open option / GDAL_GEOREF_SOURCES config. option to all JPEG2000 drivers and GTiff to control which sources of georeferencing can be used and their respective priority AIGRID driver: * fix 2.1.0 regression when reading statistics (.sta) file with only 3 values, and fix <2.1 behaviour to read them in LSB order (#6633) AAIGRID driver: * auto-detect Float64 when the nodata value is not representable in the Float32 range ADRG driver: * handle north and south polar zones (ZNA 9 and 18) (#6783) ASRP driver: * fix georeferencing of polar arc zone images (#6560) BPG driver: * declare GDALRegister_BPG as C exported for building as a plugin (#6693) DIMAP driver: * DIMAP: for DIMAP 2, read RPC from RPC_xxxxx.XML file (#6539) * DIMAP/Pleiades metadata reader: take into tiling to properly shift RPC (#6293) * add support for tiled DIMAP 2 datasets (#6293) DODS driver: * fix crash on URL that are not DODS servers (#6718) DTED driver: * correctly create files at latitudes -80, -75, -70 and -50 (#6859) ECW driver: * Add option ECW_ALWAYS_UPWARD=TRUE/FALSE to work around issues with "Downward" oriented images (#6516). ENVI driver: * on closing, pad image file with trailing nul bytes if needed (#6662) * add read/write support for rotated geotransform (#1778) GeoRaster driver: * fix report of rotation (#6593) * support for JP2-F compression (#6861) * support direct loading of JPEG-F when blocking=no (#6861) * default blocking increased from 256x256 to 512x512 (#6861) GPKG driver: * implement tiled gridded elevation data extension * add VERSION creation option * check if transaction COMMIT is successful (#6667) * fix crash on overview building on big overview factor (#6668) * fix crash when opening an empty raster with USE_TILE_EXTENT=YES * fix gpkg_zoom_other registration GTiff driver: * support SPARSE_OK=YES in CreateCopy() mode (and in update mode with the SPARSE_OK=YES open option), by actively detecting blocks filled with 0/nodata about to be written * When writing missing blocks (i.e. non SPARSE case), use the nodata value when defined. Otherwise fallback to 0 as before. * in FillEmptyTiles() (i.e. in the TIFF non-sparse mode), avoid writing zeroes to file so as to speed up file creation when filesystem supports ... sparse files * add write support for half-precision floating point (Float32 with NBITS=16) * handle storing (and reading) band color interpretation in GDAL internal metadata when it doesn't match the capabilities of the TIFF format, such as B,G,R ordering (#6651) * Fix RasterIO() reported when downsampling a RGBA JPEG compressed TIFF file (#6943) * Switch search order in GTIFGetOGISDefn() - Look for gdal_datum.csv before datum.csv (#6531) * optimize IWriteBlock() to avoid reloading tile/strip from disk in multiband contig/pixel-interleave layouts when all blocks are dirty * fix race between empty block filling logic and background compression threads when using Create() interface and NUM_THREADS creation option (#6582) * use VSIFTruncateL() to do file extension * optimize reading and writing of 1-bit rasters * fix detection of blocks larger than 2GB on opening on 32-bit builds * fix saving and loading band description (#6592) * avoid reading external metadata file that could be related to the target filename when using Create() or CreateCopy() (#6594) * do not generate erroneous ExtraSamples tag when translating from a RGB UInt16, without explicit PHOTOMETRIC=RGB (#6647) * do not set a PCSCitationGeoKey = 'LUnits = ...' as the PROJCS citation on reading * fix creating an image with the Create() interface with BLOCKYSIZE > image height (#6743) * fix so that GDAL_DISABLE_READDIR_ON_OPEN = NO / EMPTY_DIR is properly honoured and doesn't cause a useless directory listing * make setting GCPs when geotransform is already set work (with warning about unsetting the geotransform), and vice-versa) (#6751) * correctly detect error return of TIFFReadRGBATile() and TIFFReadRGBAStrip() * in the YCBCR RGBA interface case, only expose RGB bands, as the alpha is always 255 * don't check free disk space when outputting to /vsistdout/ (#6768) * make GetUnitType() use VERT_CS unit as a fallback (#6675) * in COPY_SRC_OVERVIEWS=YES mode, set nodata value on overview bands * read GCPs in ESRI <GeodataXform> .aux.xml * explicitly write YCbCrSubsampling tag, so as to avoid (latest version of) libtiff to try reading the first strip to guess it. Helps performance for cloud optimized geotiffs * map D_North_American_1927 datum citation name to OGC North_American_Datum_1927 so that datum is properly recognized (#6863) * Internal libtiff. Resync with CVS (post 4.0.7) * Internal libtiff: fix 1.11 regression that prevents from reading one-strip files that have no StripByteCounts tag (#6490) GRASS driver: * plugin configure: add support for GRASS 7.2 (#6785) * plugin makefile: do not clone datum tables and drivers (#2953) * use Rast_get_window/Rast_set_window for GRASS 7 (#6853) GRIB driver: * Add (minimalistic) support for template 4.15 needed to read Wide Area Forecast System (WAFS) products (#5768) * **Partial** resynchronization with degrib-2.0.3, mostly to get updated tables (related to #5768) * adds MRMS grib2 decoder table (http://www.nssl.noaa.gov/projects/mrms/operational/tables.php) (github #160) * enable PNG decoding on Unix (#5661, github #160) * remove explicitly JPEG2000 decompression through Jasper and use generic GDAL code so that other drivers can be triggered * fix a few crashes on malformed files GTX driver: * add a SHIFT_ORIGIN_IN_MINUS_180_PLUS_180 open option HDF4 driver: * Fixed erroneous type casting in HDF4Dataset::AnyTypeToDouble() that breaks reading georeferencing and other metadata HDF5 driver: * correct number of GCPs to avoid dummy trailing (0,0)->(0,0,0) and remove +180 offset applied to GCP longitude. Add instead a heuristics to determine if the product is crossing the antimeridian, and a HDF5_SHIFT_GCPX_BY_180 config option to be able to override the heuristics (#6666) HFA driver: * fix reading and writing of TOWGS84 parameters (github #132) * export overview type from HFA files to GDAL metadata as OVERVIEWS_ALGORITHM (github #135) * make .ige initialization use VSIFTruncateL() to be faster on Windows * add support for TMSO and HOM Variant A projections (#6615) * Add elevation units read from HFA files metadata (github #169) * set binning type properly according to layerType being thematic or not (#6854) Idrisi driver: * use geotransform of source dataset even if it doesn't have a SRS (#6727) * make Create() zero-initialize the .rst file (#6873) ILWIS driver: * avoid IniFile::Load() to set the bChanged flag, so as to avoid a rewrite of files when just opening datasets ISCE driver: * fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL 2.1.0 (#6556) * fix misbehaviour on big endian hosts * add support for reading and writing georeferencing (#6630, #6634) * make parsing of properties case insensitive (#6637) ISIS3 driver: * add write support * add mask band support on read * get label in json:ISIS3 metadata domain JPEGLS driver: JP2ECW driver: * fix crash when translating a Float64 raster (at least with SDK 3.3) JP2KAK driver: * add support for Kakadu v7.9. v7.8 should not be used. It has a bug fixed in v7.9 * catch exceptions in jp2_out.write_header() JP2OpenJPEG driver: * add a USE_TILE_AS_BLOCK=YES open option that can help with whole image conversion * prevent endless looping in openjpeg in case of short write * for single-line organized images, such as found in some GRIB2 JPEG2000 images, use a Wx1 block size to avoid huge performance issues (#6719) * ignore warnings related to empty tag-trees. JPIPKAK driver: * fix random crashes JPIP in multi-tread environment (#6809) KEA driver: * Add support for Get/SetLinearBinning (#6855) KMLSuperOverlay driver: * recognize simple document made of GroundOverlay (#6712) * Add FORMAT=AUTO option. Uses PNG for semi-transparent areas, else JPG. (#4745) LAN driver: * remove wrong byte-swapping for big-endian hosts MAP driver: * change logic to detect image file when its path is not absolute MBTiles driver: * on opening if detecting 3 bands, expose 4 bands since there might be transparent border tiles (#6836) * fix setting of minzoom when computing overviews out of order * do not open .mbtiles that contain vector tiles, which are not supported by the driver MEM driver: * disable R/W mutex for tiny performance increase in resampled RasterIO * add support for overviews * add support for mask bands MRF driver: * bug fix in PNG and JPEG codecs * Fixing a problem with setting NoData for MRFs generated with Create * fix plugin building (#6498) * rename CS variable so as to avoid build failure on Solaris 11 (#6559) * Allow MRF to write the data file directly to an S3 bucket. * Allow relative paths when MRF is open via the metadata string. * Add support for spacing (unused space) between tiles. Defaults to none. * Read a single LERC block as an MRF file. MSG driver: * fix incorrect georeference calculation for msg datasets (github #129) NetCDF driver: * add support for reading SRS from srid attribute when it exists and has content like urn:ogc:def:crs:EPSG::XXXX (#6613) * fix crash on datasets with 1D variable with 0 record (#6645) * fix erroneous detection of a non-longitude X axis as a longitude axis that caused a shift of 360m on the georeferencing (#6759) * read/write US_survey_foot unit for linear units of projection * apply 'add_offset' and 'scale_factor' on x and y variables when present, such as in files produced by NOAA from the new GOES-16 (GOES-R) satellite (github #200) * add a HONOUR_VALID_RANGE=YES/NO open option to control whether pixel values outside of the validity range should be set to the nodata value (#6857) * fix crash on int64/uint64 dimensions and variables, and add support for them (#6870) NITF driver: * add support for writing JPEG2000 compressed images with JP2OpenJPEG driver * fix writing with JP2KAK driver (emit codestream only instead of JP2 format) * fix setting of NBPR/NBPC/NPPBH/NPPBV fields for JPEG2000 (fixes #4322); in JP2ECW case, make sure that the default PROFILE=NPJE implies 1024 block size at the NITF level * implement creation of RPC00B TRE for RPC metadata in CreateCopy() mode * add support for reading&writing _rpc.txt files * nitf_spec.xml: Add support for MTIRPB TRE in NITF image segment. Also makes minor change to BLOCKA to include default values (github #127) * nitf_spec.xml: add IMASDA and IMRFCA TREs * GetFileList(): Small optimization to avoid useless file probing. NWT_GRD: * detect short writes OpenJPEG driver: * support direct extracting of GeoRaster JP2-F BLOB (#6861) PCIDSK driver: * handle Exceptions returned from destructor and check access rights in setters (github #183) PDF driver: * implement loading/saving of metadata from/into PAM (#6600) * implement reading from/writing to PAM for geotransform and projection (#6603) * prevent crashes on dataset reopening in case of short write PLScenes driver: * add a METADATA open option PostgisRaster driver: * fix potential crash when one tile has a lower number of bands than the max of the table (#6267) R driver: * fix out-of-memory (oom) with corrupt R file Raw drivers: * prevent crashes on dataset closing in case of short write RMF driver: * fix wrong counter decrement that caused compressed RMF to be incorrectly decompressed (github #153) * fix load/store inversion of cm and dm units in MTW files (github #162) * fix reading nodata for non-double data type (github #174) ROIPAC driver: * add support for reading/writing .flg files (#6504) * fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL >= 2.0.0 (#6591) * fix for big endian hosts RS2 driver: * add support for reading RPC from product.xml SAFE driver: * fix handling of SLC Products by providing access to measurements as subdatasets (#6514) Sentinel2 driver: * add support for new "Safe Compact" encoding of L1C products (fixes #6745) SQLite driver: * Add read support for RasterLite2 coverages in SQLite driver SRTMHGT driver: * open directly .hgt.zip files * accept filenames like NXXEYYY.SRTMGL1.hgt (#6614) * handle files for latitude >= 50 (#6840) VRT driver: * add default pixel functions: real, imag, complex, mod, phase, conj, etc... for complex data types (github #141) * avoid useless floating point values in SrcRect / DstRect (#6568) * avoid buffer initialization in RasterIO() when possible (replace ancient and likely broken concept of bEqualAreas) * make CheckCompatibleForDatasetIO() return FALSE on VRTDerivedRasterBands (#6599) * VRT warp: fix issue with partial blocks at the right/bottom and dest nodata values that are different per band (#6581) * fix performance issue when nodata set at band level and non-nearest resampling used (#6628) * VRTComplexSource: do temp computations on double to avoid precision issues when band data type is Int32/UInt32/CInt32/Float64/CFloat64 (#6642) * VRT derived band: add the capability to define pixel functions in Python * CreateCopy(): detect short writes * Fix linking error on VRTComplexSource::RasterIOInternal<float>() (#6748) * avoid recursion in xml:VRT metadata (#6767) * prevent 'Destination buffer too small' error when calling GetMetadata('xml:VRT') on a in-memory VRT copied from a VRT * fix 2.1 regression that can cause crash in VRTSimpleSource::GetFileList() (#6802) WMS driver: * Added support for open options to WMS minidrivers * Refactored the multi-http code to make it possible to do range requests. * Added a minidriver_mrf, which reads from remote MRFs using range requests. * Made the minidriver_arcgis work with an ImageService, not only MapService. * Added static cache of server response. * Allow tiledWMS to work in off-line mode by including the server response in the .wms file itself. * honour GDAL_HTTP_USERAGENT config option when it is set and <UserAgent> is missing (#6825) * WMS/WMTS: better deal with tiles with different band count (grayscale, gray+alpha, palette, rgb, rgba) (github #208) * Make HTTPS options apply to tWMS minidriver init, better HTTP error reporting WMTS driver: * do not take into account WGS84BoundingBox/BoundingBox that would be the result of the densified reprojection of the bbox of the most precise tile matrix * add TILEMATRIX / ZOOM_LEVEL open options * accept tiles of small dimensions (github #210) XYZ driver: == OGR 2.2.0 - Overview of Changes == Core: * Layer algebra: Add KEEP_LOWER_DIMENSION_GEOMETRIES=YES/NO option to Intersection, Union and Identity. Default is yes, but it is set to no unless result layer is of geom type unknown. If set to no, result features which would have lower dim geoms are skipped if operating on two layers with same geom dim. * Fix crash/corrupted values when running importFromWkb() on POLYGON M/POLYGON ZM geometries (#6562) * Add OGR_WKT_PRECISION config option that defaults to 15 to select the number of decimals when outputting to WKT * Make OGRFeature::SetField(string) accept JSon serialized arrays for the String/Integer/Integer64/RealList types; add reciprocal OGRFeature::GetFieldAsSerializedJSon() for those types * OGRGeometryFactory::transformWithOptions(): better deal with reprojection from polar projection to WGS84, and projections crossing the antimeridian to WGS84, by splitting geometries prior to reprojection (#6705) * LinearRing transformTo(): force last point to be identical to first one in case it is not. * GML geometry parsing: avoid 'Cannot add a compound curve inside a compound curve' error (#6777) * OGR SQL: fix IN filtering on MapInfo indexed columns (#6798) * OGR SQL: add support for LIMIT and OFFSET keywords * OGR SQL: add comparisons on date / datetime (#6810) * OGR SQL: increase efficiency of DISTINCT operator * OGREnvelope: change initialization to fix issue when getting MULTIPOINT(0 0,1 1) envelope (#6841) * OGRParse: fix parsing logic to avoid false positive detection of string as datetime (#6867) OGRSpatialReference: * Upgrade to EPSG database v9.0 (#6772) * OGRCT: upgrade LIBNAME of mingw and cygwin to libproj-9.dll and cygproj-9.dll to be up-to-date with proj 4.9.X (recommended method is using ./configure --with-static-proj4 instead) (#6501) * importFromESRI(): fix import of multi line MERCATOR SRS (#6523) * morphToESRI(): correctly compute standard_parallel_1 of Mercator(2SP) projection from scale factor of Mercator(1SP) (#6456, #4861) * exportToProj4(): recognize explicit OSR_USE_ETMERC=NO to avoid using etmerc with proj >= 4.9.3 * importFromProj4(): do not set a AUTHORITY node for strings like '+init=epsg:XXXX +some_param=val' * importFromProj4(): be robust with missing proj.4 epsg dictionary when importing '+init=epsg:xxxx +other_parm=value' * AutoIdentifyEPSG(): add identification of EPSG:3995 (Arctic Polar Stereographic on WGS84) and EPSG:3031 (Antarctic Polar Stereographic on WGS84) * OGRCoordinateTransformation: avoid potential bugs in proj.4 on NaN input * importFromEPSG(): take into account DX,DY,DZ,RX,RY,RZ,DS columns in pcs.csv to add per-PCS TOWGS84 overrides (geotiff #52) * Coordinate transformation: prevent unnecessary coordinate transformations (github #184, #185) Utilities: * ogr2ogr: do not return error on ogr2ogr --utility_version * ogr2ogr: keep -append and -overwrite when -update follows * ogr2ogr: fix heuristics to detect likely absence of '-f' (#6561) * ogr2ogr: do not emit a warning when updating/overwriting a destination dataset that is not a Shapefile and if -f is not specified (#6561) * ogr2ogr: make overwriting of lots of PG tables less prone to PostgreSQL 'out of shared memory' errors, by committing transactions after each table recreation * ogr2ogr: prevent 'ogr2ogr same.shp same.shp' (#1465) * ogr2ogr: add a -limit option to limit the number of features read per layer * GDALVectorTranslate(): imply update mode if access mode not explicitly specified and hDstDS != NULL (#6612) * ogrlineref: Fix crash if no new layer name provided * ogrtindex: add -src_srs_name, -src_srs_format and -t_srs option to allow indexing files that have different projections Multidriver: * PGeo/FileGDB/OpenFileGDB: OGRCreateFromShapeBin(): do not error out on empty lines/polygons * GPKG/SQLite/PG/FileGDB/MEM: properly set/reset field subtype with AlterFieldDefn() (#6689) * DXF, GeoJSON, GML, KML, LIBKML, ODS, Shape, XLSX: report operating system error if file creation fails (#2253) AmigoCloud driver: * add option to receive an AmigoCloud API key in the connection string, fix page size (github #137) Carto: * renamed from CartoDB * fix CartoDB'fication() by removing manual creation of the_geom_webmercator mercator, and also attach created sequence to table.cartodb_id (#6565) CSV driver: * add read/write support for String/Integer/Integer64/RealList types as serialized JSon arrays DGN driver: * avoid crash with -ftrapv on 250K_NF44NE_Area_V7.dgn (relates to #6806) * use coordinate delta encoding from the attribute records array (6806) DXF driver: * sanitize layer name on export to avoid forbidden characters * reader: convert GeometryCollection to MultiPoint, MultiLineString or MultiPolygon when possible (QGIS #10485) * add font of TEXT and MTEXT to OGR style string (https://github.com/OSGeo/gdal/pull/198) DWG driver: * compilation fixes with Teigha 4.2.2 ElasticSearch driver: * add support for ElasticSearch 5.0 and fix 2.X support (#6724) * implement translation from SQL to E.S. query language FileGDB driver: * add support to read curve geometries (#5890) * support opening '.' directory GeoJSON driver: * writer: add a RFC7946=YES creation option (#6705) * read and write 'name' and 'description' members at FeatureCollection level * fix field type detection when first value of a field is null (#6517) * improve/fix field type promotion * fix wrong behaviour when there's a 'id' at Feature level and 'id' or 'ID' field in properties (#6538) * in case top level id is a negative integer, put the value in a 'id' attribute (#6538) * ESRI Json reader: support multilinestring from esriGeometryPolyline * ESRI Json reader: do not set field width of 2147483647 (#6529) * ESRI Json reader: support reading M and ZM geometries * Add CPL_json_object_object_get() and use it, to avoid deprecation warnings on json_object_object_get() * TopoJSON reader: sanitize invalid polygons (such as found in the 'TopoJSON' layer of http://bl.ocks.org/mbostock/raw/4090846/us.json) * writer: fix segfaults on NULL geometry with -lco WRITE_BBOX=YES (#6698) * writer: fix crash if NATIVE_MEDIA_TYPE creation option is specified alone * Add support of TopoJSON without 'transform' element (github #192) * don't set SRS if 'crs' set to null (github #206) GML driver: * remove hack for CityGML regarding forcing srsDimension to 3 if not specified. Instead add a GML_SRS_DIMENSION_IF_MISSING config option that can be set to 3 if needed (#6597) * consider srsName with URL like 'http://www.opengis.net/def/crs/EPSG/0/' as following EPSG axis order. Add SWAP_COORDINATES=YES/NO/AUTO (and GML_SWAP_COORDINATES configuration option) to override all auto guessing (#6678) * add a SRSNAME_FORMAT=SHORT/OGC_URN/OGC_URL dataset creation option * OGR_G_ExportToGMLEx(): add a COORD_SWAP option * Writer: make ogr:FeatureCollection be a substitutionGroup of AbstractFeature for GML 3.2 output, so as to be compatible of GMLJP2 v2 * GML and NAS: improve error reporting (mention feature id and gml_id) when parsing fails because of geometry decoding error * GML and NAS: add GML_SKIP_CORRUPTED_FEATURES and NAS_SKIP_CORRUPTED_FEATURES configuration options to avoid stopping reading a layer because of a corrupted geometry * Add support for Japanese GML FGD v4.1 format (github #204) GPKG driver: * list all tables/views by default (useful for non spatial tables not registered as aspatial). Add ASPATIAL_VARIANT layer creation option to select the way how to create a non-spatial table as aspatial. * Create non-spatial tables by default conforming to GPKG v1.2 'attributes' * use OGR_CURRENT_DATE config option if defined as the value of the 'last_change' of column of 'gpkg_contents' so as to be able to have binary identical output * update last_change column when content of raster or vector tables has changed * do not emit error when running ExecuteSQL() with a spatial filter on an empty layer (#6639) * update schema to reflect CURRENT_TIMESTAMP -> 'now' changes (github #155) * better table and column quoting. * Robustify layer deletion (make it accessible through DROP TABLE xxx and DELLAYER:xxx syntax) and renaming * implement DeleteField(), AlterFieldDefn(), ReorderFields() * make GetExtent() save extent if not already cached * add special SQL 'RECOMPUTE EXTENT ON layer_name' to force recomputation of extent * check identifier unicity on layer creation * add possibility to disable foreign key check by setting OGR_GPKG_FOREIGN_KEY_CHECK=NO config option * add HasSpatialIndex(tblname,geomcolname) SQL function * don't show Spatialite vgpkg_ virtual tables (#6707) * SQLite/GPKG: add explicit error message when trying to open a read-only WAL-enabled database (#6776) * SQLite/GPKG: make sure when closing a WAL-enabled database opened in read-only mode to reopen it in read-write mode so that the -wal and -shm files are removed (#6776) * make GetFeature() works on non conformant tables that have no integer primary key field (#6799), and be robust to non standard column types * remove triggers related to metadata tables that cause issues and have been removed by latest revisions of the spec. * declare feature id column of features tables and tile pyramid user data tables as NOT NULL (#6807) * add a gpkg_ogr_contents table to store feature count. * speed-up GetFeatureCount() with only a spatial filter set * improve column recognition for SQL result layer * add/override ST_Transform() and SridFromAuthCRS() from Spatialite to make them work with gpkg_spatial_ref_sys * add ImportFromEPSG() * make ST_Min/MaxX/Y(), ST_SRID(), ST_GeometryType() and ST_IsEmpty() work with Spatialite geometries * improve performance of spatial index creation and use on multi-gigabyte databases * better support of spatial views, by adding a special behaviour if a column is named OGC_FID * avoid potential denial of services by adding LIMIT clauses * slightly more efficient implementation of GetExtent() if extent in gpkg_contents is empty * create a dummy 'ogr_empty_table' features table in case we have no 'features' or 'tiles' table, so as to be conformant with Req 17 of the GeoPackage specification * add DEFAULT '' to metadata column of gpkg_metadata table * accept opening a .gpkg without vector content and without gpkg_geometry_columns table in vector mode if we also open in update mode; remove capability of opening a .gpkg without vector content but with gpkg_geometry_columns table in vector mode if we only open in read-only mode; fix creation of a vector layer in a database if it initially lacks a gpkg_geometry_columns table * fix appending a raster to a vector database without pre-existing raster support tables * add minimalistic support for definition_12_063 column in gpkg_spatial_ref_sys, so that insertion of new SRS doesn't fail * use GEOMETRYCOLLECTION instead of GEOMCOLLECTION for SQL and gpkg_geometry_columns.geometry_type_name * do not warn if gpkg_metadata extension declared GPX driver: * ignore wpt/rtept/trkpt with empty content for lat or lon ILI driver: * ILI1: fix crash in JoinSurfaceLayer() when the multicurve of the feature of the poSurfaceLineLayer layer is empty (#6688) * ILI1: make polygon reconstruction in Surface layers robust to curves not in natural order (#6728) * ILI2: assign FID to features (#6839) * ILI2: fix crashing bug in Create() if model file not specified KML driver: * add a DOCUMENT_ID datasource creation option to set the id of the root <Document> node LIBKML driver: * fix crash when reading <gx:TimeStamp> or <gx:TimeSpan> elements (#6518) * add a DOCUMENT_ID datasource creation option to set the id of the root <Document> node * emit style related errors as warnings to make datasets openable by SWIG bindings (#6850) MITAB driver: * limit (width, precision) of numeric fields on creation to (20,16) for compatibility with MapInfo (#6392) * add support for oblique stereographic (#6598) * Adds the authority code for Irish national grid (Ireland_1965) (github #149) * fix spelling for Euref_89 and add EPSG code (#6816) MSSQLSpatial driver: * Fix bulk insert with table names containing spaces (#6527) * Build optional mssql plugin with SQL Native Client support for MSSQL Bulk Copy * Fix MSSQL select layer to recognize geometry column with sqlncli (#6641) MySQL driver: * fix spatial filtering on recent mysql by adding a SRID in the rectangle geometry * do not force NOT NULL constraint on geometry field if no spatial index is used NAS driver: * support multiple 'anlass' in updates NWT_GRD: * add write support (#6533) OCI driver: * Add options for faster feature loading (#6606) * add WORKSPACE open option * correctly handle OFTInteger64 case in loader layer (bug found by cppcheck multiCondition) * support for long identifiers (up to 128 long) when running of 12.2 or + (#6866) * OCILOB VSIL driver: new driver to streams in and out of Oracle BLOB as a GDAL large virtual file system (#6861) ODS driver: * fix FID filtering (#6788) OGDI driver: * make GetNextRawFeature() report an error when it is not end of layer * better error reporting when the layer list cannot be established * catch non-fatal OGDI errors emitted by OGDI 3.2.0 if OGDI_STOP_ON_ERROR environment variable is set to NO, and emit them as CPLError()s * display OGDI error message if opening fails OpenFileGDB driver: * do not error out on geometries that have a declared M array but that is missing (#6528) * add support to read curve geometries (#5890) * transcode UTF-16 strings found in column names, alias, etc... to UTF-8 (instead of using only their ASCII byte) (#6544) * do not emit an error on a empty table whose declaration has M settings (#6564) * support opening '.' directory * improve detection of some form of TINs from MULTIPATCH, and for MultiPatch layers, try to select a better geometry type for those layers (#5888) * fix bug when field description offset is beyond 4GB (#6830) OSM driver: * fix 'too many tags in relation' error when parsing .osm files * allow key=value entries in closed_ways_are_polygons= configuration (#6476) * allow OSM_SQLITE_CACHE config option to be greater than 2047 PG driver: * fix insertion of binary/bytea content in non-copy mode (#6566) * fix errors caused by missing geometry_columns/spatial_ref_sys tables in non PostGIS databases, that prevent reading more than 500 features (QGIS #10904) * avoid errors with field default expressions like 'foo'::text (#6872) PLScenes driver: * add HTTP retry logic (#6655) * V0 API: workaround limitations on filtering on image_statistics.image_quality (#6657) * add support for Data V1 API S57 driver: * Fix ogr2ogr -f S57 (#6549) * fix crashes if the s57objectclasses.csv resource file contains invalid lines * remove data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv and move their content into main s57attributes.csv and s57objectclasses.csv files (#6673) * Update s57 attributes and object classes according to s-57 reference (github #202) * add POSACC and QUAPOS fields for geometric primitive layers (github #205) SDE driver: * rename driver to OGR_SDE. Fix build (#6714) SEGY driver: * Accept SEGY files with ASCII headers that have nul terminated strings (#6674) Shapefile driver: * auto-repack by default at dataset closing and FlushCache()/SyncToDisk() time. Controlled by AUTO_REPACK open and layer creation options (that default to YES) * generate .dbf end-of-file 0x1A character by default. Add DBF_EOF_CHAR layer creation options / open options to control that behaviour * writing: use strerrno() for better error messages (QGIS #13468) * change REPACK implementation on Windows to be robust to remaining file descriptors openend on the .shp/.shx/.dbf (#6672, QGIS #15570) * Fix issue in DBFCloneEmpty() one a one field DBF not yet written to disk * add call to AutoIdentifyEPSG() when reading a .prj * support reading .dbf with substantial padding after last field definition. * when rewriting the geometry of the last record in the .shp, do it at the file offset it previously used (#6787) SOSI driver: * make registration of driver work again (2.1.0 regression) (#6500) * update to latest version of https://github.com/kartverket/gdal. Make SOSI driver support more geometry types (including curved geometries) plus provides some improvements on attribute-type mapping. Fix some memory errors/leaks. Disable by default non working creation code (#6503) SQLite/Spatialite driver: * do not emit error when running ExecuteSQL() with a spatial filter on an empty layer (#6639) * add read/write support for String/Integer/Integer64/RealList types as serialized JSon arrays * Spatialite: avoid crash when creating layer with geom_type = wkbCurve (fixes #6660) * Spatialite: do not report some BLOB columns as geometry columns of tables/views (when found before the geometry column(s)) (#6695, #6659) * fix update of features with multiple geometry columns (#6696) * speed-up dataset closing when creating a database with many spatial layers * Spatialite: avoid spatial views to cause layers 'layer_name(geometry_name)' to be publicly listed (#6740) * Spatialite: speed-up creation of database with INIT_WITH_EPSG=NO that is slow without transaction (not sure why as the table is empty...) * use AUTOINCREMENT for feature id column * allow OGR_SQLITE_CACHE to be set to > 2047 (MB) without overflowing * SQLite/GPKG: use SQLITE_OPEN_NOMUTEX flag to open databases. * GPKG/SQLite: fix ExecuteSQL() to work with a statement with 2 SELECT and ORDER BY clause (#6832) * SQLite/GPKG: change default page_size to 4096 bytes. * Update layer statistics for Spatialite 4 DB (#6838) * Remove traces of support of SQLite < 3.6.0 * SQLite dialect: properly quote column names when needed (github #214) VFK driver: * allow reading multiple VFK files into single DB support amendment VFK files * recreate DB in the case that it's outdated (VFK DB created by previous versions of GDAL) * allow reading DB as valid datasource (#6509) * new tables in backend SQLite database (geometry_columns/spatial_ref_sys) to enable reading DB datasource by SQLite driver * new configuration option OGR_VFK_DB_READ to enable reading backend database by VFK driver VRT driver: * add support for 'm' attribute in PointFromColumns mode (#6505) WFS driver: * invalidate underlying layer when SetIgnoredFields() is called (QGIS #15112) * don't crash on empty <Keyword/> declaration (#6586) * fix potential nullptr dereference on dataset without layer (github #179) XLSX driver: * only list worksheets (and no charts) as layers (#6680) == SWIG Language Bindings == All bindings: * map osr.GetTargetLinearUnits() (#6627) * allow wkbCurve/wkbSurface as valid values for the geom type of GeomFieldDefn object * map GDALIdentifyDriverEx() Java bindings: * Fix SWIG Java bindings for GNM (#6434) * Fix crash on GetDefaultHistogram() if the C++ method returns an error (#6812) Perl bindings: * return value always from GetGeomFieldIndex (#6506) * the Warp method requires a list of datasets (#6521) * when 'use bigint' is in effect, int var is a ref. * Separate the module building in the CPAN distribution and in the GDAL source tree. The CPAN distribution will be developed at https://github.com/ajolma * Fix the Extension method in Driver per RFC 46 * The Inv method of GeoTransform did not return a new object. Allow a single point in Apply method of GeoTransform. * Test for existence of PDL and require it if available. * allow decimation/replication in Piddle i/o. * support resampling in ReadTile. * Polygonize: Require explicit 8 for Connectedness to set 8-connectedness and allow 8CONNECTED as an option. * use Safefree to free memory allocated in Perl (#6796) Python bindings: * release the GIL before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open()) * add outputType option to gdal.Rasterize() * fix build issues when CXX is defined in the environment * gdal.VectorTranslate(): add spatSRS option * when enabling ogr.UseExceptions(), use the GDAL error message in the exception text (if available), when the exception is linked to an error value in the OGRErr return code * gdal.VectorTranslate(): accept a single string as value of the layers option (instead of iterating over each of its characters) * Regenerate Python bindings with SWIG 3.0.8 to avoid issue with Python 3.5. Add backward compatibility in Band.ComputeStatistics() to accept 0/1 as input instead of the expected bool value (#6749) * fix gdal.DEMProcessingOptions(zeroForFlat=True) (#6775) * fix 'import osgeo.gdal_array' with python3 and SWIG 3.0.10 (#6801) * allow gdal.FileFromMemBuffer() to use buffer > 2GB (#6828) * accept unicode strings as field name argument in Feature (like SetField, GetField, etc...) and FeatureDefn methods = GDAL/OGR 2.1.0 Release Notes = == In a nutshell... == * New GDAL/raster drivers: - CALS: read/write driver for CALS Type I rasters - DB2 driver: read/write support for DB2 database (Windows only) - ISCE: read/write driver (#5991) - MRF: read/write driver (#6342) - SAFE: read driver for ESA SENTINEL-1 SAR products (#6054) - SENTINEL2: read driver for ESA SENTINEL-2 L1B/LC1/L2A products - WMTS: read driver for OGC WMTS services * New OGR/vector drivers: - AmigoCloud: read/write support for AmigoCloud mapping platform - DB2 driver: read/write support for DB2 database (Windows only) - MongoDB: read/write driver - netCDF: read/write driver - VDV: read/write VDV-451/VDV-452 driver, with specialization for the Austrian official open government street graph format * Significantly improved drivers: - CSV: new options, editing capabilities of existing file - ElasticSearch: read support and support writing any geometry type - GeoJSON: editing capabilities of existing file, "native data" (RFC 60) support - MBTiles: add raster write support. fixes in open support - PDF: add PDFium library as a possible back-end. - PLScenes: add support for V1 API - VRT: on-the-fly pan-sharpening - GTiff: multi-threaded compression for some compression methods * Port library: add /vsis3/, /vsis3_streaming/, /vsicrypt/ virtual file systems * RFC 26: Add hash-set band block cache implementation for very larger rasters (WMS, WMTS, ...) http://trac.osgeo.org/gdal/wiki/rfc26_blockcache * RFC 48: Geographical networks support (GNM) https://trac.osgeo.org/gdal/wiki/rfc48_geographical_networks_support * RFC 58: Add DeleteNoDataValue(): https://trac.osgeo.org/gdal/wiki/rfc58_removing_dataset_nodata_value * RFC 59.1: Make GDAL/OGR utilities available as library functions: https://trac.osgeo.org/gdal/wiki/rfc59.1_utilities_as_a_library For gdalinfo, gdal_translate, gdalwarp, ogr2ogr, gdaldem, nearblack, gdalgrid, gdal_rasterize, gdalbuildvrt Available in C, Python, Perl and Java bindings. * RFC 60: Improved round-tripping in OGR https://trac.osgeo.org/gdal/wiki/rfc60_improved_roundtripping_in_ogr Implemented in GeoJSON driver * RFC 61: Support for measured geometries. https://trac.osgeo.org/gdal/wiki/rfc61_support_for_measured_geometries Implemented in Shapefile, PostgreSQL/PostGIS, PGDump, MEM, SQLite, GeoPackage, FileGDB, OpenFileGDB, CSV, VRT * Upgrade to EPSG database v8.8 * General sanitization pass to clean-up code, fix a lot of compiler warnings, as well as issues pointed by static code analyzers, such as Coverity Scan (credits to Kurt Schwehr for tackling a large part of them) or CLang Static Analyzer. * Fixes in a number of drivers to be more robust against corrupted files (most found with American Fuzzy Lop): RIK, INGR, Northwood, HF2, CEOS, GTiff, GXF, BMP, NITF, HFA, VRT, FIT, CEOS2, NWT_GRD/NWT_GRC, MITAB, RPFTOC, DBF/Shape, XYZ, VFK, DXF, NAS, GSAG, GS7BG, OpenFileGDB, RMF, AIGRID, OpenAIR, EHDR, ISO8211, FAST, USGSDEM, DGN, SGI, OpenJPEG, PCRaster, BSB, ADRG, SRP, JPEG, Leveller, VICAR, PCIDSK, XPM as well as in portability library (CPL), algorithms and raster core. * Driver removed: - Google Maps Engine (GME) (#6261) == New installed files == * gdal_pansharpen.py * data/vdv452.xml * data/vdv452.xsd * data/netcdf_config.xsd * /path/where/bash-completion/scripts/are/installed/gdal-bash-completion.sh == Backward compatibility issues == See MIGRATION_GUIDE.TXT == GDAL/OGR 2.1.0 - General Changes == Build(Unix): * Allow plugin for HDF4, HDF5, GeoRaster, FileGDB, OCI and PG * Add a --with-gnm option to enable GNM * Add --enable-lto switch to turn on link time optimization (GCC >= 5) * Add --with-spatialite=dlopen --with-spatialite-soname=libspatialite.so[.X] syntax to allow linking against spatialite through dlopen() mechanism (#6386) * Add a lot of warning options when available in the compiler. * Change detection test of podofo to work with podofo 0.9.3 * Fix gdalserver compilation with recent GNU libc (such as in Arch Linux) (#6073) * Add pkg-config support for libkml fork (#6077) * Update config.sub and config.guess to their latest upstream versions so as to be able to detect new architectures (android, ppcle64, etc..) (#6101) * Fix python package installation in custom prefixes (#4563) * configure: check that CXX is really a working compiler (#4436) * Build support for Kakadu 7.7 * thinplatespline.cpp: avoid using optimized version of VizGeorefSpline2DBase_func4() with ICC versions that fail on it (#6350) * Add bash completion for GDAL/OGR utilities and scripts (#6381) * Add support for ODBC autodetection with mingw64 (#6000) * Remove macos 9 and older support (#6133). * Remove dist_docs, burnpath and pszUpdableINST_DATA in gcore/gdaldrivermanager.cpp as they are have not been used for a while (#6140). * Removed --without-ogr configure flag (#6117). Always build with OGR. Build(Windows): * Add support for Visual Studio 2015 (a.k.a MSVC_VER=1900 or VC 14) * Add KAKFLAGS to nmake.opt with KDU_{MAJOR,MINOR,PATCH}_VERSION define-s. * nmake.opt: make it less error prone to define SETARGV when paths include spaces (#6032) * nmake.opt: add CL.EXE compiler option /FC to display full path of source code file in diagnostics * Remove WinCE support (#6133) * nmake.opt: add /I flag to the INSTALL command so that xcopy will be smart enough to create a directory when copying files and avoid prompting for user input (https://github.com/OSGeo/gdal/pull/95) * Enable mssql spatial blugin build, use BCP as the default option for the sqlncli enabled builds * Add support to find MrSID 9.5 dll * Fix Windows build with recent MySQL versions and Visual Studio 2015 (#6457) Build(all): * Compatibility with C++11 and C++14 * Optional crypto++ dependency for /vsicrypt/ support * Optional mongocxx dependency for MongoDB support Other: * Disable copy constructor and assignment operators in classes OGRFieldDefn, OGRGeomFieldDefn, OGRFeature, GDALMultiDomainMetadata, GDALDefaultOverviews, GDALOpenInfoGDALDataset, GDALRasterBlock, GDALRasterBand and GDALDriver (#6100) == GDAL 2.1.0 - Overview of Changes == Port: * Add /vsicrypt/ virtual file system for reading/creating/update encrypted files on the fly, with random access capabilities * Add /vsis3/ and /vsis3_streaming/ virtual file systems to read/write objects from AWS S3 buckets * /vsizip/: avoid returning the previous file list of an already opened .zip if it has changed afterwards (#6005) * /vsizip/: use CP437 on Windows when ICONV support is available (#6410) * /vsimem/: implement append mode (#6049) * /vsistdin/: fix caching of first 1MB / VRT driver: read XML content from /vsistdin/ in a streaming compatible way (#6061) * /vsistdout/: flush when closing the handle (#6149) * Add VSIReadDirEx() with a limit on the number of files before giving up and corresponding VSIFilesystemHandler::ReadDirEx() virtual function * cpl_vsil_cache: rework to be able to work on very large files on 32bit systems * Add CPLThreadLocaleC class to use thread-specific locale settings (when available) * VSIWin32Handle::Flush(): add VSI_FLUSH config option that can be set to TRUE to force FlushFileBuffers(). (hack related to #5556) * Fix VSIL append mode in Windows (#6065) * Make CPLCreateMutexEx() support CPL_MUTEX_REGULAR; and fix CPL_MUTEX_ADAPTIVE to avoid continuing in code path for creation of recursive mutex * Add CPLWorkerThreadPool API * Add CPLGetThreadLocalConfigOption() * Fix CPL_LSBINT16PTR() and CPL_LSBINT32PTR() to work with non-byte pointer (#6090) * CPLRecodeStub(): add special case for CP437 -> UTF-8 when input is only printable ASCII * CPLHTTPFetch(): add LOW_SPEED_TIME and LOW_SPEED_LIMIT options * CPLGetValueType(): improve heuristics to avoid detecting some WKB strings as real numbers (#6128) * Add CPL_SHA256* and CPL_HMAC_SHA256 functions * Make CPLVirtualMemFileMapNew() work on all systems having mmap(), and thus GeoTIFF GTIFF_VIRTUAL_MEM_IO optimization too * Add VSI_MALLOC_VERBOSE() and similar macros to output an error message in case of failed alloc * CPLParseXMLString(): limit depth of elements to 10000 * Win32 file management: handle files whose total file path length is greater than 255 characters in Open() and Stat() * Make CPLFormFilename(absolute_path, "..", NULL) truncate when possible * Add VSIGetDiskFreeSpace() * Implement CPLIsInf() for Solaris 11 * Add a CPL_FINAL macro that expands to C++11 final keyword when C++11 is enabled, and use it in a few places * Avoid CPLEscapeString (CPLES_URL) encoding some characters unnecessarily (#5526) * Add CPLTestBool(), CPLTestBoolean(), CPLFetchBool() * Fix CPLGetValueType() to recognize D1 as a string and not a real number (#6305) * CPLFormFilename(): use '/' directory separator on Windows on /vsicurl_streaming/ files (#6310) * Add hack enabled by -DDEBUG_BOOL to detect implicit int->bool conversions that dislike MSVC (#6325) * Add hack to '#define NULL nullptr' when NULL_AS_NULLPTR is defined. Must be used together with -std=c++11 -Wzero-as-null-pointer-constant with GCC to detect misuses of NULL/nullptr (#6327) * Add VSIError mechanism to store errors related to filesystem calls, and use it for /vsis3/. Add new CPLE_ error numbers. (https://github.com/OSGeo/gdal/pull/98) * Fix CPLsscanf() to be conformant regarding how space/tab/... separators are handled, so as to fix OGR GMT to be able to read coordinates separated by tabulations (#6453) Core: * Change default value of GDAL_CACHEMAX to 5% of usable physical RAM * Allow open options name to be prefixed by @ to be silently ignored when not existing in driver options (RFC 60) * Honour NBITS metadata item when doing RasterIO() with non-nearest resampling (#6024) * GDALClientServer: pass open options to INSTR_Open call * Improve performance of GDALCopyWords() float->byte/int16/uint16 by using SSE2 * Decorate prototypes of RasterIO() related operations with CPL_WARN_UNUSED_RESULT * Avoid deadlock when writing 2 datasets in 2 threads (#6163) * Remove obsolete symbols __pure_virtual(), GDALCreateProjDef(), GDALReprojectToLongLat(), GDALReprojectFromLongLat() and GDALDestroyProjDef() * Remove obsolete non-template-based implementation of GDALCopyWords(). We don't support anymore such compilers * GDALJP2Box::ReadBoxData(): do not abort if memory allocation fails * Add GDALAdjustValueToDataType() in C API, and use it in GDALTranslate() and GDALWarp() * GDALDefaultOverviews::CreateMaskBand(): fix logic error related to writing per-band mask * Block cache: fix excessive memory consumption when dealing with datasets with different block sizes (#6226) * EXIFExtractMetadata(): fix potential 1-byte write buffer stack overflow * MDReader: do no attempt reading side-car files on /vsisubfile/ (#6241) * GDALCheckBandCount(): allow by default a maximum number of bands to 65536 * GDALOpenInfo: add StealSiblingFiles() and AreSiblingFilesLoaded() methods * GDALOpenInfo::GetSiblingFiles(): give up after GDAL_READDIR_LIMIT_ON_OPEN (default=1000) files in the directory * GDALDefaultOverviews: add TransferSiblingFiles() method * GDALDriver::DefaultCreateCopy(): fix logic related to propagation of NBITS / PIXELTYPE metadata items as creation options * Reorder driver registration a bit so that formats with efficient identification are probed first * Add GDALIdentifyEnum (GDAL_IDENTIFY_UNKNOWN, GDAL_IDENTIFY_FALSE or GDAL_IDENTIFY_TRUE) for return values of Identify() * GDALLoadRPCFile(): load .rpc files from Ikonos products that have extra end-of-line character (#6341) * Export GDALRegenerateOverviewsMultiBand() symbol, but mostly for plugins (#6342) * Add GDAL_NO_AUTOLOAD to compile out the body of AutoLoadDriver (#6380) * Avoid ComputeStatistics(), GetHistogram() and ComputeRasterMinMax() to use only the first column of blocks in approximation mode for a raster whose shape of blocks is a square (#6378) * Add GDALGetDataTypeSizeBits() and GDALGetDataTypeSizeBytes(). * GDALReadOziMapFile(): fix reading Ozi external files from virtual file systems (https://github.com/OSGeo/gdal/pull/114) * Add CPLSetCurrentErrorHandlerCatchDebug() to enable custom error handlers not to intercept debug messages Algorithms: * RPC transformer: provide SSE2 accelerated transformer * RPC: fix off-by-half pixel computation of (pixel, line), and in bilinear and bicubic RPC DEM interpolation; fix off-by-one pixel registration for Pleiades RPC (#5993) * RPC: make RPCInverseTransformPoint() check convergence has been reached (#6162) * RPC DEM: optimize when DEM is in NAD83 or any other geodetic CS that transform as a no-op to WGS 84 * RPC DEM: take into account vertical datum of the DEM when present to convert DEM elevations to ellipsoidal heights (#6084) * RPC DEM: do fallback cubic -> bilinear and bilinear -> near on DEM edges * RPC DEM: improve inverse transformer to validate error threshold and improve convergence (#6162, #6377) * RPC: fix issues with dateline (#6472) * TPS solver: discard duplicated GCP to avoid avoidable error, warning when 2 GCPs have same (pixel,line) but different (X,Y) or the reverse * Warper: rework multithreaded computations to use a thread pool rather than forking threads each time * Warper: avoid really excessive processing time for some warping with target areas completely off the source raster (especially when involving RPC) (#6182) * Warper: CreateKernelMask(): fix potential 32 bit integer overflow when using warp memory value > 2GB (#6448) * gdal_grid: add linear interpolation algorithm * gdal_grid: add invdistnn algorithm, variation on the existing inverse distance weighting algorithm with quadtree to search for points only in the neighborhood (#6038) * gdal_grid: fix crash in optimized mode with GCC 4.4 on 64bit (#5987) * gdal_grid: compile gdalgrid AVX optimization for Windows when supported by compiler * Add GDALTriangulationXXXX() API through libqhull * Sieve filter: fix crash on nodata polygons (#6096) * Sieve filter: improvement to walk through the biggest neighbour chain until we find a polygon larger than the threshold (#6296) * GDALFPolygonize(): factor implementation with integer case * GDALComputeMedianCutPCT(): fix to make it work with rasters with more than 2 billion pixels (#6146) * Overview: Make average and gauss methods aware of transparent color table entries (#6371) Utilities: * gdalinfo: workaround bug in proj <= 4.9.1 on datasets with a SRS with a vertical shift grid (#6083) * gdal_translate: detect more reliably if specified bands are not in default order * gdal_translate: fix -a_nodata with negative values on rasters that have PIXELTYPE=SIGNEDBYTE; fix preserving PIXELTYPE=SIGNEDBYTE with VRT * gdal_translate: fix random behaviour when -scale is used without source bounds (#6455) * gdal_rasterize: do on-the-fly reprojection of input vector onto output raster * gdal_rasterize: rasterize: always create output with 1/2 cell buffer of input geometry envelope (#6058) * gdal_rasterize: add the -dialect option * gdal_rasterize: accept NaN as a valid value for -init / -burn (#6467) * gdalwarp: add -doo option to specify open options of (existing) output dataset * gdalwarp: if RPC_DEM warping option is specified, use exact transformer by default (#5993) * gdalwarp: make it honour nodata value of existing dataset (if -dstNoData isn't explicitly specified) * gdalwarp: do not propagate STATISTICS_ of second or following source datasets * gdalwarp: do not emit warning when using -cutline with a SRS and the source raster has RPC or GEOLOCATION * gdalwarp: check that the cutline is valid after transformation/reprojection * gdalwarp: better deal when dealing with a mix of RGB and RGBA datasets as input * gdalwarp: fix -srcnodata to not put garbage values as target dstnodata (#6315) * gdalwarp: densify cutline to avoid invalid geometry after reprojection to source raster, especially in the RPC case (#6375) * gdalserver: add a -nofork mode (Unix only for now), so that multiple clients can connect to the same dataset. Useful for safe 'concurrent' updates * add gdal_pansharpen.py script * gdal2tiles.py: fix error on a raster with less than 3 bands that resulted in a 'IndexError: list index out of range' * gdal2tiles.py: Change EPSG:3785 / EPSG:900913 into EPSG:3857 (#5622) * gdal2tiles.py: add Leaflet template (https://github.com/OSGeo/gdal/pull/71) * gdal2tiles.py: add -q switch for quiet mode * gdaldem: correctly deal with NaN as nodata value (#6066) * gdaldem color-relief: deal with the case of repeated entries with the same value and the input raster has pixels that match that value exactly (#6422) * gdaladdo: emit error message if passed an invalid dataset name (#6240) * gdaladdo: do not silence warnings when opening in update mode, if the open is successful * gdalbuildvrt: fix potential crash when using -b switch (#6095) * gdalbuildvrt: accept nan as value for -srcnodata and -vrtnodata * gdalbuildvrt: return non zero return code if the flush of the VRT to disk failed * gdal_merge.py: takes again into account -n flag (#6067) * gdalbuildvrt / gdal_translate / VRT: use floating point values for source and destination offsets and sizes (#6127) * gdalmove.py: fix to run with GDAL 2.0 gdal.InvGeoTransform() signature * gdal_calc.py: Add * from gdalnumeric to gdal_calc.py eval namespace again, to fix 2.0 regression that made for example 'log10(A)' to no longer work (https://github.com/OSGeo/gdal/pull/121) Python samples: * gdalpythonserver.py: update to protocol 3 AAIGRID: * when writing with floating-point values, ensure at least one value has a decimal point to avoid issues with some readers (#6060) BMP driver: * BMP: avoid too big color table allocation in case of wrong iClrUsed value BLX driver: * converted to support VirtualIO ECRG driver: * change subdataset definition to make sure that they only consist of frames of same scale (#6043) * fix base34 decoding and Windows filename handling (#6271) ECW driver: * use wide char Open API on Windows when GDAL_FILENAME_IS_UTF8=YES (https://github.com/OSGeo/gdal/pull/70) * fix reading of ECW in /vsi file systems (with SDK >= 4) (#6482) GeoRaster driver: * fix deflate read error depending on endianness (#6252) GIF driver: * libgif: partial resync with giflib master (but keep ABI of 4.1.6) * fix crash when CreateCopy a source with one color in the color table GMT driver: * validate raster dimensions on opening, and acquire mutex in dataset destructor GPKG driver: * write support: fix various issues in update scenarios when interacting with the GDAL block cache that could result in lost/corrupted band data to be written in tiles (#6309) * write support: fix potential use of freed sqlite temporary DB handle when generating overviews with partial tiles (#6335) * write support: fix potential crash in scenarios when block cache is full (#6365) * write support: fix inversion of row/column in one of the SQL request involved in partial tiles management (#6339) * fix generation of files with 1-band paletted input dataset. Also improve the logic to retrieve color palette when forcing BAND_COUNT=1 on opening (#6450) GTiff driver: * add NUM_THREADS creation and open options to enable multi-threaded compression * fix GTiffDataset::IsBlockAvailable() wrong behaviour when compiling against internal libtiff, when a BigTIFF file has a TileByteCounts with LONG/4-byte counts and not LONG8 (#6001) * Correctly take into account overridden linear units for a geotiff with a EPSG PCS code (#6210, #4954) * make VirtualMemIO() work with non native endianness * improve VirtualMemIO() performance in tiled Contig read to single band copy * improve single band tiled VirtualMemIO reading * improve DirectIO() to work on tiled uncompressed geotiff, for non-resampling and nearest resampling cases. Also improve performance of resampling cases on un-tiled files * fix DirectIO() mode with complex types and inverted endianness (#6198) * optimize writing of 12-bit values * implement lazy loading of .aux.xml and .tfw/.wld * Internal libtiff: update to CVS HEAD post libtiff 4.0.6 * make SetColorInterpretation(GCI_AlphaBand) work on a 5 band or more GeoTIFF (#6102) * automatically set PHOTOMETRIC=RGB if manually assigning color interpretation Red,Green,Blue to band 1,2,3 before directory crystallization (#6272) * add GEOTIFF_KEYS_FLAVOR=ESRI_PE creation option to write EPSG:3857 in a ESRI compatible way (#5924) * call XTIFFInitialize() in LibgeotiffOneTimeInit() as the former isn't thread-safe, so better call it from the later which is thread-safe (#6163) * fix reading and writing angular units different from degree (namely arc-second, arc-minute, grad, gon and radian) (#6171) * do not use VirtualMemIO optimization on compressed /vsimem/ files (#6195) * correctly set GTRasterTypeGeoKey=RasterPixelIsPoint if AREA_OR_POINT=Point but there is no SRS set (#6225) * improve a bit error detection when writing * check free space before writing (only for big, non sparse, uncompressed) * do not read large 'one row' JBIG compressed files with the scanline API (#6264) * Fix SetMetadata() to properly clear existing PAM metadata (complement to #5807) * prevent potential out of bounds read/write to TIFFTAG_EXTRASAMPLES (#6282) * do not use first directory as potential mask, to avoid assertion in GTiffDataset::SetDirectory() (#6287) * reject files with strips/tiles/scanlines bigger than 2 GB to avoid 32 bit integer overflow. Also in case of files with Contig PlanarConfiguration do not make reading one block for band 2 OK when reading for band 1 issued an error (#6288) * GTIFFWriteDirectory(): avoid memory leak of codec related memory (#2055) * Make ALPHA=NO in CreateCopy() cancel alpha color interpretation even if present in source raster * fix problem with implicit overviews of JPEG-compressed files (#6308) * fix compilation problem with internal libtiff if DEFER_STRILE_LOAD isn't defined (which is not the default configuration) (https://github.com/OSGeo/gdal/pull/90) * use more appropriate error message when 4GB threshold is reached with external overviews, and try to make doc of BIGTIFF=IF_NEEDED/IF_SAFER clearer (#6353) HDF4 driver: * Skip quotation mark when parsing HDF-EOS metadata. HDF5 driver: * add Komsat Mission ID to possible value for HDF5 SAR product (https://github.com/OSGeo/gdal/pull/103) HF2 driver: * fix reading side of the driver to work on architectures where char is unsigned, like PPC or ARM (#6082) HFA driver: * when reading projection, preserve EPSG code if AutoIdentifyEPSG() identified the SRS, even if a PE string is present (#6079) * keep TOWGS84 even when using ESRI PE string (#6158) * fix crashes on corrupted files (#6208, #6286) INGR driver: * check that RLE decoding produced the expected number of bytes and error out otherwise; test that 'random' line seeking actually works JPEGLS driver: * fix build error (#6430) JP2ECW driver: * honour psExtraArg->eResampleAlg when upsampling (#6022) JP2KAK driver: * honour psExtraArg->eResampleAlg when upsampling (#6022) * try rounded dimensions to decide if the data is being requested exactly at a sub-resolution * support v7.7 on Unix (additional changes potentially needed on Windows) JP2OpenJPEG driver: * Safer multi-threaded use JPEG driver: * add USE_INTERNAL_OVERVIEWS open option (default to TRUE as in recent versions, can be set to FALSE to hide internal overviews KMLSuperOverlay driver: * fix 2.0 regression with some RasterIO() requests involving resampling (#6311) * recognize datasets that have a intermediate <Folder> that forms a <Document><Folder><Region/><GroundOverlay/> structure (#6343) LCP driver: * fix wrong use of endianness macros to fix behaviour on big-endian hots Leveller driver: * update to v9 read support (#5632,) MBTiles driver: * add raster write support * fix so that datasets whose lowest min zoom level is 16 or above are recognized * be robust to invalid 'bounds' at dataset opening (#6458) MEM driver: * avoid 32 bit overflows NetCDF driver: * add support for reading NC4 unsigned short attributes and variables (#6337) * read correctly nodata values in [128,255] range for (unsigned) Byte data type (#6175) * implement Get/SetUnitType() using the standard units attribute (https://github.com/OSGeo/gdal/pull/96) * optimize IReadBlock() and CheckData() handling of partial blocks in the x axis by re-using the GDAL block buffer instead of allocating a new temporary buffer for each block (#5950) * full read/write support for new NetCDF4 types NC_UBYTE, NC_USHORT, NC_UINT and NC_STRING for variables (except for NC_STRING) and attributes (https://github.com/OSGeo/gdal/pull/99) * add support for the geostationary projection (#6030) * fix one byte heap write overflow in NCDFTokenizeArray() (#6231) * fix potential buffer overflows with uses of nc_inq_varname(), nc_inq_attname() and nc_get_att_text() (#6227) * validate that gridmapping:GeoTransform has 6 values (#6244) * fix wrong use of deallocator when writing a GEOLOCATION array, and other issues * limit number of bands reported to 32768 by default * validate raster dimensions * validate content of NC_GLOBAL#GDAL variable (#6245) NGSGEOID driver: * make it work on > 2GB file NITF driver: * data/nitf_spec.xml: Add CSCCGA, MENSRB, SENSRB, STREOB, ENGRDA, EXPLTB and PATCHB TREs (https://github.com/OSGeo/gdal/pull/81, #6285, https://github.com/OSGeo/gdal/pull/86) * fix parser to properly deal with variable length items not in first nesting level Northwoord driver: * fix computation of intermediate color table values on non-Intel platforms (#6091) NTv2 driver: * support reading/writing/appending to files with big-endian order (cf https://github.com/OSGeo/proj.4/issues/345) OpenJPEG driver: * do not expose block dimensions larger than dataset dimensions to avoid wasting memory (#6233) PCIDSK driver: * Remove the old driver (#6172) PDF driver: * add PDFium library as a possible back-end. Initial support contributed by Klokan Technologies GmbH (http://www.maptiler.com/) * workaround a bug of PoDoFo 0.9.0 by avoiding loading of vector content in raster-only mode (doesn't prevent the crash if reading the vector layers) (#6069) * make OGC BP registration work with media box where bottom_y is negative and top_y = 0 (in non rotated case) * make OGC BP registration work with media box where min_x != 0 (in non-rotated case) * correctly take into account non-meter linear units with OGC BP encoding, and add support for US FOOT (#6292) PDS driver: * change default values of PDS_SampleProjOffset_Shift and PDS_LineProjOffset_Shift to 0.5 (#5941) * fix nodata value for UInt16 to be 0 (#6064) * accept 'ODL_VERSION_ID = ODL3' in header (#6279) PGChip driver: * driver removed from sources PLScenes driver: * PLScenes V0: avoid opening raster to generate dummy .aux.xml PNG driver: * Support writing 1, 2 or 4 bit single band * Add NBITS creation option * fix XML of creation option list * Update internal libpng to 1.2.56 PostgisRaster driver: * avoid Identify() to recognize OGR PostgreSQL connection strings with schemas option and cause loud 'QuietDelete' (#6034) Raw drivers: * better support for direct read of more than 2GB in single gulp (untested though) RIK driver: * fix Identify() method to recognize again non-RIK3 RIK datasets (#6078) RMF driver: * implement GetNoDataValue() * add read/write access to new RMF format for files larger than 4 Gb (version=0x201) (https://github.com/OSGeo/gdal/pull/11) ROIPAC driver: * Support offset and scale band (#6189) VICAR driver: * change PDS_SampleProjOffset_Shift and PDS_LineProjOffset_Shift default values to 0.5 (#5941) * fix loss of precision in scale and offset VRT driver: * expose implicit 'virtual' overviews for VRT whose bands are made of a single SimpleSource/ComplexSource * gdalvrt.xsd: Add capitalized versions of true and false (#6014) * GetSingleSimpleSource(): check there's a single source (#6025) * honour VRTRasterBand NBITS metadata with SimpleSource and ComplexSource * properly take into account nodata value declared at VRT band level when doing resampling with non-nearest * honour relativeToVRT when using AddBand() to add a VRTRawRasterBand (https://github.com/OSGeo/gdal/pull/67) * VRT warp: fix crash with implicit overviews and destination alpha band (#6081) * make GetDefaultHistogram() on a sourced raster band save the result in the VRT (#6088) * serialize NODATA and NoDataValue items with %.16g, e.g. so as to be able to hold large int32 nodata values (#6151) * VRTSourcedRasterBand: make ComputeRasterMinMax() and ComputeStatistics() forward bApproxOK to overview band (useful for implicit overviews) * make CreateCopy() preserve NBITS metadata item * avoid loading sibling file list if not available * VRT raw: don't truncate last figure of ImageOffset if there are left space padding (#6290) * VRTWarpedDataset::SetMetadataItem(): fix crash when calling with name=SrcOvrLevel and value=NULL (#6397) * Warped VRT: fix deadlock in situation where warped VRT datasets are read in multiple threads and the block cache reaches saturation. Also add a GDAL_ENABLE_READ_WRITE_MUTEX config option that can be set to NO to disable the read/write mutex mechanism in cases where it would deadlock (#6400) WMS driver: * add a IIP (Internet Imaging Protocol) minidriver * limit number of zoom levels for ArcGIS MapServer JSon (#6186) * determine a resolution that will not result in a number that is larger than the maximum size of an integer. Any value that exceeds the maximum size of an integer will raise an invalid dataset dimensions error. (https://github.com/OSGeo/gdal/pull/89) XYZ driver: * be more robust to not exactly equal X and Y spacing (#6461) == OGR 2.1.0 - Overview of Changes == Core: * Add OGREditableLayer class to add editing capabilities to drivers with none or limited editing capabilities * OGRGeometry: add DelaunayTriangulation() method (GEOS >= 3.4) * OGRGeometry and derived classes: implement copy constructor and assignment operator (#5990) * OGRGeometry: Fix result of Equals on POINT EMPTY with POINT(0 0) * OGRFeature SetField(): more type conversions allowed, particularly with array types * OGRFeature::SetGeometry()/SetGeometryDirectly(): make it work when passed geometry is the currently installed geometry (#6312) * OGR SQL: do not silently skip NULL values in the first records when evaluating a SELECT DISTINCT (#6020) * OGR SQL: correctly sort NULL values in first positions (#6155) * OGR SQL: fix CAST(x AS bigint) to return an evaluated int64 node, and not int32 (#6479) * OGR SQL: handle 'fid' as Integer64 in where clause, and allow CAST(fid AS bigint) in selected columns (#6484) * Add OGRUpdateFieldType() * Decorate a few functions/methods of the OGR layer API with CPL_WARN_UNUSED_RESULT * WKT export: use 15 significant figures, instead of 15 figures after decimal point (#6145) * WKT export: do not append .0 after non-finite values (#6319) * Fix typo in definition of name of OGR_FD_ReorderFieldDefns (final s was missing) * OGRLayer::SetIgnoredFields(): properly reset state of non first geometry fields (#6283) * Make OGRLayer::SetSpatialFilter(GetSpatialFilter()) work with non empty spatial filter (#6284) * OGRLayerDecorator: add missing CreateGeomField() * OGRLayer::Erase(): do not discard input geometries that have no intersection with method layer (#6322) * OGRLayer::Erase(): Speedup = ~70% * Add OGRPreparedGeometryContains() * Use prepared geometry intersects as pretest in layer Intersection, Union, and Identity methods if requested. Use prepared geometry containment as pretest in layer Intersection method if requested. * Bail out from layer algebra methods if GEOS calls fail and not SKIP_FAILURES. * OGR_G_SetPoints(): error out if padfX or padfY == NULL, do not change coordinate dimension to 3D when pabyZ == NULL, fix optimization on linestring to call setPoints() only if the strides are the ones of a double, not 0 as incorrectly done before (#6344) * OGRParseDate(): more strict validation to reject invalide dates (#6452) OGRSpatialReference: * Upgrade to EPSG database v8.8 * Add support for SCH (Spherical Cross-track Height) projection * Optimize reprojection typically between WGS84 based SRS and WebMercator * Correctly transform Mercator_2SP and _1SP to ESRI Mercator, and back from ESRI Mercator to Mercator_2SP (#4861) * No longer enforce C locale if running against latest proj that is locale safe (4.9.2 or later) * EPSGGetPCSInfo(): use pcs.override.csv in priority over pcs.csv to read projection name, UOM, UOMAngle, GeogCS, etc... (#6026) * morphToESRI(): use GCS_WGS_1972 as GCS name for EPSG:4322 (#6027) * morphToESRI(): use Mercator_Auxiliary_Sphere projection for EPSG:3857. morphFromESRI(): map Mercator_Auxiliary_Sphere to EPSG:3857 (#5924) * Align hard-coded WKT of well known GCS definitions of WGS84, WGS72, NAD27 and NAD83 with the WKT of their EPSG def (#6080) * morphFromESRI(): special case with PROJCS name 'WGS_84_Pseudo_Mercator' (#6134) * OSR C API: fix declarations of OSRSetAxes() and OSRSetWagner(), and add missing OSRSetHOMAC(), OSRSetMercator2SP() and OSRSetTPED() (#6183) * Recognize EPSG 9835 method (Lambert Cylindrical Equal Area (Ellipsoidal)), needed for EPSG:6933 PCS for example * importFromProj4/exportToProj4(): rework linear unit conversion between WKT name/values and proj4 unit name, and extend its scope in WKT to proj4 conversions * OSR ESRI .prj: add support for reading custom ellipsoid in Parameters line * on import of +proj=geos, if +sweep=x is used then store it as a proj4 extension node (#6030) Utilities: * ogrinfo / ogr2ogr: implement @filename syntax for -sql and -where * ogr2ogr: prevent the -gt setting from overriding transaction group size of 1 set by skipfailures earlier (#2409) * ogr2ogr: warn if -zfield field does not exist in source layer * ogr2ogr -skip: rollback dataset transaction in case of failure (#6328) * ogr2ogr: fix -append with a source dataset with a mix of existing and non existing layers in the target datasource (#6345) * ogr2ogr: imply quiet mode if /vsistdout/ is used as destination filename * ogr2ogr: make -dim and -nlt support measure geometry types CartoDB: * fix GetNextFeature() on a newly create table (#6109) * defer 'CartoDBfycation' at layer closing * optimize feature insertion with multiple rows INSERT CSV driver: * add editing capabilities of existing files * add X_POSSIBLE_NAMES, Y_POSSIBLE_NAMES, Z_POSSIBLE_NAMES, GEOM_POSSIBLE_NAMES and KEEP_GEOM_COLUMNS open options * add HEADERS open option to force OGR to handle numeric column names. (PR #63) * add EMPTY_STRING_AS_NULL=YES/NO open option * implement compatibility enhancements for GeoCSV specification (#5989) * fix detection of TAB delimiter in allCountries.csv when the first line has a comma (#6086) * fix issues with leading single quote, and missing first line after ResetReading(), when parsing allCountries.txt (#6087) * speed-up GetFeatureCount() on allCountries.txt * on CreateDataSource() with a .csv name, do not try to open other existing .csv files in the directory * make CreateGeomField() returns OGRERR_NONE in case of success instead of OGRERR_FAILURE (#6280) * avoid adding trailing comma in header line when writing 'WKT,a_single_field' DGN driver: * add partial 3D transformation support for cell headers DXF driver: * detect files without .dxf extension (#5994) * fix handling of ELLIPSE with Z extrusion axis = -1 (#5705) * take into account full definition of spline entity (degree, control points, weights and knots) when stroking splines (#6436) * better handling of various object coordinate systems found in dxf files for point, line, polyline, spline and ellipse entities. Add anchor position to text styles. Remove polygon/polyface mesh parsing from polyline entity (#6459) ElasticSearch driver: * use get /_stats instead of /_status for ElasticSearch 2.0 compatibility (#6346) FileGDB driver: * make CreateFeature() honour user set FID, and implement more fine grained transaction for Linux/Unix * give a hint of using FileGDB SDK 1.4 is FileGDB compression is used GeoJSON driver: * Add editing capabilities of existing files * Add ARRAY_AS_STRING=YES open option * Use '%.17g' formatting by default for floating-point numbers and add SIGNIFICANT_FIGURES layer creation option (#6291) * add a json_ex_get_object_by_path() function * fix crash on null / non-json object features (#6166) * serialize string values that are valid JSon dictionary or array as it (ie do not quote them) * make sure there's enough space to write the FeatureCollection bbox (#6262). Also avoid duplicating FeatureCollection bbox if source has one (trunk only) * Export POINT EMPTY as having a null geometry, instead as of being POINT(0 0) (#6349) * Do not 'promote' a null field to OFTString type if it had another type before (#6351) GME driver: * Driver removed. Maps Engine being shut down at the end of January 2016. GML driver: * VFR: add new attribute DatumVzniku (v1.6) * VFR: fix ST_UVOH type handling * VFR: fix ZpusobyOchrany attributes (data types and names) * VFR: fix CisloDomovni attributes (Integer->IntegerList) * VFR: fix TEA attributes of StavebniObjekty * add NAMESPACE_DECL=YES option to OGR_G_ExportToGMLEx() to add xmlns:gml=http://www.opengis.net/gml or http://www.opengis.net/gml/3.2 declaration; Also accept GML2 or GML32 as valid valiues for FORMAT option (#6214) * serialize in .gfs file the name of the geometry element when it is 'geometry' since this is a particular case (#6247) * fix logic error in BuildJointClassFromScannedSchema() (#6302) GPKG driver: * make it accept files with non standard extension if they still have the correct application_id (#6396); also accept the .gpkx extension that may be used for extended geopackages * emit warning when generating a database without .gpkg/.gpkx extension (#6396) * as GPKG 1.1 uses a different application_id, emit a more specific warning if the application id starts with GPxx (but is not GP10). Add GPKG_WARN_UNRECOGNIZED_APPLICATION_ID config option to avoid the warning * correct scope of gpkg_geom_XXXXX extensions to be read-write, and allow reading geometry types CURVE or SURFACE * avoid trying to insert a gpkg_geom_XXXX extension if already done (#6402) * writer: implement strategy to flush partial_tiles temporary database when it becomes too big (#6462) * writer: when writing to GoogleMapsCompatible tiling scheme, better deal with source rasters in EPSG:4326 with latitude = +/-90 (#6463) * fix generation of files with 1-band paletted input dataset. Also improve the logic to retrieve color palette when forcing BAND_COUNT=1 on opening (#6450) GPX driver: * fix crash when parsing a 'time' extension element at route/track level (2.0 regression, #6237) ILI driver: * ILI1: Support for Surface polygon rings spread over multiple geometry records * ILI1: add string TID support (https://github.com/OSGeo/gdal/pull/91) * Fix crash with models using types derived from INTERLIS * Fix memory leaks (#6178) JML driver: * remove arbitrary limitation preventing from reading geometries with <gml:coordinates> larger than 10 MB (#6338) KML driver: * fix crash on KML files without content but with nested folders (#6486) LIBKML driver: * for documents without folder, use document name when available as name of layer (#6409) Memory driver: * add support for sparse feature IDs * add ADVERTIZE_UTF8 layer creation option MITAB driver: * Add support for block sizes other than 512 bytes in .map files, for MapInfo 15.2 compatibility (#6298) * write correct datum id for EPSG:3857 * read MID files with TAB delimiter and empty first field (#5405) * use projection code 29 when exporting non-Polar Lambert Azimuthal Equal Area (#5220) * fix crashes when parsing invalid MIF geometries (#6273) MSSQLSpatial driver: * Implement MSSQL bulk insert (#4792) * do not treat a primary key that is not of integer type as the FID (#6235) NTF driver: * fix potential buffer overflows when reading too short lines (#6277) ODBC driver: * remove limitations to 500 columns ODS driver: * fix loss of precision in formula computation OpenFileGDB driver: * do not emit warning if SDC/CDF table detected and that FileGDB driver is present * fix min/max on columns without indices (#6150) * build correct geometry for a multi-part wkbMultiLineStringZ (#6332) * add support for reading SHPT_GENERALPOINT (#6478) OSM driver: * correct fields ids for the (non frequently used) Node message * do not override 'our' osm_id (the node, way or relation id) with a tag named 'osm_id' (#6347) * properly deal with polygons in other_relations geometrycollection (#6475) PG driver: * Add PRELUDE_STATEMENTS and CLOSING_STATEMENTS open option to be for example able to specify options, like statement_timeout, with pg_bouncer * Fix 2.0 regression when overwriting several existing PostGIS layers with ogr2ogr (#6018) * Update PG, PGDump and CartoDB drivers to correctly export POINT EMPTY for PostGIS 2.2 * avoid resetting error potentially emitted by ExecuteSQL() (#6194) * sanitize management of quoting for FID column at layer creation * fix to get SRID on result layer with PostGIS 2.2 * in copy mode (the default on layer creation), do not truncate the concatenated string list to the field width (#6356) * make such that GEOMETRY_NAME layer creation option is honoured in ogr2ogr when the source geometry field has a not-null constraint (#6366) * read and set DESCRIPTION metadata item from/into pg_description system table; add DESCRIPTION layer creation option * support int2[] and numeric[] types, better map float4[] type * remove code that was intended to handled binary cursors as it cannot be triggered * fix append of several layers in PG_USE_COPY mode and within transaction (ogr2ogr -append use case) (#6411) PGDump driver: * fix issue with case of ogc_fid field in case the FID layer creation option is not set by user or by ogr2ogr (related to #6232) * in copy mode (the default on layer creation), do not truncate the concatenated string list to the field width (#6356) * make such that GEOMETRY_NAME layer creation option is honoured in ogr2ogr when the source geometry field has a not-null constraint (#6366) * set DESCRIPTION metadata item from/into pg_description system table; add DESCRIPTION layer creation option Shapefile driver: * accept opening standalone .dbf files whose header length is not a multiple of 32 bytes (#6035) * fix REPACK crash on shapefile without .dbf (#6274) * add capability to restore/build a missing .shx file when defining SHAPE_RESTORE_SHX to TRUE (#5035) * avoid CreateLayer() to error out when passed wkbUnknown | wkb25D (#6473) SQLite/Spatialite driver: * support file:xxx URI syntax (derived from patch by joker99, #6150) * fix heuristics in OGRSQLiteSelectLayer::GetExtent() to not be used when there's a sub SELECT (#6062) * fix crash on GetLayerByName('non_existing_table(geom_column)') (#6103) * fix OGRSQLiteSelectLayerCommonBehaviour::GetBaseLayer() to no longer 'eat' consecutive characters in layer name (#6107) * Spatialite: turn debug messages warning about update not being supported because of missing or too old spatialite version as errors, and return NULL to the caller (#6199) * fix memleak in OGRSQLiteTableLayer destructor when updating geometry_columns_time * VFS: increase mxPathname to 2048 by default, and provide OGR_SQLITE_VFS_MAXPATHNAME config option to be able to configure that higher if that would be needed. Useful when dealing with very long names like /vsicurl/.... with AWS S3 security tokens * VFS: do not probe -wal files on /vsicurl/ SXF driver: * fix wrong use of endianness macros to fix behaviour on big-endian hots * add recoding from CP1251 for TEXT attribute that is now decoded * fix various issues (#6357) VRT driver: * implement CloseDependentDatasets() * fix editing with 'direct' geometry mode which could cause attribute column to be empty (#6289) * fix crash with a OGRVRTWarpedLayer using a source layer that would have non geometry column (unlikely to happen currently as this would require explicit disabling it, but more likey with following commit that createe VRT non-spatial layer implictly when the source is non-spatial, linked to #6336) * avoid creating an implicit wkbUnknown geometry field when the source has no geometry column and there's no XML elements related to geometry fields (#6336) XLSX driver: * fix reading sheets with more than > 26 columns and 'holes' (#6363) XPlane driver: * extend ICAO identifiers to 5 digits (#6003) == SWIG Language Bindings == All bindings: * add a options parameter to gdal.ReprojectImage() to pass warp options * Change ReadRaster and WriteRaster to use GIntBig and the *IOEx-methods * prevent NULL file pointer from being passed to VSIF*L functions * make gdal.Rename() accept Unicode strings * add SpatialReference.GetAxisName() and SpatialReference.GetAxisOrientation() (#6441) * add SpatialReference.GetAngularUnitsName() (#6445) Java bindings: * Fix typemap for input parameter of type GIntBig (fixes GetFeature(long), DeleteFeature(long), etc...) (#6464) * Bump minimal java version to 1.5 in case SWIG generates anotations (#6433, patch by Bas Couwenberg) * GNUmakefile: add -f in rm commands * GNUmakefile: add support for all hardening buildflags Perl bindings: * Fix #6050: string formatting in croak. * Perl Makefile.PL: add support for all hardening buildflags (#5998) * use strict and warnings in overridden constructors. * add $VERSION to ogr_perl.i (OGR.pm), which is required by pause.perl.org. * Add some basic module info for CPAN. * Bugfix for Geo::OGR::Feature->new(). * Add many utility level algorithms as methods to various classes. * New class for XML stuff * New Makefile.PL, which can download and build GDAL. This allows automatic testing of the CPAN module. * Wrap VSIStdoutSetRedirection and allow creating datasets via an object, which can write and close. * Geo::OGR::Driver and Geo::OGR::DataSource are now Perl wrappers for respective GDAL classes. * Add to the error stack also errors from the bindings * Fix sending utf8 from Perl to GDAL. Should also remove some "uninitialized value" warnings. * ReadTile and WriteTile methods for Dataset, ReadTile accepts now tile size and scaling algorithm. * Improved Parent - Child management. * Improved support for 64bit ints. * Measures support in Geometry class. * Many new tests Python bindings: * make Feature.ExportToJson() output boolean value for a boolean field * support floating point coordinates for the source windows of Band.ReadRaster() and Band.ReadAsArray() * fix build with SWIG 3.0.6 (#6045) * make gdal.OpenEx() throw a Python exception in case of failed open when exceptions are enables with gdal.UseExceptions() (#6075) * Disable opening a NumPy dataset with a filename returned by gdal_array.GetArrayFilename(() unless GDAL_ARRAY_OPEN_BY_FILENAME is set to TRUE * disable the warning about using deprecated wkb25DBit constant as it uses a trick that prevents the bindings from being used by py2exe / pyinstaller (#6364)
2018-02-19 13:05:17 +01:00
PATCHDIR= ${.CURDIR}/../../geography/gdal-lib/patches
DISTINFO_FILE= ${.CURDIR}/../../geography/gdal-lib/distinfo
CHECK_PORTABILITY_SKIP+= mkbindist.sh
gdal-lib: Force the use of openjpeg over jasper for JPEG 2000 support Add pkg-config to USE_TOOLS to ensure openjpeg gets detected properly. See https://github.com/mdadams/jasper/issues/208 bump PKGREVISION
2019-07-16 23:24:47 +02:00
USE_TOOLS+= gmake pkg-config
Update to 2.2.3 * Add Makefile.common to prepare for geography/py-gdal Changelog: = GDAL/OGR 2.2.3 Release Notes = The 2.2.3 release is a bug fix release. == Build == * nmake.opt: Ensure PDB is included in release DLL if WITH_PDB requested (#7055) == Port == * /vsicurl/ and derived filesystems: redirect ReadDir() to ReadDirEx() (#7045) * /vsicurl/: enable redirection optimization on signed URLs of Google Cloud Storage. Helps for the PLScenes driver (#7067) * /vsicurl/: fix 2.2 regression regarding retrieval of file size of FTP file (#7088) * /vsis3/: fix to avoid invalid content to be sent if Write() writes more than 50 MB in a single call (#7058) * /vsis3/: fix Seek(Tell(), SEEK_SET) fails if current position is not 0 (#7062) * /vsis3/: fix support of non-ASCII characters in keys (#7143, #7146) * Fix CPLCopyTree() that doesn't properly on MSVC 2015 (and possibly other platforms) (#7070) == Algorithms == * RPC transformer: set output coordinates to HUGE_VAL when failure occurs, so that a following coordinate transformation can detect the error too (#7090) * GDALGrid() with linear algorithm: avoid assertions/segmentation fault when GDALTriangulationFindFacetDirected() fails (#7101) * GDALComputeProximity(): fix int32 overflow when computing distances on large input datasets (#7102) * GDALResampleChunk32R_Gauss: fix potential out of bounds access (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4056) == GDAL utilities == * gdalwarp: fix issue with GDALAdjustValueToDataType(Float32, +/- inf) that didn't preserve infinity, which affected gdalwarp -dstnodata inf (#7097) * gdalwarp -crop_to_cutline: reduce number of iterations to find the appropriate densification (#7119) * gdal_contour: return with non-0 code if field creation or contour generation failed (#7147) == GDAL drivers == GeoRaster driver: * fix int overflow (#6999) GPKG driver: * speed-up statistics retrieval on non-Byte datasets (#7096) GRIB driver: * if GRIB_ADJUST_LONGITUDE_RANGE config option is set to YES, adjust the longitude range to be close to [-180,180] when possible for products whose left origin is close to 180deg. (#7103) GSAG driver: * avoid assertion on some files with nul characters (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3726) GTiff driver: * make sure that -co PHOTOMETRIC=RGB overrides the color interpretation of the first 3 bands of the source datasets (#7064) * on reading use GeogTOWGS84GeoKey to override the defaults TOWGS84 values coming from EPSG code (#7144) * fix potential crash when reading in RGBA mode with internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4157) * fix potential crash with old-jpeg compression and internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4180) HTTP driver: * do not open the underlying dataset with GDAL_OF_SHARED, to avoid later assertion JP2KAK driver: * fix unix build with Kakadu 7.A and later (#7048, #7081) JPEG driver: * Add compatibility with libjpeg-turbo 1.5.2 that honours max_memory_to_use (cf https://github.com/libjpeg-turbo/libjpeg-turbo/issues/162) JP2OpenJPEG driver: * Add support for openjpeg 2.3 (#7074) netCDF driver: * fix raster read as nodata with Byte datatype, (valid_range={0,255} or _Unsigned = True) and negative _FillValue (#7069) * be more tolerant on the formatting of standard parallel (space separated instead of {x,y,...} syntax), and accept up to 2/1000 error on spacing to consider a regular grid, to be able to read files provided by the national weather institute of Netherlands (KNMI) (#7086) PDF driver: * round to upper integer when computing a DPI such that page size remains within limits accepted by Acrobat (#7083) Sentinel2 driver: * add support for direct opening of .zip files of new safe_compact L1C products (#7085) VRT driver: * avoid error being emitted when opening a VRTRawRasterBand in a .zip files (#7056) == OGR core == * Fix OGR[Curve]Polygon::Intersects(OGRPoint*) to return true when point is on polygon boundary (#7091) * importFromWkt(): fix import of GEOMETRYCOLLECTION ending with POINT EMPTY or LINESTRING EMPTY (#7128, 2.1 regression) == OGR utilities == * ogrtindex: fix crash when using -f SQLITE -t_srs XXXX (#7053) == OGR drivers == Amigocloud driver: * Fixed data field types (https://github.com/OSGeo/gdal/pull/246) * Output list of datasets if dataset id is not provided. * Implemented waiting for job to complete on the server. This will improve write to AmigoCloud reliability. * Updated documentation. FileGDB driver: * remove erroneous ODsCCreateGeomFieldAfterCreateLayer capability declaration (https://github.com/OSGeo/gdal/pull/247) GeoJSON driver: * ESRIJson: recognize documents that lack geometry fields (#7071) * ESRIJson: recognize documents starting with a very long fieldAliases list (#7107) GeoRSS driver: * fix detection of field type (#7108) GML driver: * fix FORCE_SRS_DETECTION=YES effect on feature count and SRS reporting on gml files with .gfs (#7046) * do not try to open kml files (#7061) * GML geometry parsing: fix robustness issue with gml:PolyhedralSurface * do not report gml:name / gml:description of features as layer metadata GPKG driver: * do not try to update extent on gpkg_contents after GetExtent() on a empty layer of a datasource opened in read-only mode GTM driver: * fix null pointer dereference in case of read error (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4047) KML / LIBKML drivers: MDB driver: * fix multi-thread support (https://issues.qgis.org/issues/16039) MITAB driver * do not emit error if the .ind file is missing, just a debug message (#7094) MySQL: * fix build with MariaDB 10.2 (#7079) OCI driver: * initialize in multi-threaded compatible mode to fix QGIS related crashes (https://issues.qgis.org/issues/17311), and fix a few memleaks ODS driver: * avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050) PLScenes driver: * backport data/plscenesconf.json from trunk to add SkySatScene fields Shapefile driver: * Fix GetFeatureCount() to properly take into account spatial filter when attribute filter also in effect (#7123) SQLite/Spatialite driver: * don't invalidate statistics when running a PRAGMA (https://issues.qgis.org/issues/17424) * SQLite dialect: support SQLite 3.21, and LIKE, <>, IS NOT, IS NOT NULL, IS NULL and IS operators (#7149) XLS driver: * workaround opening filenames with incompatible character set on Windows (https://issues.qgis.org/issues/9301) XLSX driver: * fix non working detection of Date/Time fields in some documents (#7073) * fix opening of documents with x: namespace in xl/workbook.xml (#7110) * avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050) == SWIG bindings == * map GRA_Max, GRA_Min, GRA_Med, GRA_Q1 and GRA_Q3 (#7153) == Python bindings == * remove 'from . import _gdal_array' line from gdal_array.py that is not necessary with normal execution of the bindings, and cause errors with PyInstaller (#7044) = GDAL/OGR 2.2.2 Release Notes = The 2.2.2 release is a bug fix release. == Build == * Windows build: always build the PDF driver, even when none of poppler/podofo/pdfium are available, in which case it is write-only (#6938) * Compilation fixes on Ubuntu 16.04 with explicit --std=c++03 (#6952) == Port == * /vsigzip/: make Eof() detect end of stream when receive a Z_BUF_ERROR error. Fixes #6944. * Fix memleak in VSIGSFSHandler::GetFileList() * /vsigzip/: avoid trying to write a .gz.properties file on a /vsicurl/ file (#7016) * Fix registration of /vsicrypt/ file system handler when compiled as a plugin (#7030) * CPLStrtod(): parse string like '-1.#IND0000000' as NaN instead of -1 (#7031) == Algorithms == * Warper: when operating on single-band, skip target pixels whose source center pixel is nodata (2.2 regression, #7001) * Warper: avoid blocking when number of rows is 1 and NUM_THREADS > 1 (#7041). Also limit the number of threads so that each one processes at least 65536 pixels == GDAL core == * GDALGCPsToGeoTransform(): add GDAL_GCPS_TO_GEOTRANSFORM_APPROX_OK=YES and GDAL_GCPS_TO_GEOTRANSFORM_APPROX_THRESHOLD=threshold_in_pixel configuration option (#6995) * Fix issue with GDALProxyRasterBand::FlushCache() not flushing everything required (#7040) * RawDataset::IRasterIO(): don't assume all bands are RawRasterBand == GDAL utilities == * gdal2tiles.py: fix GDAL 2.2 regression where tilemapresource.xml was no longer generated (#6966) * gdal_translate/DefaultCreateCopy(): do not destroy target file in case of failed copy wen using -co APPEND_SUBDATASET=YES (#7019) * gdal_translate: make -b mask[,xx] use the appropriate band data type (#7028) == GDAL drivers == GeoRaster driver: * add support for GCP (#6973) GPKG driver: * do not error out if extent in gpkg_contents is present but invalid (#6976) * fix opening subdatasets with absolute filenames on Windows (https://issues.qgis.org/issues/16997) * fix possible assertion / corruption when creating a raster GeoPackage (#7022) GSAG driver: * fix reading issue that could cause a spurious 0 value to be read and shift all following values (#6992) GTiff driver: * fix reading subsampled JPEG-in-TIFF when block height = 8 (#6988) * when reading a COMPD_CS (and GTIFF_REPORT_COMPD_CS=YES), set the name from the GTCitationGeoKey (#7011) GTX driver: * do not emit error when opening with GDAL_PAM_ENABLED=NO (#6996) HF2 driver: * fix reading tiles that are 1-pixel wide (2.1 regression, #6949) IDRISI driver: * Fix memleak in IDRISI Open() ISIS3 driver: * make sure that -co USE_SRC_HISTORY=NO -co ADD_GDAL_HISTORY=NO results in remove of History section (#6968) * fix logic to initialize underlying GeoTIFF file in IWriteBlock(), and implement Fill() (#7040) JPEG2000 driver: * Fix build failure in C++03 mode on Jasper inclusion in RHEL 6 (#6951) * Fix build failure with recent Jasper that no longer define uchar JP2OpenJPEG driver: * Add support for building against OpenJPEG 2.2 (#7002) * fix performance issues with small images with very small tile size, such as some Sentinel2 quicklooks (#7012) * properly use the opj_set_decode_area() API (#7018) netCDF driver: * avoid vector vs raster variable confusion that prevents reading Sentinel3 datasets, and avoid invalid geolocation array to be reported (#6974) PDF driver: * add support for Poppler 0.58 (#7033) PDS driver: * fix parsing of labels with nested list constructs (2.2 regression, #6970) SRTMHGT driver: * recognizes the .hgt.gz extension (#7016) VRT driver: * avoid stack buffer read overflow with complex data type and scale = 0. (oss-fuzz#2468) * fix uninitialized buffer in areas without sources when using non pixel packed spacing (#6965) * Warped VRT: correctly take into account cutline for implicit overviews; also avoid serializing a duplicate CUTLINE warping options in warped .vrt (#6954) * Warped VRT: fix implicit overview when output geotransform is not the same as the transformer dst geotransform (#6972) * fix IGetDataCoverageStatus() in the case of non-simple sources, which unbreaks gdalenhance -equalize (#6987) == OGR core == * OGRCompondCurve::addCurveDirectly(): try reversing non contiguous curves (for GML/NAS) * OGR API SPY: fix the way we map dataset handles to variable name, to avoid invalid reuses of variable still active * OGRParseDate(): avoid false-positive time detection, in particular for GeoJSON (#7014) * OGRCurve::get_isClosed(): do not take into account M component (#7017) * OGRLineString::setPoint() and addPoint() with a OGRPoint* argument. properly takes into account ZM, Z and M dimensions (#7017) == OGRSpatialReference == * Fix OGRSpatialReference::IsSame() to return FALSE when comparing EPSG:3857 (web_mercator) and EPSG:3395 (WGS84 Mercator) (#7029) * importFromProj4(): implement import of Hotine Oblique Mercator Two Points Natural Origin, and fix OGRSpatialReference::Validate() for that formulation (#7042) == OGR utilities == * ogr2ogr: fix small memory leak when using -limit switch * ogr2ogr: make -f GMT work again (#6993) == OGR drivers == AVCBin driver: * fix 2.1 regression regarding attributes fetching (#6950) DXF driver: * fix reading files where INSERT is the last entity and DXF_MERGE_BLOCK_GEOMETRIES is false (#7006) * avoid segfault when creating a DXF file with only the 'blocks' layer (#6998) * fix potential null pointer dereference in PrepareLineStyle (#7038) GeoJSON driver: * fix 2.2 regression regarding lack of setting the FeatureCollection CRS on feature geometries (fixes https://github.com/r-spatial/sf/issues/449#issuecomment-319369945) * prevent infinite recursion when reading geocouch spatiallist with properties.properties.properties (#7036) GML driver: * fix field type detection logic to avoid a field with xsi:nil=true to be identified as a string (#7027) * JPGIS FGD v4: fix logic so that coordinate order reported is lon/lat (https://github.com/OSGeo/gdal/pull/241) GMLAS driver: * fix memleak in swe:DataRecord handling * avoid false-positive warning about attribute found in document but ignored according to configuration, when the attribute is actually not present in the document but has a default value (#6956) * (quick-and-not-prefect) fix to avoid a <xs:any> to prevent all other fields from being set (#6957) * get the srsName when set on the srsName of the gml:pos of a gml:Point (#6962) * CityGML related fixes: better take into account .xsd whose namespace prefix is not in the document, fix discovery of substitution elements, handle gYear data type (#6975) GPKG driver: * fix feature count after SetFeature() with sqlite < 3.7.8 (#6953) * do not take into account gpkg_contents for the extent of vector layers when it is present but invalid (#6976) * fix logic to detect FID on SQL result layer when several FID fields are selected (#7026) KML / LIBKML drivers: * read documents with an explicit kml prefix (#6981) MSSQLSpatial driver: * fix issues on x64 (#6930) * properly format DELETE statement with schema (#7039) OCI driver: * check view with unquoted table name (#5552) OpenFileGDB driver: * properly read GeneralPolygon with M component whose all values are set to NaN (#7017) OSM driver: * increase string buffer for osm XML files (#6964) Shapefile driver: * use VSIStatL() in Create() to properly work with /vsimem/ directories (#6991) * fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (#7031) SQLite/Spatialite driver: * escape integer primary key column name on table creation (#7007) == C# bindings == * Implement RasterIO extensions (#6915) == Python bindings == * fix reference count issue on gdal.VSIStatL() when it returns None, that can cause Python crashes if None refcount drops to zero * add C-API to create driver instances for use in SWIG (https://trac.osgeo.org/osgeo4w/ticket/466) == Security oriented fixes == Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them. (oss-fuzz#XXXX is a shortcut to https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=XXXX) * NAS: avoid assertion / null ptr deref and possibly other failures on corrupted files. (oss-fuzz#2366, oss-fuzz#2580) * NTF: avoid null ptr deref. Related to https://oss-fuzz.com/v2/testcase-detail/6527758899347456 * VSIMemHandle::Read(): avoid unwanted unsigned int overflow that cause a later heap buffer overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5227283000328192. * MBTiles: fix use after free. (oss-fuzz#2388) * netCDF: fix stack buffer overflow when building vector layers from netCDF variables if they don't have the expected number of dimensions. (oss-fuzz#2400) * DXF: prevent infinite loop / excessive memory allocation on truncated file. (#oss-fuzz#2403) * DXF: avoid excessive memory allocation. (oss-fuzz#2444) * XYZ: fix write heap-buffer-overflow (oss-fuzz#2426) * GTiff: make IGetDataCoverageStatus() properly set the current TIFF directory, and only implement it for 'normal' bands. To avoid heap buffer overflows. (oss-fuzz#2438) * GTiff: avoid null pointer derefrence when requested implicit overviews on corrupted JPEG-compressed file. (oss-fuzz#2441) * GTiff: avoid potential issue on corrupted files (oss-fuzz#2481) * GTiff: don't override member nBlocksPerRow member of GTiffJPEGOverviewBand with unrelated value, in case of single-strip case. (oss-fuzz#3020) * PCIDSK: for band interleave channel, correctly use pixel_offset in case it is different from pixel_size. (oss-fuzz#2440) * MITAB: fix skipping of blank lines when reading MIF geometries, and avoid potential very loop loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4516730841858048 * CPLKeywordParser: avoid potential heap buffer overflow read. (oss-fuzz#2706) * CPLKeywordParser: avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/5733173726019584 * morphFromESRI(): fixes heap-after-free uses. (oss-fuzz#2864) * OGR_VRT: fix null pointer dereference on GetExtent() on an invalid layer. (oss-fuzz#3017) * OGR_VRT: avoid crash on cyclic VRT. (oss-fuzz#3123) * OSM: avoid potential write heap buffer overflow on corrupted PBF files. (oss-fuzz#3022) * FAST: fix potential read heap buffer overflow. (oss-fuzz#3025) * CPG: avoid null pointer dereference on failed open. (oss-fuzz#3136) = GDAL/OGR 2.2.1 Release Notes = The 2.2.1 release is a bug fix release. == Build == * fix compilation without BIGTIFF_SUPPORT (#6890) * configure: detect if std::isnan() is available. Helps compilation on some MacOSX setups, combined together --without-cpp11. Refs https://github.com/macports/macports-ports/pull/480 * fix compilation against ancient Linux kernel headers (#6909) * fix detection of 64bit file API with clang 5 (#6912) * configure: use .exe extension when building with mingw64* toolchains (fixes #6919) * mongoDB: compilation fix on Windows == Port == * /vsicurl/: fix occasional inappropriate failures in Read() with some combinations of initial offset, file size and read request size (#6901) * Add a VSICurlClearCache() function (bound to SWIG as gdal.VSICurlClearCache()) to be able to clear /vsicurl/ related caches (#6937) == Algorithms == * GDALRasterize(): avoid hang in some cases with all_touched option (#5580) * gdal_rasterize: fix segfault when rasterizing onto a raster with RPC (#6922) == GDAL utilities == * ogr_merge.py: fix '-single -o out.shp in.shp' case (#6888) == GDAL drivers == AIGRID driver: * fix handling on raw 32-bit AIG blocks ENVISAT driver: * fix 2.2 regression in initialization of members of MerisL2FlagBand. (#6929) GeoRaster driver: * Fix memory allocation failure (#6884) * add support for JP2-F in BLOB compression (corrections on geo-reference) (#6861) GPKG driver: * avoid corruption of gpkg_tile_matrix when building overviews, down to a level where they are smaller than the tile size (#6932) GTIFF driver: * Internal libtiff: fix libtiff 4.0.8 regression regarding creating of single strip uncompressed TIFF files (#6924) netCDF driver: * add support for radian and microradian units for geostationnary projection (https://github.com/OSGeo/gdal/pull/220) NWT_GRC driver: * Fix handling of alpha values in GRC color table (#6905) * Handle case of 0-len GRC class names (#6907) VRT driver: * speed-up SerializeToXML() in case of big number of bands XYZ driver: * fix 2.2 regression where the driver hangs on some dataset with missing samples (#6934) == OGR utilities == * ogr2ogr/GDALVectorTranslate(): fix crash when using -f PDF -a_srs (#6920) == OGR drivers == GeoJSON driver: * ESRIJson: avoid endless looping on servers that don't support resultOffset (#6895) * ESRIJson: use 'latestWkid' in priority over 'wkid' when reading 'spatialReference' (https://github.com/OSGeo/gdal/pull/218) * GeoJSON writer: accept writing ZM or M geometry by dropping the M component (#6935) GPKG driver: * make driver robust to difference of cases between table_name in gpkg_contents/gpkg_geometry_columns and name in sqlite_master (#6916) MITAB driver: * recognize Reseau_National_Belge_1972 / EPSG:31370 on writing (#6903) MySQL driver: * fix compilation issue with Arch Linux and mariadb 10.1.23 (fixes #6899) PG driver: * do not be confused by a 'geometry' table in a non-PostGIS enabled database (#6896) PLScenes: * remove support for V0. Deprecate V1 API. Only Data V1 is supported ( #6933) == Perl bindings == * Backport the fix to #6142 Install man page according to GDALmake.opt if INSTALL_BASE is set. * always return something from non-void functions (#6898) == Python bindings == * Accept callback = 0 since SWIG generates it as the default argument of BandRasterIONumPy(). Fixes https://github.com/OSGeo/gdal/pull/219 * Fix 2.2 regression preventing use of callback function in Band.ComputeStatistics() (#6927) == Security oriented fixes == Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them (some have been found locally, so no related ticket) * Fix CPLErrorSetState to ensure it does not write beyond DEFAULT_LAST_ERR_MSG_SIZE and correctly null-terminates last message. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1529. * Open() and Stat() methods of a number of virtual file systems: check that the filename starts with the full prefix. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1543. * VRT pixel functions: fix crash with 'complex' when source count is < 2. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1549 * OpenAIR: fix potential out-of-buffer read if we need to ingest 30000 bytes * Several fixes in importFromWkb() and importFromWkt() * GDALDataset and GDALRasterBand::ReportError(): fix crash if dataset name has a % character * NASAKeywordHandler::SkipWhite(): fix out of bounds read * MITAB: ParseTABFileFields(): fix out of bounds read. * MITAB: ParseMIFHeader(): fix memory leak and out-of-bounds read on corrupted file * MITAB: ParseMIFHeader(): fix memory leaks on corrupted files * MITAB: avoid potentially veryyyy long loop when stroking arcs. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1644 * MITAB: avoid heap-buffer-overflow in MITABCoordSys2TABProjInfo(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1653 * OSARDataset::Open(): fix crash if pOpenInfo->fpL == NULL. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1565 * OGRESRIJSONReadPolygon(): fix crash in error code path * DXF: prevent null ptr deref and out-of-bounds read on corrupted file * DXF: TranslateSPLINE(): sanitize integer values read to avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1843 * KML::unregisterLayerIfMatchingThisNode(): use memmove() instead of memcpy(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1580 * KML: fix crash on weird structure with recursively empty folders. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1591 * KML: fix null ptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1634 * OGRCurveCollection::importBodyFromWkb(): fix potential crash if the curve in the collection has not the same dimension has the collection dimension. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1584 * OGRCompoundCurve::importFromWkb(): avoid potential stack overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5192348843638784 * TIGER: fix potential stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1586 and 2020 * TIGER: avoid potential infinite looping. Fixes https://oss-fuzz.com/v2/testcase-detail/4724815883665408 * VFK: avoid out-of-bounds read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1596 and 2074 * CPLHexToBinary(): avoid reading outside of hex2char array on on-ASCII input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1606 * OGR PDS: avoid int32 overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=160 * GeoRSS: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1611. * VSIArchiveFilesystemHandler::SplitFilename(): improve performance. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1597 * OGRGeometryFactory::organizePolygons(): fix crash on empty polygons. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1620 * JML: fix null pointer dereference on invalid files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1622 * Shape: prevent null ptr deref on truncated MultiPointM geometry. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1638 * /vsisubfile/: avoid Tell() to return negative values. And make VSIIngestFile() more robust to unsigned overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1642 * GTM: avoid useless recursive opening of files when provided with a gzip-compressed input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1650 * GTiff: fix heap-buffer-overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1677 * GTiff: avoid heap-buffer-overfow on corrupted State Plane citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2050 * GTiff: avoid potential stack buffer overflow on corrupted Imagine citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2105 * GTiff: prevent heap overflow and fix reading of multi-band band-interleaved 16/24-bit float datasets. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2264 * GTiff: fix potential infinite loop when parsing some 24-bit float denormalized numbers. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2315 * Internal libjson-c: fix stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1682 * ILI1/ILI2: fix null pointer dereference when opening filename ','. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1689 * ILI1: fix various crashes on corrupted files (including, but not limited to, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1760, 1784, 1926) * ILI2: use proper cast operator. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1813 * ILI2: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1993 * ILI2: fix crash due to unhandled exception. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2191 * morphFromESRI(): fix heap-use-after-free issue. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1690 * morphFromESRI(): prevent potential null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1783 and 1867 * SEGUKOOA: fix inversion of leap year that caused index-out-of-bound reading on day 366 of leap years (2.2 regression). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1698 * CPLParseXMLString(): make it error out on invalid XML file even under CPLTurnErrorIntoWarning() mode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1703. * GML / NAS: fix memory leak in error code path, and potential heap-buffer-read-overflow * NTF: fix various issues: heap & stack buffer-overflow, null ptr derefs, memory leaks. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1819 , 1820 , 1823, 1833, 1854, 1862, 1910, 1931, 1961, 1990, 1995, 1996, 2003, 2033, 2052, 2077, 2084, 2103, 2130, 2135, 2146, 2166, 2185, 2187, https://oss-fuzz.com/v2/testcase-detail/4696417694121984 * OGRCreateFromMultiPatch(): avoid assertion on NaN coordinates. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1930 * GXF: validate nGType to avoid later out-of-bound read in GXFReadRawScanlineFrom(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1975 * GXF: fix int overflow and avoid excessive memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2207 * DGN: prevent heap-buffer-overflow read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1988 * COSAR: fix leak of file descriptor. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2019 * ISO8211: prevent stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2022 * WEBP: prevent int32 overflow and too large memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2059 * IRIS: fix heap-buffer-overflow in some cases of nDataTypeCode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2069 * E00GRID: avoid heap and stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2090 , 2182, 2237, 2327 * VICAR: fix null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2116 * VICAR: avoid use-after-free and heap-buffer-overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/4825577427042304 * VICAR: fix potential endless loop on broken files. Fixes https://oss-fuzz.com/v2/testcase-detail/6261508172414976 * REC: fix nullptr deref * REC: fix potential stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2165 * GDALGetJPEG2000Structure() / DumpGeoTIFFBox(): fix memory leak. * DumpGeoTIFFBox(): reject GeoJP2 boxes with a TIFF with band_count > 1 * DumpJPK2CodeStream(): avoid potentially very long loop * GDALGetJPEG2000Structure(): avoid bad performance on corrupted JP2 boxes. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2214 * GPKG: fix potential heap-buffer overflow in GPkgHeaderFromWKB(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2150 * GPKG: fix potential null ptr deref. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2240 * GPKG: avoid potential division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2253 * SEGUKOOA: prevent read beyond end of buffer. (#6921) * SRP: avoid potential stack buffer overflow and excessive memory allocation/processing time * CPLUnixTimeToYMDHMS(): avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4949567697059840 * Selafin: fix double frees. Fixes https://oss-fuzz.com/v2/testcase-detail/6429713822121984 * CEOS: fix heap buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2259 * CEOS: fix memleak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2279 * FAST: avoid null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2290 * netCDF: avoid stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2302 GDAL/OGR 2.2.0 Release Notes = == In a nutshell... == * New GDAL/raster drivers: - DERIVED driver: read-support. Expose subdatasets in a a new metadata domain, called DERIVED_SUBDATASETS - JP2Lura driver: read/create support for JPEG-2000 images using Luratech JP2 Library - PRF: add read-only support for PHOTOMOD PRF file format driver (github #173) - RRASTER driver: read-support .grd/.gri files handled by the R 'raster' package (#6249) * New OGR/vector drivers: - CAD driver: read support for DWG R2000 files (GSoC 2016 project) - DGNv8 driver: read-write support for DGN 8.0 format (using Teigha ODA libraries) - GMLAS driver: read-write support. XML/GML driver driven by Application Schemas. * New utility script: ogrmerge.py to merge several vector datasets into a single one * New /vsigs/ and /vsigs_streaming/ virtual file systems to read Google Cloud Storage non-public files * Significantly improved drivers: - NWT_GRD: write support (#6533) - FileGDB/OpenFileGDB: add support to read curve geometries (#5890) - VRT derived band: add the capability to define pixel functions in Python - Add read support for RasterLite2 coverages in SQLite driver - GPKG: implement tiled gridded elevation data extension - ISIS3: add write support and improve read support * RFC 63: Add GDALRasterBand::GetDataCoverageStatus() and implement it in GTiff and VRT drivers https://trac.osgeo.org/gdal/wiki/rfc63_sparse_datasets_improvements * RFC 64: Triangle, Polyhedral surface and TIN https://trac.osgeo.org/gdal/wiki/rfc64_triangle_polyhedralsurface_tin ==> this RFC introduces potential backward incompatible behaviour. Consult MIGRATION_GUIDE.txt * RFC 66: OGR random layer read/write capabilities https://trac.osgeo.org/gdal/wiki/rfc66_randomlayerreadwrite * RFC 67: add null field state for OGR features, in addition to unset fields https://trac.osgeo.org/gdal/wiki/rfc67_nullfieldvalues ==> this RFC introduces potential backward incompatible behaviour. Consult MIGRATION_GUIDE.txt * Upgrade to EPSG database v9.0 (#6772) * Python bindings: Global Interpreter Lock (GIL) released before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open()) * Continued major efforts on sanitization of code base * Remove bridge and vb6 bindings (#6640) * GNM built by default == Installed files == * Removed: data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv * Added plscenesconf.json, gmlasconf.xsd == Backward compatibility issues == See MIGRATION_GUIDE.TXT == GDAL/OGR 2.2.0 - General Changes == Build(Unix): * improve detection of packaged libfyba (SOSI) --with-sosi, as in Ubuntu 16.04 (#6488) * Sort files in static library to make the build reproducible (#6520) * fix libqhull include path when it is /usr/local/include/libqhull (#6522) * FileGDB: compilation fix on Linux in C++11 mode * configure: make pdfium detection not fail if there are just warnings. And make configure fail if --with-pdfium was required but failed (#6653) * Make ./configure --with-xerces fail if not found * Don't install script documentation in INST_BIN (github #157) * configure: auto-detect webp without requiring explicit --with-webp * configure: use pkg-config for HDF5 detection so that works out of the box on recent Ubuntu * auto-detect JDK 8 on Ubuntu * MDB: allow libjvm.so to be dlopen'd with --with-jvm-lib=dlopen (Unix only, github #177) * configure: delete temporary directories on the mac * configure: make sure --with-macosx-framework is correctly defined * configure: error out if --with-ld-shared is specified (#6769) * configure: remove bashism. * configure: fix --without-mrf (#6811) * configure: take into account CXXFLAGS and LDFLAGS in a few more cases (cryptopp, podofo, libdap) * Vagrant: all lxc and Hyper-V provider support; use vagrant-cachier for package caching * configure: update DWG support to work with Teigha libraries * Internal libgeotiff: hide symbols in --with-hide-internal-symbols mode * Shape: do not export Shapelib symbols for builds --with-hide-internal-symbols (#6860) Build(Windows): * Try to avoid confusion between libqhull io.h and mingw own io.h (#6590) * update script to generate most recent Visual C++ project files (#6635) * fix broken and missing dependencies in makefile.vc * add a way to use an external zlib (github #171) * Rename makegdal_gen.bat to generate_vcxproj.bat * generate_vcxproj.bat: Set correct value of PlatformToolset property based on specified Visual C++ version. Add NMAKE command line parameters: MSVC_VER based on specified Visual C++ version, DEBUG=1 WITH_PDB=1 to Debug build configuration. * generate_vcxproj.bat: generate project for autotest/cpp (Ticket #6815) * Add WIN64=1 to NMAKE command line options. * Add HDF4_INCLUDE option (#6805) * Add MSVC compiler option /MP to build with parallel processes. * Add ZLIB_LIB missing from EXTERNAL_LIBS Build(all): * make Xerces 3.1 the minimal version * drop support for PostgreSQL client library older than 7.4, or non security maintained releases older than 8.1.4, 8.0.8, 7.4.13, 7.3.15 == GDAL 2.2.0 - Overview of Changes == Port: * Export VSICreateCachedFile() as CPL_DLL so as to enable building JP2KAK as a plugin * Added possibility to find GDAL_DATA path using INST_DATA definition without execution GDALAllRegister if GDAL_DATA placed in version named directory on Linux (#6543) * Unix filesystem: make error message about failed open to report the filename (#6545) * File finder: Remove hardcoded find location (/usr/local/share/gdal) (#6543) * Win32 filesystem handler: make Truncate() turn on sparse files when doing file extension * Add VSIFGetRangeStatusL() and VSISupportsSparseFiles() * GDAL_NO_HARDCODED_FIND compilation option (#6543,#6531) to block file open calls (for sandboxed systems) * Add VSIMallocAligned(), VSIMallocAlignedAuto() and VSIFreeAligned() APIs * /vsizip / /vsitar: support alternate syntax /vsitar/{/path/to/the/archive}/path/inside/the/tar/file so as not to be dependent on file extension and enable chaining * Optimize opening of /vsitar/my.tar.gz/my_single_file * /vsizip/ : support creating non-ASCII filenames inside a ZIP (#6631) * VSI Unix file system: fix behaviour in a+ mode that made MRF caching not work on Mac and other BSD systems * Fix deadlock at CPLWorkerThreadPool destruction (#6646) * Windows: honour GDAL_FILENAME_IS_UTF8 setting to call LoadLibraryW() (#6650) * CPLFormFilename(): always use / path separator for /vsimem, even on Windows * /vsimem/: add trick to limit the file size, so as to be able to test how drivers handle write errors * /vsimem/: fix potential crash when closing -different- handles pointing to the same file from different threads (#6683) * CPLHTTPFetch(): add MAX_FILE_SIZE option * CPLHTTPFetch(): add a CAINFO option to set the path to the CA bundle file. As a fallback also honour the CURL_CA_BUNDLE and SSL_CERT_FILE environment variables used by the curl binary, which makes this setting also available for /vsicurl/, /vsicurl_streaming/, /vsis3/ and /vsis3_streaming/ file systems (#6732) * CPLHTTPFetch(): don't disable peer certificate verification when doing https (#6734) * CPLHTTPFetch(): cleanly deal with multiple headers passed with HEADERS and separated with newlines * CPLHTTPFetch(): add a CONNECTTIMEOUT option * CPLHTTPFetch(): add a GDAL_HTTP_HEADER_FILE / HEADER_FILE option. * CPLHTTPSetOptions(): make redirection of POST requests to still be POST requests after redirection (#6849) * /vsicurl/: take CPL_VSIL_CURL_ALLOWED_EXTENSIONS into account even if GDAL_DISABLE_READDIR_ON_OPEN is defined (#6681) * /vsicurl/: get modification time if available from GET or HEAD results * /vsis3/: add a AWS_REQUEST_PAYER=requester configuration option (github #186) * CPLParseXMLString(): do not reset error state * Windows: fix GetFreeDiskSpace() * Fix GetDiskFreeSpace() on 32bit Linux to avoid 32bit overflow when free disk space is above 4 GB (#6750) * Fix CPLPrintUIntBig() to really print a unsigned value and not a signed one * Add CPL_HAS_GINT64, GINT64_MIN/MAX, GUINT64_MAX macros (#6747) * Add CPLGetConfigOptions(), CPLSetConfigOptions(), CPLGetThreadLocalConfigOptions() and CPLSetThreadLocalConfigOptions() to help improving compatibility between osgeo.gdal python bindings and rasterio (related to https://github.com/mapbox/rasterio/pull/969) * MiniXML serializer: fix potential buffer overflow. Core: * Proxy dataset: add consistency checks in (the unlikely) case the proxy and underlying dataset/bands would not share compatible characteristics * GDALPamDataset::TryLoadXML(): do not reset error context when parsing .aux.xml * PAM/VRT: only take into account <Entry> elements when deserializing a <ColorTable> * GDALCopyWords(): add fast copy path when src data type == dst data type == Int16 or UInt16 * GetVirtualMemAuto(): allow USE_DEFAULT_IMPLEMENTATION=NO to prevent the default implementation from being used * Nodata comparison: fix test when nodata is FLT_MIN or DBL_MIN (#6578) * GetHistogram() / ComputeRasterMinMax() / ComputeStatistics(): better deal with precision issues of nodata comparison on Float32 data type * Fast implementation of GDALRasterBand::ComputeStatistics() for GDT_Byte and GDT_UInt16 (including use of SSE2/AVX2) * Driver manage: If INST_DATA is not requested, do not check the GDAL_DATA variable. * Make sure that GDALSetCacheMax() initialize the raster block mutex (#6611) * External overview: fix incorrect overview building when generating from greater overview factors to lower ones, in compressed and single-band case (#6617) * Speed-up SSE2 implementation of GDALCopy4Words from float to byte/uint16/int16 * Add SSE2 and SSSE3 implementations of GDALCopyWords from Byte with 2,3 or 4 byte stride to packed byte * GDALCopyWords(): SSE2-accelerated Byte->Int32 and Byte->Float32 packed conversions * Fix GDALRasterBand::IRasterIO() on a VRT dataset that has resampled sources, on requests such as nXSize == nBufXSize but nXSize != dfXSize * GDALRasterBand::IRasterIO(): add small epsilon to floating-point srcX and srcY to avoid some numeric precision issues when rounding. * Add GDALRasterBand::GetActualBlockSize() (#1233) * Fix potential deadlock in multithreaded writing scenarios (#6661) * Fix thread-unsafe behaviour when using GetLockedBlock()/MarkDirty()/DropLock() lower level interfaces (#6665) * Fix multi-threading issues in read/write scenarios (#6684) * Resampled RasterIO(): so as to get consistent results, use band datatype as intermediate type if it is different from the buffer type * Add GDALIdentifyDriverEx() function (github #152) * GDALOpenInfo: add a papszAllowedDrivers member and fill it in GDALOpenEx() * GDALDefaultOverviews::BuildOverviews(): improve progress report * Average and mode overview/rasterio resampling: correct source pixel computation due to numerical precision issues when downsampling by an integral factor, and also in oversampling use cases (github #156) * Overview building: add experimental GDAL_OVR_PROPAGATE_NODATA config option that can be set to YES so that a nodata value in source samples will cause the target pixel to be zeroed. Only implemented for AVERAGE resampling right now * GDALValidateOptions(): fix check of min/max values * GMLJP2 v2: update to 2.0.1 corrigendum and add capability to set gml:RectifiedGrid/gmlcov:rangeType content. Set SRSNAME_FORMAT=OGC_URL by default when converting to GML. Add gml:boundedBy in gmljp2:GMLJP2RectifiedGridCoverage * GMLJP2 v2: ensure KML root node id unicity when converting annotations on the fly. When generating GML features, make sure that PREFIX and TARGET_NAMESPACE are unique when specifying several documents. Algorithms: * RPC transformer: speed-up DEM extraction by requesting and caching a larger buffer, instead of doing many queries of just a few pixels that can be costly with VRT for example * GDALDeserializeRPCTransformer(): for consistency, use the same default value as in GDALCreateRPCTransformer() if <PixErrThreshold> is missing (so use 0.1 instead of 0.25 as before) * TPS solver: when Armadillo fails sometimes, fallback to old method * GDALCreateGenImgProjTransformer2(): add SRC_APPROX_ERROR_IN_SRS_UNIT, SRC_APPROX_ERROR_IN_PIXEL, DST_APPROX_ERROR_IN_SRS_UNIT, DST_APPROX_ERROR_IN_PIXEL, REPROJECTION_APPROX_ERROR_IN_SRC_SRS_UNIT and REPROJECTION_APPROX_ERROR_IN_DST_SRS_UNIT transformer options, so as to be able to have approximate sub-transformers * Fix GDAL_CG_Create() to call GDALContourGenerator::Init() (#6491) * GDALContourGenerate(): handle the case where the nodata value is NaN (#6519) * GDALGridCreate(): fix hang in multi-threaded case when pfnProgress is NULL or GDALDummyProgress (#6552) * GDAL contour: fix incorrect oriented contour lines in some rare cases (#6563) * Warper: multiple performance improvements for cubic interpolation and uint16 data type * Warper: add SRC_ALPHA_MAX and DST_ALPHA_MAX warp options to control the maximum value of the alpha channel. Set now to 65535 for UInt16 (and 32767 for Int16), or to 2^NBITS-1. 255 used for other cases as before * Warper: avoid undefined behaviour when doing floating point to int conversion, that may trigger exception with some compilers (LLVM 8) (#6753) * OpenCL warper: update cubicConvolution to use same formula as CPU case (#6664) * OpenCL warper: fix compliance to the spec. Fix issues with NVidia opencl (#6624, #6669) * OpenCL warper: use GPU based over CPU based implementation when possible, use non-Intel OpenCL implementation when possible. Add BLACKLISTED_OPENCL_VENDOR and PREFERRED_OPENCL_VENDOR to customize choice of implementation Utilities: * gdalinfo -json: fix order of points in wgs84Extent.coordinates (github #166) * gdalwarp: do not densify cutlines by default when CUTLINE_BLEND_DIST is used (#6507) * gdalwarp: when -to RPC_DEM is specified, make -et default to 0 as documented (#6608) * gdalwarp: improve detection of source alpha band and auto-setting of target alpha band. Automatically set PHOTOMETRIC=RGB on target GeoTIFF when input colors are RGB * gdalwarp: add a -nosrcalpha option to wrap the alpha band as a regular band and not as the alpha band * gdalwarp: avoid cutline densification when no transform at all is involved (related to #6648) * gdalwarp: fix failure with cutline on a layer of touching polygons (#6694) * gdalwarp: allow to set UNIFIED_SRC_NODATA=NO to override the default that set it to YES * gdalwarp: fix -to SRC_METHOD=NO_GEOTRANSFORM -to DST_METHOD=NO_GEOTRANSFORM mode (#6721) * gdalwarp: add support for shifting the values of input DEM when source and/or target SRS references a proj.4 vertical datum shift grid * gdalwarp: fix crash when -multi and -to RPC_DEM are used together (#6869) * gdal_translate: when using -projwin with default nearest neighbour resampling, align on integer source pixels (#6610) * gdal_translate & gdalwarp: lower the default value of GDAL_MAX_DATASET_POOL_SIZE to 100 on MacOSX (#6604) * gdal_translate: avoid useless directory scanning on GeoTIFF files * gdal_translate: make "-a_nodata inf -ot Float32" work without warning * gdal_translate: set nodata value on vrtsource on scale / unscale / expand cases (github #199) * GDALTranslate(): make it possible to create a anonymous target VRT from a (anonymous) memory source * gdaldem: speed-up computations for src type = Byte/Int16/UInt16 and particularly for hillshade * gdaldem hillshade: add a -multidirectional option * GDALDEMProcessing() API: fix -alt support (#6847) * gdal_polygonize.py: explicitly set output layer geometry type to be polygon (#6530) * gdal_polygonize.py: add support for -b mask[,band_number] option to polygonize a mask band * gdal_rasterize: make sure -3d, -burn and -a are exclusive * gdal_rasterize: fix segfaults when rasterizing into an ungeoreferenced raster, or when doing 'gdal_rasterize my.shp my.tif' with a non existing my.tif (#6738) * gdal_rasterize: fix crash when rasterizing empty polygon (#6844) * gdal_grid: add a smoothing parameter to invdistnn algorithm (github #196) * gdal_retile.py: add a -overlap switch * gdal2tiles.py: do not crash on empty tiles generation (#6057) * gdal2tiles.py: handle requested tile at too low zoom to get any data (#6795) * gdal2tiles: fix handling of UTF-8 filenames (#6794) * gdal2xyz: use %d formatting for Int32/UInt32 data types (#6644) * gdal_edit.py: add -scale and -offset switches (#6833) * gdaltindex: emit warning in -src_srs_format WKT when WKT is too large * gdalbuildvrt: add a -oo switch to specify dataset open options Python samples: * add validate_cloud_optimized_geotiff.py * add validate_gpkg.py Multi-driver: * Add GEOREF_SOURCES open option / GDAL_GEOREF_SOURCES config. option to all JPEG2000 drivers and GTiff to control which sources of georeferencing can be used and their respective priority AIGRID driver: * fix 2.1.0 regression when reading statistics (.sta) file with only 3 values, and fix <2.1 behaviour to read them in LSB order (#6633) AAIGRID driver: * auto-detect Float64 when the nodata value is not representable in the Float32 range ADRG driver: * handle north and south polar zones (ZNA 9 and 18) (#6783) ASRP driver: * fix georeferencing of polar arc zone images (#6560) BPG driver: * declare GDALRegister_BPG as C exported for building as a plugin (#6693) DIMAP driver: * DIMAP: for DIMAP 2, read RPC from RPC_xxxxx.XML file (#6539) * DIMAP/Pleiades metadata reader: take into tiling to properly shift RPC (#6293) * add support for tiled DIMAP 2 datasets (#6293) DODS driver: * fix crash on URL that are not DODS servers (#6718) DTED driver: * correctly create files at latitudes -80, -75, -70 and -50 (#6859) ECW driver: * Add option ECW_ALWAYS_UPWARD=TRUE/FALSE to work around issues with "Downward" oriented images (#6516). ENVI driver: * on closing, pad image file with trailing nul bytes if needed (#6662) * add read/write support for rotated geotransform (#1778) GeoRaster driver: * fix report of rotation (#6593) * support for JP2-F compression (#6861) * support direct loading of JPEG-F when blocking=no (#6861) * default blocking increased from 256x256 to 512x512 (#6861) GPKG driver: * implement tiled gridded elevation data extension * add VERSION creation option * check if transaction COMMIT is successful (#6667) * fix crash on overview building on big overview factor (#6668) * fix crash when opening an empty raster with USE_TILE_EXTENT=YES * fix gpkg_zoom_other registration GTiff driver: * support SPARSE_OK=YES in CreateCopy() mode (and in update mode with the SPARSE_OK=YES open option), by actively detecting blocks filled with 0/nodata about to be written * When writing missing blocks (i.e. non SPARSE case), use the nodata value when defined. Otherwise fallback to 0 as before. * in FillEmptyTiles() (i.e. in the TIFF non-sparse mode), avoid writing zeroes to file so as to speed up file creation when filesystem supports ... sparse files * add write support for half-precision floating point (Float32 with NBITS=16) * handle storing (and reading) band color interpretation in GDAL internal metadata when it doesn't match the capabilities of the TIFF format, such as B,G,R ordering (#6651) * Fix RasterIO() reported when downsampling a RGBA JPEG compressed TIFF file (#6943) * Switch search order in GTIFGetOGISDefn() - Look for gdal_datum.csv before datum.csv (#6531) * optimize IWriteBlock() to avoid reloading tile/strip from disk in multiband contig/pixel-interleave layouts when all blocks are dirty * fix race between empty block filling logic and background compression threads when using Create() interface and NUM_THREADS creation option (#6582) * use VSIFTruncateL() to do file extension * optimize reading and writing of 1-bit rasters * fix detection of blocks larger than 2GB on opening on 32-bit builds * fix saving and loading band description (#6592) * avoid reading external metadata file that could be related to the target filename when using Create() or CreateCopy() (#6594) * do not generate erroneous ExtraSamples tag when translating from a RGB UInt16, without explicit PHOTOMETRIC=RGB (#6647) * do not set a PCSCitationGeoKey = 'LUnits = ...' as the PROJCS citation on reading * fix creating an image with the Create() interface with BLOCKYSIZE > image height (#6743) * fix so that GDAL_DISABLE_READDIR_ON_OPEN = NO / EMPTY_DIR is properly honoured and doesn't cause a useless directory listing * make setting GCPs when geotransform is already set work (with warning about unsetting the geotransform), and vice-versa) (#6751) * correctly detect error return of TIFFReadRGBATile() and TIFFReadRGBAStrip() * in the YCBCR RGBA interface case, only expose RGB bands, as the alpha is always 255 * don't check free disk space when outputting to /vsistdout/ (#6768) * make GetUnitType() use VERT_CS unit as a fallback (#6675) * in COPY_SRC_OVERVIEWS=YES mode, set nodata value on overview bands * read GCPs in ESRI <GeodataXform> .aux.xml * explicitly write YCbCrSubsampling tag, so as to avoid (latest version of) libtiff to try reading the first strip to guess it. Helps performance for cloud optimized geotiffs * map D_North_American_1927 datum citation name to OGC North_American_Datum_1927 so that datum is properly recognized (#6863) * Internal libtiff. Resync with CVS (post 4.0.7) * Internal libtiff: fix 1.11 regression that prevents from reading one-strip files that have no StripByteCounts tag (#6490) GRASS driver: * plugin configure: add support for GRASS 7.2 (#6785) * plugin makefile: do not clone datum tables and drivers (#2953) * use Rast_get_window/Rast_set_window for GRASS 7 (#6853) GRIB driver: * Add (minimalistic) support for template 4.15 needed to read Wide Area Forecast System (WAFS) products (#5768) * **Partial** resynchronization with degrib-2.0.3, mostly to get updated tables (related to #5768) * adds MRMS grib2 decoder table (http://www.nssl.noaa.gov/projects/mrms/operational/tables.php) (github #160) * enable PNG decoding on Unix (#5661, github #160) * remove explicitly JPEG2000 decompression through Jasper and use generic GDAL code so that other drivers can be triggered * fix a few crashes on malformed files GTX driver: * add a SHIFT_ORIGIN_IN_MINUS_180_PLUS_180 open option HDF4 driver: * Fixed erroneous type casting in HDF4Dataset::AnyTypeToDouble() that breaks reading georeferencing and other metadata HDF5 driver: * correct number of GCPs to avoid dummy trailing (0,0)->(0,0,0) and remove +180 offset applied to GCP longitude. Add instead a heuristics to determine if the product is crossing the antimeridian, and a HDF5_SHIFT_GCPX_BY_180 config option to be able to override the heuristics (#6666) HFA driver: * fix reading and writing of TOWGS84 parameters (github #132) * export overview type from HFA files to GDAL metadata as OVERVIEWS_ALGORITHM (github #135) * make .ige initialization use VSIFTruncateL() to be faster on Windows * add support for TMSO and HOM Variant A projections (#6615) * Add elevation units read from HFA files metadata (github #169) * set binning type properly according to layerType being thematic or not (#6854) Idrisi driver: * use geotransform of source dataset even if it doesn't have a SRS (#6727) * make Create() zero-initialize the .rst file (#6873) ILWIS driver: * avoid IniFile::Load() to set the bChanged flag, so as to avoid a rewrite of files when just opening datasets ISCE driver: * fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL 2.1.0 (#6556) * fix misbehaviour on big endian hosts * add support for reading and writing georeferencing (#6630, #6634) * make parsing of properties case insensitive (#6637) ISIS3 driver: * add write support * add mask band support on read * get label in json:ISIS3 metadata domain JPEGLS driver: JP2ECW driver: * fix crash when translating a Float64 raster (at least with SDK 3.3) JP2KAK driver: * add support for Kakadu v7.9. v7.8 should not be used. It has a bug fixed in v7.9 * catch exceptions in jp2_out.write_header() JP2OpenJPEG driver: * add a USE_TILE_AS_BLOCK=YES open option that can help with whole image conversion * prevent endless looping in openjpeg in case of short write * for single-line organized images, such as found in some GRIB2 JPEG2000 images, use a Wx1 block size to avoid huge performance issues (#6719) * ignore warnings related to empty tag-trees. JPIPKAK driver: * fix random crashes JPIP in multi-tread environment (#6809) KEA driver: * Add support for Get/SetLinearBinning (#6855) KMLSuperOverlay driver: * recognize simple document made of GroundOverlay (#6712) * Add FORMAT=AUTO option. Uses PNG for semi-transparent areas, else JPG. (#4745) LAN driver: * remove wrong byte-swapping for big-endian hosts MAP driver: * change logic to detect image file when its path is not absolute MBTiles driver: * on opening if detecting 3 bands, expose 4 bands since there might be transparent border tiles (#6836) * fix setting of minzoom when computing overviews out of order * do not open .mbtiles that contain vector tiles, which are not supported by the driver MEM driver: * disable R/W mutex for tiny performance increase in resampled RasterIO * add support for overviews * add support for mask bands MRF driver: * bug fix in PNG and JPEG codecs * Fixing a problem with setting NoData for MRFs generated with Create * fix plugin building (#6498) * rename CS variable so as to avoid build failure on Solaris 11 (#6559) * Allow MRF to write the data file directly to an S3 bucket. * Allow relative paths when MRF is open via the metadata string. * Add support for spacing (unused space) between tiles. Defaults to none. * Read a single LERC block as an MRF file. MSG driver: * fix incorrect georeference calculation for msg datasets (github #129) NetCDF driver: * add support for reading SRS from srid attribute when it exists and has content like urn:ogc:def:crs:EPSG::XXXX (#6613) * fix crash on datasets with 1D variable with 0 record (#6645) * fix erroneous detection of a non-longitude X axis as a longitude axis that caused a shift of 360m on the georeferencing (#6759) * read/write US_survey_foot unit for linear units of projection * apply 'add_offset' and 'scale_factor' on x and y variables when present, such as in files produced by NOAA from the new GOES-16 (GOES-R) satellite (github #200) * add a HONOUR_VALID_RANGE=YES/NO open option to control whether pixel values outside of the validity range should be set to the nodata value (#6857) * fix crash on int64/uint64 dimensions and variables, and add support for them (#6870) NITF driver: * add support for writing JPEG2000 compressed images with JP2OpenJPEG driver * fix writing with JP2KAK driver (emit codestream only instead of JP2 format) * fix setting of NBPR/NBPC/NPPBH/NPPBV fields for JPEG2000 (fixes #4322); in JP2ECW case, make sure that the default PROFILE=NPJE implies 1024 block size at the NITF level * implement creation of RPC00B TRE for RPC metadata in CreateCopy() mode * add support for reading&writing _rpc.txt files * nitf_spec.xml: Add support for MTIRPB TRE in NITF image segment. Also makes minor change to BLOCKA to include default values (github #127) * nitf_spec.xml: add IMASDA and IMRFCA TREs * GetFileList(): Small optimization to avoid useless file probing. NWT_GRD: * detect short writes OpenJPEG driver: * support direct extracting of GeoRaster JP2-F BLOB (#6861) PCIDSK driver: * handle Exceptions returned from destructor and check access rights in setters (github #183) PDF driver: * implement loading/saving of metadata from/into PAM (#6600) * implement reading from/writing to PAM for geotransform and projection (#6603) * prevent crashes on dataset reopening in case of short write PLScenes driver: * add a METADATA open option PostgisRaster driver: * fix potential crash when one tile has a lower number of bands than the max of the table (#6267) R driver: * fix out-of-memory (oom) with corrupt R file Raw drivers: * prevent crashes on dataset closing in case of short write RMF driver: * fix wrong counter decrement that caused compressed RMF to be incorrectly decompressed (github #153) * fix load/store inversion of cm and dm units in MTW files (github #162) * fix reading nodata for non-double data type (github #174) ROIPAC driver: * add support for reading/writing .flg files (#6504) * fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL >= 2.0.0 (#6591) * fix for big endian hosts RS2 driver: * add support for reading RPC from product.xml SAFE driver: * fix handling of SLC Products by providing access to measurements as subdatasets (#6514) Sentinel2 driver: * add support for new "Safe Compact" encoding of L1C products (fixes #6745) SQLite driver: * Add read support for RasterLite2 coverages in SQLite driver SRTMHGT driver: * open directly .hgt.zip files * accept filenames like NXXEYYY.SRTMGL1.hgt (#6614) * handle files for latitude >= 50 (#6840) VRT driver: * add default pixel functions: real, imag, complex, mod, phase, conj, etc... for complex data types (github #141) * avoid useless floating point values in SrcRect / DstRect (#6568) * avoid buffer initialization in RasterIO() when possible (replace ancient and likely broken concept of bEqualAreas) * make CheckCompatibleForDatasetIO() return FALSE on VRTDerivedRasterBands (#6599) * VRT warp: fix issue with partial blocks at the right/bottom and dest nodata values that are different per band (#6581) * fix performance issue when nodata set at band level and non-nearest resampling used (#6628) * VRTComplexSource: do temp computations on double to avoid precision issues when band data type is Int32/UInt32/CInt32/Float64/CFloat64 (#6642) * VRT derived band: add the capability to define pixel functions in Python * CreateCopy(): detect short writes * Fix linking error on VRTComplexSource::RasterIOInternal<float>() (#6748) * avoid recursion in xml:VRT metadata (#6767) * prevent 'Destination buffer too small' error when calling GetMetadata('xml:VRT') on a in-memory VRT copied from a VRT * fix 2.1 regression that can cause crash in VRTSimpleSource::GetFileList() (#6802) WMS driver: * Added support for open options to WMS minidrivers * Refactored the multi-http code to make it possible to do range requests. * Added a minidriver_mrf, which reads from remote MRFs using range requests. * Made the minidriver_arcgis work with an ImageService, not only MapService. * Added static cache of server response. * Allow tiledWMS to work in off-line mode by including the server response in the .wms file itself. * honour GDAL_HTTP_USERAGENT config option when it is set and <UserAgent> is missing (#6825) * WMS/WMTS: better deal with tiles with different band count (grayscale, gray+alpha, palette, rgb, rgba) (github #208) * Make HTTPS options apply to tWMS minidriver init, better HTTP error reporting WMTS driver: * do not take into account WGS84BoundingBox/BoundingBox that would be the result of the densified reprojection of the bbox of the most precise tile matrix * add TILEMATRIX / ZOOM_LEVEL open options * accept tiles of small dimensions (github #210) XYZ driver: == OGR 2.2.0 - Overview of Changes == Core: * Layer algebra: Add KEEP_LOWER_DIMENSION_GEOMETRIES=YES/NO option to Intersection, Union and Identity. Default is yes, but it is set to no unless result layer is of geom type unknown. If set to no, result features which would have lower dim geoms are skipped if operating on two layers with same geom dim. * Fix crash/corrupted values when running importFromWkb() on POLYGON M/POLYGON ZM geometries (#6562) * Add OGR_WKT_PRECISION config option that defaults to 15 to select the number of decimals when outputting to WKT * Make OGRFeature::SetField(string) accept JSon serialized arrays for the String/Integer/Integer64/RealList types; add reciprocal OGRFeature::GetFieldAsSerializedJSon() for those types * OGRGeometryFactory::transformWithOptions(): better deal with reprojection from polar projection to WGS84, and projections crossing the antimeridian to WGS84, by splitting geometries prior to reprojection (#6705) * LinearRing transformTo(): force last point to be identical to first one in case it is not. * GML geometry parsing: avoid 'Cannot add a compound curve inside a compound curve' error (#6777) * OGR SQL: fix IN filtering on MapInfo indexed columns (#6798) * OGR SQL: add support for LIMIT and OFFSET keywords * OGR SQL: add comparisons on date / datetime (#6810) * OGR SQL: increase efficiency of DISTINCT operator * OGREnvelope: change initialization to fix issue when getting MULTIPOINT(0 0,1 1) envelope (#6841) * OGRParse: fix parsing logic to avoid false positive detection of string as datetime (#6867) OGRSpatialReference: * Upgrade to EPSG database v9.0 (#6772) * OGRCT: upgrade LIBNAME of mingw and cygwin to libproj-9.dll and cygproj-9.dll to be up-to-date with proj 4.9.X (recommended method is using ./configure --with-static-proj4 instead) (#6501) * importFromESRI(): fix import of multi line MERCATOR SRS (#6523) * morphToESRI(): correctly compute standard_parallel_1 of Mercator(2SP) projection from scale factor of Mercator(1SP) (#6456, #4861) * exportToProj4(): recognize explicit OSR_USE_ETMERC=NO to avoid using etmerc with proj >= 4.9.3 * importFromProj4(): do not set a AUTHORITY node for strings like '+init=epsg:XXXX +some_param=val' * importFromProj4(): be robust with missing proj.4 epsg dictionary when importing '+init=epsg:xxxx +other_parm=value' * AutoIdentifyEPSG(): add identification of EPSG:3995 (Arctic Polar Stereographic on WGS84) and EPSG:3031 (Antarctic Polar Stereographic on WGS84) * OGRCoordinateTransformation: avoid potential bugs in proj.4 on NaN input * importFromEPSG(): take into account DX,DY,DZ,RX,RY,RZ,DS columns in pcs.csv to add per-PCS TOWGS84 overrides (geotiff #52) * Coordinate transformation: prevent unnecessary coordinate transformations (github #184, #185) Utilities: * ogr2ogr: do not return error on ogr2ogr --utility_version * ogr2ogr: keep -append and -overwrite when -update follows * ogr2ogr: fix heuristics to detect likely absence of '-f' (#6561) * ogr2ogr: do not emit a warning when updating/overwriting a destination dataset that is not a Shapefile and if -f is not specified (#6561) * ogr2ogr: make overwriting of lots of PG tables less prone to PostgreSQL 'out of shared memory' errors, by committing transactions after each table recreation * ogr2ogr: prevent 'ogr2ogr same.shp same.shp' (#1465) * ogr2ogr: add a -limit option to limit the number of features read per layer * GDALVectorTranslate(): imply update mode if access mode not explicitly specified and hDstDS != NULL (#6612) * ogrlineref: Fix crash if no new layer name provided * ogrtindex: add -src_srs_name, -src_srs_format and -t_srs option to allow indexing files that have different projections Multidriver: * PGeo/FileGDB/OpenFileGDB: OGRCreateFromShapeBin(): do not error out on empty lines/polygons * GPKG/SQLite/PG/FileGDB/MEM: properly set/reset field subtype with AlterFieldDefn() (#6689) * DXF, GeoJSON, GML, KML, LIBKML, ODS, Shape, XLSX: report operating system error if file creation fails (#2253) AmigoCloud driver: * add option to receive an AmigoCloud API key in the connection string, fix page size (github #137) Carto: * renamed from CartoDB * fix CartoDB'fication() by removing manual creation of the_geom_webmercator mercator, and also attach created sequence to table.cartodb_id (#6565) CSV driver: * add read/write support for String/Integer/Integer64/RealList types as serialized JSon arrays DGN driver: * avoid crash with -ftrapv on 250K_NF44NE_Area_V7.dgn (relates to #6806) * use coordinate delta encoding from the attribute records array (6806) DXF driver: * sanitize layer name on export to avoid forbidden characters * reader: convert GeometryCollection to MultiPoint, MultiLineString or MultiPolygon when possible (QGIS #10485) * add font of TEXT and MTEXT to OGR style string (https://github.com/OSGeo/gdal/pull/198) DWG driver: * compilation fixes with Teigha 4.2.2 ElasticSearch driver: * add support for ElasticSearch 5.0 and fix 2.X support (#6724) * implement translation from SQL to E.S. query language FileGDB driver: * add support to read curve geometries (#5890) * support opening '.' directory GeoJSON driver: * writer: add a RFC7946=YES creation option (#6705) * read and write 'name' and 'description' members at FeatureCollection level * fix field type detection when first value of a field is null (#6517) * improve/fix field type promotion * fix wrong behaviour when there's a 'id' at Feature level and 'id' or 'ID' field in properties (#6538) * in case top level id is a negative integer, put the value in a 'id' attribute (#6538) * ESRI Json reader: support multilinestring from esriGeometryPolyline * ESRI Json reader: do not set field width of 2147483647 (#6529) * ESRI Json reader: support reading M and ZM geometries * Add CPL_json_object_object_get() and use it, to avoid deprecation warnings on json_object_object_get() * TopoJSON reader: sanitize invalid polygons (such as found in the 'TopoJSON' layer of http://bl.ocks.org/mbostock/raw/4090846/us.json) * writer: fix segfaults on NULL geometry with -lco WRITE_BBOX=YES (#6698) * writer: fix crash if NATIVE_MEDIA_TYPE creation option is specified alone * Add support of TopoJSON without 'transform' element (github #192) * don't set SRS if 'crs' set to null (github #206) GML driver: * remove hack for CityGML regarding forcing srsDimension to 3 if not specified. Instead add a GML_SRS_DIMENSION_IF_MISSING config option that can be set to 3 if needed (#6597) * consider srsName with URL like 'http://www.opengis.net/def/crs/EPSG/0/' as following EPSG axis order. Add SWAP_COORDINATES=YES/NO/AUTO (and GML_SWAP_COORDINATES configuration option) to override all auto guessing (#6678) * add a SRSNAME_FORMAT=SHORT/OGC_URN/OGC_URL dataset creation option * OGR_G_ExportToGMLEx(): add a COORD_SWAP option * Writer: make ogr:FeatureCollection be a substitutionGroup of AbstractFeature for GML 3.2 output, so as to be compatible of GMLJP2 v2 * GML and NAS: improve error reporting (mention feature id and gml_id) when parsing fails because of geometry decoding error * GML and NAS: add GML_SKIP_CORRUPTED_FEATURES and NAS_SKIP_CORRUPTED_FEATURES configuration options to avoid stopping reading a layer because of a corrupted geometry * Add support for Japanese GML FGD v4.1 format (github #204) GPKG driver: * list all tables/views by default (useful for non spatial tables not registered as aspatial). Add ASPATIAL_VARIANT layer creation option to select the way how to create a non-spatial table as aspatial. * Create non-spatial tables by default conforming to GPKG v1.2 'attributes' * use OGR_CURRENT_DATE config option if defined as the value of the 'last_change' of column of 'gpkg_contents' so as to be able to have binary identical output * update last_change column when content of raster or vector tables has changed * do not emit error when running ExecuteSQL() with a spatial filter on an empty layer (#6639) * update schema to reflect CURRENT_TIMESTAMP -> 'now' changes (github #155) * better table and column quoting. * Robustify layer deletion (make it accessible through DROP TABLE xxx and DELLAYER:xxx syntax) and renaming * implement DeleteField(), AlterFieldDefn(), ReorderFields() * make GetExtent() save extent if not already cached * add special SQL 'RECOMPUTE EXTENT ON layer_name' to force recomputation of extent * check identifier unicity on layer creation * add possibility to disable foreign key check by setting OGR_GPKG_FOREIGN_KEY_CHECK=NO config option * add HasSpatialIndex(tblname,geomcolname) SQL function * don't show Spatialite vgpkg_ virtual tables (#6707) * SQLite/GPKG: add explicit error message when trying to open a read-only WAL-enabled database (#6776) * SQLite/GPKG: make sure when closing a WAL-enabled database opened in read-only mode to reopen it in read-write mode so that the -wal and -shm files are removed (#6776) * make GetFeature() works on non conformant tables that have no integer primary key field (#6799), and be robust to non standard column types * remove triggers related to metadata tables that cause issues and have been removed by latest revisions of the spec. * declare feature id column of features tables and tile pyramid user data tables as NOT NULL (#6807) * add a gpkg_ogr_contents table to store feature count. * speed-up GetFeatureCount() with only a spatial filter set * improve column recognition for SQL result layer * add/override ST_Transform() and SridFromAuthCRS() from Spatialite to make them work with gpkg_spatial_ref_sys * add ImportFromEPSG() * make ST_Min/MaxX/Y(), ST_SRID(), ST_GeometryType() and ST_IsEmpty() work with Spatialite geometries * improve performance of spatial index creation and use on multi-gigabyte databases * better support of spatial views, by adding a special behaviour if a column is named OGC_FID * avoid potential denial of services by adding LIMIT clauses * slightly more efficient implementation of GetExtent() if extent in gpkg_contents is empty * create a dummy 'ogr_empty_table' features table in case we have no 'features' or 'tiles' table, so as to be conformant with Req 17 of the GeoPackage specification * add DEFAULT '' to metadata column of gpkg_metadata table * accept opening a .gpkg without vector content and without gpkg_geometry_columns table in vector mode if we also open in update mode; remove capability of opening a .gpkg without vector content but with gpkg_geometry_columns table in vector mode if we only open in read-only mode; fix creation of a vector layer in a database if it initially lacks a gpkg_geometry_columns table * fix appending a raster to a vector database without pre-existing raster support tables * add minimalistic support for definition_12_063 column in gpkg_spatial_ref_sys, so that insertion of new SRS doesn't fail * use GEOMETRYCOLLECTION instead of GEOMCOLLECTION for SQL and gpkg_geometry_columns.geometry_type_name * do not warn if gpkg_metadata extension declared GPX driver: * ignore wpt/rtept/trkpt with empty content for lat or lon ILI driver: * ILI1: fix crash in JoinSurfaceLayer() when the multicurve of the feature of the poSurfaceLineLayer layer is empty (#6688) * ILI1: make polygon reconstruction in Surface layers robust to curves not in natural order (#6728) * ILI2: assign FID to features (#6839) * ILI2: fix crashing bug in Create() if model file not specified KML driver: * add a DOCUMENT_ID datasource creation option to set the id of the root <Document> node LIBKML driver: * fix crash when reading <gx:TimeStamp> or <gx:TimeSpan> elements (#6518) * add a DOCUMENT_ID datasource creation option to set the id of the root <Document> node * emit style related errors as warnings to make datasets openable by SWIG bindings (#6850) MITAB driver: * limit (width, precision) of numeric fields on creation to (20,16) for compatibility with MapInfo (#6392) * add support for oblique stereographic (#6598) * Adds the authority code for Irish national grid (Ireland_1965) (github #149) * fix spelling for Euref_89 and add EPSG code (#6816) MSSQLSpatial driver: * Fix bulk insert with table names containing spaces (#6527) * Build optional mssql plugin with SQL Native Client support for MSSQL Bulk Copy * Fix MSSQL select layer to recognize geometry column with sqlncli (#6641) MySQL driver: * fix spatial filtering on recent mysql by adding a SRID in the rectangle geometry * do not force NOT NULL constraint on geometry field if no spatial index is used NAS driver: * support multiple 'anlass' in updates NWT_GRD: * add write support (#6533) OCI driver: * Add options for faster feature loading (#6606) * add WORKSPACE open option * correctly handle OFTInteger64 case in loader layer (bug found by cppcheck multiCondition) * support for long identifiers (up to 128 long) when running of 12.2 or + (#6866) * OCILOB VSIL driver: new driver to streams in and out of Oracle BLOB as a GDAL large virtual file system (#6861) ODS driver: * fix FID filtering (#6788) OGDI driver: * make GetNextRawFeature() report an error when it is not end of layer * better error reporting when the layer list cannot be established * catch non-fatal OGDI errors emitted by OGDI 3.2.0 if OGDI_STOP_ON_ERROR environment variable is set to NO, and emit them as CPLError()s * display OGDI error message if opening fails OpenFileGDB driver: * do not error out on geometries that have a declared M array but that is missing (#6528) * add support to read curve geometries (#5890) * transcode UTF-16 strings found in column names, alias, etc... to UTF-8 (instead of using only their ASCII byte) (#6544) * do not emit an error on a empty table whose declaration has M settings (#6564) * support opening '.' directory * improve detection of some form of TINs from MULTIPATCH, and for MultiPatch layers, try to select a better geometry type for those layers (#5888) * fix bug when field description offset is beyond 4GB (#6830) OSM driver: * fix 'too many tags in relation' error when parsing .osm files * allow key=value entries in closed_ways_are_polygons= configuration (#6476) * allow OSM_SQLITE_CACHE config option to be greater than 2047 PG driver: * fix insertion of binary/bytea content in non-copy mode (#6566) * fix errors caused by missing geometry_columns/spatial_ref_sys tables in non PostGIS databases, that prevent reading more than 500 features (QGIS #10904) * avoid errors with field default expressions like 'foo'::text (#6872) PLScenes driver: * add HTTP retry logic (#6655) * V0 API: workaround limitations on filtering on image_statistics.image_quality (#6657) * add support for Data V1 API S57 driver: * Fix ogr2ogr -f S57 (#6549) * fix crashes if the s57objectclasses.csv resource file contains invalid lines * remove data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv and move their content into main s57attributes.csv and s57objectclasses.csv files (#6673) * Update s57 attributes and object classes according to s-57 reference (github #202) * add POSACC and QUAPOS fields for geometric primitive layers (github #205) SDE driver: * rename driver to OGR_SDE. Fix build (#6714) SEGY driver: * Accept SEGY files with ASCII headers that have nul terminated strings (#6674) Shapefile driver: * auto-repack by default at dataset closing and FlushCache()/SyncToDisk() time. Controlled by AUTO_REPACK open and layer creation options (that default to YES) * generate .dbf end-of-file 0x1A character by default. Add DBF_EOF_CHAR layer creation options / open options to control that behaviour * writing: use strerrno() for better error messages (QGIS #13468) * change REPACK implementation on Windows to be robust to remaining file descriptors openend on the .shp/.shx/.dbf (#6672, QGIS #15570) * Fix issue in DBFCloneEmpty() one a one field DBF not yet written to disk * add call to AutoIdentifyEPSG() when reading a .prj * support reading .dbf with substantial padding after last field definition. * when rewriting the geometry of the last record in the .shp, do it at the file offset it previously used (#6787) SOSI driver: * make registration of driver work again (2.1.0 regression) (#6500) * update to latest version of https://github.com/kartverket/gdal. Make SOSI driver support more geometry types (including curved geometries) plus provides some improvements on attribute-type mapping. Fix some memory errors/leaks. Disable by default non working creation code (#6503) SQLite/Spatialite driver: * do not emit error when running ExecuteSQL() with a spatial filter on an empty layer (#6639) * add read/write support for String/Integer/Integer64/RealList types as serialized JSon arrays * Spatialite: avoid crash when creating layer with geom_type = wkbCurve (fixes #6660) * Spatialite: do not report some BLOB columns as geometry columns of tables/views (when found before the geometry column(s)) (#6695, #6659) * fix update of features with multiple geometry columns (#6696) * speed-up dataset closing when creating a database with many spatial layers * Spatialite: avoid spatial views to cause layers 'layer_name(geometry_name)' to be publicly listed (#6740) * Spatialite: speed-up creation of database with INIT_WITH_EPSG=NO that is slow without transaction (not sure why as the table is empty...) * use AUTOINCREMENT for feature id column * allow OGR_SQLITE_CACHE to be set to > 2047 (MB) without overflowing * SQLite/GPKG: use SQLITE_OPEN_NOMUTEX flag to open databases. * GPKG/SQLite: fix ExecuteSQL() to work with a statement with 2 SELECT and ORDER BY clause (#6832) * SQLite/GPKG: change default page_size to 4096 bytes. * Update layer statistics for Spatialite 4 DB (#6838) * Remove traces of support of SQLite < 3.6.0 * SQLite dialect: properly quote column names when needed (github #214) VFK driver: * allow reading multiple VFK files into single DB support amendment VFK files * recreate DB in the case that it's outdated (VFK DB created by previous versions of GDAL) * allow reading DB as valid datasource (#6509) * new tables in backend SQLite database (geometry_columns/spatial_ref_sys) to enable reading DB datasource by SQLite driver * new configuration option OGR_VFK_DB_READ to enable reading backend database by VFK driver VRT driver: * add support for 'm' attribute in PointFromColumns mode (#6505) WFS driver: * invalidate underlying layer when SetIgnoredFields() is called (QGIS #15112) * don't crash on empty <Keyword/> declaration (#6586) * fix potential nullptr dereference on dataset without layer (github #179) XLSX driver: * only list worksheets (and no charts) as layers (#6680) == SWIG Language Bindings == All bindings: * map osr.GetTargetLinearUnits() (#6627) * allow wkbCurve/wkbSurface as valid values for the geom type of GeomFieldDefn object * map GDALIdentifyDriverEx() Java bindings: * Fix SWIG Java bindings for GNM (#6434) * Fix crash on GetDefaultHistogram() if the C++ method returns an error (#6812) Perl bindings: * return value always from GetGeomFieldIndex (#6506) * the Warp method requires a list of datasets (#6521) * when 'use bigint' is in effect, int var is a ref. * Separate the module building in the CPAN distribution and in the GDAL source tree. The CPAN distribution will be developed at https://github.com/ajolma * Fix the Extension method in Driver per RFC 46 * The Inv method of GeoTransform did not return a new object. Allow a single point in Apply method of GeoTransform. * Test for existence of PDL and require it if available. * allow decimation/replication in Piddle i/o. * support resampling in ReadTile. * Polygonize: Require explicit 8 for Connectedness to set 8-connectedness and allow 8CONNECTED as an option. * use Safefree to free memory allocated in Perl (#6796) Python bindings: * release the GIL before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open()) * add outputType option to gdal.Rasterize() * fix build issues when CXX is defined in the environment * gdal.VectorTranslate(): add spatSRS option * when enabling ogr.UseExceptions(), use the GDAL error message in the exception text (if available), when the exception is linked to an error value in the OGRErr return code * gdal.VectorTranslate(): accept a single string as value of the layers option (instead of iterating over each of its characters) * Regenerate Python bindings with SWIG 3.0.8 to avoid issue with Python 3.5. Add backward compatibility in Band.ComputeStatistics() to accept 0/1 as input instead of the expected bool value (#6749) * fix gdal.DEMProcessingOptions(zeroForFlat=True) (#6775) * fix 'import osgeo.gdal_array' with python3 and SWIG 3.0.10 (#6801) * allow gdal.FileFromMemBuffer() to use buffer > 2GB (#6828) * accept unicode strings as field name argument in Feature (like SetField, GetField, etc...) and FeatureDefn methods = GDAL/OGR 2.1.0 Release Notes = == In a nutshell... == * New GDAL/raster drivers: - CALS: read/write driver for CALS Type I rasters - DB2 driver: read/write support for DB2 database (Windows only) - ISCE: read/write driver (#5991) - MRF: read/write driver (#6342) - SAFE: read driver for ESA SENTINEL-1 SAR products (#6054) - SENTINEL2: read driver for ESA SENTINEL-2 L1B/LC1/L2A products - WMTS: read driver for OGC WMTS services * New OGR/vector drivers: - AmigoCloud: read/write support for AmigoCloud mapping platform - DB2 driver: read/write support for DB2 database (Windows only) - MongoDB: read/write driver - netCDF: read/write driver - VDV: read/write VDV-451/VDV-452 driver, with specialization for the Austrian official open government street graph format * Significantly improved drivers: - CSV: new options, editing capabilities of existing file - ElasticSearch: read support and support writing any geometry type - GeoJSON: editing capabilities of existing file, "native data" (RFC 60) support - MBTiles: add raster write support. fixes in open support - PDF: add PDFium library as a possible back-end. - PLScenes: add support for V1 API - VRT: on-the-fly pan-sharpening - GTiff: multi-threaded compression for some compression methods * Port library: add /vsis3/, /vsis3_streaming/, /vsicrypt/ virtual file systems * RFC 26: Add hash-set band block cache implementation for very larger rasters (WMS, WMTS, ...) http://trac.osgeo.org/gdal/wiki/rfc26_blockcache * RFC 48: Geographical networks support (GNM) https://trac.osgeo.org/gdal/wiki/rfc48_geographical_networks_support * RFC 58: Add DeleteNoDataValue(): https://trac.osgeo.org/gdal/wiki/rfc58_removing_dataset_nodata_value * RFC 59.1: Make GDAL/OGR utilities available as library functions: https://trac.osgeo.org/gdal/wiki/rfc59.1_utilities_as_a_library For gdalinfo, gdal_translate, gdalwarp, ogr2ogr, gdaldem, nearblack, gdalgrid, gdal_rasterize, gdalbuildvrt Available in C, Python, Perl and Java bindings. * RFC 60: Improved round-tripping in OGR https://trac.osgeo.org/gdal/wiki/rfc60_improved_roundtripping_in_ogr Implemented in GeoJSON driver * RFC 61: Support for measured geometries. https://trac.osgeo.org/gdal/wiki/rfc61_support_for_measured_geometries Implemented in Shapefile, PostgreSQL/PostGIS, PGDump, MEM, SQLite, GeoPackage, FileGDB, OpenFileGDB, CSV, VRT * Upgrade to EPSG database v8.8 * General sanitization pass to clean-up code, fix a lot of compiler warnings, as well as issues pointed by static code analyzers, such as Coverity Scan (credits to Kurt Schwehr for tackling a large part of them) or CLang Static Analyzer. * Fixes in a number of drivers to be more robust against corrupted files (most found with American Fuzzy Lop): RIK, INGR, Northwood, HF2, CEOS, GTiff, GXF, BMP, NITF, HFA, VRT, FIT, CEOS2, NWT_GRD/NWT_GRC, MITAB, RPFTOC, DBF/Shape, XYZ, VFK, DXF, NAS, GSAG, GS7BG, OpenFileGDB, RMF, AIGRID, OpenAIR, EHDR, ISO8211, FAST, USGSDEM, DGN, SGI, OpenJPEG, PCRaster, BSB, ADRG, SRP, JPEG, Leveller, VICAR, PCIDSK, XPM as well as in portability library (CPL), algorithms and raster core. * Driver removed: - Google Maps Engine (GME) (#6261) == New installed files == * gdal_pansharpen.py * data/vdv452.xml * data/vdv452.xsd * data/netcdf_config.xsd * /path/where/bash-completion/scripts/are/installed/gdal-bash-completion.sh == Backward compatibility issues == See MIGRATION_GUIDE.TXT == GDAL/OGR 2.1.0 - General Changes == Build(Unix): * Allow plugin for HDF4, HDF5, GeoRaster, FileGDB, OCI and PG * Add a --with-gnm option to enable GNM * Add --enable-lto switch to turn on link time optimization (GCC >= 5) * Add --with-spatialite=dlopen --with-spatialite-soname=libspatialite.so[.X] syntax to allow linking against spatialite through dlopen() mechanism (#6386) * Add a lot of warning options when available in the compiler. * Change detection test of podofo to work with podofo 0.9.3 * Fix gdalserver compilation with recent GNU libc (such as in Arch Linux) (#6073) * Add pkg-config support for libkml fork (#6077) * Update config.sub and config.guess to their latest upstream versions so as to be able to detect new architectures (android, ppcle64, etc..) (#6101) * Fix python package installation in custom prefixes (#4563) * configure: check that CXX is really a working compiler (#4436) * Build support for Kakadu 7.7 * thinplatespline.cpp: avoid using optimized version of VizGeorefSpline2DBase_func4() with ICC versions that fail on it (#6350) * Add bash completion for GDAL/OGR utilities and scripts (#6381) * Add support for ODBC autodetection with mingw64 (#6000) * Remove macos 9 and older support (#6133). * Remove dist_docs, burnpath and pszUpdableINST_DATA in gcore/gdaldrivermanager.cpp as they are have not been used for a while (#6140). * Removed --without-ogr configure flag (#6117). Always build with OGR. Build(Windows): * Add support for Visual Studio 2015 (a.k.a MSVC_VER=1900 or VC 14) * Add KAKFLAGS to nmake.opt with KDU_{MAJOR,MINOR,PATCH}_VERSION define-s. * nmake.opt: make it less error prone to define SETARGV when paths include spaces (#6032) * nmake.opt: add CL.EXE compiler option /FC to display full path of source code file in diagnostics * Remove WinCE support (#6133) * nmake.opt: add /I flag to the INSTALL command so that xcopy will be smart enough to create a directory when copying files and avoid prompting for user input (https://github.com/OSGeo/gdal/pull/95) * Enable mssql spatial blugin build, use BCP as the default option for the sqlncli enabled builds * Add support to find MrSID 9.5 dll * Fix Windows build with recent MySQL versions and Visual Studio 2015 (#6457) Build(all): * Compatibility with C++11 and C++14 * Optional crypto++ dependency for /vsicrypt/ support * Optional mongocxx dependency for MongoDB support Other: * Disable copy constructor and assignment operators in classes OGRFieldDefn, OGRGeomFieldDefn, OGRFeature, GDALMultiDomainMetadata, GDALDefaultOverviews, GDALOpenInfoGDALDataset, GDALRasterBlock, GDALRasterBand and GDALDriver (#6100) == GDAL 2.1.0 - Overview of Changes == Port: * Add /vsicrypt/ virtual file system for reading/creating/update encrypted files on the fly, with random access capabilities * Add /vsis3/ and /vsis3_streaming/ virtual file systems to read/write objects from AWS S3 buckets * /vsizip/: avoid returning the previous file list of an already opened .zip if it has changed afterwards (#6005) * /vsizip/: use CP437 on Windows when ICONV support is available (#6410) * /vsimem/: implement append mode (#6049) * /vsistdin/: fix caching of first 1MB / VRT driver: read XML content from /vsistdin/ in a streaming compatible way (#6061) * /vsistdout/: flush when closing the handle (#6149) * Add VSIReadDirEx() with a limit on the number of files before giving up and corresponding VSIFilesystemHandler::ReadDirEx() virtual function * cpl_vsil_cache: rework to be able to work on very large files on 32bit systems * Add CPLThreadLocaleC class to use thread-specific locale settings (when available) * VSIWin32Handle::Flush(): add VSI_FLUSH config option that can be set to TRUE to force FlushFileBuffers(). (hack related to #5556) * Fix VSIL append mode in Windows (#6065) * Make CPLCreateMutexEx() support CPL_MUTEX_REGULAR; and fix CPL_MUTEX_ADAPTIVE to avoid continuing in code path for creation of recursive mutex * Add CPLWorkerThreadPool API * Add CPLGetThreadLocalConfigOption() * Fix CPL_LSBINT16PTR() and CPL_LSBINT32PTR() to work with non-byte pointer (#6090) * CPLRecodeStub(): add special case for CP437 -> UTF-8 when input is only printable ASCII * CPLHTTPFetch(): add LOW_SPEED_TIME and LOW_SPEED_LIMIT options * CPLGetValueType(): improve heuristics to avoid detecting some WKB strings as real numbers (#6128) * Add CPL_SHA256* and CPL_HMAC_SHA256 functions * Make CPLVirtualMemFileMapNew() work on all systems having mmap(), and thus GeoTIFF GTIFF_VIRTUAL_MEM_IO optimization too * Add VSI_MALLOC_VERBOSE() and similar macros to output an error message in case of failed alloc * CPLParseXMLString(): limit depth of elements to 10000 * Win32 file management: handle files whose total file path length is greater than 255 characters in Open() and Stat() * Make CPLFormFilename(absolute_path, "..", NULL) truncate when possible * Add VSIGetDiskFreeSpace() * Implement CPLIsInf() for Solaris 11 * Add a CPL_FINAL macro that expands to C++11 final keyword when C++11 is enabled, and use it in a few places * Avoid CPLEscapeString (CPLES_URL) encoding some characters unnecessarily (#5526) * Add CPLTestBool(), CPLTestBoolean(), CPLFetchBool() * Fix CPLGetValueType() to recognize D1 as a string and not a real number (#6305) * CPLFormFilename(): use '/' directory separator on Windows on /vsicurl_streaming/ files (#6310) * Add hack enabled by -DDEBUG_BOOL to detect implicit int->bool conversions that dislike MSVC (#6325) * Add hack to '#define NULL nullptr' when NULL_AS_NULLPTR is defined. Must be used together with -std=c++11 -Wzero-as-null-pointer-constant with GCC to detect misuses of NULL/nullptr (#6327) * Add VSIError mechanism to store errors related to filesystem calls, and use it for /vsis3/. Add new CPLE_ error numbers. (https://github.com/OSGeo/gdal/pull/98) * Fix CPLsscanf() to be conformant regarding how space/tab/... separators are handled, so as to fix OGR GMT to be able to read coordinates separated by tabulations (#6453) Core: * Change default value of GDAL_CACHEMAX to 5% of usable physical RAM * Allow open options name to be prefixed by @ to be silently ignored when not existing in driver options (RFC 60) * Honour NBITS metadata item when doing RasterIO() with non-nearest resampling (#6024) * GDALClientServer: pass open options to INSTR_Open call * Improve performance of GDALCopyWords() float->byte/int16/uint16 by using SSE2 * Decorate prototypes of RasterIO() related operations with CPL_WARN_UNUSED_RESULT * Avoid deadlock when writing 2 datasets in 2 threads (#6163) * Remove obsolete symbols __pure_virtual(), GDALCreateProjDef(), GDALReprojectToLongLat(), GDALReprojectFromLongLat() and GDALDestroyProjDef() * Remove obsolete non-template-based implementation of GDALCopyWords(). We don't support anymore such compilers * GDALJP2Box::ReadBoxData(): do not abort if memory allocation fails * Add GDALAdjustValueToDataType() in C API, and use it in GDALTranslate() and GDALWarp() * GDALDefaultOverviews::CreateMaskBand(): fix logic error related to writing per-band mask * Block cache: fix excessive memory consumption when dealing with datasets with different block sizes (#6226) * EXIFExtractMetadata(): fix potential 1-byte write buffer stack overflow * MDReader: do no attempt reading side-car files on /vsisubfile/ (#6241) * GDALCheckBandCount(): allow by default a maximum number of bands to 65536 * GDALOpenInfo: add StealSiblingFiles() and AreSiblingFilesLoaded() methods * GDALOpenInfo::GetSiblingFiles(): give up after GDAL_READDIR_LIMIT_ON_OPEN (default=1000) files in the directory * GDALDefaultOverviews: add TransferSiblingFiles() method * GDALDriver::DefaultCreateCopy(): fix logic related to propagation of NBITS / PIXELTYPE metadata items as creation options * Reorder driver registration a bit so that formats with efficient identification are probed first * Add GDALIdentifyEnum (GDAL_IDENTIFY_UNKNOWN, GDAL_IDENTIFY_FALSE or GDAL_IDENTIFY_TRUE) for return values of Identify() * GDALLoadRPCFile(): load .rpc files from Ikonos products that have extra end-of-line character (#6341) * Export GDALRegenerateOverviewsMultiBand() symbol, but mostly for plugins (#6342) * Add GDAL_NO_AUTOLOAD to compile out the body of AutoLoadDriver (#6380) * Avoid ComputeStatistics(), GetHistogram() and ComputeRasterMinMax() to use only the first column of blocks in approximation mode for a raster whose shape of blocks is a square (#6378) * Add GDALGetDataTypeSizeBits() and GDALGetDataTypeSizeBytes(). * GDALReadOziMapFile(): fix reading Ozi external files from virtual file systems (https://github.com/OSGeo/gdal/pull/114) * Add CPLSetCurrentErrorHandlerCatchDebug() to enable custom error handlers not to intercept debug messages Algorithms: * RPC transformer: provide SSE2 accelerated transformer * RPC: fix off-by-half pixel computation of (pixel, line), and in bilinear and bicubic RPC DEM interpolation; fix off-by-one pixel registration for Pleiades RPC (#5993) * RPC: make RPCInverseTransformPoint() check convergence has been reached (#6162) * RPC DEM: optimize when DEM is in NAD83 or any other geodetic CS that transform as a no-op to WGS 84 * RPC DEM: take into account vertical datum of the DEM when present to convert DEM elevations to ellipsoidal heights (#6084) * RPC DEM: do fallback cubic -> bilinear and bilinear -> near on DEM edges * RPC DEM: improve inverse transformer to validate error threshold and improve convergence (#6162, #6377) * RPC: fix issues with dateline (#6472) * TPS solver: discard duplicated GCP to avoid avoidable error, warning when 2 GCPs have same (pixel,line) but different (X,Y) or the reverse * Warper: rework multithreaded computations to use a thread pool rather than forking threads each time * Warper: avoid really excessive processing time for some warping with target areas completely off the source raster (especially when involving RPC) (#6182) * Warper: CreateKernelMask(): fix potential 32 bit integer overflow when using warp memory value > 2GB (#6448) * gdal_grid: add linear interpolation algorithm * gdal_grid: add invdistnn algorithm, variation on the existing inverse distance weighting algorithm with quadtree to search for points only in the neighborhood (#6038) * gdal_grid: fix crash in optimized mode with GCC 4.4 on 64bit (#5987) * gdal_grid: compile gdalgrid AVX optimization for Windows when supported by compiler * Add GDALTriangulationXXXX() API through libqhull * Sieve filter: fix crash on nodata polygons (#6096) * Sieve filter: improvement to walk through the biggest neighbour chain until we find a polygon larger than the threshold (#6296) * GDALFPolygonize(): factor implementation with integer case * GDALComputeMedianCutPCT(): fix to make it work with rasters with more than 2 billion pixels (#6146) * Overview: Make average and gauss methods aware of transparent color table entries (#6371) Utilities: * gdalinfo: workaround bug in proj <= 4.9.1 on datasets with a SRS with a vertical shift grid (#6083) * gdal_translate: detect more reliably if specified bands are not in default order * gdal_translate: fix -a_nodata with negative values on rasters that have PIXELTYPE=SIGNEDBYTE; fix preserving PIXELTYPE=SIGNEDBYTE with VRT * gdal_translate: fix random behaviour when -scale is used without source bounds (#6455) * gdal_rasterize: do on-the-fly reprojection of input vector onto output raster * gdal_rasterize: rasterize: always create output with 1/2 cell buffer of input geometry envelope (#6058) * gdal_rasterize: add the -dialect option * gdal_rasterize: accept NaN as a valid value for -init / -burn (#6467) * gdalwarp: add -doo option to specify open options of (existing) output dataset * gdalwarp: if RPC_DEM warping option is specified, use exact transformer by default (#5993) * gdalwarp: make it honour nodata value of existing dataset (if -dstNoData isn't explicitly specified) * gdalwarp: do not propagate STATISTICS_ of second or following source datasets * gdalwarp: do not emit warning when using -cutline with a SRS and the source raster has RPC or GEOLOCATION * gdalwarp: check that the cutline is valid after transformation/reprojection * gdalwarp: better deal when dealing with a mix of RGB and RGBA datasets as input * gdalwarp: fix -srcnodata to not put garbage values as target dstnodata (#6315) * gdalwarp: densify cutline to avoid invalid geometry after reprojection to source raster, especially in the RPC case (#6375) * gdalserver: add a -nofork mode (Unix only for now), so that multiple clients can connect to the same dataset. Useful for safe 'concurrent' updates * add gdal_pansharpen.py script * gdal2tiles.py: fix error on a raster with less than 3 bands that resulted in a 'IndexError: list index out of range' * gdal2tiles.py: Change EPSG:3785 / EPSG:900913 into EPSG:3857 (#5622) * gdal2tiles.py: add Leaflet template (https://github.com/OSGeo/gdal/pull/71) * gdal2tiles.py: add -q switch for quiet mode * gdaldem: correctly deal with NaN as nodata value (#6066) * gdaldem color-relief: deal with the case of repeated entries with the same value and the input raster has pixels that match that value exactly (#6422) * gdaladdo: emit error message if passed an invalid dataset name (#6240) * gdaladdo: do not silence warnings when opening in update mode, if the open is successful * gdalbuildvrt: fix potential crash when using -b switch (#6095) * gdalbuildvrt: accept nan as value for -srcnodata and -vrtnodata * gdalbuildvrt: return non zero return code if the flush of the VRT to disk failed * gdal_merge.py: takes again into account -n flag (#6067) * gdalbuildvrt / gdal_translate / VRT: use floating point values for source and destination offsets and sizes (#6127) * gdalmove.py: fix to run with GDAL 2.0 gdal.InvGeoTransform() signature * gdal_calc.py: Add * from gdalnumeric to gdal_calc.py eval namespace again, to fix 2.0 regression that made for example 'log10(A)' to no longer work (https://github.com/OSGeo/gdal/pull/121) Python samples: * gdalpythonserver.py: update to protocol 3 AAIGRID: * when writing with floating-point values, ensure at least one value has a decimal point to avoid issues with some readers (#6060) BMP driver: * BMP: avoid too big color table allocation in case of wrong iClrUsed value BLX driver: * converted to support VirtualIO ECRG driver: * change subdataset definition to make sure that they only consist of frames of same scale (#6043) * fix base34 decoding and Windows filename handling (#6271) ECW driver: * use wide char Open API on Windows when GDAL_FILENAME_IS_UTF8=YES (https://github.com/OSGeo/gdal/pull/70) * fix reading of ECW in /vsi file systems (with SDK >= 4) (#6482) GeoRaster driver: * fix deflate read error depending on endianness (#6252) GIF driver: * libgif: partial resync with giflib master (but keep ABI of 4.1.6) * fix crash when CreateCopy a source with one color in the color table GMT driver: * validate raster dimensions on opening, and acquire mutex in dataset destructor GPKG driver: * write support: fix various issues in update scenarios when interacting with the GDAL block cache that could result in lost/corrupted band data to be written in tiles (#6309) * write support: fix potential use of freed sqlite temporary DB handle when generating overviews with partial tiles (#6335) * write support: fix potential crash in scenarios when block cache is full (#6365) * write support: fix inversion of row/column in one of the SQL request involved in partial tiles management (#6339) * fix generation of files with 1-band paletted input dataset. Also improve the logic to retrieve color palette when forcing BAND_COUNT=1 on opening (#6450) GTiff driver: * add NUM_THREADS creation and open options to enable multi-threaded compression * fix GTiffDataset::IsBlockAvailable() wrong behaviour when compiling against internal libtiff, when a BigTIFF file has a TileByteCounts with LONG/4-byte counts and not LONG8 (#6001) * Correctly take into account overridden linear units for a geotiff with a EPSG PCS code (#6210, #4954) * make VirtualMemIO() work with non native endianness * improve VirtualMemIO() performance in tiled Contig read to single band copy * improve single band tiled VirtualMemIO reading * improve DirectIO() to work on tiled uncompressed geotiff, for non-resampling and nearest resampling cases. Also improve performance of resampling cases on un-tiled files * fix DirectIO() mode with complex types and inverted endianness (#6198) * optimize writing of 12-bit values * implement lazy loading of .aux.xml and .tfw/.wld * Internal libtiff: update to CVS HEAD post libtiff 4.0.6 * make SetColorInterpretation(GCI_AlphaBand) work on a 5 band or more GeoTIFF (#6102) * automatically set PHOTOMETRIC=RGB if manually assigning color interpretation Red,Green,Blue to band 1,2,3 before directory crystallization (#6272) * add GEOTIFF_KEYS_FLAVOR=ESRI_PE creation option to write EPSG:3857 in a ESRI compatible way (#5924) * call XTIFFInitialize() in LibgeotiffOneTimeInit() as the former isn't thread-safe, so better call it from the later which is thread-safe (#6163) * fix reading and writing angular units different from degree (namely arc-second, arc-minute, grad, gon and radian) (#6171) * do not use VirtualMemIO optimization on compressed /vsimem/ files (#6195) * correctly set GTRasterTypeGeoKey=RasterPixelIsPoint if AREA_OR_POINT=Point but there is no SRS set (#6225) * improve a bit error detection when writing * check free space before writing (only for big, non sparse, uncompressed) * do not read large 'one row' JBIG compressed files with the scanline API (#6264) * Fix SetMetadata() to properly clear existing PAM metadata (complement to #5807) * prevent potential out of bounds read/write to TIFFTAG_EXTRASAMPLES (#6282) * do not use first directory as potential mask, to avoid assertion in GTiffDataset::SetDirectory() (#6287) * reject files with strips/tiles/scanlines bigger than 2 GB to avoid 32 bit integer overflow. Also in case of files with Contig PlanarConfiguration do not make reading one block for band 2 OK when reading for band 1 issued an error (#6288) * GTIFFWriteDirectory(): avoid memory leak of codec related memory (#2055) * Make ALPHA=NO in CreateCopy() cancel alpha color interpretation even if present in source raster * fix problem with implicit overviews of JPEG-compressed files (#6308) * fix compilation problem with internal libtiff if DEFER_STRILE_LOAD isn't defined (which is not the default configuration) (https://github.com/OSGeo/gdal/pull/90) * use more appropriate error message when 4GB threshold is reached with external overviews, and try to make doc of BIGTIFF=IF_NEEDED/IF_SAFER clearer (#6353) HDF4 driver: * Skip quotation mark when parsing HDF-EOS metadata. HDF5 driver: * add Komsat Mission ID to possible value for HDF5 SAR product (https://github.com/OSGeo/gdal/pull/103) HF2 driver: * fix reading side of the driver to work on architectures where char is unsigned, like PPC or ARM (#6082) HFA driver: * when reading projection, preserve EPSG code if AutoIdentifyEPSG() identified the SRS, even if a PE string is present (#6079) * keep TOWGS84 even when using ESRI PE string (#6158) * fix crashes on corrupted files (#6208, #6286) INGR driver: * check that RLE decoding produced the expected number of bytes and error out otherwise; test that 'random' line seeking actually works JPEGLS driver: * fix build error (#6430) JP2ECW driver: * honour psExtraArg->eResampleAlg when upsampling (#6022) JP2KAK driver: * honour psExtraArg->eResampleAlg when upsampling (#6022) * try rounded dimensions to decide if the data is being requested exactly at a sub-resolution * support v7.7 on Unix (additional changes potentially needed on Windows) JP2OpenJPEG driver: * Safer multi-threaded use JPEG driver: * add USE_INTERNAL_OVERVIEWS open option (default to TRUE as in recent versions, can be set to FALSE to hide internal overviews KMLSuperOverlay driver: * fix 2.0 regression with some RasterIO() requests involving resampling (#6311) * recognize datasets that have a intermediate <Folder> that forms a <Document><Folder><Region/><GroundOverlay/> structure (#6343) LCP driver: * fix wrong use of endianness macros to fix behaviour on big-endian hots Leveller driver: * update to v9 read support (#5632,) MBTiles driver: * add raster write support * fix so that datasets whose lowest min zoom level is 16 or above are recognized * be robust to invalid 'bounds' at dataset opening (#6458) MEM driver: * avoid 32 bit overflows NetCDF driver: * add support for reading NC4 unsigned short attributes and variables (#6337) * read correctly nodata values in [128,255] range for (unsigned) Byte data type (#6175) * implement Get/SetUnitType() using the standard units attribute (https://github.com/OSGeo/gdal/pull/96) * optimize IReadBlock() and CheckData() handling of partial blocks in the x axis by re-using the GDAL block buffer instead of allocating a new temporary buffer for each block (#5950) * full read/write support for new NetCDF4 types NC_UBYTE, NC_USHORT, NC_UINT and NC_STRING for variables (except for NC_STRING) and attributes (https://github.com/OSGeo/gdal/pull/99) * add support for the geostationary projection (#6030) * fix one byte heap write overflow in NCDFTokenizeArray() (#6231) * fix potential buffer overflows with uses of nc_inq_varname(), nc_inq_attname() and nc_get_att_text() (#6227) * validate that gridmapping:GeoTransform has 6 values (#6244) * fix wrong use of deallocator when writing a GEOLOCATION array, and other issues * limit number of bands reported to 32768 by default * validate raster dimensions * validate content of NC_GLOBAL#GDAL variable (#6245) NGSGEOID driver: * make it work on > 2GB file NITF driver: * data/nitf_spec.xml: Add CSCCGA, MENSRB, SENSRB, STREOB, ENGRDA, EXPLTB and PATCHB TREs (https://github.com/OSGeo/gdal/pull/81, #6285, https://github.com/OSGeo/gdal/pull/86) * fix parser to properly deal with variable length items not in first nesting level Northwoord driver: * fix computation of intermediate color table values on non-Intel platforms (#6091) NTv2 driver: * support reading/writing/appending to files with big-endian order (cf https://github.com/OSGeo/proj.4/issues/345) OpenJPEG driver: * do not expose block dimensions larger than dataset dimensions to avoid wasting memory (#6233) PCIDSK driver: * Remove the old driver (#6172) PDF driver: * add PDFium library as a possible back-end. Initial support contributed by Klokan Technologies GmbH (http://www.maptiler.com/) * workaround a bug of PoDoFo 0.9.0 by avoiding loading of vector content in raster-only mode (doesn't prevent the crash if reading the vector layers) (#6069) * make OGC BP registration work with media box where bottom_y is negative and top_y = 0 (in non rotated case) * make OGC BP registration work with media box where min_x != 0 (in non-rotated case) * correctly take into account non-meter linear units with OGC BP encoding, and add support for US FOOT (#6292) PDS driver: * change default values of PDS_SampleProjOffset_Shift and PDS_LineProjOffset_Shift to 0.5 (#5941) * fix nodata value for UInt16 to be 0 (#6064) * accept 'ODL_VERSION_ID = ODL3' in header (#6279) PGChip driver: * driver removed from sources PLScenes driver: * PLScenes V0: avoid opening raster to generate dummy .aux.xml PNG driver: * Support writing 1, 2 or 4 bit single band * Add NBITS creation option * fix XML of creation option list * Update internal libpng to 1.2.56 PostgisRaster driver: * avoid Identify() to recognize OGR PostgreSQL connection strings with schemas option and cause loud 'QuietDelete' (#6034) Raw drivers: * better support for direct read of more than 2GB in single gulp (untested though) RIK driver: * fix Identify() method to recognize again non-RIK3 RIK datasets (#6078) RMF driver: * implement GetNoDataValue() * add read/write access to new RMF format for files larger than 4 Gb (version=0x201) (https://github.com/OSGeo/gdal/pull/11) ROIPAC driver: * Support offset and scale band (#6189) VICAR driver: * change PDS_SampleProjOffset_Shift and PDS_LineProjOffset_Shift default values to 0.5 (#5941) * fix loss of precision in scale and offset VRT driver: * expose implicit 'virtual' overviews for VRT whose bands are made of a single SimpleSource/ComplexSource * gdalvrt.xsd: Add capitalized versions of true and false (#6014) * GetSingleSimpleSource(): check there's a single source (#6025) * honour VRTRasterBand NBITS metadata with SimpleSource and ComplexSource * properly take into account nodata value declared at VRT band level when doing resampling with non-nearest * honour relativeToVRT when using AddBand() to add a VRTRawRasterBand (https://github.com/OSGeo/gdal/pull/67) * VRT warp: fix crash with implicit overviews and destination alpha band (#6081) * make GetDefaultHistogram() on a sourced raster band save the result in the VRT (#6088) * serialize NODATA and NoDataValue items with %.16g, e.g. so as to be able to hold large int32 nodata values (#6151) * VRTSourcedRasterBand: make ComputeRasterMinMax() and ComputeStatistics() forward bApproxOK to overview band (useful for implicit overviews) * make CreateCopy() preserve NBITS metadata item * avoid loading sibling file list if not available * VRT raw: don't truncate last figure of ImageOffset if there are left space padding (#6290) * VRTWarpedDataset::SetMetadataItem(): fix crash when calling with name=SrcOvrLevel and value=NULL (#6397) * Warped VRT: fix deadlock in situation where warped VRT datasets are read in multiple threads and the block cache reaches saturation. Also add a GDAL_ENABLE_READ_WRITE_MUTEX config option that can be set to NO to disable the read/write mutex mechanism in cases where it would deadlock (#6400) WMS driver: * add a IIP (Internet Imaging Protocol) minidriver * limit number of zoom levels for ArcGIS MapServer JSon (#6186) * determine a resolution that will not result in a number that is larger than the maximum size of an integer. Any value that exceeds the maximum size of an integer will raise an invalid dataset dimensions error. (https://github.com/OSGeo/gdal/pull/89) XYZ driver: * be more robust to not exactly equal X and Y spacing (#6461) == OGR 2.1.0 - Overview of Changes == Core: * Add OGREditableLayer class to add editing capabilities to drivers with none or limited editing capabilities * OGRGeometry: add DelaunayTriangulation() method (GEOS >= 3.4) * OGRGeometry and derived classes: implement copy constructor and assignment operator (#5990) * OGRGeometry: Fix result of Equals on POINT EMPTY with POINT(0 0) * OGRFeature SetField(): more type conversions allowed, particularly with array types * OGRFeature::SetGeometry()/SetGeometryDirectly(): make it work when passed geometry is the currently installed geometry (#6312) * OGR SQL: do not silently skip NULL values in the first records when evaluating a SELECT DISTINCT (#6020) * OGR SQL: correctly sort NULL values in first positions (#6155) * OGR SQL: fix CAST(x AS bigint) to return an evaluated int64 node, and not int32 (#6479) * OGR SQL: handle 'fid' as Integer64 in where clause, and allow CAST(fid AS bigint) in selected columns (#6484) * Add OGRUpdateFieldType() * Decorate a few functions/methods of the OGR layer API with CPL_WARN_UNUSED_RESULT * WKT export: use 15 significant figures, instead of 15 figures after decimal point (#6145) * WKT export: do not append .0 after non-finite values (#6319) * Fix typo in definition of name of OGR_FD_ReorderFieldDefns (final s was missing) * OGRLayer::SetIgnoredFields(): properly reset state of non first geometry fields (#6283) * Make OGRLayer::SetSpatialFilter(GetSpatialFilter()) work with non empty spatial filter (#6284) * OGRLayerDecorator: add missing CreateGeomField() * OGRLayer::Erase(): do not discard input geometries that have no intersection with method layer (#6322) * OGRLayer::Erase(): Speedup = ~70% * Add OGRPreparedGeometryContains() * Use prepared geometry intersects as pretest in layer Intersection, Union, and Identity methods if requested. Use prepared geometry containment as pretest in layer Intersection method if requested. * Bail out from layer algebra methods if GEOS calls fail and not SKIP_FAILURES. * OGR_G_SetPoints(): error out if padfX or padfY == NULL, do not change coordinate dimension to 3D when pabyZ == NULL, fix optimization on linestring to call setPoints() only if the strides are the ones of a double, not 0 as incorrectly done before (#6344) * OGRParseDate(): more strict validation to reject invalide dates (#6452) OGRSpatialReference: * Upgrade to EPSG database v8.8 * Add support for SCH (Spherical Cross-track Height) projection * Optimize reprojection typically between WGS84 based SRS and WebMercator * Correctly transform Mercator_2SP and _1SP to ESRI Mercator, and back from ESRI Mercator to Mercator_2SP (#4861) * No longer enforce C locale if running against latest proj that is locale safe (4.9.2 or later) * EPSGGetPCSInfo(): use pcs.override.csv in priority over pcs.csv to read projection name, UOM, UOMAngle, GeogCS, etc... (#6026) * morphToESRI(): use GCS_WGS_1972 as GCS name for EPSG:4322 (#6027) * morphToESRI(): use Mercator_Auxiliary_Sphere projection for EPSG:3857. morphFromESRI(): map Mercator_Auxiliary_Sphere to EPSG:3857 (#5924) * Align hard-coded WKT of well known GCS definitions of WGS84, WGS72, NAD27 and NAD83 with the WKT of their EPSG def (#6080) * morphFromESRI(): special case with PROJCS name 'WGS_84_Pseudo_Mercator' (#6134) * OSR C API: fix declarations of OSRSetAxes() and OSRSetWagner(), and add missing OSRSetHOMAC(), OSRSetMercator2SP() and OSRSetTPED() (#6183) * Recognize EPSG 9835 method (Lambert Cylindrical Equal Area (Ellipsoidal)), needed for EPSG:6933 PCS for example * importFromProj4/exportToProj4(): rework linear unit conversion between WKT name/values and proj4 unit name, and extend its scope in WKT to proj4 conversions * OSR ESRI .prj: add support for reading custom ellipsoid in Parameters line * on import of +proj=geos, if +sweep=x is used then store it as a proj4 extension node (#6030) Utilities: * ogrinfo / ogr2ogr: implement @filename syntax for -sql and -where * ogr2ogr: prevent the -gt setting from overriding transaction group size of 1 set by skipfailures earlier (#2409) * ogr2ogr: warn if -zfield field does not exist in source layer * ogr2ogr -skip: rollback dataset transaction in case of failure (#6328) * ogr2ogr: fix -append with a source dataset with a mix of existing and non existing layers in the target datasource (#6345) * ogr2ogr: imply quiet mode if /vsistdout/ is used as destination filename * ogr2ogr: make -dim and -nlt support measure geometry types CartoDB: * fix GetNextFeature() on a newly create table (#6109) * defer 'CartoDBfycation' at layer closing * optimize feature insertion with multiple rows INSERT CSV driver: * add editing capabilities of existing files * add X_POSSIBLE_NAMES, Y_POSSIBLE_NAMES, Z_POSSIBLE_NAMES, GEOM_POSSIBLE_NAMES and KEEP_GEOM_COLUMNS open options * add HEADERS open option to force OGR to handle numeric column names. (PR #63) * add EMPTY_STRING_AS_NULL=YES/NO open option * implement compatibility enhancements for GeoCSV specification (#5989) * fix detection of TAB delimiter in allCountries.csv when the first line has a comma (#6086) * fix issues with leading single quote, and missing first line after ResetReading(), when parsing allCountries.txt (#6087) * speed-up GetFeatureCount() on allCountries.txt * on CreateDataSource() with a .csv name, do not try to open other existing .csv files in the directory * make CreateGeomField() returns OGRERR_NONE in case of success instead of OGRERR_FAILURE (#6280) * avoid adding trailing comma in header line when writing 'WKT,a_single_field' DGN driver: * add partial 3D transformation support for cell headers DXF driver: * detect files without .dxf extension (#5994) * fix handling of ELLIPSE with Z extrusion axis = -1 (#5705) * take into account full definition of spline entity (degree, control points, weights and knots) when stroking splines (#6436) * better handling of various object coordinate systems found in dxf files for point, line, polyline, spline and ellipse entities. Add anchor position to text styles. Remove polygon/polyface mesh parsing from polyline entity (#6459) ElasticSearch driver: * use get /_stats instead of /_status for ElasticSearch 2.0 compatibility (#6346) FileGDB driver: * make CreateFeature() honour user set FID, and implement more fine grained transaction for Linux/Unix * give a hint of using FileGDB SDK 1.4 is FileGDB compression is used GeoJSON driver: * Add editing capabilities of existing files * Add ARRAY_AS_STRING=YES open option * Use '%.17g' formatting by default for floating-point numbers and add SIGNIFICANT_FIGURES layer creation option (#6291) * add a json_ex_get_object_by_path() function * fix crash on null / non-json object features (#6166) * serialize string values that are valid JSon dictionary or array as it (ie do not quote them) * make sure there's enough space to write the FeatureCollection bbox (#6262). Also avoid duplicating FeatureCollection bbox if source has one (trunk only) * Export POINT EMPTY as having a null geometry, instead as of being POINT(0 0) (#6349) * Do not 'promote' a null field to OFTString type if it had another type before (#6351) GME driver: * Driver removed. Maps Engine being shut down at the end of January 2016. GML driver: * VFR: add new attribute DatumVzniku (v1.6) * VFR: fix ST_UVOH type handling * VFR: fix ZpusobyOchrany attributes (data types and names) * VFR: fix CisloDomovni attributes (Integer->IntegerList) * VFR: fix TEA attributes of StavebniObjekty * add NAMESPACE_DECL=YES option to OGR_G_ExportToGMLEx() to add xmlns:gml=http://www.opengis.net/gml or http://www.opengis.net/gml/3.2 declaration; Also accept GML2 or GML32 as valid valiues for FORMAT option (#6214) * serialize in .gfs file the name of the geometry element when it is 'geometry' since this is a particular case (#6247) * fix logic error in BuildJointClassFromScannedSchema() (#6302) GPKG driver: * make it accept files with non standard extension if they still have the correct application_id (#6396); also accept the .gpkx extension that may be used for extended geopackages * emit warning when generating a database without .gpkg/.gpkx extension (#6396) * as GPKG 1.1 uses a different application_id, emit a more specific warning if the application id starts with GPxx (but is not GP10). Add GPKG_WARN_UNRECOGNIZED_APPLICATION_ID config option to avoid the warning * correct scope of gpkg_geom_XXXXX extensions to be read-write, and allow reading geometry types CURVE or SURFACE * avoid trying to insert a gpkg_geom_XXXX extension if already done (#6402) * writer: implement strategy to flush partial_tiles temporary database when it becomes too big (#6462) * writer: when writing to GoogleMapsCompatible tiling scheme, better deal with source rasters in EPSG:4326 with latitude = +/-90 (#6463) * fix generation of files with 1-band paletted input dataset. Also improve the logic to retrieve color palette when forcing BAND_COUNT=1 on opening (#6450) GPX driver: * fix crash when parsing a 'time' extension element at route/track level (2.0 regression, #6237) ILI driver: * ILI1: Support for Surface polygon rings spread over multiple geometry records * ILI1: add string TID support (https://github.com/OSGeo/gdal/pull/91) * Fix crash with models using types derived from INTERLIS * Fix memory leaks (#6178) JML driver: * remove arbitrary limitation preventing from reading geometries with <gml:coordinates> larger than 10 MB (#6338) KML driver: * fix crash on KML files without content but with nested folders (#6486) LIBKML driver: * for documents without folder, use document name when available as name of layer (#6409) Memory driver: * add support for sparse feature IDs * add ADVERTIZE_UTF8 layer creation option MITAB driver: * Add support for block sizes other than 512 bytes in .map files, for MapInfo 15.2 compatibility (#6298) * write correct datum id for EPSG:3857 * read MID files with TAB delimiter and empty first field (#5405) * use projection code 29 when exporting non-Polar Lambert Azimuthal Equal Area (#5220) * fix crashes when parsing invalid MIF geometries (#6273) MSSQLSpatial driver: * Implement MSSQL bulk insert (#4792) * do not treat a primary key that is not of integer type as the FID (#6235) NTF driver: * fix potential buffer overflows when reading too short lines (#6277) ODBC driver: * remove limitations to 500 columns ODS driver: * fix loss of precision in formula computation OpenFileGDB driver: * do not emit warning if SDC/CDF table detected and that FileGDB driver is present * fix min/max on columns without indices (#6150) * build correct geometry for a multi-part wkbMultiLineStringZ (#6332) * add support for reading SHPT_GENERALPOINT (#6478) OSM driver: * correct fields ids for the (non frequently used) Node message * do not override 'our' osm_id (the node, way or relation id) with a tag named 'osm_id' (#6347) * properly deal with polygons in other_relations geometrycollection (#6475) PG driver: * Add PRELUDE_STATEMENTS and CLOSING_STATEMENTS open option to be for example able to specify options, like statement_timeout, with pg_bouncer * Fix 2.0 regression when overwriting several existing PostGIS layers with ogr2ogr (#6018) * Update PG, PGDump and CartoDB drivers to correctly export POINT EMPTY for PostGIS 2.2 * avoid resetting error potentially emitted by ExecuteSQL() (#6194) * sanitize management of quoting for FID column at layer creation * fix to get SRID on result layer with PostGIS 2.2 * in copy mode (the default on layer creation), do not truncate the concatenated string list to the field width (#6356) * make such that GEOMETRY_NAME layer creation option is honoured in ogr2ogr when the source geometry field has a not-null constraint (#6366) * read and set DESCRIPTION metadata item from/into pg_description system table; add DESCRIPTION layer creation option * support int2[] and numeric[] types, better map float4[] type * remove code that was intended to handled binary cursors as it cannot be triggered * fix append of several layers in PG_USE_COPY mode and within transaction (ogr2ogr -append use case) (#6411) PGDump driver: * fix issue with case of ogc_fid field in case the FID layer creation option is not set by user or by ogr2ogr (related to #6232) * in copy mode (the default on layer creation), do not truncate the concatenated string list to the field width (#6356) * make such that GEOMETRY_NAME layer creation option is honoured in ogr2ogr when the source geometry field has a not-null constraint (#6366) * set DESCRIPTION metadata item from/into pg_description system table; add DESCRIPTION layer creation option Shapefile driver: * accept opening standalone .dbf files whose header length is not a multiple of 32 bytes (#6035) * fix REPACK crash on shapefile without .dbf (#6274) * add capability to restore/build a missing .shx file when defining SHAPE_RESTORE_SHX to TRUE (#5035) * avoid CreateLayer() to error out when passed wkbUnknown | wkb25D (#6473) SQLite/Spatialite driver: * support file:xxx URI syntax (derived from patch by joker99, #6150) * fix heuristics in OGRSQLiteSelectLayer::GetExtent() to not be used when there's a sub SELECT (#6062) * fix crash on GetLayerByName('non_existing_table(geom_column)') (#6103) * fix OGRSQLiteSelectLayerCommonBehaviour::GetBaseLayer() to no longer 'eat' consecutive characters in layer name (#6107) * Spatialite: turn debug messages warning about update not being supported because of missing or too old spatialite version as errors, and return NULL to the caller (#6199) * fix memleak in OGRSQLiteTableLayer destructor when updating geometry_columns_time * VFS: increase mxPathname to 2048 by default, and provide OGR_SQLITE_VFS_MAXPATHNAME config option to be able to configure that higher if that would be needed. Useful when dealing with very long names like /vsicurl/.... with AWS S3 security tokens * VFS: do not probe -wal files on /vsicurl/ SXF driver: * fix wrong use of endianness macros to fix behaviour on big-endian hots * add recoding from CP1251 for TEXT attribute that is now decoded * fix various issues (#6357) VRT driver: * implement CloseDependentDatasets() * fix editing with 'direct' geometry mode which could cause attribute column to be empty (#6289) * fix crash with a OGRVRTWarpedLayer using a source layer that would have non geometry column (unlikely to happen currently as this would require explicit disabling it, but more likey with following commit that createe VRT non-spatial layer implictly when the source is non-spatial, linked to #6336) * avoid creating an implicit wkbUnknown geometry field when the source has no geometry column and there's no XML elements related to geometry fields (#6336) XLSX driver: * fix reading sheets with more than > 26 columns and 'holes' (#6363) XPlane driver: * extend ICAO identifiers to 5 digits (#6003) == SWIG Language Bindings == All bindings: * add a options parameter to gdal.ReprojectImage() to pass warp options * Change ReadRaster and WriteRaster to use GIntBig and the *IOEx-methods * prevent NULL file pointer from being passed to VSIF*L functions * make gdal.Rename() accept Unicode strings * add SpatialReference.GetAxisName() and SpatialReference.GetAxisOrientation() (#6441) * add SpatialReference.GetAngularUnitsName() (#6445) Java bindings: * Fix typemap for input parameter of type GIntBig (fixes GetFeature(long), DeleteFeature(long), etc...) (#6464) * Bump minimal java version to 1.5 in case SWIG generates anotations (#6433, patch by Bas Couwenberg) * GNUmakefile: add -f in rm commands * GNUmakefile: add support for all hardening buildflags Perl bindings: * Fix #6050: string formatting in croak. * Perl Makefile.PL: add support for all hardening buildflags (#5998) * use strict and warnings in overridden constructors. * add $VERSION to ogr_perl.i (OGR.pm), which is required by pause.perl.org. * Add some basic module info for CPAN. * Bugfix for Geo::OGR::Feature->new(). * Add many utility level algorithms as methods to various classes. * New class for XML stuff * New Makefile.PL, which can download and build GDAL. This allows automatic testing of the CPAN module. * Wrap VSIStdoutSetRedirection and allow creating datasets via an object, which can write and close. * Geo::OGR::Driver and Geo::OGR::DataSource are now Perl wrappers for respective GDAL classes. * Add to the error stack also errors from the bindings * Fix sending utf8 from Perl to GDAL. Should also remove some "uninitialized value" warnings. * ReadTile and WriteTile methods for Dataset, ReadTile accepts now tile size and scaling algorithm. * Improved Parent - Child management. * Improved support for 64bit ints. * Measures support in Geometry class. * Many new tests Python bindings: * make Feature.ExportToJson() output boolean value for a boolean field * support floating point coordinates for the source windows of Band.ReadRaster() and Band.ReadAsArray() * fix build with SWIG 3.0.6 (#6045) * make gdal.OpenEx() throw a Python exception in case of failed open when exceptions are enables with gdal.UseExceptions() (#6075) * Disable opening a NumPy dataset with a filename returned by gdal_array.GetArrayFilename(() unless GDAL_ARRAY_OPEN_BY_FILENAME is set to TRUE * disable the warning about using deprecated wkb25DBit constant as it uses a trick that prevents the bindings from being used by py2exe / pyinstaller (#6364)
2018-02-19 13:05:17 +01:00
USE_LANGUAGES= c99 c++
USE_LIBTOOL= yes
USE_PKGLOCALEDIR= yes
GNU_CONFIGURE= yes
CONFIGURE_ARGS+= --with-geos
CONFIGURE_ARGS+= --with-libjson-c=internal
CONFIGURE_ARGS+= --with-libtiff=${BUILDLINK_PREFIX.tiff}
CONFIGURE_ARGS+= --with-libz=${BUILDLINK_PREFIX.zlib}/include
CONFIGURE_ARGS+= --with-netcdf=${BUILDLINK_PREFIX.netcdf}
CONFIGURE_ARGS+= --with-webp=${BUILDLINK_PREFIX.libwebp}
CONFIGURE_ARGS+= --with-xerces
CONFIGURE_ARGS+= --with-xerces-inc=${BUILDLINK_PREFIX.xerces-c}/include
CONFIGURE_ARGS+= --with-xerces-lib=-L${BUILDLINK_PREFIX.xerces-c}/lib\ -lxerces-c
gdal-lib: Force the use of openjpeg over jasper for JPEG 2000 support Add pkg-config to USE_TOOLS to ensure openjpeg gets detected properly. See https://github.com/mdadams/jasper/issues/208 bump PKGREVISION
2019-07-16 23:24:47 +02:00
# force OpenJPEG for JPEG 2000 support
CONFIGURE_ARGS+= --without-jasper
CONFIGURE_ARGS+= --with-openjpeg
Reference in a new issue Copy permalink