There was build problem after pkg-config was aded to USE_TOOLS if
PKG_DEVELOPER is enabled, causing WRKDIR reference.
Fix method is dirty (or quick) hack.
There was build problem after pkg-config was aded to USE_TOOLS if
PKG_DEVELOPER is enabled, causing WRKDIR reference.
Fix method is dirty (or quick) hack.
The null provider is a rather-unusual provider that has constructs that
intentionally do nothing. This may sound strange, and indeed these constructs
do not need to be used in most cases, but they can be useful in various
situations to help orchestrate tricky behavior or work around limitations.
v2.64.0
ENHANCEMENTS:
data-source/aws_directory_service_directory: connect_settings connect_ips attribute now set
resource/aws_directory_service_directory: connect_settings connect_ips attribute now set
resource/aws_iot_topic_rule: Add step_functions configuration block
resource/aws_ses_event_destination: Support resource import
BUG FIXES:
data-source/aws_elasticsearch_domain: processing is now correctly set
resource/aws_acm_certificate: Update pending DNS validation record creation time from 1 minute to 5 minutes (better support for certificates with high amount of Subject Alternative Names)
resource/aws_api_gateway_method_settings: settings now properly set
resource/aws_autoscaling_group: Ignore ordering differences for tags argument (prevent unexpected differences from version 2.63.0)
resource/aws_codebuild_project: Enable drift detection for environment_variable argument
resource/aws_codebuild_project: Prevent inconsistent final plan errors with source configuration block
resource/aws_ecs_task_definition: Ensure efs_volume_configuration changes are properly detected
resource/aws_lb_cookie_stickiness_policy: cookie_expiration_policy now properly set
resource/aws_lightsail_instance: ram_size now properly set
resource/aws_load_balancer_backend_server_policy: instance_port now properly set
resource/aws_load_balancer_listener_policy: load_balancer_port now properly set
resource/aws_opsworks_application: environment secure now properly set
resource/aws_security_group_rule: Correctly set description after state refresh when source_security_group_id refers to a security group across accounts
resource/aws_ses_active_receipt_rule_set: Recreate resource when destroyed outside of Terraform
resource/aws_ses_event_destination: Correctly refresh entire resource state (prevent unexpected differences from version 2.63.0 and properly perform drift detection)
resource/aws_ses_receipt_rule: Recreate resource when destroyed outside of Terraform
resource/aws_sns_topic: Attributes of type schema.TypeInt are now correctly set
v0.12.26
ENHANCEMENTS:
backend/remote: Can now accept -target options when creating a plan using remote operations, if supported by the target server. (Server-side support for this in Terraform Cloud and Terraform Enterprise will follow in forthcoming releases of each.)
cli: A special new lifecycle mode for provider plugins where they are assumed to be controlled by an external process outside of Terraform. This is for automated provider plugin testing only, and is not an end-user feature.
pkgsrc changes:
- Update patches due file renames
Changes:
1.52.0
------
Special thanks to Martin Michlmayr for proof reading and correcting all
the docs and Edward Barker for helping re-write the front page.
* New backends
* Tardigrade backend for use with storj.io (Caleb Case)
* Union re-write to have multiple writable remotes (Max Sum)
* Seafile for Seafile server (Fred @creativeprojects)
* New commands
* backend: command for backend specific commands (see backends)
(Nick Craig-Wood)
* cachestats: Deprecate in favour of `rclone backend stats cache:`
(Nick Craig-Wood)
* dbhashsum: Deprecate in favour of `rclone hashsum DropboxHash` (Nick Craig-Wood)
* New Features
* Add `--header-download` and `--header-upload` flags for setting
HTTP headers when uploading/downloading (Tim Gallant)
* Add `--header` flag to add HTTP headers to every HTTP transaction
(Nick Craig-Wood)
* Add `--check-first` to do all checking before starting transfers
(Nick Craig-Wood)
* Add `--track-renames-strategy` for configurable matching criteria for
`--track-renames` (Bernd Schoolmann)
* Add `--cutoff-mode` hard,soft,catious (Shing Kit Chan & Franklyn Tackitt)
* Filter flags (eg `--files-from -`) can read from stdin (fishbullet)
* Add `--error-on-no-transfer` option (Jon Fautley)
* Implement `--order-by xxx,mixed` for copying some small and some big
files (Nick Craig-Wood)
* Allow `--max-backlog` to be negative meaning as large as possible
(Nick Craig-Wood)
* Added `--no-unicode-normalization` flag to allow Unicode filenames to
remain unique (Ben Zenker)
* Allow `--min-age`/`--max-age` to take a date as well as a duration
(Nick Craig-Wood)
* Add rename statistics for file and directory renames (Nick Craig-Wood)
* Add statistics output to JSON log (reddi)
* Make stats be printed on non-zero exit code (Nick Craig-Wood)
* When running `--password-command` allow use of stdin (Sébastien Gross)
* Stop empty strings being a valid remote path (Nick Craig-Wood)
* accounting: support WriterTo for less memory copying (Nick Craig-Wood)
* build
* Update to use go1.14 for the build (Nick Craig-Wood)
* Add `-trimpath` to release build for reproduceable builds
(Nick Craig-Wood)
* Remove GOOS and GOARCH from Dockerfile (Brandon Philips)
* config
* Fsync the config file after writing to save more reliably
(Nick Craig-Wood)
* Add `--obscure` and `--no-obscure` flags to `config create`/`update`
(Nick Craig-Wood)
* Make `config show` take `remote:` as well as `remote` (Nick Craig-Wood)
* copyurl: Add `--no-clobber` flag (Denis)
* delete: Added `--rmdirs` flag to delete directories as well (Kush)
* filter: Added `--files-from-raw` flag (Ankur Gupta)
* genautocomplete: Add support for fish shell (Matan Rosenberg)
* log: Add support for syslog LOCAL facilities (Patryk Jakuszew)
* lsjson: Add `--hash-type` parameter and use it in lsf to speed up
hashing (Nick Craig-Wood)
* rc
* Add `-o`/`--opt` and `-a`/`--arg` for more structured input
(Nick Craig-Wood)
* Implement `backend/command` for running backend specific commands
remotely (Nick Craig-Wood)
* Add `mount/mount` command for starting `rclone mount` via the
API (Chaitanya)
* rcd: Add Prometheus metrics support (Gary Kim)
* serve http
* Added a `--template` flag for user defined markup (calistri)
* Add Last-Modified headers to files and directories (Nick Craig-Wood)
* serve sftp: Add support for multiple host keys by repeating `--key` flag
(Maxime Suret)
* touch: Add `--localtime` flag to make `--timestamp` localtime not UTC
(Nick Craig-Wood)
* Bug Fixes
* accounting
* Restore "Max number of stats groups reached" log line (Michał Matczuk)
* Correct exitcode on Transfer Limit Exceeded flag. (Anuar Serdaliyev)
* Reset bytes read during copy retry (Ankur Gupta)
* Fix race clearing stats (Nick Craig-Wood)
* copy: Only create empty directories when they don't exist on the remote
(Ishuah Kariuki)
* dedupe: Stop dedupe deleting files with identical IDs (Nick Craig-Wood)
* oauth
* Use custom http client so that `--no-check-certificate` is honored by
oauth token fetch (Mark Spieth)
* Replace deprecated oauth2.NoContext (Lars Lehtonen)
* operations
* Fix setting the timestamp on Windows for multithread copy
(Nick Craig-Wood)
* Make rcat obey `--ignore-checksum` (Nick Craig-Wood)
* Make `--max-transfer` more accurate (Nick Craig-Wood)
* rc
* Fix dropped error (Lars Lehtonen)
* Fix misplaced http server config (Xiaoxing Ye)
* Disable duplicate log (ElonH)
* serve dlna
* Cds: don't specify childCount at all when unknown (Dan Walters)
* Cds: use modification time as date in dlna metadata (Dan Walters)
* serve restic: Fix tests after restic project removed vendoring
(Nick Craig-Wood)
* sync
* Fix incorrect "nothing to transfer" message using `--delete-before`
(Nick Craig-Wood)
* Only create empty directories when they don't exist on the remote
(Ishuah Kariuki)
* Mount
* Add `--async-read` flag to disable asynchronous reads (Nick Craig-Wood)
* Ignore `--allow-root` flag with a warning as it has been removed upstream
(Nick Craig-Wood)
* Warn if `--allow-non-empty` used on Windows and clarify docs
(Nick Craig-Wood)
* Constrain to go1.13 or above otherwise bazil.org/fuse fails to compile
(Nick Craig-Wood)
* Fix fail because of too long volume name (evileye)
* Report 1PB free for unknown disk sizes (Nick Craig-Wood)
* Map more rclone errors into file systems errors (Nick Craig-Wood)
* Fix disappearing cwd problem (Nick Craig-Wood)
* Use ReaddirPlus on Windows to improve directory listing performance (Nick Craig-Wood)
* Send a hint as to whether the filesystem is case insensitive or not (Nick Craig-Wood)
* Add rc command `mount/types` (Nick Craig-Wood)
* Change maximum leaf name length to 1024 bytes (Nick Craig-Wood)
* VFS
* Add `--vfs-read-wait` and `--vfs-write-wait` flags to control time
waiting for a sequential read/write (Nick Craig-Wood)
* Change default `--vfs-read-wait` to 20ms (it was 5ms and not configurable)
(Nick Craig-Wood)
* Make `df` output more consistent on a rclone mount. (Yves G)
* Report 1PB free for unknown disk sizes (Nick Craig-Wood)
* Fix race condition caused by unlocked reading of Dir.path (Nick Craig-Wood)
* Make File lock and Dir lock not overlap to avoid deadlock (Nick Craig-Wood)
* Implement lock ordering between File and Dir to eliminate deadlocks
(Nick Craig-Wood)
* Factor the vfs cache into its own package (Nick Craig-Wood)
* Pin the Fs in use in the Fs cache (Nick Craig-Wood)
* Add SetSys() methods to Node to allow caching stuff on a node
(Nick Craig-Wood)
* Ignore file not found errors from Hash in Read.Release (Nick Craig-Wood)
* Fix hang in read wait code (Nick Craig-Wood)
* Local
* Speed up multi thread downloads by using sparse files on Windows
(Nick Craig-Wood)
* Implement `--local-no-sparse` flag for disabling sparse files
(Nick Craig-Wood)
* Implement `rclone backend noop` for testing purposes (Nick Craig-Wood)
* Fix "file not found" errors on post transfer Hash calculation
(Nick Craig-Wood)
* Cache
* Implement `rclone backend stats` command (Nick Craig-Wood)
* Fix Server Side Copy with Temp Upload (Brandon McNama)
* Remove Unused Functions (Lars Lehtonen)
* Disable race tests until bbolt is fixed (Nick Craig-Wood)
* Move methods used for testing into test file (greatroar)
* Add Pin and Unpin and canonicalised lookup (Nick Craig-Wood)
* Use proper import path go.etcd.io/bbolt (Robert-André Mauchin)
* Crypt
* Calculate hashes for uploads from local disk (Nick Craig-Wood)
* This allows crypted Jottacloud uploads without using local disk
* This means crypted s3/b2 uploads will now have hashes
* Added `rclone backend decode`/`encode` commands to replicate functionality
of `cryptdecode` (Anagh Kumar Baranwal)
* Get rid of the unused Cipher interface as it obfuscated the code
(Nick Craig-Wood)
* Azure Blob
* Implement streaming of unknown sized files so `rcat` is now supported
(Nick Craig-Wood)
* Implement memory pooling to control memory use (Nick Craig-Wood)
* Add `--azureblob-disable-checksum` flag (Nick Craig-Wood)
* Retry `InvalidBlobOrBlock` error as it may indicate block concurrency
problems (Nick Craig-Wood)
* Remove unused `Object.parseTimeString()` (Lars Lehtonen)
* Fix permission error on SAS URL limited to container (Nick Craig-Wood)
* B2
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Ignore directory markers at the root also (Nick Craig-Wood)
* Force the case of the SHA1 to lowercase (Nick Craig-Wood)
* Remove unused `largeUpload.clearUploadURL()` (Lars Lehtonen)
* Box
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Implement About to read size used (Nick Craig-Wood)
* Add token renew function for jwt auth (David Bramwell)
* Added support for interchangeable root folder for Box backend (Sunil Patra)
* Remove unnecessary iat from jws claims (David)
* Drive
* Follow shortcuts by default, skip with `--drive-skip-shortcuts`
(Nick Craig-Wood)
* Implement `rclone backend shortcut` command for creating shortcuts
(Nick Craig-Wood)
* Added `rclone backend` command to change `service_account_file` and
`chunk_size` (Anagh Kumar Baranwal)
* Fix missing files when using `--fast-list` and `--drive-shared-with-me`
(Nick Craig-Wood)
* Fix duplicate items when using `--drive-shared-with-me` (Nick Craig-Wood)
* Extend `--drive-stop-on-upload-limit` to respond to
`teamDriveFileLimitExceeded`. (harry)
* Don't delete files with multiple parents to avoid data loss
(Nick Craig-Wood)
* Server side copy docs use default description if empty (Nick Craig-Wood)
* Dropbox
* Make error insufficient space to be fatal (harry)
* Add info about required redirect url (Elan Ruusamäe)
* Fichier
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Implement custom pacer to deal with the new rate limiting (buengese)
* FTP
* Fix lockup when using concurrency limit on failed connections
(Nick Craig-Wood)
* Fix lockup on failed upload when using concurrency limit (Nick Craig-Wood)
* Fix lockup on Close failures when using concurrency limit (Nick Craig-Wood)
* Work around pureftp sending spurious 150 messages (Nick Craig-Wood)
* Google Cloud Storage
* Add support for `--header-upload` and `--header-download` (Nick Craig-Wood)
* Add `ARCHIVE` storage class to help (Adam Stroud)
* Ignore directory markers at the root (Nick Craig-Wood)
* Googlephotos
* Make the start year configurable (Daven)
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Create feature/favorites directory (Brandon Philips)
* Fix "concurrent map write" error (Nick Craig-Wood)
* Don't put an image in error message (Nick Craig-Wood)
* HTTP
* Improved directory listing with new template from Caddy project (calisro)
* Jottacloud
* Implement `--jottacloud-trashed-only` (buengese)
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Use `RawURLEncoding` when decoding base64 encoded login token (buengese)
* Implement cleanup (buengese)
* Update docs regarding cleanup, removed remains from old auth, and added
warning about special mountpoints. (albertony)
* Mailru
* Describe 2FA requirements (valery1707)
* Onedrive
* Implement `--onedrive-server-side-across-configs` (Nick Craig-Wood)
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Fix occasional 416 errors on multipart uploads (Nick Craig-Wood)
* Added maximum chunk size limit warning in the docs (Harry)
* Fix missing drive on config (Nick Craig-Wood)
* Make error `quotaLimitReached` to be fatal (harry)
* Opendrive
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Pcloud
* Added support for interchangeable root folder for pCloud backend
(Sunil Patra)
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Fix initial config "Auth state doesn't match" message (Nick Craig-Wood)
* Premiumizeme
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Prune unused functions (Lars Lehtonen)
* Putio
* Add support for `--header-upload` and `--header-download` (Nick Craig-Wood)
* Make downloading files use the rclone http Client (Nick Craig-Wood)
* Fix parsing of remotes with leading and trailing / (Nick Craig-Wood)
* Qingstor
* Make `rclone cleanup` remove pending multipart uploads older than 24h
(Nick Craig-Wood)
* Try harder to cancel failed multipart uploads (Nick Craig-Wood)
* Prune `multiUploader.list()` (Lars Lehtonen)
* Lint fix (Lars Lehtonen)
* S3
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Use memory pool for buffer allocations (Maciej Zimnoch)
* Add SSE-C support for AWS, Ceph, and MinIO (Jack Anderson)
* Fail fast multipart upload (Michał Matczuk)
* Report errors on bucket creation (mkdir) correctly (Nick Craig-Wood)
* Specify that Minio supports URL encoding in listings (Nick Craig-Wood)
* Added 500 as retryErrorCode (Michał Matczuk)
* Use `--low-level-retries` as the number of SDK retries
(Aleksandar Janković)
* Fix multipart abort context (Aleksandar Jankovic)
* Replace deprecated `session.New()` with `session.NewSession()`
(Lars Lehtonen)
* Use the provided size parameter when allocating a new memory pool
(Joachim Brandon LeBlanc)
* Use rclone's low level retries instead of AWS SDK to fix listing retries
(Nick Craig-Wood)
* Ignore directory markers at the root also (Nick Craig-Wood)
* Use single memory pool (Michał Matczuk)
* Do not resize buf on put to memBuf (Michał Matczuk)
* Improve docs for `--s3-disable-checksum` (Nick Craig-Wood)
* Don't leak memory or tokens in edge cases for multipart upload
(Nick Craig-Wood)
* Seafile
* Implement 2FA (Fred)
* SFTP
* Added `--sftp-pem-key` to support inline key files (calisro)
* Fix post transfer copies failing with 0 size when using
`set_modtime=false` (Nick Craig-Wood)
* Sharefile
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Sugarsync
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Swift
* Add support for `--header-upload` and `--header-download` (Nick Craig-Wood)
* Fix cosmetic issue in error message (Martin Michlmayr)
* Union
* Implement multiple writable remotes (Max Sum)
* Fix server-side copy (Max Sum)
* Implement ListR (Max Sum)
* Enable ListR when upstreams contain local (Max Sum)
* WebDAV
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
* Fix `X-OC-Mtime` header for Transip compatibility (Nick Craig-Wood)
* Report full and consistent usage with `about` (Yves G)
* Yandex
* Add support for `--header-upload` and `--header-download` (Tim Gallant)
Changelog:
Monday, May 25, 2020
Bugfixes:
Old ZSK can be withdrawn too early during a ZSK rollover if maximum zone TTL is computed automatically
Server responds SERVFAIL to ANY queries on empty non-terminal nodes
Improvements:
Also module onlinesign returns minimized responses to ANY queries
Linking against libcap-ng can be disabled via a configure option
1.18.67
api-change:quicksight: Update quicksight command to latest version
api-change:macie: Update macie command to latest version
api-change:elasticache: Update elasticache command to latest version
api-change:ec2: Update ec2 command to latest version
api-change:dlm: Update dlm command to latest version
api-change:ssm: Update ssm command to latest version
1.18.66
api-change:iotsitewise: Update iotsitewise command to latest version
api-change:autoscaling: Update autoscaling command to latest version
1.18.65
api-change:codebuild: Update codebuild command to latest version
api-change:s3: Update s3 command to latest version
api-change:ec2: Update ec2 command to latest version
api-change:synthetics: Update synthetics command to latest version
1.18.64
api-change:codedeploy: Update codedeploy command to latest version
api-change:securityhub: Update securityhub command to latest version
api-change:chime: Update chime command to latest version
api-change:medialive: Update medialive command to latest version
api-change:backup: Update backup command to latest version
api-change:application-autoscaling: Update application-autoscaling command to latest version
api-change:appmesh: Update appmesh command to latest version
1.18.63
api-change:health: Update health command to latest version
bugfix:s3: Mute warnings for not restored glacier deep archive objects if --ignore-glacier-warnings option enabled. Fixes#4039
api-change:transcribe: Update transcribe command to latest version
api-change:ec2: Update ec2 command to latest version
api-change:chime: Update chime command to latest version
1.18.62
api-change:qldb: Update qldb command to latest version
api-change:ecs: Update ecs command to latest version
api-change:dynamodb: Update dynamodb command to latest version
api-change:macie2: Update macie2 command to latest version
api-change:chime: Update chime command to latest version
api-change:ec2: Update ec2 command to latest version
1.18.61
api-change:ecr: Update ecr command to latest version
api-change:glue: Update glue command to latest version
api-change:cloudformation: Update cloudformation command to latest version
api-change:sts: Update sts command to latest version
1.18.60
api-change:imagebuilder: Update imagebuilder command to latest version
api-change:ec2: Update ec2 command to latest version
1.18.59
api-change:elasticache: Update elasticache command to latest version
api-change:macie2: Update macie2 command to latest version
1.18.58
api-change:iotsitewise: Update iotsitewise command to latest version
api-change:workmail: Update workmail command to latest version
1.13.17
api-change:elasticache: [botocore] Update elasticache client to latest version
api-change:dlm: [botocore] Update dlm client to latest version
api-change:quicksight: [botocore] Update quicksight client to latest version
api-change:ssm: [botocore] Update ssm client to latest version
api-change:ec2: [botocore] Update ec2 client to latest version
api-change:macie: [botocore] Update macie client to latest version
1.13.16
api-change:autoscaling: [botocore] Update autoscaling client to latest version
api-change:iotsitewise: [botocore] Update iotsitewise client to latest version
1.13.15
api-change:synthetics: [botocore] Update synthetics client to latest version
api-change:codebuild: [botocore] Update codebuild client to latest version
api-change:s3: [botocore] Update s3 client to latest version
api-change:ec2: [botocore] Update ec2 client to latest version
1.13.14
api-change:backup: [botocore] Update backup client to latest version
api-change:codedeploy: [botocore] Update codedeploy client to latest version
api-change:securityhub: [botocore] Update securityhub client to latest version
api-change:chime: [botocore] Update chime client to latest version
api-change:medialive: [botocore] Update medialive client to latest version
api-change:application-autoscaling: [botocore] Update application-autoscaling client to latest version
api-change:appmesh: [botocore] Update appmesh client to latest version
1.13.13
api-change:transcribe: [botocore] Update transcribe client to latest version
api-change:ec2: [botocore] Update ec2 client to latest version
api-change:health: [botocore] Update health client to latest version
api-change:chime: [botocore] Update chime client to latest version
1.13.12
api-change:chime: [botocore] Update chime client to latest version
api-change:qldb: [botocore] Update qldb client to latest version
api-change:ec2: [botocore] Update ec2 client to latest version
api-change:ecs: [botocore] Update ecs client to latest version
api-change:dynamodb: [botocore] Update dynamodb client to latest version
api-change:macie2: [botocore] Update macie2 client to latest version
1.13.11
api-change:sts: [botocore] Update sts client to latest version
api-change:ecr: [botocore] Update ecr client to latest version
api-change:glue: [botocore] Update glue client to latest version
api-change:cloudformation: [botocore] Update cloudformation client to latest version
1.13.10
api-change:ec2: [botocore] Update ec2 client to latest version
api-change:imagebuilder: [botocore] Update imagebuilder client to latest version
1.13.9
api-change:elasticache: [botocore] Update elasticache client to latest version
api-change:macie2: [botocore] Update macie2 client to latest version
1.13.8
api-change:workmail: [botocore] Update workmail client to latest version
api-change:iotsitewise: [botocore] Update iotsitewise client to latest version
enchancement:Endpoints: [botocore] Improved endpoint resolution for clients with unknown regions
1.16.17
api-change:elasticache: Update elasticache client to latest version
api-change:dlm: Update dlm client to latest version
api-change:quicksight: Update quicksight client to latest version
api-change:ssm: Update ssm client to latest version
api-change:ec2: Update ec2 client to latest version
api-change:macie: Update macie client to latest version
1.16.16
api-change:autoscaling: Update autoscaling client to latest version
api-change:iotsitewise: Update iotsitewise client to latest version
1.16.15
api-change:synthetics: Update synthetics client to latest version
api-change:codebuild: Update codebuild client to latest version
api-change:s3: Update s3 client to latest version
api-change:ec2: Update ec2 client to latest version
1.16.14
api-change:backup: Update backup client to latest version
api-change:codedeploy: Update codedeploy client to latest version
api-change:securityhub: Update securityhub client to latest version
api-change:chime: Update chime client to latest version
api-change:medialive: Update medialive client to latest version
api-change:application-autoscaling: Update application-autoscaling client to latest version
api-change:appmesh: Update appmesh client to latest version
1.16.13
api-change:transcribe: Update transcribe client to latest version
api-change:ec2: Update ec2 client to latest version
api-change:health: Update health client to latest version
api-change:chime: Update chime client to latest version
1.16.12
api-change:chime: Update chime client to latest version
api-change:qldb: Update qldb client to latest version
api-change:ec2: Update ec2 client to latest version
api-change:ecs: Update ecs client to latest version
api-change:dynamodb: Update dynamodb client to latest version
api-change:macie2: Update macie2 client to latest version
1.16.11
api-change:sts: Update sts client to latest version
api-change:ecr: Update ecr client to latest version
api-change:glue: Update glue client to latest version
api-change:cloudformation: Update cloudformation client to latest version
1.16.10
api-change:ec2: Update ec2 client to latest version
api-change:imagebuilder: Update imagebuilder client to latest version
1.16.9
api-change:elasticache: Update elasticache client to latest version
api-change:macie2: Update macie2 client to latest version
1.16.8
api-change:workmail: Update workmail client to latest version
api-change:iotsitewise: Update iotsitewise client to latest version
enchancement:Endpoints: Improved endpoint resolution for clients with unknown regions
This had a SUBST block to fix /usr/share to ${PREFIX}/share. It was
doing this to files where the pattern didn't match, and also missed
fixin up /usr/bin. Split into a bin and a share SUBST, and loosen the
regexp (specifically, to match /syncthing rather than just
/syncthing-gtk). Tested to basically work on NetBSD 8. Desktop files
etc. appear much better than before.
(Found by newish SUBST noop check.)
0.26.3
Improved readability of logged incoming data.
Threads are given unique names now to aid debugging.
Fixed a regression where get_service_info() called within a listener add_service method would deadlock, timeout and incorrectly return None.
This module allows you to perform IP subnet calculations, there is support for
both IPv4 and IPv6 CIDR notation.
This is required by dane(1) from security/sshfp.
dnspython is a DNS toolkit for Python. It supports almost all record types. It
can be used for queries, zone transfers, and dynamic updates. It supports TSIG
authenticated messages and EDNS0.
dnspython provides both high and low level access to DNS. The high level
classes perform queries for data of a given name, type, and class, and return
an answer set. The low level classes allow direct manipulation of DNS zones,
messages, names, and records.
0.8.0:
[FEATURE] Added ASGI application
[FEATURE] Add support for parsing timestamps in Prometheus exposition format.
[FEATURE] Add target_info to registries
[ENHANCEMENT] Handle empty and slashes in label values for pushgateway
[ENHANCEMENT] Various updates for latest OpenMetrics draft spec
[ENHANCEMENT] Add HELP output for auto-created metrics
[ENHANCEMENT] Use mmap.PAGESIZE constant as value for first read.
[ENHANCEMENT] Add __repr__ method to metric objects, make them debug friendly.
[ENHANCEMENT] Add observability check to metrics
[BUGFIX] Fix urlparse in python >= 3.7.6
[BUGFIX] Cleaning up name before appending unit on name
[BUGFIX] Allow for OSError on Google App Engine
ntopng 4.0:
Breakthroughs
* Plugins engine to tap into flows, hosts and other network elements
* Migration to Bootstrap 4 and Font Awesome 5 for a renewed ntopng look-and-feel with light and dark themes
* Processes and containers monitoring thanks to the eBPF integration via libebpfflow https://github.com/ntop/libebpfflow
* Active monitoring of hosts ICMP/ICMPv6/HTTP/HTTPS Round Trip Times (RTT)
New features
* X.509 client certificate authentication
* ERSPAN transparent ethernet bridging
* Webhook export module for exporting alarms
* Identifications of the hosts in broadcast domain
* Category Lists editor to manage ip/domain lists
* Handling of PEN fields from nProbe
* Added anomalous flows to the looking glass
* Visibility of ICMP port-unreachable flows IPv4
* TCP states filtering (est., connecting, closed and rst)
* Ability to serialize local hosts in the broadcast domain via MAC address
* Japanese, portugese/brazilian localization
* Added process memory, cpu load, InfluxDB, Redis status pages and charts
* Implement ntopng Plugins, self contained modules to extend the ntopng functionalities
* Implement ZMQ/Suricata companion interface
* SSL traffic analysis and alerts via JA3 fingerprint, unsafe ciphers detection
* SSH traffic analysis and alerts via HASSH fingerprint
* Host traffic profile generation via the (MUD) Manufacturer Usage Descriptor
* Experimental Prometheus timeseries export
* Introduce the System interface to manage system wide settings and status
* Read events from Suricata and generate alerts
* SNMP network topology visualization
* Automatic ntopng update check and upgrade
* Calculate host anomaly score and trigger alerts when it exceeds a threshold
* Add ability to extract timeseries data with a click
* Initial Marketplace droplet using Fabric
* Alerts on duplex status change on SNMP interface
Improvements
* View interfaces are now optimized for big networks and use less memory
* Systemd macros are now used to start/restart the ntopng services
* Handles n2disk traffic extractions from recording processes non managed by ntopng
* Interface in/out now available also for non PF_RING interfaces (read from /proc)
* Automatic InfluxDB rollup support
* MDNS discovery improvements
* Rework of the alerts engine and api for efficient engaged alerts triggering
* Faster ZMQ communication to nProbe thanks to the implementation of a binary TLV format
* Stats update for ZMQ interfaces is now based on the idle/active flows timeout
* Timeseries export improvements via queues, detect if InfluxDB is down and stop the export
* Implemented reusable Lua engine to reduce the overhead of periodic scripts
* Improve Lua error handling
* Exclude certain categories from Elephant/Long lived flows alerts
nEdge
* Ability to set up port forwarding
* Support for Ubuntu 18.04
* Fix users and other prefs deleted during nEdge data reset
* Japanese localization
* Block unsupported L3 protocols (currently only ARP and IPv4 are supported)
* DNS mapping port to avoid conflicts with system programs
Fixes
* Fixed export to mysql on shutdown in case of Pcap file in community mode
* Fixed failing SYN-scan detection
* Fixed ZMQ decompression errors with large templates
* Fixed possible XSS in login.lua referer param and `runtime.lua`
* Update geolocation due to changes in the library usage policy
* Fixes to support browsers dark mode
* Option `--zmq-encryption-key <pub key>` can be used with `-I <endpoint>` to encrypt data hi hierarchical mode
* Fixed nIndex missing data while performing some queries and throughput calculation
nDPI 3.2:
New Features
* New API calls
* Protocol detection: ndpi_is_protocol_detected
* Categories: ndpi_load_categories_file / ndpi_load_category
* JSON/TLV serialization: ndpi_serialize_string_boolean / ndpi_serialize_uint32_boolean
* Patricia tree: ndpi_load_ipv4_ptree
* Module initialization: ndpi_init_detection_module / ndpi_finalize_initalization
* Base64 encoding: ndpi_base64_encode
* JSON exprot: ndpi_flow2json
* Print protocol: ndpi_get_l4_proto_name / ndpi_get_l4_proto_info
* Libfuzz integration
* Implemented Community ID hash (API call ndpi_flowv6_flow_hash and ndpi_flowv4_flow_hash)
* Detection of RCE in HTTP GET requests via PCRE
* Integration of the libinjection library to detect SQL injections and XSS type attacks in HTTP requests
New Supported Protocols and Services
* TLS
* Added ALPN support
* Added export of supported version in TLS header
* Added Telnet dissector with metadata extraction
* Added Zabbix dissector
* Added POP3/IMAP metadata extraction
* Added FTP user/password extraction
* Added NetBIOS metadata extraction
* Added Kerberos metadata extraction
* Implemented SQL Injection and XSS attack detection
* Host-based detection improvements and changes
* Added Microsoft range
* Added twitch.tv website
* Added brasilbandalarga.com.br and .eaqbr.com.br as EAQ
* Added 20.180.0.0/14, 20.184.0.0/13 range as Skype
* Added 52.84.0.0/14 range as Amazon
* Added ^pastebin.com
* Changed 13.64.0.0/11 range from Skype to Microsoft
* Refreshed Whatsapp server list, added *whatsapp-*.fbcdn.net IPs
* Added public DNSoverHTTPS servers
Improvements
* Reworked and improved the TLS dissector
* Reworked Kerberos dissector
* Improved DNS response decoding
* Support for DNS continuous flow dissection
* Improved Python bindings
* Improved Ethereum support
* Improved categories detection with streaming and HTTP
* Support for IP-based detection to compute the application protocol
* Renamed protocol 104 to IEC60870 (more meaningful)
* Added failed authentication support with FTP
* Renamed DNSoverHTTPS to handle bot DoH and DoT
* Implemented stacked DPI decoding
* Improvements for CapWAP and Bloomberg
* Improved SMB dissection
* Improved SSH dissection
* Added capwap support
* Modified API signatures for ndpi_ssl_version2str / ndpi_detection_giveup
* Removed ndpi_pref_http_dont_dissect_response / ndpi_pref_dns_dont_dissect_response (replaced by ndpi_extra_dissection_possible)
Fixes
* Fixed memory invalid access in SMTP and leaks in TLS
* Fixed a few memory leaks
* Fixrd invalid memory access in a few protocol dissectors (HTTP, memcached, Citrix, STUN, DNS, Amazon Video, TLS, Viber)
* Fixed IPv6 address format across the various platforms/distributions
* Fixed infinite loop in ndpi_workflow_process_packet
* Fixed SHA1 certificate detection
* Fixed custom protocol detection
* Fixed SMTP dissection (including email)
* Fixed Telnet dissection and invalid password report
* Fixed invalid category matching in HTTP
* Fixed Skype and STUN false positives
* Fixed SQL Injection detection
* Fixed invalid SMBv1 detection
* Fixed SSH dissection
* Fixed ndpi_ssl_version2str
* Fixed ndpi_extra_dissection_possible
* Fixed out of bounds read in ndpi_match_custom_category
Misc
* ndpiReader
* CSV output enhancements
* Added tunnelling decapsulation
* Improved HTTP reporting
nDPI 3.0:
New Features
* nDPI now reports the protocol ASAP even when specific fields have not yet been dissected because such packets have not yet been observed. This is important for inline applications that can immediately act on traffic. Applications that need full dissection need to call the new API function ndpi_extra_dissection_possible() to check if metadata dissection has been completely performed or if there is more to read before declaring it completed.
* TLS (formerly identified as SSL in nDPI v2.x) is now dissected more deeply, certificate validity is extracted as well certificate SHA-1.
* nDPIreader can now export data in CSV format with option `-C`
* Implemented Sequence of Packet Length and Time (SPLT) and Byte Distribution (BD) as specified by Cisco Joy (https://github.com/cisco/joy). This allows malware activities on encrypted TLS streams. Read more at https://blogs.cisco.com/security/detecting-encrypted-malware-traffic-without-decryption
* Available as library and in `ndpiReader` with option `-J`
* Promoted usage of protocol categories rather than protocol identifiers in order to classify protocols. This allows application protocols to be clustered in families and thus better managed by users/developers rather than using hundred of protocols unknown to most of the people.
* Added Inter-Arrival Time (IAT) calculation used to detect protocol misbehaviour (e.g. slow-DoS detection)
* Added data analysis features for computign metrics such as entropy, average, stddev, variance on a single and consistent place that will prevent when possible. This should ease traffic analysis on monitoring/security applications. New API calls have been implemented such as ndpi_data_XXX() to handle these calculations.
* Initial release of Python bindings available under nDPI/python.
* Implemented search of human readable strings for promoting data exfiltration detection
* Available as library and in `ndpiReader` with option `-e`
* Fingerprints
* JA3 (https://github.com/salesforce/ja3)
* HASSH (https://github.com/salesforce/hassh)
* DHCP
* Implemented a library to serialize/deserialize data in both Type-Length-Value (TLV) and JSON format
* Used by nProbe/ntopng to exchange data via ZMQ
New Supported Protocols and Services
* DTLS (i.e. TLS over UDP)
* Hulu
* TikTok/Musical.ly
* WhatsApp Video
* DNSoverHTTPS
* Datasaver
* Line protocol
* Google Duo and Hangout merged
* WireGuard VPN
* IMO
* Zoom.us
Improvements
* TLS
* Organizations
* Ciphers
* Certificate analysis
* Added PUBLISH/SUBSCRIBE methods to SIP
* Implemented STUN cache to enhance matching of STUN-based protocols
* Dissection improvements
* Viber
* WhatsApp
* AmazonVideo
* SnapChat
* FTP
* QUIC
* OpenVPN support for UDP-based VPNs
* Facebook Messenger mobile
* Various improvements for STUN, Hangout and Duo
* Added new categories: CUSTOM_CATEGORY_ANTIMALWARE, NDPI_PROTOCOL_CATEGORY_MUSIC, NDPI_PROTOCOL_CATEGORY_VIDEO, NDPI_PROTOCOL_CATEGORY_SHOPPING, NDPI_PROTOCOL_CATEGORY_PRODUCTIVITY and NDPI_PROTOCOL_CATEGORY_FILE_SHARING
* Added NDPI_PROTOCOL_DANGEROUS classification
Fixes
* Fixed the dissection of certain invalid DNS responses
* Fixed Spotify dissection
* Fixed false positives with FTP and FTP_DATA
* Fix to discard STUN over TCP flows
* Fixed MySQL dissector
* Fix category detection due to missing initialization
* Fix DNS rsp_addr missing in some tiny responses
* Various hardening fixes
