Changes:
1.13.0
======
Additions
---------
- Support for
- `furaffinity` (#284)
- `8kun` (#582)
- `bcy` (#592)
- [blogger] implement video extraction (#587)
- [oauth] add option to specify port number used by local server (#604)
- [pixiv] add `rating` metadata field (#595)
- [pixiv] recognize tags at the end of new bookmark URLs
- [reddit] add `videos` option
- [weibo] use youtube-dl to download from m3u8 manifests
- implement `parent-directory` option (#551)
- extend filename formatting capabilities:
- implement field name alternatives (#525)
- allow multiple "special" format specifiers per replacement field (#595)
- allow for numeric list and string indices
Changes
-------
- [reddit] handle reddit-hosted images and videos natively (#551)
- [twitter] change default value for `videos` to `true`
Fixes
-----
- [cloudflare] unescape challenge URLs
- [deviantart] fix video extraction from `extended_fetch` results
- [hitomi] implement workaround for "broken" redirects
- [khinsider] fix and improve metadata extraction
- [patreon] filter duplicate files per post (#590)
- [piczel] fix extraction
- [pixiv] fix user IDs for bookmarks API calls (#596)
- [sexcom] fix image URLs
- [twitter] force old login page layout (#584, #598)
- [vsco] skip "invalid" entities
- improve functions to load/save cookies.txt files (#586)
Removals
--------
- [yaplog] remove module
Changes:
### Additions
- [4chan] add extractor for entire boards (#510)
- [realbooru] add extractors for pools, posts, and tag searches (#514)
- [instagram] implement a `videos` option (#521)
- [vsco] implement a `videos` option
- [postprocessor:metadata] implement a `bypost` option for downloading the
metadata of an entire post (#511)
### Changes
- [reddit] change the default value for `comments` to `0`
- [vsco] improve image resolutions
- make filesystem-related errors during file downloads non-fatal (#512)
### Fixes
- [foolslide] add fallback for chapter data extraction
- [instagram] ignore errors during post-page extraction
- [patreon] avoid errors when fetching user info (#508)
- [patreon] improve URL pattern for single posts
- [reddit] fix errors with `t1` submissions
- [vsco] fix user profile extraction … again
- [weibo] handle unavailable/deleted statuses
- [downloader:http] improve rate limit handling
- retain trailing zeroes in Cloudflare challenge answers
Changes:
1.11.1
======
### Fixes
- Fix inclusion of bash completion and man pages in source distributions
1.11.0
======
### Additions
- Support for
- `blogger` (#364)
- `nozomi` (#388)
- `issuu` (#413)
- `naver` (#447)
- Extractor for `twitter` search results (#448)
- Extractor for `deviantart` user profiles with configurable targets
(#377, #419)
- `--ugoira-conv-lossless` (#432)
- `cookies-update` option to allow updating cookies.txt files (#445)
- Optional `cloudflare` and `video` installation targets (#460)
- Allow executing commands with the `exec` post-processor after all files
are downloaded (#413, #421)
### Changes
- Rewrite `imgur` using its public API (#446)
- Rewrite `luscious` using GraphQL queries (#457)
- Adjust default `nijie` filenames to match `pixiv`
- Change enumeration index for gallery extractors from `page` to `num`
- Return non-zero exit status when errors occurred
- Forward proxy settings to youtube-dl downloader
- Install bash completion script into `share/bash-completion/completions`
### Fixes
- Adapt to new `instagram` page layout when logged in (#391)
- Support protected `twitter` videos (#452)
- Extend `hitomi` URL pattern and fix gallery extraction
- Restore OAuth2 authentication error messages
- Miscellaneous fixes for `patreon` (#444), `deviantart` (#455),
`sexcom` (#464), `imgur` (#467), `simplyhentai`
Changes:
1.10.6
------
# Additions
- `--exec` command-line option to specify a command to run after each
file download (#421)
# Changes
- Include titles in `gfycat` default filenames (#434)
# Fixes
- Fetch working download URLs for `deviantart` (#436)
- Various fixes and improvements for `yaplog` blogs (#443)
- Fix image URL generation for `hitomi` galleries
- Miscellaneous fixes for `behance` and `xvideos`
Changes:
1.10.5
------
### Additions
- `instagram.highlights` option to include highlighted stories when
downloading user profiles (#329)
- Support for `/user/` URLs on `reddit` (#350)
- Support for `imgur` user profiles and favorites (#420)
- Additional metadata fields on `nijie`(#423)
### Fixes
- Improve handling of private `deviantart` artworks (#414) and
429 status codes (#424)
- Prevent fatal errors when trying to open download-archive files (#417)
- Detect and ignore unavailable videos on `weibo` (#427)
- Update the `scope` of new `reddit` refresh-tokens (#428)
- Fix inconsistencies with the `reddit.comments` option (#429)
- Extend URL patterns for `hentaicafe` manga and `pixiv` artworks
- Improve detection of unavailable albums on `luscious` and `imgbb`
- Miscellaneous fixes for `tsumino`
Changes:
1.10.3
------
### Additions
- Provide `filename` metadata for all `deviantart` files (#392, #400)
- Implement a `ytdl.outtmpl` option to let youtube-dl handle filenames by
itself (#395)
- Support `seiga` mobile URLs (#401)
### Fixes
- Extract more than the first 32 posts from `piczel` galleries (#396)
- Fix filenames of archives created with `--zip` (#397)
- Skip unavailable images and videos on `flickr` (#398)
- Fix filesystem paths on Windows with Python 3.6 and lower (#402)
Changes:
1.10.2
======
### Additions
- Support for `instagram` stories and IGTV (#371, #373)
- Support for individual `imgbb` images (#363)
- `deviantart.quality` option to set the JPEG compression quality for newer
images (#369)
- `enumerate` option for `extractor.skip` (#306)
- `adjust-extensions` option to control filename extension adjustments
- `path-remove` option to remove control characters etc. from filesystem
paths
### Changes
- Rename `restrict-filenames` to `path-restrict`
- Adjust `pixiv` metadata and default filename format (#366)
- Set `filename` to `"{category}_{user[id]}_{id}{suffix}.{extension}"` to
restore the old default
- Improve and optimize directory and filename generation
### Fixes
- Allow the `classify` post-processor to handle files with unknown filename
extension (#138)
- Fix rate limit handling for OAuth APIs (#368)
- Fix artwork and scraps extraction on `deviantart` (#376, #392)
- Distinguish between `imgur` album and gallery URLs (#380)
- Prevent crash when using `--ugoira-conv` (#382)
- Handle multi-image posts on `patreon` (#383)
- Miscellaneous fixes for `*reactor`, `simplyhentai`
Changes:
1.10.0
------
### Warning
- Prior to version 1.10.0 all cache files were created world readable
(mode `644`) leading to possible sensitive information disclosure on
multi-user systems.
It is recommended to restrict access permissions of already existing files
(`/tmp/.gallery-dl.cache`) with `chmod 600`.
Windows users should not be affected.
### Additions
- Support for
- `vsco` (#331)
- `imgbb` (#361)
- `adultempire` (#340)
- `restrict-filenames` option to create Windows-compatible filenames on any
platform (#348)
- `forward-cookies` option to control cookie forwarding to youtube-dl (#352)
### Changes
- The default cache file location on non-Windows systems is now
`$XDG_CACHE_HOME/gallery-dl/cache.sqlite3` or
`~/.cache/gallery-dl/cache.sqlite3`
- New cache files are created with mode `600`
- `exhentai` extractors will always use `e-hentai.org` as domain
### Fixes
- Logging into `exhentai` now collects more cookies than just `ipb_member_id`
and `ipb_pass_hash`
- Better handling of `exhentai` image limits and errors (#356, #360)
- Try to prevent ZIP file corruption (#355)
- Miscellaneous fixes for `behance`, `ngomik`
Changes:
1.9.0
-----
### Additions
- Support for
- `erolord` (#326)
- Add login support for `instagram` (#195)
- Add `--no-download` and `extractor.*.download` disable file downloads (#220)
- Add `-A/--abort` to specify the number of consecutive download skips before
aborting
- Interpret `-1` as infinite retries (#300)
- Implement custom log message formats per log-level (#304)
- Implement an `mtime` post-processor that sets file modification times
according to metadata fields (#332)
- Implement a `twitter.content` option to enable tweet text extraction
(#333, #338)
- Enable `date-min/-max/-format` options for `tumblr` (#337)
### Changes
- Set file modification times according to their `Last-Modified` header when
downloading (#236, #277)
- Use `--no-mtime` or `downloader.*.mtime` to disable this behavior
- Duplicate download URLs are no longer silently ignored (controllable with
`extractor.*.image-unique`)
- Deprecate `--abort-on-skip`
### Fixes
- Retry downloads on OpenSSL exceptions (#324)
- Ignore unavailable pins on `sexcom` instead of raising an exception (#325)
- Use Firefox's SSL/TLS ciphers to prevent Cloudflare CAPTCHAs (#342)
- Improve folder name matching on `deviantart` (#343)
- Forward cookies to `youtube-dl` to allow downloading private videos
- Miscellaneous fixes for `35photo`, `500px`, `newgrounds`, `simplyhentai`
Changes:
1.8.7
-----
Additions:
- Support for
- `vanillarock` (#254)
- `nsfwalbum` (#287)
- `artist` and `tags` metadata for `hentaicafe` (#238)
- `description` metadata for `instagram` (#310)
- Format string option to replace a substring with another
- `R<old>/<new>/` (#318)
Changes:
- Delete empty archives created by the `zip` post-processor (#316)
Fixes:
- Handle `hitomi` Game CG galleries correctly (#321)
- Miscellaneous fixes for `deviantart`, `hitomi`, `pururin`, `kissmanga`,
`keenspot`, `mangoxo`, `imagefap`
Changes:
1.8.5
-----
### Additions
- Support for
- `keenspot` (#223)
- `sankakucomplex` (#258)
- `folders` option for `deviantart` to add a list of containing folders to
each file's metadata (#276)
- `captcha` option for `kissmanga` and `readcomiconline` to control CAPTCHA
handling (#279)
- `filename` metadata for files downloaded with youtube-dl (#291)
### Changes
- Adjust `wallhaven` extractors to new page layout:
- use API and add `api-key` option
- removed traditional login support
- Provide original filenames for `patreon` downloads (#268)
- Use e-hentai.org or exhentai.org depending on input URL (#278)
### Fixes
- Fix pagination over `sankaku` popular listings (#265)
- Fix folder and collection extraction on `deviantart` (#271)
- Detect "AreYouHuman" redirects on `readcomiconline` (#279)
- Miscellaneous fixes for `hentainexus`, `livedoor`, `ngomik`
Changes:
### Additions
- Support for
- `patreon` (#226)
- `hentainexus` (#256)
- `date` metadata fields for `pixiv` (#248), `instagram` (#250),
`exhentai`, and `newgrounds`
### Changes
- Improved `flickr` metadata and video extraction (#246)
### Fixes
- Download original GIF animations from `deviantart` (#242)
- Ignore missing `edge_media_to_comment` fields on `instagram` (#250)
- Fix serialization of `datetime` objects for `--write-metadata` (#251, #252)
- Allow multiple post-processor command-line options at once (#253)
- Prevent crash on `booru` sites when no tags are available (#259)
- Fix extraction on `instagram` after `rhx_gis` field removal (#266)
- Avoid Cloudflare CAPTCHAs for Python interpreters built
against OpenSSL < 1.1.1
- Miscellaneous fixes for `luscious`
Changes:
1.8.3
-----
### Additions
- Support for
- `plurk` (#212)
- `sexcom` (#147)
- `--clear-cache`
- `date` metadata fields for `deviantart`, `twitter`, and `tumblr` (#224, #232)
### Changes
- Standalone executables are now built using PyInstaller:
- uses the latest CPython interpreter (Python 3.7.3)
- available on several platforms (Windows, Linux, macOS)
- includes the `certifi` CA bundle, `youtube-dl`, and `pyOpenSSL` on Windows
### Fixes
- Patch `urllib3`'s default list of SSL/TLS ciphers to prevent Cloudflare
CAPTCHAs (#227)
- Windows users need to install `pyOpenSSL` for this to take effect
- Provide fallback URLs for `twitter` images (#237)
- Send `Referer` headers when downloading from `hitomi` (#239)
- Updated login procedure on `mangoxo`
pkgsrc changes:
- Configuration files and man pages were accidentally installed
in etc/ and share/man. Add a SUBST_SED in order to avoid hard-coded
paths and install bash completions in share/bash_completions.d.
Changes:
1.8.2
=====
Additions
---------
- Support for
- `pixnet` (#177)
- `wikiart` (#179)
- `mangoxo` (#184)
- `yaplog` (#190)
- `livedoor` (#190)
- Login support for `mangoxo` (#184) and `twitter` (#214)
Changes
-------
- Increased required `Requests` version to 2.11.0
Fixes
-----
- Improved image quality on `reactor` sites (#210)
- Support `imagebam` galleries with more than 100 images (#219)
- Updated Cloudflare bypass code
Changes:
1.8.0
-----
### Added
- Support for:
- `weibo`
- `pururin` (#174)
- `fashionnove` (#175)
- `shopify` sites in general (#175)
- Snap packaging (#169, #170, #187, #188)
- Automatic Cloudflare DDoS protection bypass
- Extractor and Job information for logging format strings
- `dynastyscans` image and search extractors (#163)
- `deviantart` scraps extractor (#168)
- `artstation` extractor for artwork listings (#172)
- `smugmug` video support and improved image format selection (#183)
### Changed
- More metadata for `nhentai` galleries
- Combined `myportfolio` extractors into one
- Renamed `name` metadata field to `filename` and removed the original `filename` field
- Simplified and improved internal data structures
- Optimized creation of child extractors
### Fixed
- Filter empty `tumblr` URLs (#165)
- Filter ads and improve connection speed on `hentaifoundry`
- Show proper error messages if `luscious` galleries are unavailable
- Miscellaneous issues on `mangahere`, `ngomik`, `simplyhentai`, `imgspice`
### Removed
- `seaotterscans`
Changes:
1.7.0
-----
- Added support for:
- `photobucket` (#117)
- `hentaifox` (#160)
- `tsumino` (#161)
- Added the ability to dynamically generate extractors based on a user's
config file for
- `mastodon` instances (#144)
- `foolslide` based sites
- `foolfuuka` based archives
- Added an extractor for `behance` collections (#157)
- Added login support for `luscious` (#159) and `tsumino` (#161)
- Added an option to stop downloading if the `exhentai` image limit is
exceeded (#141)
- Fixed extraction issues for `behance` and `mangapark`
Changes:
1.6.3
-----
- Added `metadata` post-processor to write image metadata to an external
file (#135)
- Added option to reverse chapter order of manga extractors (#149)
- Added authentication support for `danbooru` (#151)
- Added tag metadata for `exhentai` and `hbrowse` galleries
- Improved `*reactor` extractors (#148)
- Fixed extraction issues for `nhentai` (#156), `pinterest`, `mangapark`
Changes:
1.6.2
-----
- Added support for:
- `instagram` - https://www.instagram.com (#134)
- Added support for multiple items on sta.sh pages (#113)
- Added option to download `tumblr` avatars (#137)
- Changed defaults for visited post types and inline media on `tumblr`
- Improved inline extraction of `tumblr` posts (#133, #137)
- Improved error handling and retry behavior of all API calls
- Improved handling of missing fields in format strings (#136)
- Fixed hash extraction for unusual `tumblr` URLs (#129)
- Fixed image subdomains for `hitomi` galleries (#142)
- Fixed and improved miscellaneous issues for `kissmanga` (#20), `luscious`,
`mangapark`, `readcomiconline`
Changes:
1.6.0
-----
- Add wallhaven extractor
- Add yuki extractor
- Add a ytdl (youtube-dl) downloader to download media via youtube-dl
(Unfortunately at the moment youtube-dl package is not a multipackage (we do
not have py{27,34,35,37}-youtube-dl so this will work only if youtube-dl was
built with the same PYTHON_VERSION_DEFAULT of gallery-dl))
- Add '--no-check-certificate' command-line option
- Misc bug fixes and improvements
Changes:
1.5.3
-----
- Added support for:
+ hentaicafe - https://hentai.cafe/ (#101)
+ bobx - http://www.bobx.com/dark/
- Added black-/whitelist options for post-processor modules
- Added support for tumblr inline videos (#102)
- Fixed extraction of smugmug albums without owner (#100)
- Fixed issues when using default config values with reddit extractors (#104)
- Fixed pagination for user favorites on sankaku (#106)
• Fixed a crash when processing deviantart journals (#108)
Changes:
1.5.1
-----
- Added support for:
* `piczel` - https://piczel.tv/
- Added support for related pins on `pinterest`
- Fixed accessing "offensive" galleries on `exhentai` (#97)
- Fixed extraction issues for `mangadex`, `komikcast` and `behance`
- Removed original-image functionality from `tumblr`, since "raw"
images are no longer accessible
Changes:
1.4.2
-----
- Added image-pool extractors for safebooru and rule34
- Added option for extended tag information on booru sites (#92)
- Added support for DeviantArt's new URL format
- Added support for mangapark mirrors
- Changed imagefap extractors to use HTTPS
- Fixed crash when skipping downloads for files without known extension
Changes:
1.4.1
-----
- Added an ugoira post-processor to convert pixiv animations to WebM
- Added --zip and --ugoira-conv command-line options
- Changed how ugoira frame information is handled
* instead of being written to a separate file, it is now made
available as metadata field of the ZIP archive
- Fixed manga and chapter titles for mangadex
- Fixed file deletion by post-processors
Changes:
1.4.0
-----
* Added support for:
- `simplyhentai` (#89)
* Added extractors for
- `pixiv` search results and followed users
- `deviantart` search results and popular listings
* Added post-processors to perform actions on downloaded files
* Added options to configure logging behavior
* Added OAuth support for `smugmug`
* Changed `pixiv` extractors to use the AppAPI,
this breaks `favorite` archive IDs and changes some metadata fields
* Changed the default filename format for `tumblr` and renamed `offset` to
`num`
* Fixed a possible UnicodeDecodeError during installation (#86)
* Fixed extraction of `mangadex` manga with more than 100 chapters (#84)
* Fixed miscellaneous issues for `imgur`, `reddit`, `komikcast`, `mangafox`
and `imagebam`
Changes:
1.3.5
-----
- Added support for:
* smugmug - https://www.smugmug.com/
- Added title information for mangadex chapters
- Improved the pinterest API implementation (#83)
- Improved error handling for deviantart and tumblr
- Removed gomanga and puremashiro
Changes:
1.3.2
-----
* Added extractors for `artstation` albums, challenges and search results
* Improved URL and metadata extraction for `hitomi`and `nhentai`
* Fixed page transitions for `danbooru` API results (#82)
Changes:
1.3.0
-----
* Added `--proxy` to explicitly specify a proxy server (#76)
* Added options to customize archive ID formats and undefined replacement
fields
* Changed various archive ID formats to improve their behavior for favorites
/ bookmarks / etc.
* Affected modules are `deviantart`, `flickr`, `tumblr`, `pixiv` and
all ...boorus
* Improved `sankaku` and `idolcomplex` support by
* respecting `page` and `next` URL parameters (#79)
* bypassing the page-limit for unauthenticated users
* Improved `directlink` metadata by properly unquoting it
* Fixed `pixiv` ugoira extraction (#78)
* Fixed miscellaneous extraction issues for `mangastream` and `tumblr`
* Removed `yeet`, `chronos`, `coreimg`, `hosturimage`, `imageontime`,
`img4ever`, `imgmaid`, `imgupload`
Changes:
1.2.0
-----
* Added support for:
- `paheal` (#69)
- `komikcast` (#70)
- `subapics` (#70)
* Added `--download-archive` to record downloaded files in an archive file
* Added `--write-log` to write logging output to a file
* Added a filetype check on download completion to fix incorrectly assigned filename extensions (#63)
* Added the `tumblr:...` pseudo URI scheme to support custom domains for Tumblr blogs (#71)
* Added fallback URLs for `tumblr` images (#64)
* Added support for `reddit`-hosted images (#68)
* Improved the input file format by allowing comments and per-URL options
* Fixed OAuth 1.0 signature generation for Python 3.3 and 3.4 (#75)
* Fixed smaller issues for `luscious`, `hentai2read`, `hentaihere` and `imgur`
* Removed the `batoto` module
Changes:
1.1.2
-----
* Added support for:
* `puremashiro` (#66)
* `idolcomplex`
* Added an option to filter reblogs on `tumblr` (#61)
* Added OAuth user authentication for `tumblr` (#65)
* Added support for `slideshare` mobile URLs (#67)
* Improved pagination for various ...booru sites to work around page limits
* Fixed chapter information parsing for certain manga on `kissmanga` (#58) and `batoto` (#60)
Changes:
1.1.1
-----
* Added support for:
`slideshare` - <https://www.slideshare.net/> (#54)
* Added pool- and post-extractors for `sankaku`
* Added OAuth user authentication for `deviantart`
* Updated `luscious` to support `members.luscious.net` URLs (#55)
* Updated `mangahere` to use their new domain name (mangahere.cc) and mobile URLs
* Updated `gelbooru` to not be restricted to the first 20,000 images (#56)
* Fixed extraction issues for `nhentai` and `khinsider`
Changes:
1.1.0
-----
* Added the `-r/--limit-rate` command-line option to set a maximum download
rate
* Added the `--sleep` command-line option to specify the number of seconds
to sleep before each download
* Updated `gelbooru` to no longer use their now disabled API
* Fixed SWF extraction for `sankaku` (#52)
* Fixed extraction issues for `hentai2read` and `khinsider`
* Removed the deprecated `--images` and `--chapters` options
* Removed the `mangazuki` module
Changes:
1.0.2
-----
* Added an option to set a custom user-agent string
* Improved retry behavior for failed HTTP requests
* Improved `seiga` by providing better metadata and getting more than the
latest 200 images
* Improved `tumblr` by adding support for all post types, scanning for inline
images and following external links] (#48)
* Fixed extraction issues for `hbrowse`, `khinsider` and `senmanga`
Changes:
- Added support for: xvideos (#45)
- Fixed exception handling during file downloads which could lead to a
premature exit
- Fixed an issue with tumblr where not all images would be downloaded when
using tags (#48)
- Fixed extraction issues for imgbox (#47), mangastream (#49), and
mangahere
