Changes in 2.8.3 2016/12/8
Bug
* Avoid frozen string errors on Ruby 2.3 - fixed by RJ Walsh #336
* SSL timeouts not working on JRuby - fixed by Dave Golombek #341
* Do not reset SSL connection if unnecessary - fixed by Dave Golombek #344
* Ignore text around PEM file in JRuby fixed by Scott Kolb #352
Changes
* Add HTTPClient#tcp_keepalive= for enabling TCP keepalive. false by default. #350
Changes in 2.8.2
Aug 15, 2016 - version 2.8.2
* Bug
o 2.8.1 introduced JRuby + SSL connection problem; in some cases it cannot
connect to trusted TLS server. 2.8.1 failed to load multiple CA
certificates in a file. #327.
Aug 16, 2016 - version 2.8.2.1
* Bug
o 2.8.1 introduced another bug that causes NPE from JRuby when JRuby
program loads httpclient and uses OpenSSL::X509::Store outside of
httpclient. 2.8.3 fixed this problem. #325
Aug 28, 2016 - version 2.8.2.3
* Bug
o 2.8.2 fixed VERIFY_NONE at JRuby but the fix was not enough.
Sep 11, 2016 - version 2.8.2.4
* Bug
o 2.8.2 caused unexpected resulting value change of
OpenSSL::X509::Store#add_cert method. Fixed.
Changes in 2.8.1
Aug 8, 2016 - version 2.8.1
* Changes
o Use TLSv1.2 always on JRuby #320
o Do not reset keep-alive connection by configuration change #315
o Add strict_response_size_check option #316 false by default, meaning it
behavies like browsers by default.
o Add MIME type for XML #308
* Bug
o Direct access to SSLConfig#cert_store in JRuby was broken from 2.7
#276#317
o OpenSSL::SSL::VERIFY_NONE does not work in JRuby #319
o Allow receiving response body in block when follow_redirects => true. #304
o Fix blocking issue with request_async when Encoding.default_internal is
set. #307
o Apply timeouts for chunked transfer encoding #309
Changes in 2.8.0
Apr 24, 2016 - version 2.8.0
* Changes
o Force using RSA 2048bit CA cert set
Use RSA 2048bit CA cert set every time if it runs with OpenSSL (==
except JRuby.)
Old openssl (<1.0.1p or <1.0.2d) cannot handle this CA set and causes
SSL connection failure against some SSL servers including AWS S3
API. For such case you can manually specify RSA 1024bit CA cert set as a
workaround.
c = HTTPClient.new { |c| c.ssl_config.add_trust_ca("cacert1024.pem") }
c.get("https://www.ruby-lang.org/")
RSA 1024bit CA cert set is not maintained over years so you should
consider updating OpenSSL version so that HTTPClient uses RSA 2048 bit
CA cert set.
Changes in 2.7.2
Apr 22, 2016 - version 2.7.2
* Changes
o Use RSA 1024bit CA cert when linked to old openssl
Based on comments to #297 this commit silently (without warning) accepts
RSA 1024bit certificate set when runtime ruby is liked with old OpenSSL
(<1.0.1p or <1.0.2d.)
If you're unsure that your OpenSSL is patched or not, and want to make
sure to use RSA 2048bit certificate set, please call
HTTPClient::SSLConfig#add_trust_ca("cacert.pem").
c = HTTPClient.new { |c| c.ssl_config.add_trust_ca("cacert.pem") }
c.get("https://www.ruby-lang.org/")
I'm going to remove RSA 1024bit certificate set and bump httpclient
version to 2.8.0 soon after I release this as 2.7.2. I believe almost
all OpenSSL installation is patched quickly these days so it should not
cause SSL connectivity problem.
pkgsrc changes:
* Add completely support for pkg_alternatives (adding ALTERNATIVES).
* Remove ruby18 specific description in Makefile.
Changes in 2.7.1
Jan 1, 2016 - version 2.7.1
* Changes
- Symbol aware header key comparison
Normalizes symbol to String for header key.
https://github.com/nahi/httpclient/issues/278
- Show warning message only once
Added HTTPClient::Util.warning that cached warning message and supress
the message if it's already dumped.
- Use Timeout.timeout instead of Kernel.timeout that is deprecated from
ruby 2.3.0.
* Bug fixes
- [JRuby] Set ssl_version properly
(with post 2.6.0 fix: bin/httpclient one-liner broken)
## Changes
### Changes in 2.6.0
This release includes internal CookieManager implementation change. It
involves compatibility layer but for the case your library depends on internal
implementation it also provides a way to restore the implementation. See below
for more details.
* Changes
* feat: use http-cookie if available for better Cookies spec compliance.
Instead of WebAgent 0.6.2 that is not maintained over 10 years. To omit
maintaining that library use http-cookie for better spec compliance and
healthy development.
This introduces following incompatibility from existing cookies
implementation.
* Expired cookies are not saved. With the old implementation expired
cookies are saved in file and not be sent to the server. With the new
implementation the expired cookies are not saved to the file and not
be sent to the server.
* Cookie#domain returns dot-less domain for domain cookies. Instead,
Cookie#dot_domain returns with dot.
http-cookie is used by default if available but you can restore original
CookieManager behavior by loading 'httpclient/webagent-cookie' feature
before 'httpclient' like this;
```ruby
require 'httpclient/webagent-cookie'
require 'httpclient'
```
The new implementation dumps warnings to help you migrate to http-cookie.
Please follow the suggestion to avoid future compatibility.
```ruby
e.g.
WebAgent::Cookie is deprecated and will be replaced with HTTP::Cookie in the near future. Please use Cookie#origin= instead of Cookie#url= for the replacement.
Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.
CookieManager#find is deprecated and will be removed in near future. Use HTTP::Cookie.cookie_value(CookieManager#cookies) instead
```
* feat: Message#previous to get responses in negotiation
HTTP::Message#previous keeps previous response in negotiation. For
redirection, authorization negotiation and retry from custom filter.
Closes#234.
* feat: Add JSONClient
JSONClient auto-converts Hash <-> JSON in request and response.
* For POST or PUT request, convert Hash body to JSON String with
'application/json; charset=utf-8' header.
* For response, convert JSON String to Hash when content-type is
'(application|text)/(x-)?json'
This commit include bin/jsonclient that works as same as bin/httpclient
not with HTTPClient but with JSONClient.
* feat: Add download command
```
% httpclient download http://host/path > file
```
* Bug fixes
* fix: duplicated query params by follow_redirect
When the original request has query and the server returns redirection
response with Location, HTTPClient wrongly adds query to the new URI. In
such case the Location header could include query part;
```
e.g.
http://originalhost/api/call?limit=10
-> Location: http://otherhost/api/call?limit=10
```
HTTPClient should just hit the new location '/api/call?limit=10' not
'/api/call?limit=10&limit=10'. Closes#236.
* fix: NTLM & Basic dual auth
When a server returns two or more WWW-Authenticate headers and the first
one is NTLM, say WWW-Authenticate: NTLM and WWW-Authenticate: Basic in
this order, HTTPClient sent Basic Authorization header after finishing
NTLM auth negotiation.
NTLM auth is a connection authentication scheme so HTTPClient deleted
the internal auth negotiation state so that NTLM authenticator does not
do anything after the negotiation has completed. In such case, for the
subsequent requests, NTLM authenticator does nothing but Basic
authenticator sends Basic Authorization header to the server that is
already negotiated via NTLM authenticator. This can cause authentication
failure.
This commit changes the internal state handling not to delete the state
but introduce :done state. NTLM authenticator returns :skip for the
request to the server that auth negotiation has completed. WWWAuth skips
other authenticator to avoid above issue. Closes#157.
* fix: transplant IO positions to new request in negotiation
In authorization negotiation HTTP::Message for request is generated for
each request, of course, but HTTPClient did not care the IO position
recorded in the previous requests in the subsequent requests. Closes#130.
* fix: avoid inconsistent Content-Length and actual body
If lengths of all posted arguments are known HTTPClient sends
'Content-Length' as a sum length of all arguments. But the length of
actual body was wrong because it read as much as possible regardless of
what IO#size returned. So if the file is getting bigger while HTTPClient
is processing a request the request has inconsistent Content-Length and
body.
This bug is found, and the fix is proposed both by @Teshootub7. Thank
you very much for patient trouble shooting! Fixes#117.
* fix: KeepAliveDisconnected race condition
As details explained in #84, current HTTPClient's KeepAliveDisconnected
handling has a race condition bug that allows a client to have
invalidated connection two or more times. This could be a cause of #185.
To avoid this, make HTTPClient acquire new connection for retry of
KeepAliveDisconnected. Closes#84. Closes#185.
### Changes in 2.5.3
This release includes behavior changes of POST and PUT requests that has
nil as a body. See changes below. Emtpty String as a body is not affected.
* Changes
* Update cacert. "Certificate data from Mozilla as of: Tue Oct 28 22:03:58 2014"
-> Reverted in 2.5.3.3 because it caused unexpected SSLError. See
https://github.com/nahi/httpclient/issues/230
* Allow no content POST and PUT.
Previously POST or PUT with :body => nil meant that 'POST or PUT with 0
length entity body'. But sometimes you need to POST or PUT actually no
content which should not have Content-Type nor Content-Length.
It could be incompatible change for user who POST/PUT-ed with empty body
but it should be rare, actually WEBrick cannot handle such 'no content'
POST and PUT. #128.
* Add default_header property.
:default_header is for providing default headers Hash that all HTTP
requests should have, such as custom 'Authorization' header in API. You
can override :default_header with :header Hash parameter in HTTP request
methods.
* raise if redirect res does not have Location header. #155.
* Bug fixes
* Avoid NPE by a cookie without domain=.
The root cause is still uncertain though. Closes#123
* Suppress verify_callback warning.
Because OpenSSL can try multiple certificate chains and some of it can
fail, and one of them succeeds. For that case warning is irrelevant.
Let it warn only in $DEBUG mode. #221.
### Changes in 2.5.2
Oct 29, 2014 - version 2.5.2
* Changes
* Add :force_basic_auth config - #166, #179, #181.
Generally HTTP client must send Authorization header after it gets 401
error from server from security reason. But in some situation (e.g.
API client) you might want to send Authorization from the beginning.
You can turn on/off force_basic_auth flag for sending Authorization
header from the beginning. (Of cource, if a request URI matches with
the URI you set in set_auth method)
Syntax:
```ruby
HTTPClient.new(:force_basic_auth => true)
# or
c = HTTPClient.new
c.force_basic_auth = true
```
* Add :base_url to HTTPClient configuration.
Passing path to get, post, etc. is recognized as a request to
:base_url + uri. If you pass full URL :base_url is ignored.
```ruby
api = HTTPClient.new(:base_url => 'https://api.example.com/v1')
api.get("/users.json") # => Get https://api.example.com/v1/users.json
api.get("https://localhost/path") # => https://localhost/path
```
### Changes in 2.5.1
Oct 19, 2014 - version 2.5.1
* Changes
* Allow to specify :query in POST, PUT, DELETE and OPTIONS requests.
Closes#83.
* Allow to specify :body in OPTIONS request. Closes#136.
### Changes in 2.5.0
Oct 17, 2014 - version 2.5.0
**IMPORTANT CHANGES**
This version changes (again) default SSL options to help
BEAST/CRIME/POODLE Attack prevension.
* Disabled SSLv3 in favor of POODLE Attack prevention.
* Enabled 1/n-1 fragment in favor of BEAST Attack prevention.
* No TLS compression in favor of CRIME Attack prevention.
You can restore the previous SSL configuration like this;
```ruby
client = HTTPClient.new
client.ssl_config.ssl_version = :SSLv23
client.ssl_config.options = OpenSSL::SSL::OP_ALL | OpenSSL::SSL::OP_NO_SSLv2
```
* Changes
* Change default SSL options. See above.
* Keep cause error of KeepAliveDisconnected. It allows caller to
investigate the cause of KeepAliveDisconnected.
### Changes in 2.4.0
Jun 8, 2014 - version 2.4.0
**IMPORTANT CHANGES**
This version changes default SSL version to :auto (same as nil) to use SSL/TLS
version negotiation. Former versions use SSLv3 as default that does not connect
via TLS. This change makes underlying OpenSSL library decide which SSL/TLS
version to use but SSLv2 is disabled.
This change makes your secure connection safer but if you see SSL connection
failure with this version try specifying SSL version to use SSLv3 like;
```
client = HTTPClient.new
client.ssl_config.ssl_version = :SSLv3
```
* Bug fixes
* Avoid unnecessary connection retries for OAuth error.
[#203](https://github.com/nahi/httpclient/issues/203)
* Make authentication drivers Thread-safe. Note that HTTPClient instance is
Thread-safe for authentication state update but it shares authentication
state across threads by design. If you don't want to share authentication
state, such as for using different authentication username/password pair
per thread, create HTTPClient instance for each Thread.
[#200](https://github.com/nahi/httpclient/issues/200)
* Avoid chunked String recycle in callback block.
[#193](https://github.com/nahi/httpclient/issues/193)
* Do not send empty 'oauth_token' in signed request for compatibility.
[#188](https://github.com/nahi/httpclient/issues/188)
* Ignore negative Content-Length header from server.
[#175](https://github.com/nahi/httpclient/issues/175)
* Fix incorrect use of absolute URL for HTTPS proxy requests.
[#168](https://github.com/nahi/httpclient/issues/168)
* Handle UTF characters in chunked bodies.
[#167](https://github.com/nahi/httpclient/issues/167)
* A new cookie never be accepted if an HTTPClient has the same expired cookie.
[#154](https://github.com/nahi/httpclient/issues/154)
* Allow spaces in NO_PROXY environment like; "hosta, hostb"
[#141](https://github.com/nahi/httpclient/issues/141)
* Avoid HttpClient::Message::Body#dump causes Encoding::CompatibilityError.
[#140](https://github.com/nahi/httpclient/issues/140)
* Changes
* Change default SSL version to :auto to use version negotiation.
[#186](https://github.com/nahi/httpclient/issues/186),
[#204](https://github.com/nahi/httpclient/issues/204)
* Allow to pass client private key passphrase in SSLConfig.
[#201](https://github.com/nahi/httpclient/issues/201)
* Convert README to markdown syntax
[#198](https://github.com/nahi/httpclient/issues/198)
* Update default CA certificates: change the source from JDK's to Firefox's.
The file is downloaded from
https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
(Certificate data from Mozilla as of: Tue Apr 22 08:29:31 2014)
[#195](https://github.com/nahi/httpclient/issues/195)
* Callback block can be defined as to get 2 arguments to retrieve the
response object.
[#194](https://github.com/nahi/httpclient/issues/194)
* Remove [] from given address for IPv6 compat.
[#176](https://github.com/nahi/httpclient/issues/176)
* Update API endpoints to those of Twitter REST API v1.1.
[#150](https://github.com/nahi/httpclient/issues/150)
= Changes in 2.3.3 =
February 24, 2013 - version 2.3.3
* Changes
* #144 Add User-Agent field by default. You can remove the header by
setting nil to HTTPClient#agent_name.
= Changes in 2.3.2 =
January 5, 2013 - version 2.3.2
* Changes
* #138 Revert Timeout change unintentionally included in v2.3.1. It's
reported that the change causes background processes not terminated
properly.
= Changes in 2.3.1 =
January 1, 2013 - version 2.3.1
* Changes
* #137 Signing key is expiring for cacert_sha1.p7s.
Deleted p7s signature check for default cacerts. Sorry for many troubles
in the past. This feature is not useful without having online/real-time
CA certs update but I don't think I can implement it in near future.
Users depend on this signature check (who puts cacert.p7s in R/W
filesystem and ssl_config.rb in R/O filesystem) should take care the
tampering by themself.
* Bug fixes
* #122 Support IPv6 address in URI
= Changes in 2.3.0 =
October 10, 2012 - version 2.3.0
* Features
* Added debug mode CLI. bin/httpclient is installed as CLI.
Usage: 1) % httpclient get https://www.google.co.jp/ q=ruby
Usage: 2) %httpclient
For 1) it issues a GET request to the given URI and shows the wiredump
and the parsed result. For 2) it invokes irb shell with the binding
that has a HTTPClient as 'self'. You can call HTTPClient instance
methods like;
> get "https://www.google.co.jp/", :q => :ruby
* #119 Addressable gem support (only if it exists); should handle IRI
properly.
* Bug fixes
* #115 Cookies couldn't work properly if the path in an URI is ommited.
* #112, #117 Proper handling of sized IO (the IO object that responds to
:size) for chunked POST. HTTPClient did read till EOF even if the
given IO has :size method.
* Handle '303 See Other' properly. RFC2616 says it should be redirected
with GET.
* #116 Fix "100-continue" support. It was just ignored.
* #118 Support for boolean values when making POST/PUT requests with
multiipart/form Content-Type.
* #110 Allows leading dots in no_proxy hostname suffixes.
== Changes
= Changes in 2.2.7 =
August 14, 2012 - version 2.2.7
* Bug fixes
* Fix arity incompatibility introduced in 2.2.6. It broke Webmock.
Thanks Andrew France for the report!
= Changes in 2.2.6 =
August 14, 2012 - version 2.2.6
* Bug fixes
* Make get_content doesn't raise a BadResponseError for perfectly good
responses like 304 Not Modified. Thanks to Florian Hars.
* Add 'Content-Type: application/x-www-form-urlencoded' for the PUT
request that has urlencoded entity-body.
* Features
* Add HTTPClient::IncludeClient by Jonathan Rochkind, a mix-in for easily
adding a thread-safe lazily initialized class-level HTTPClient object
to your class.
* Proxy DigestAuth support. Thanks to Alexander Kotov and Florian Hars.
* Accept an array of strings (and IO-likes) as a query value
e.g. `{ x: 'a', y: [1,2,3] }` is encoded into `"x=a&y=1&y=2&y=3"`.
Thanks to Akinori MUSHA.
* Allow body for DELETE method.
* Allow :follow_redirect => true for HEAD request.
* Fill request parameters request_method, request_uri and request_query
as part of response Message::Header.
= Changes in 2.2.5 =
May 06, 2012 - version 2.2.5
* Bug fixes
* Added Magic encoding comment to hexdump.rb to avoid encoding error.
* Add workaround for JRuby issue on Windows (JRUBY-6136)
On Windows, calling File#size fails with an Unknown error (20047).
This workaround uses File#lstat instead.
* Require open-uri only on ruby 1.9, since it is not needed on 1.8.
* Features
* Allow symbol Header name for HTTP request.
* Dump more SSL certificate information under $DEBUG.
* Add HTTPClient::SSLConfig#ssl_version property.
* Add 'Accept: */*' header to request by default. Rails requies it.
It doesn't override given Accept header from API.
* Add HTTPClient::SSLConfig#set_default_paths. This method makes
HTTPClient instance to use OpenSSL's default trusted CA certificates.
* Allow to set Date header manually.
ex. clent.get(uri, :header => {'Date' => Time.now.httpdate})
= Changes in 2.2.4 =
Dec 08, 2011 - version 2.2.4
* Bug fixes
* Do not recycle buffer String object for yielding. When the response is
not chunked and the size of the response > 16KB, API with block style
yields recycled String object for each yields.
* Set VERSION string in User-Agent header. $Id$ didn't work long time...
Bugs are reported by Seamus Abshere. Thanks!
While here, install document again.
== Changes
= Changes in 2.2.2 =
Oct 17, 2011 - version 2.2.2
* Bug fixes
* Do not sort query params on request: Wrongly sorted query params for
easier debugging but the order of request parameter should be
preserved. #65
* Changes
* Set responce String encoding if possible. Parse content-type response
header with some helps from OpenURI::Meta and set response String
encoding. #26
* Improve connection cache strategy. Reuse cached session in MRU order,
not in LRU. MRU is more server friendly than LRU because it reduces
number of cached sessions when a number of requests drops after an
usaage spike.
With reusing sessions in LRU order, all sessions are equally checked if
it's closed or not, as far as there's a request to the same site. With
reusing sessions in MRU order, old cold sessions are kept in cache long
time even if there's a request to the same site. To avoid this leakage,
this version adds keep_alive_timeout property and let SessionManager
scrub all sessions with checking the timeout for each session. When the
session expires against the last used time, it's closed and collected.
keep_alive_timeout is 15[sec] by default. The value is from the default
value for KeepAliveTimeout of Apache httpd 2. #68#69
== Changes
= Changes in 2.2.1 =
Jun 2, 2011 - version 2.2.1
* Bug fixes
* For Lighttpd + PUT/POST support, do not send a request using chunked
encoding when IO respond to :size, File for example.
- There is no need to send query with Transfer-Encoding: chuncked when
IO respond to :size.
- Lighttpd does not support PUT, POST with Transfer-Encoding: chuncked.
You will see that the lighty respond with 200 OK, but there is a file
whose size is zero.
LIMITATION:
timeout occurs certainly when you send very large file and
@send_timeout is default since HTTPClient::Session#query() assumes
that *all* write are finished in @send_timeout sec not each write.
WORKAROUND:
increment @send_timeout and @receive_timeout or set @send_timeout and
@receive_timeout to 0 not to be timeout.
This fix is by TANABE Ken-ichi <nabeken@tknetworks.org>. Thanks!
* Allow empty http_proxy ENV variable. Just treat it the same as if it's
nil/unset. This fix is by Ash Berlin <ash_github@firemirror.com>.
Thanks!
* Check EOF while reading chunked response and close the session. It
raised NoMethodError.
* Changes
* Updated trusted CA certificates file (cacert.p7s and cacert_sha1.p7s).
CA certs are imported from
'Java(TM) SE Runtime Environment (build 1.6.0_25-b06)'.
* Changed default chunk size from 4K to 16K. It's used for reading size
at a time.
switch to use gem.
= Changes in 2.2.0 =
Apr 8, 2011 - version 2.2.0
* Features
* Add HTTPClient#cookies as an alias of #cookie_manager.cookies.
* Add res.cookies method. It returns parsed cookie in response header.
It's different from client.cookie_manager.cookies. Manager keeps
persistent cookies in it.
* Add res.headers method which returns a Hash of headers.
Hash key and value are both String. Each key has a single value so you
can't extract exact value when a message has multiple headers like
'Set-Cookie'. Use header['Set-Cookie'] for that purpose.
(It returns an Array always)
* Allow keyword style argument for HTTPClient#get, post, etc.
Introduced keywords are: :body, :query, and :header.
You can write
HTTPClient.get(uri, :header => {'X-custom' => '1'})
instead of;
HTTPClient.get(uri, nil, {'X-custom' => '1'})
* Add new keyword argument :follow_redirect to get/post. Now you can
follow redirection response with passing :follow_redirect => true.
* [INCOMPAT] Rename HTTPClient::HTTP::Message#body to #http_body, then
add #body as an alias of #content. It's incompatible change though
users rarely depends on this method. (I've never seen such a case)
Users who are using req.body and/or res.body should follow this
change. (req.http_body and res.http_body)
* Bug fixes
* Reenable keep-alive for chunked response.
This feature was disabled by c206b687952e1ad3e20c20e69bdbd1a9cb38609e at
2008-12-09. I should have written a test for keep-alive. Now I added it.
Thanks Takahiro Nishimura(@dr_taka_n) for finding this bug.
= Changes in 2.1.7 =
Mar 22, 2011 - version 2.1.7
* Features
* Add MD5-sess auth support. Thanks to wimm-dking. (#47)
* Add SNI support. (Server Name Indication of HTTPS connection) (#49)
* Add GSSAPI auth support using gssapi gem. Thanks to zenchild. (#50)
* NTLM logon to exchange Web Services. [experimental] Thanks to curzonj and mccraigmccraig (#52)
* Add HTTPOnly cookie support. Thanks to nbrosnahan. (#55)
* Add HTTPClient#socket_local for specifying local binding hostname and port of TCP socket. Thanks to icblenke.
