(with post 2.6.0 fix: bin/httpclient one-liner broken)
## Changes
### Changes in 2.6.0
This release includes internal CookieManager implementation change. It
involves compatibility layer but for the case your library depends on internal
implementation it also provides a way to restore the implementation. See below
for more details.
* Changes
* feat: use http-cookie if available for better Cookies spec compliance.
Instead of WebAgent 0.6.2 that is not maintained over 10 years. To omit
maintaining that library use http-cookie for better spec compliance and
healthy development.
This introduces following incompatibility from existing cookies
implementation.
* Expired cookies are not saved. With the old implementation expired
cookies are saved in file and not be sent to the server. With the new
implementation the expired cookies are not saved to the file and not
be sent to the server.
* Cookie#domain returns dot-less domain for domain cookies. Instead,
Cookie#dot_domain returns with dot.
http-cookie is used by default if available but you can restore original
CookieManager behavior by loading 'httpclient/webagent-cookie' feature
before 'httpclient' like this;
```ruby
require 'httpclient/webagent-cookie'
require 'httpclient'
```
The new implementation dumps warnings to help you migrate to http-cookie.
Please follow the suggestion to avoid future compatibility.
```ruby
e.g.
WebAgent::Cookie is deprecated and will be replaced with HTTP::Cookie in the near future. Please use Cookie#origin= instead of Cookie#url= for the replacement.
Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.
CookieManager#find is deprecated and will be removed in near future. Use HTTP::Cookie.cookie_value(CookieManager#cookies) instead
```
* feat: Message#previous to get responses in negotiation
HTTP::Message#previous keeps previous response in negotiation. For
redirection, authorization negotiation and retry from custom filter.
Closes#234.
* feat: Add JSONClient
JSONClient auto-converts Hash <-> JSON in request and response.
* For POST or PUT request, convert Hash body to JSON String with
'application/json; charset=utf-8' header.
* For response, convert JSON String to Hash when content-type is
'(application|text)/(x-)?json'
This commit include bin/jsonclient that works as same as bin/httpclient
not with HTTPClient but with JSONClient.
* feat: Add download command
```
% httpclient download http://host/path > file
```
* Bug fixes
* fix: duplicated query params by follow_redirect
When the original request has query and the server returns redirection
response with Location, HTTPClient wrongly adds query to the new URI. In
such case the Location header could include query part;
```
e.g.
http://originalhost/api/call?limit=10
-> Location: http://otherhost/api/call?limit=10
```
HTTPClient should just hit the new location '/api/call?limit=10' not
'/api/call?limit=10&limit=10'. Closes#236.
* fix: NTLM & Basic dual auth
When a server returns two or more WWW-Authenticate headers and the first
one is NTLM, say WWW-Authenticate: NTLM and WWW-Authenticate: Basic in
this order, HTTPClient sent Basic Authorization header after finishing
NTLM auth negotiation.
NTLM auth is a connection authentication scheme so HTTPClient deleted
the internal auth negotiation state so that NTLM authenticator does not
do anything after the negotiation has completed. In such case, for the
subsequent requests, NTLM authenticator does nothing but Basic
authenticator sends Basic Authorization header to the server that is
already negotiated via NTLM authenticator. This can cause authentication
failure.
This commit changes the internal state handling not to delete the state
but introduce :done state. NTLM authenticator returns :skip for the
request to the server that auth negotiation has completed. WWWAuth skips
other authenticator to avoid above issue. Closes#157.
* fix: transplant IO positions to new request in negotiation
In authorization negotiation HTTP::Message for request is generated for
each request, of course, but HTTPClient did not care the IO position
recorded in the previous requests in the subsequent requests. Closes#130.
* fix: avoid inconsistent Content-Length and actual body
If lengths of all posted arguments are known HTTPClient sends
'Content-Length' as a sum length of all arguments. But the length of
actual body was wrong because it read as much as possible regardless of
what IO#size returned. So if the file is getting bigger while HTTPClient
is processing a request the request has inconsistent Content-Length and
body.
This bug is found, and the fix is proposed both by @Teshootub7. Thank
you very much for patient trouble shooting! Fixes#117.
* fix: KeepAliveDisconnected race condition
As details explained in #84, current HTTPClient's KeepAliveDisconnected
handling has a race condition bug that allows a client to have
invalidated connection two or more times. This could be a cause of #185.
To avoid this, make HTTPClient acquire new connection for retry of
KeepAliveDisconnected. Closes#84. Closes#185.
### Changes in 2.5.3
This release includes behavior changes of POST and PUT requests that has
nil as a body. See changes below. Emtpty String as a body is not affected.
* Changes
* Update cacert. "Certificate data from Mozilla as of: Tue Oct 28 22:03:58 2014"
-> Reverted in 2.5.3.3 because it caused unexpected SSLError. See
https://github.com/nahi/httpclient/issues/230
* Allow no content POST and PUT.
Previously POST or PUT with :body => nil meant that 'POST or PUT with 0
length entity body'. But sometimes you need to POST or PUT actually no
content which should not have Content-Type nor Content-Length.
It could be incompatible change for user who POST/PUT-ed with empty body
but it should be rare, actually WEBrick cannot handle such 'no content'
POST and PUT. #128.
* Add default_header property.
:default_header is for providing default headers Hash that all HTTP
requests should have, such as custom 'Authorization' header in API. You
can override :default_header with :header Hash parameter in HTTP request
methods.
* raise if redirect res does not have Location header. #155.
* Bug fixes
* Avoid NPE by a cookie without domain=.
The root cause is still uncertain though. Closes#123
* Suppress verify_callback warning.
Because OpenSSL can try multiple certificate chains and some of it can
fail, and one of them succeeds. For that case warning is irrelevant.
Let it warn only in $DEBUG mode. #221.
### Changes in 2.5.2
Oct 29, 2014 - version 2.5.2
* Changes
* Add :force_basic_auth config - #166, #179, #181.
Generally HTTP client must send Authorization header after it gets 401
error from server from security reason. But in some situation (e.g.
API client) you might want to send Authorization from the beginning.
You can turn on/off force_basic_auth flag for sending Authorization
header from the beginning. (Of cource, if a request URI matches with
the URI you set in set_auth method)
Syntax:
```ruby
HTTPClient.new(:force_basic_auth => true)
# or
c = HTTPClient.new
c.force_basic_auth = true
```
* Add :base_url to HTTPClient configuration.
Passing path to get, post, etc. is recognized as a request to
:base_url + uri. If you pass full URL :base_url is ignored.
```ruby
api = HTTPClient.new(:base_url => 'https://api.example.com/v1')
api.get("/users.json") # => Get https://api.example.com/v1/users.json
api.get("https://localhost/path") # => https://localhost/path
```
### Changes in 2.5.1
Oct 19, 2014 - version 2.5.1
* Changes
* Allow to specify :query in POST, PUT, DELETE and OPTIONS requests.
Closes#83.
* Allow to specify :body in OPTIONS request. Closes#136.
### Changes in 2.5.0
Oct 17, 2014 - version 2.5.0
**IMPORTANT CHANGES**
This version changes (again) default SSL options to help
BEAST/CRIME/POODLE Attack prevension.
* Disabled SSLv3 in favor of POODLE Attack prevention.
* Enabled 1/n-1 fragment in favor of BEAST Attack prevention.
* No TLS compression in favor of CRIME Attack prevention.
You can restore the previous SSL configuration like this;
```ruby
client = HTTPClient.new
client.ssl_config.ssl_version = :SSLv23
client.ssl_config.options = OpenSSL::SSL::OP_ALL | OpenSSL::SSL::OP_NO_SSLv2
```
* Changes
* Change default SSL options. See above.
* Keep cause error of KeepAliveDisconnected. It allows caller to
investigate the cause of KeepAliveDisconnected.
### Changes in 2.4.0
Jun 8, 2014 - version 2.4.0
**IMPORTANT CHANGES**
This version changes default SSL version to :auto (same as nil) to use SSL/TLS
version negotiation. Former versions use SSLv3 as default that does not connect
via TLS. This change makes underlying OpenSSL library decide which SSL/TLS
version to use but SSLv2 is disabled.
This change makes your secure connection safer but if you see SSL connection
failure with this version try specifying SSL version to use SSLv3 like;
```
client = HTTPClient.new
client.ssl_config.ssl_version = :SSLv3
```
* Bug fixes
* Avoid unnecessary connection retries for OAuth error.
[#203](https://github.com/nahi/httpclient/issues/203)
* Make authentication drivers Thread-safe. Note that HTTPClient instance is
Thread-safe for authentication state update but it shares authentication
state across threads by design. If you don't want to share authentication
state, such as for using different authentication username/password pair
per thread, create HTTPClient instance for each Thread.
[#200](https://github.com/nahi/httpclient/issues/200)
* Avoid chunked String recycle in callback block.
[#193](https://github.com/nahi/httpclient/issues/193)
* Do not send empty 'oauth_token' in signed request for compatibility.
[#188](https://github.com/nahi/httpclient/issues/188)
* Ignore negative Content-Length header from server.
[#175](https://github.com/nahi/httpclient/issues/175)
* Fix incorrect use of absolute URL for HTTPS proxy requests.
[#168](https://github.com/nahi/httpclient/issues/168)
* Handle UTF characters in chunked bodies.
[#167](https://github.com/nahi/httpclient/issues/167)
* A new cookie never be accepted if an HTTPClient has the same expired cookie.
[#154](https://github.com/nahi/httpclient/issues/154)
* Allow spaces in NO_PROXY environment like; "hosta, hostb"
[#141](https://github.com/nahi/httpclient/issues/141)
* Avoid HttpClient::Message::Body#dump causes Encoding::CompatibilityError.
[#140](https://github.com/nahi/httpclient/issues/140)
* Changes
* Change default SSL version to :auto to use version negotiation.
[#186](https://github.com/nahi/httpclient/issues/186),
[#204](https://github.com/nahi/httpclient/issues/204)
* Allow to pass client private key passphrase in SSLConfig.
[#201](https://github.com/nahi/httpclient/issues/201)
* Convert README to markdown syntax
[#198](https://github.com/nahi/httpclient/issues/198)
* Update default CA certificates: change the source from JDK's to Firefox's.
The file is downloaded from
https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
(Certificate data from Mozilla as of: Tue Apr 22 08:29:31 2014)
[#195](https://github.com/nahi/httpclient/issues/195)
* Callback block can be defined as to get 2 arguments to retrieve the
response object.
[#194](https://github.com/nahi/httpclient/issues/194)
* Remove [] from given address for IPv6 compat.
[#176](https://github.com/nahi/httpclient/issues/176)
* Update API endpoints to those of Twitter REST API v1.1.
[#150](https://github.com/nahi/httpclient/issues/150)
= Changes in 2.3.2 =
January 5, 2013 - version 2.3.2
* Changes
* #138 Revert Timeout change unintentionally included in v2.3.1. It's
reported that the change causes background processes not terminated
properly.
= Changes in 2.3.1 =
January 1, 2013 - version 2.3.1
* Changes
* #137 Signing key is expiring for cacert_sha1.p7s.
Deleted p7s signature check for default cacerts. Sorry for many troubles
in the past. This feature is not useful without having online/real-time
CA certs update but I don't think I can implement it in near future.
Users depend on this signature check (who puts cacert.p7s in R/W
filesystem and ssl_config.rb in R/O filesystem) should take care the
tampering by themself.
* Bug fixes
* #122 Support IPv6 address in URI
= Changes in 2.3.0 =
October 10, 2012 - version 2.3.0
* Features
* Added debug mode CLI. bin/httpclient is installed as CLI.
Usage: 1) % httpclient get https://www.google.co.jp/ q=ruby
Usage: 2) %httpclient
For 1) it issues a GET request to the given URI and shows the wiredump
and the parsed result. For 2) it invokes irb shell with the binding
that has a HTTPClient as 'self'. You can call HTTPClient instance
methods like;
> get "https://www.google.co.jp/", :q => :ruby
* #119 Addressable gem support (only if it exists); should handle IRI
properly.
* Bug fixes
* #115 Cookies couldn't work properly if the path in an URI is ommited.
* #112, #117 Proper handling of sized IO (the IO object that responds to
:size) for chunked POST. HTTPClient did read till EOF even if the
given IO has :size method.
* Handle '303 See Other' properly. RFC2616 says it should be redirected
with GET.
* #116 Fix "100-continue" support. It was just ignored.
* #118 Support for boolean values when making POST/PUT requests with
multiipart/form Content-Type.
* #110 Allows leading dots in no_proxy hostname suffixes.
== Changes
= Changes in 2.2.7 =
August 14, 2012 - version 2.2.7
* Bug fixes
* Fix arity incompatibility introduced in 2.2.6. It broke Webmock.
Thanks Andrew France for the report!
= Changes in 2.2.6 =
August 14, 2012 - version 2.2.6
* Bug fixes
* Make get_content doesn't raise a BadResponseError for perfectly good
responses like 304 Not Modified. Thanks to Florian Hars.
* Add 'Content-Type: application/x-www-form-urlencoded' for the PUT
request that has urlencoded entity-body.
* Features
* Add HTTPClient::IncludeClient by Jonathan Rochkind, a mix-in for easily
adding a thread-safe lazily initialized class-level HTTPClient object
to your class.
* Proxy DigestAuth support. Thanks to Alexander Kotov and Florian Hars.
* Accept an array of strings (and IO-likes) as a query value
e.g. `{ x: 'a', y: [1,2,3] }` is encoded into `"x=a&y=1&y=2&y=3"`.
Thanks to Akinori MUSHA.
* Allow body for DELETE method.
* Allow :follow_redirect => true for HEAD request.
* Fill request parameters request_method, request_uri and request_query
as part of response Message::Header.
= Changes in 2.2.5 =
May 06, 2012 - version 2.2.5
* Bug fixes
* Added Magic encoding comment to hexdump.rb to avoid encoding error.
* Add workaround for JRuby issue on Windows (JRUBY-6136)
On Windows, calling File#size fails with an Unknown error (20047).
This workaround uses File#lstat instead.
* Require open-uri only on ruby 1.9, since it is not needed on 1.8.
* Features
* Allow symbol Header name for HTTP request.
* Dump more SSL certificate information under $DEBUG.
* Add HTTPClient::SSLConfig#ssl_version property.
* Add 'Accept: */*' header to request by default. Rails requies it.
It doesn't override given Accept header from API.
* Add HTTPClient::SSLConfig#set_default_paths. This method makes
HTTPClient instance to use OpenSSL's default trusted CA certificates.
* Allow to set Date header manually.
ex. clent.get(uri, :header => {'Date' => Time.now.httpdate})
While here, install document again.
== Changes
= Changes in 2.2.2 =
Oct 17, 2011 - version 2.2.2
* Bug fixes
* Do not sort query params on request: Wrongly sorted query params for
easier debugging but the order of request parameter should be
preserved. #65
* Changes
* Set responce String encoding if possible. Parse content-type response
header with some helps from OpenURI::Meta and set response String
encoding. #26
* Improve connection cache strategy. Reuse cached session in MRU order,
not in LRU. MRU is more server friendly than LRU because it reduces
number of cached sessions when a number of requests drops after an
usaage spike.
With reusing sessions in LRU order, all sessions are equally checked if
it's closed or not, as far as there's a request to the same site. With
reusing sessions in MRU order, old cold sessions are kept in cache long
time even if there's a request to the same site. To avoid this leakage,
this version adds keep_alive_timeout property and let SessionManager
scrub all sessions with checking the timeout for each session. When the
session expires against the last used time, it's closed and collected.
keep_alive_timeout is 15[sec] by default. The value is from the default
value for KeepAliveTimeout of Apache httpd 2. #68#69
== Changes
= Changes in 2.2.1 =
Jun 2, 2011 - version 2.2.1
* Bug fixes
* For Lighttpd + PUT/POST support, do not send a request using chunked
encoding when IO respond to :size, File for example.
- There is no need to send query with Transfer-Encoding: chuncked when
IO respond to :size.
- Lighttpd does not support PUT, POST with Transfer-Encoding: chuncked.
You will see that the lighty respond with 200 OK, but there is a file
whose size is zero.
LIMITATION:
timeout occurs certainly when you send very large file and
@send_timeout is default since HTTPClient::Session#query() assumes
that *all* write are finished in @send_timeout sec not each write.
WORKAROUND:
increment @send_timeout and @receive_timeout or set @send_timeout and
@receive_timeout to 0 not to be timeout.
This fix is by TANABE Ken-ichi <nabeken@tknetworks.org>. Thanks!
* Allow empty http_proxy ENV variable. Just treat it the same as if it's
nil/unset. This fix is by Ash Berlin <ash_github@firemirror.com>.
Thanks!
* Check EOF while reading chunked response and close the session. It
raised NoMethodError.
* Changes
* Updated trusted CA certificates file (cacert.p7s and cacert_sha1.p7s).
CA certs are imported from
'Java(TM) SE Runtime Environment (build 1.6.0_25-b06)'.
* Changed default chunk size from 4K to 16K. It's used for reading size
at a time.
switch to use gem.
= Changes in 2.2.0 =
Apr 8, 2011 - version 2.2.0
* Features
* Add HTTPClient#cookies as an alias of #cookie_manager.cookies.
* Add res.cookies method. It returns parsed cookie in response header.
It's different from client.cookie_manager.cookies. Manager keeps
persistent cookies in it.
* Add res.headers method which returns a Hash of headers.
Hash key and value are both String. Each key has a single value so you
can't extract exact value when a message has multiple headers like
'Set-Cookie'. Use header['Set-Cookie'] for that purpose.
(It returns an Array always)
* Allow keyword style argument for HTTPClient#get, post, etc.
Introduced keywords are: :body, :query, and :header.
You can write
HTTPClient.get(uri, :header => {'X-custom' => '1'})
instead of;
HTTPClient.get(uri, nil, {'X-custom' => '1'})
* Add new keyword argument :follow_redirect to get/post. Now you can
follow redirection response with passing :follow_redirect => true.
* [INCOMPAT] Rename HTTPClient::HTTP::Message#body to #http_body, then
add #body as an alias of #content. It's incompatible change though
users rarely depends on this method. (I've never seen such a case)
Users who are using req.body and/or res.body should follow this
change. (req.http_body and res.http_body)
* Bug fixes
* Reenable keep-alive for chunked response.
This feature was disabled by c206b687952e1ad3e20c20e69bdbd1a9cb38609e at
2008-12-09. I should have written a test for keep-alive. Now I added it.
Thanks Takahiro Nishimura(@dr_taka_n) for finding this bug.
= Changes in 2.1.7 =
Mar 22, 2011 - version 2.1.7
* Features
* Add MD5-sess auth support. Thanks to wimm-dking. (#47)
* Add SNI support. (Server Name Indication of HTTPS connection) (#49)
* Add GSSAPI auth support using gssapi gem. Thanks to zenchild. (#50)
* NTLM logon to exchange Web Services. [experimental] Thanks to curzonj and mccraigmccraig (#52)
* Add HTTPOnly cookie support. Thanks to nbrosnahan. (#55)
* Add HTTPClient#socket_local for specifying local binding hostname and port of TCP socket. Thanks to icblenke.
