oxen-io
/
oxen-core
mirror of https://github.com/oxen-io/oxen-core.git
2
1
Fork 0
Code Issues Projects Releases Wiki Activity

Replace epee http client with curl-based client 

In short: epee's http client is garbage, standard violating, and
unreliable.

This completely removes the epee http client support and replaces it
with cpr, a curl-based C++ wrapper.  rpc/http_client.h wraps cpr for RPC
requests specifically, but it is also usable directly.

This replacement has a number of advantages:

- requests are considerably more reliable.  The epee http client code
  assumes that a connection will be kept alive forever, and returns a
  failure if a connection is ever closed.  This results in some very
  annoying things: for example, preparing a transaction and then waiting
  a long tim before confirming it will usually result in an error
  communication with the daemon.  This is just terribly behaviour: the
  right thing to do on a connection failure is to resubmit the request.

- epee's http client is broken in lots of other ways: for example, it
  tries throwing SSL at the port to see if it is HTTPS, but this is
  protocol violating and just breaks (with a several second timeout) on
  anything that *isn't* epee http server (for example, when lokid is
  behind a proxying server).

- even when it isn't doing the above, the client breaks in other ways:
  for example, there is a comment (replaced in this PR) in the Trezor PR
  code that forces a connection close after every request because epee's
  http client doesn't do proper keep-alive request handling.

- it seems noticeably faster to me in practical use in this PR; both
  simple requests (for example, when running `lokid status`) and
  wallet<->daemon connections are faster, probably because of crappy
  code in epee.  (I think this is also related to the throw-ssl-at-it
  junk above: the epee client always generates an ssl certificate during
  static initialization because it might need one at some point).

- significantly reduces the amount of code we have to maintain.

- removes all the epee ssl option code: curl can handle all of that just
  fine.

- removes the epee socks proxy code; curl can handle that just fine.
  (And can do more: it also supports using HTTP/HTTPS proxies).

- When a cli wallet connection fails we know show why it failed (which
  now is an error message from curl), which could have all sorts of
  reasons like hostname resolution failure, bad ssl certificate, etc.
  Previously you just got a useless generic error that tells you
  nothing.

Other related changes in this PR:

- Drops the check-for-update and download-update code.  To the best of
my knowledge these have never been supported in loki-core and so it
didn't seem worth the trouble to convert them to use cpr for the
requests.

- Cleaned up node_rpc_proxy return values: there was an inconsistent mix
  of ways to return errors and how the returned strings were handled.
  Instead this cleans it up to return a pair<bool, val>, which (with
  C++17) can be transparently captured as:

    auto [success, val] = node.whatever(req);

  This drops the failure message string, but it was almost always set to
  something fairly useless (if we want to resurrect it we could easily
  change the first element to be a custom type with a bool operator for
  success, and a `.error` attribute containing some error string, but
  for the most part the current code wasn't doing much useful with the
  failure string).

- changed local detection (for automatic trusted daemon determination)
  to just look for localhost, and to not try to resolve anything.
  Trusting non-public IPs does not work well (e.g. with lokinet where
  all .loki addresses resolve to a local IP).

- ssl fingerprint option is removed; this isn't supported by curl
  (because it is essentially just duplicating what a custom cainfo
  bundle does)

- --daemon-ssl-allow-chained is removed; it wasn't a useful option (if
  you don't want chaining, don't specify a cainfo chain).

- --daemon-address is now a URL instead of just host:port.  (If you omit
  the protocol, http:// is prepended).

- --daemon-host and --daemon-port are now deprecated and produce a
  warning (in simplewallet) if used; the replacement is to use
  --daemon-address.

- --daemon-ssl is deprecated; specify --daemon-address=https://whatever
  instead.

- the above three are now hidden from --help

- reordered the wallet connection options to make more logical sense.
This commit is contained in:
Jason Rhinelander 2020-07-26 17:29:49 -03:00
parent 9c3e75805d
commit fb0aff57f6
83 changed files with 1549 additions and 5993 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.drone.jsonnet
View File

@ -1,7 +1,7 @@
local default_deps_base='libsystemd-dev libboost-filesystem-dev libboost-thread-dev libboost-date-time-dev libgtest-dev ' +
'libboost-serialization-dev libboost-program-options-dev libunbound-dev nettle-dev libevent-dev libminiupnpc-dev ' +
'libunwind8-dev libsodium-dev libssl-dev libreadline-dev libhidapi-dev libusb-1.0-0-dev libprotobuf-dev protobuf-compiler python3 ' +
'pkg-config libsqlite3-dev qttools5-dev';
'pkg-config libsqlite3-dev qttools5-dev libcurl4-openssl-dev';
local default_deps='g++ ' + default_deps_base; // g++ sometimes needs replacement
local gtest_filter='-AddressFromURL.Failure:DNSResolver.DNSSEC*:is_hdd.linux_os_root';

3
.gitmodules vendored
View File

@ -22,3 +22,6 @@
[submodule "external/libuv"]
path = external/libuv
url = https://github.com/libuv/libuv.git
[submodule "external/cpr"]
path = external/cpr
url = https://github.com/jagerman/cpr.git

87
contrib/epee/include/net/abstract_http_client.h
View File

@ -1,87 +0,0 @@
// Copyright (c) 2006-2013, Andrey N. Sabelnikov, www.sabelnikov.net
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of the Andrey N. Sabelnikov nor the
// names of its contributors may be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
// WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER BE LIABLE FOR ANY
// DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
// (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
// LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
// ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
// SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#pragma once
#include <string>
#include <optional>
#include "net/net_ssl.h"
#include "net/http_base.h"
#include "net/http_auth.h"
namespace epee
{
namespace net_utils
{
inline const char* get_hex_vals()
{
static constexpr const char hexVals[16] = {'0','1','2','3','4','5','6','7','8','9','A','B','C','D','E','F'};
return hexVals;
}
inline const char* get_unsave_chars()
{
//static constexpr char unsave_chars[] = "\"<>%\\^[]`+$,@:;/!#?=&";
static constexpr const char unsave_chars[] = "\"<>%\\^[]`+$,@:;!#&";
return unsave_chars;
}
bool is_unsafe(unsigned char compare_char);
std::string dec_to_hex(char num, int radix);
int get_index(const char *s, char c);
std::string hex_to_dec_2bytes(const char *s);
std::string convert(char val);
std::string convert_to_url_format(std::string_view uri);
std::string convert_from_url_format(std::string_view uri);
namespace http
{
class abstract_http_client
{
public:
abstract_http_client() {}
virtual ~abstract_http_client() {}
bool set_server(const std::string& address, std::optional<login> user, ssl_options_t ssl_options = ssl_support_t::e_ssl_support_autodetect);
virtual void set_server(std::string host, std::string port, std::optional<login> user, ssl_options_t ssl_options = ssl_support_t::e_ssl_support_autodetect) = 0;
virtual void set_auto_connect(bool auto_connect) = 0;
virtual bool connect(std::chrono::milliseconds timeout) = 0;
virtual bool disconnect() = 0;
virtual bool is_connected(bool *ssl = NULL) = 0;
virtual bool invoke(const std::string_view uri, const std::string_view method, const std::string& body, std::chrono::milliseconds timeout, const http_response_info** ppresponse_info = NULL, const fields_list& additional_params = fields_list()) = 0;
virtual bool invoke_get(const std::string_view uri, std::chrono::milliseconds timeout, const std::string& body = std::string(), const http_response_info** ppresponse_info = NULL, const fields_list& additional_params = fields_list()) = 0;
virtual bool invoke_post(const std::string_view uri, const std::string& body, std::chrono::milliseconds timeout, const http_response_info** ppresponse_info = NULL, const fields_list& additional_params = fields_list()) = 0;
virtual uint64_t get_bytes_sent() const = 0;
virtual uint64_t get_bytes_received() const = 0;
};
class http_client_factory
{
public:
virtual ~http_client_factory() {}
virtual std::unique_ptr<abstract_http_client> create() = 0;
};
}
}
}

23
contrib/epee/include/net/abstract_tcp_server2.h
View File

@ -45,7 +45,6 @@
#include <memory>
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/asio/steady_timer.hpp>
#include "net_utils_base.h"
#include "connection_basic.hpp"
@ -98,13 +97,11 @@ namespace net_utils
/// Construct a connection with the given io_service.
explicit connection( boost::asio::io_service& io_service,
std::shared_ptr<shared_state> state,
t_connection_type connection_type,
epee::net_utils::ssl_support_t ssl_support);
t_connection_type connection_type);
explicit connection( boost::asio::ip::tcp::socket&& sock,
std::shared_ptr<shared_state> state,
t_connection_type connection_type,
epee::net_utils::ssl_support_t ssl_support);
t_connection_type connection_type);
@ -163,7 +160,6 @@ namespace net_utils
/// Buffer for incoming data.
std::array<char, 8192> buffer_;
size_t buffer_ssl_init_fill;
t_connection_context context;
@ -208,7 +204,6 @@ namespace net_utils
{
CONNECT_SUCCESS,
CONNECT_FAILURE,
CONNECT_NO_SSL,
};
public:
@ -225,11 +220,9 @@ namespace net_utils
void create_server_type_map();
bool init_server(uint32_t port, const std::string& address = "0.0.0.0",
uint32_t port_ipv6 = 0, const std::string& address_ipv6 = "::", bool use_ipv6 = false, bool require_ipv4 = true,
ssl_options_t ssl_options = ssl_support_t::e_ssl_support_autodetect);
uint32_t port_ipv6 = 0, const std::string& address_ipv6 = "::", bool use_ipv6 = false, bool require_ipv4 = true);
bool init_server(const std::string port, const std::string& address = "0.0.0.0",
const std::string port_ipv6 = "", const std::string address_ipv6 = "::", bool use_ipv6 = false, bool require_ipv4 = true,
ssl_options_t ssl_options = ssl_support_t::e_ssl_support_autodetect);
const std::string port_ipv6 = "", const std::string address_ipv6 = "::", bool use_ipv6 = false, bool require_ipv4 = true);
/// Run the server's io_service loop.
bool run_server(size_t threads_count, bool wait = true);
@ -257,11 +250,11 @@ namespace net_utils
default_remote = std::move(remote);
}
bool add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote, epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect);
try_connect_result_t try_connect(connection_ptr new_connection_l, const std::string& adr, const std::string& port, boost::asio::ip::tcp::socket &sock_, const boost::asio::ip::tcp::endpoint &remote_endpoint, const std::string &bind_ip, uint32_t conn_timeout, epee::net_utils::ssl_support_t ssl_support);
bool connect(const std::string& adr, const std::string& port, uint32_t conn_timeot, t_connection_context& cn, const std::string& bind_ip = "0.0.0.0", epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect);
bool add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote);
try_connect_result_t try_connect(connection_ptr new_connection_l, const std::string& adr, const std::string& port, boost::asio::ip::tcp::socket &sock_, const boost::asio::ip::tcp::endpoint &remote_endpoint, const std::string &bind_ip, uint32_t conn_timeout);
bool connect(const std::string& adr, const std::string& port, uint32_t conn_timeot, t_connection_context& cn, const std::string& bind_ip = "0.0.0.0");
template<class t_callback>
bool connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeot, const t_callback &cb, const std::string& bind_ip = "0.0.0.0", epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect);
bool connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeot, const t_callback &cb, const std::string& bind_ip = "0.0.0.0");
typename t_protocol_handler::config_type& get_config_object()
{

194
contrib/epee/include/net/abstract_tcp_server2.inl
View File

@ -82,23 +82,20 @@ PRAGMA_WARNING_DISABLE_VS(4355)
template<class t_protocol_handler>
connection<t_protocol_handler>::connection( boost::asio::io_service& io_service,
std::shared_ptr<shared_state> state,
t_connection_type connection_type,
ssl_support_t ssl_support
t_connection_type connection_type
)
: connection(boost::asio::ip::tcp::socket{io_service}, std::move(state), connection_type, ssl_support)
: connection(boost::asio::ip::tcp::socket{io_service}, std::move(state), connection_type)
{
}
template<class t_protocol_handler>
connection<t_protocol_handler>::connection( boost::asio::ip::tcp::socket&& sock,
std::shared_ptr<shared_state> state,
t_connection_type connection_type,
ssl_support_t ssl_support
t_connection_type connection_type
)
:
connection_basic(std::move(sock), state, ssl_support),
connection_basic(std::move(sock), state),
m_protocol_handler(this, check_and_get(state), context),
buffer_ssl_init_fill(0),
m_connection_type( connection_type ),
m_throttle_speed_in("speed_in", "throttle_speed_in"),
m_throttle_speed_out("speed_out", "throttle_speed_out"),
@ -177,8 +174,7 @@ PRAGMA_WARNING_DISABLE_VS(4355)
const boost::uuids::uuid random_uuid = boost::uuids::random_generator()();
context = t_connection_context{};
bool ssl = m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled;
context.set_details(random_uuid, std::move(real_remote), is_income, ssl);
context.set_details(random_uuid, std::move(real_remote), is_income);
boost::system::error_code ec;
auto local_ep = socket().local_endpoint(ec);
@ -202,21 +198,11 @@ PRAGMA_WARNING_DISABLE_VS(4355)
reset_timer(std::chrono::milliseconds(m_local ? NEW_CONNECTION_TIMEOUT_LOCAL : NEW_CONNECTION_TIMEOUT_REMOTE), false);
// first read on the raw socket to detect SSL for the server
buffer_ssl_init_fill = 0;
if (is_income && m_ssl_support != epee::net_utils::ssl_support_t::e_ssl_support_disabled)
socket().async_receive(boost::asio::buffer(buffer_),
boost::asio::socket_base::message_peek,
strand_.wrap(
boost::bind(&connection<t_protocol_handler>::handle_receive, self,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred)));
else
async_read_some(boost::asio::buffer(buffer_),
strand_.wrap(
boost::bind(&connection<t_protocol_handler>::handle_read, self,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred)));
socket().async_read_some(boost::asio::buffer(buffer_),
strand_.wrap(
boost::bind(&connection<t_protocol_handler>::handle_read, self,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred)));
#if !defined(_WIN32) || !defined(__i686)
// not supported before Windows7, too lazy for runtime check
// Just exclude for 32bit windows builds
@ -390,7 +376,7 @@ PRAGMA_WARNING_DISABLE_VS(4355)
}else
{
reset_timer(get_timeout_from_bytes_read(bytes_transferred), false);
async_read_some(boost::asio::buffer(buffer_),
socket().async_read_some(boost::asio::buffer(buffer_),
strand_.wrap(
boost::bind(&connection<t_protocol_handler>::handle_read, connection<t_protocol_handler>::shared_from_this(),
boost::asio::placeholders::error,
@ -427,80 +413,6 @@ PRAGMA_WARNING_DISABLE_VS(4355)
}
//---------------------------------------------------------------------------------
template<class t_protocol_handler>
void connection<t_protocol_handler>::handle_receive(const boost::system::error_code& e,
std::size_t bytes_transferred)
{
TRY_ENTRY();
if (e)
{
// offload the error case
handle_read(e, bytes_transferred);
return;
}
reset_timer(get_timeout_from_bytes_read(bytes_transferred), false);
buffer_ssl_init_fill += bytes_transferred;
if (buffer_ssl_init_fill <= get_ssl_magic_size())
{
socket().async_receive(boost::asio::buffer(buffer_.data() + buffer_ssl_init_fill, buffer_.size() - buffer_ssl_init_fill),
boost::asio::socket_base::message_peek,
strand_.wrap(
boost::bind(&connection<t_protocol_handler>::handle_receive, connection<t_protocol_handler>::shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred)));
return;
}
// detect SSL
if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect)
{
if (is_ssl({buffer_.data(), buffer_ssl_init_fill}))
{
MDEBUG("That looks like SSL");
m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_enabled; // read/write to the SSL socket
}
else
{
MDEBUG("That does not look like SSL");
m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_disabled; // read/write to the raw socket
}
}
if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled)
{
// Handshake
if (!handshake(boost::asio::ssl::stream_base::server))
{
MERROR("SSL handshake failed");
m_want_close_connection = true;
m_ready_to_close = true;
bool do_shutdown = false;
{
std::lock_guard lock{m_send_que_lock};
if(!m_send_que.size())
do_shutdown = true;
}
if(do_shutdown)
shutdown();
return;
}
}
async_read_some(boost::asio::buffer(buffer_),
strand_.wrap(
boost::bind(&connection<t_protocol_handler>::handle_read, connection<t_protocol_handler>::shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred)));
// If an error occurs then no new asynchronous operations are started. This
// means that all shared_ptr references to the connection object will
// disappear and the object will be destroyed automatically after this
// handler returns. The connection class's destructor closes the socket.
CATCH_ENTRY_L0("connection<t_protocol_handler>::handle_receive", void());
}
//---------------------------------------------------------------------------------
template<class t_protocol_handler>
bool connection<t_protocol_handler>::call_run_once_service_io()
{
TRY_ENTRY();
@ -669,7 +581,7 @@ PRAGMA_WARNING_DISABLE_VS(4355)
CHECK_AND_ASSERT_MES( size_now == m_send_que.front().size(), false, "Unexpected queue size");
reset_timer(get_default_timeout(), false);
using namespace boost::placeholders;
async_write(boost::asio::buffer(m_send_que.front().data(), size_now ) ,
boost::asio::async_write(socket(), boost::asio::buffer(m_send_que.front().data(), size_now ) ,
strand_.wrap(
boost::bind(&connection<t_protocol_handler>::handle_write, self, _1, _2)
)
@ -773,12 +685,6 @@ PRAGMA_WARNING_DISABLE_VS(4355)
// Initiate graceful connection closure.
m_timer.cancel();
boost::system::error_code ignored_ec;
if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled)
{
const shared_state &state = static_cast<const shared_state&>(get_state());
if (!state.stop_signal_sent)
socket_.shutdown(ignored_ec);
}
socket().shutdown(boost::asio::ip::tcp::socket::shutdown_both, ignored_ec);
if (!m_host.empty())
{
@ -873,7 +779,7 @@ PRAGMA_WARNING_DISABLE_VS(4355)
do_send_handler_write_from_queue(e, m_send_que.front().size() , m_send_que.size()); // (((H)))
CHECK_AND_ASSERT_MES( size_now == m_send_que.front().size(), void(), "Unexpected queue size");
using namespace boost::placeholders;
async_write(boost::asio::buffer(m_send_que.front().data(), size_now) ,
boost::asio::async_write(socket(), boost::asio::buffer(m_send_que.front().data(), size_now) ,
strand_.wrap(
boost::bind(&connection<t_protocol_handler>::handle_write, connection<t_protocol_handler>::shared_from_this(), _1, _2)
)
@ -961,8 +867,7 @@ PRAGMA_WARNING_DISABLE_VS(4355)
//---------------------------------------------------------------------------------
template<class t_protocol_handler>
bool boosted_tcp_server<t_protocol_handler>::init_server(uint32_t port, const std::string& address,
uint32_t port_ipv6, const std::string& address_ipv6, bool use_ipv6, bool require_ipv4,
ssl_options_t ssl_options)
uint32_t port_ipv6, const std::string& address_ipv6, bool use_ipv6, bool require_ipv4)
{
TRY_ENTRY();
m_stop_signal_sent = false;
@ -973,9 +878,6 @@ PRAGMA_WARNING_DISABLE_VS(4355)
m_use_ipv6 = use_ipv6;
m_require_ipv4 = require_ipv4;
if (ssl_options)
m_state->configure_ssl(std::move(ssl_options));
std::string ipv4_failed = "";
std::string ipv6_failed = "";
try
@ -992,7 +894,7 @@ PRAGMA_WARNING_DISABLE_VS(4355)
boost::asio::ip::tcp::endpoint binded_endpoint = acceptor_.local_endpoint();
m_port = binded_endpoint.port();
MDEBUG("start accept (IPv4)");
new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, m_state->ssl_options().support));
new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type));
acceptor_.async_accept(new_connection_->socket(),
boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept_ipv4, this,
boost::asio::placeholders::error));
@ -1029,7 +931,7 @@ PRAGMA_WARNING_DISABLE_VS(4355)
boost::asio::ip::tcp::endpoint binded_endpoint = acceptor_ipv6.local_endpoint();
m_port_ipv6 = binded_endpoint.port();
MDEBUG("start accept (IPv6)");
new_connection_ipv6.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, m_state->ssl_options().support));
new_connection_ipv6.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type));
acceptor_ipv6.async_accept(new_connection_ipv6->socket(),
boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept_ipv6, this,
boost::asio::placeholders::error));
@ -1067,8 +969,7 @@ PUSH_WARNINGS
DISABLE_GCC_WARNING(maybe-uninitialized)
template<class t_protocol_handler>
bool boosted_tcp_server<t_protocol_handler>::init_server(const std::string port, const std::string& address,
const std::string port_ipv6, const std::string address_ipv6, bool use_ipv6, bool require_ipv4,
ssl_options_t ssl_options)
const std::string port_ipv6, const std::string address_ipv6, bool use_ipv6, bool require_ipv4)
{
uint32_t p = 0;
uint32_t p_ipv6 = 0;
@ -1082,7 +983,7 @@ DISABLE_GCC_WARNING(maybe-uninitialized)
MERROR("Failed to convert port no = " << port_ipv6);
return false;
}
return this->init_server(p, address, p_ipv6, address_ipv6, use_ipv6, require_ipv4, std::move(ssl_options));
return this->init_server(p, address, p_ipv6, address_ipv6, use_ipv6, require_ipv4);
}
POP_WARNINGS
//---------------------------------------------------------------------------------
@ -1261,18 +1162,11 @@ POP_WARNINGS
if (!e)
{
if (m_connection_type == e_connection_type_RPC) {
const char *ssl_message = "unknown";
switch ((*current_new_connection)->get_ssl_support())
{
case epee::net_utils::ssl_support_t::e_ssl_support_disabled: ssl_message = "disabled"; break;
case epee::net_utils::ssl_support_t::e_ssl_support_enabled: ssl_message = "enabled"; break;
case epee::net_utils::ssl_support_t::e_ssl_support_autodetect: ssl_message = "autodetection"; break;
}
MDEBUG("New server for RPC connections, SSL " << ssl_message);
MDEBUG("New server for RPC connections");
(*current_new_connection)->setRpcStation(); // hopefully this is not needed actually
}
connection_ptr conn(std::move((*current_new_connection)));
(*current_new_connection).reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, conn->get_ssl_support()));
(*current_new_connection).reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type));
current_acceptor->async_accept((*current_new_connection)->socket(),
boost::bind(accept_function_pointer, this,
boost::asio::placeholders::error));
@ -1307,18 +1201,18 @@ POP_WARNINGS
assert(m_state != nullptr); // always set in constructor
MERROR("Some problems at accept: " << e.message() << ", connections_count = " << m_state->sock_count);
misc_utils::sleep_no_w(100);
(*current_new_connection).reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, (*current_new_connection)->get_ssl_support()));
(*current_new_connection).reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type));
current_acceptor->async_accept((*current_new_connection)->socket(),
boost::bind(accept_function_pointer, this,
boost::asio::placeholders::error));
}
//---------------------------------------------------------------------------------
template<class t_protocol_handler>
bool boosted_tcp_server<t_protocol_handler>::add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote, epee::net_utils::ssl_support_t ssl_support)
bool boosted_tcp_server<t_protocol_handler>::add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote)
{
if(std::addressof(get_io_service()) == std::addressof(GET_IO_SERVICE(sock)))
{
connection_ptr conn(new connection<t_protocol_handler>(std::move(sock), m_state, m_connection_type, ssl_support));
connection_ptr conn(new connection<t_protocol_handler>(std::move(sock), m_state, m_connection_type));
if(conn->start(false, 1 < m_threads_count, std::move(real_remote)))
{
conn->get_context(out);
@ -1334,7 +1228,7 @@ POP_WARNINGS
}
//---------------------------------------------------------------------------------
template<class t_protocol_handler>
typename boosted_tcp_server<t_protocol_handler>::try_connect_result_t boosted_tcp_server<t_protocol_handler>::try_connect(connection_ptr new_connection_l, const std::string& adr, const std::string& port, boost::asio::ip::tcp::socket &sock_, const boost::asio::ip::tcp::endpoint &remote_endpoint, const std::string &bind_ip, uint32_t conn_timeout, epee::net_utils::ssl_support_t ssl_support)
typename boosted_tcp_server<t_protocol_handler>::try_connect_result_t boosted_tcp_server<t_protocol_handler>::try_connect(connection_ptr new_connection_l, const std::string& adr, const std::string& port, boost::asio::ip::tcp::socket &sock_, const boost::asio::ip::tcp::endpoint &remote_endpoint, const std::string &bind_ip, uint32_t conn_timeout)
{
TRY_ENTRY();
@ -1406,38 +1300,17 @@ POP_WARNINGS
MTRACE("Connected success to " << adr << ':' << port);
const ssl_support_t ssl_support = new_connection_l->get_ssl_support();
if (ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled || ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect)
{
// Handshake
MDEBUG("Handshaking SSL...");
if (!new_connection_l->handshake(boost::asio::ssl::stream_base::client))
{
if (ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect)
{
boost::system::error_code ignored_ec;
sock_.shutdown(boost::asio::ip::tcp::socket::shutdown_both, ignored_ec);
sock_.close();
return CONNECT_NO_SSL;
}
MERROR("SSL handshake failed");
if (sock_.is_open())
sock_.close();
return CONNECT_FAILURE;
}
}
return CONNECT_SUCCESS;
CATCH_ENTRY_L0("boosted_tcp_server<t_protocol_handler>::try_connect", CONNECT_FAILURE);
}
//---------------------------------------------------------------------------------
template<class t_protocol_handler>
bool boosted_tcp_server<t_protocol_handler>::connect(const std::string& adr, const std::string& port, uint32_t conn_timeout, t_connection_context& conn_context, const std::string& bind_ip, epee::net_utils::ssl_support_t ssl_support)
bool boosted_tcp_server<t_protocol_handler>::connect(const std::string& adr, const std::string& port, uint32_t conn_timeout, t_connection_context& conn_context, const std::string& bind_ip)
{
TRY_ENTRY();
connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, ssl_support) );
connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type) );
connections_mutex.lock();
connections_.insert(new_connection_l);
MDEBUG("connections_ size now " << connections_.size());
@ -1522,18 +1395,9 @@ POP_WARNINGS
//boost::asio::ip::tcp::endpoint remote_endpoint(boost::asio::ip::address::from_string(addr.c_str()), port);
boost::asio::ip::tcp::endpoint remote_endpoint(*iterator);
auto try_connect_result = try_connect(new_connection_l, adr, port, sock_, remote_endpoint, bind_ip_to_use, conn_timeout, ssl_support);
auto try_connect_result = try_connect(new_connection_l, adr, port, sock_, remote_endpoint, bind_ip_to_use, conn_timeout);
if (try_connect_result == CONNECT_FAILURE)
return false;
if (ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect && try_connect_result == CONNECT_NO_SSL)
{
// we connected, but could not connect with SSL, try without
MERROR("SSL handshake failed on an autodetect connection, reconnecting without SSL");
new_connection_l->disable_ssl();
try_connect_result = try_connect(new_connection_l, adr, port, sock_, remote_endpoint, bind_ip_to_use, conn_timeout, epee::net_utils::ssl_support_t::e_ssl_support_disabled);
if (try_connect_result != CONNECT_SUCCESS)
return false;
}
// start adds the connection to the config object's list, so we don't need to have it locally anymore
connections_mutex.lock();
@ -1559,10 +1423,10 @@ POP_WARNINGS
}
//---------------------------------------------------------------------------------
template<class t_protocol_handler> template<class t_callback>
bool boosted_tcp_server<t_protocol_handler>::connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeout, const t_callback &cb, const std::string& bind_ip, epee::net_utils::ssl_support_t ssl_support)
bool boosted_tcp_server<t_protocol_handler>::connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeout, const t_callback &cb, const std::string& bind_ip)
{
TRY_ENTRY();
connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, ssl_support) );
connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type) );
connections_mutex.lock();
connections_.insert(new_connection_l);
MDEBUG("connections_ size now " << connections_.size());

62
contrib/epee/include/net/connection_basic.hpp
View File

@ -49,10 +49,8 @@
#include <deque>
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include "shared_sv.h"
#include "net/net_ssl.h"
namespace epee
{
@ -61,26 +59,14 @@ namespace net_utils
class connection_basic_shared_state
{
ssl_options_t ssl_options_;
public:
boost::asio::ssl::context ssl_context;
std::atomic<long> sock_count;
std::atomic<long> sock_number;
connection_basic_shared_state()
: ssl_options_(ssl_support_t::e_ssl_support_disabled),
ssl_context(boost::asio::ssl::context::tlsv12),
sock_count(0),
: sock_count(0),
sock_number(0)
{}
void configure_ssl(ssl_options_t src)
{
ssl_options_ = std::move(src);
ssl_context = ssl_options_.create_context();
}
const ssl_options_t& ssl_options() const noexcept { return ssl_options_; }
};
/************************************************************************/
@ -114,56 +100,20 @@ class connection_basic { // not-templated base class for rapid developmet of som
/// Strand to ensure the connection's handlers are not called concurrently.
boost::asio::io_service::strand strand_;
/// Socket for the connection.
boost::asio::ssl::stream<boost::asio::ip::tcp::socket> socket_;
ssl_support_t m_ssl_support;
boost::asio::ip::tcp::socket socket_;
public:
// first counter is the ++/-- count of current sockets, the other socket_number is only-increasing ++ number generator
connection_basic(boost::asio::ip::tcp::socket&& socket, std::shared_ptr<connection_basic_shared_state> state, ssl_support_t ssl_support);
connection_basic(boost::asio::io_service &io_service, std::shared_ptr<connection_basic_shared_state> state, ssl_support_t ssl_support);
connection_basic(boost::asio::ip::tcp::socket&& socket, std::shared_ptr<connection_basic_shared_state> state);
connection_basic(boost::asio::io_service &io_service, std::shared_ptr<connection_basic_shared_state> state);
virtual ~connection_basic() noexcept(false);
//! \return `shared_state` object passed in construction (ptr never changes).
connection_basic_shared_state& get_state() noexcept { return *m_state; /* verified in constructor */ }
connection_basic(boost::asio::io_service& io_service, std::atomic<long> &ref_sock_count, std::atomic<long> &sock_number, ssl_support_t ssl);
connection_basic(boost::asio::io_service& io_service, std::atomic<long> &ref_sock_count, std::atomic<long> &sock_number);
boost::asio::ip::tcp::socket& socket() { return socket_.next_layer(); }
ssl_support_t get_ssl_support() const { return m_ssl_support; }
void disable_ssl() { m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_disabled; }
bool handshake(boost::asio::ssl::stream_base::handshake_type type)
{
//m_state != nullptr verified in constructor
return m_state->ssl_options().handshake(socket_, type);
}
template<typename MutableBufferSequence, typename ReadHandler>
void async_read_some(const MutableBufferSequence &buffers, ReadHandler &&handler)
{
if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled)
socket_.async_read_some(buffers, std::forward<ReadHandler>(handler));
else
socket().async_read_some(buffers, std::forward<ReadHandler>(handler));
}
template<typename ConstBufferSequence, typename WriteHandler>
void async_write_some(const ConstBufferSequence &buffers, WriteHandler &&handler)
{
if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled)
socket_.async_write_some(buffers, std::forward<WriteHandler>(handler));
else
socket().async_write_some(buffers, std::forward<WriteHandler>(handler));
}
template<typename ConstBufferSequence, typename WriteHandler>
void async_write(const ConstBufferSequence &buffers, WriteHandler &&handler)
{
if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled)
boost::asio::async_write(socket_, buffers, std::forward<WriteHandler>(handler));
else
boost::asio::async_write(socket(), buffers, std::forward<WriteHandler>(handler));
}
boost::asio::ip::tcp::socket& socket() { return socket_; }
// various handlers to be called from connection class:
void do_send_handler_write(const void * ptr , size_t cb);

8
contrib/epee/include/net/http_base.h
View File

@ -38,12 +38,8 @@
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "net.http"
namespace epee
namespace epee::net_utils::http
{
namespace net_utils
{
namespace http
{
enum http_method{
http_method_options,
@ -191,6 +187,4 @@ namespace net_utils
memwipe(&m_body[0], m_body.size());
}
};
}
}
}

886
contrib/epee/include/net/http_client.h
View File

@ -1,886 +0,0 @@
// Copyright (c) 2006-2013, Andrey N. Sabelnikov, www.sabelnikov.net
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of the Andrey N. Sabelnikov nor the
// names of its contributors may be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
// WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER BE LIABLE FOR ANY
// DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
// (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
// LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
// ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
// SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
#pragma once
#include <ctype.h>
#include <boost/lexical_cast.hpp>
//#include <mbstring.h>
#include <regex>
#include <algorithm>
#include <cctype>
#include <functional>
#include <memory>
#include <optional>
#include <string_view>
#include "net_helper.h"
#include "abstract_http_client.h"
#include "http_client_base.h"
#ifdef HTTP_ENABLE_GZIP
#include "gzip_encoding.h"
#endif
#include "string_tools.h"
#include "http_base.h"
#include "http_auth.h"
#include "to_nonconst_iterator.h"
#include "net_parse_helpers.h"
//#include "shlwapi.h"
//#pragma comment(lib, "shlwapi.lib")
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "net.http"
namespace epee
{
namespace net_utils
{
/*struct url
{
public:
void parse(const std::string& url_s)
{
const string prot_end("://");
string::const_iterator prot_i = search(url_s.begin(), url_s.end(),
prot_end.begin(), prot_end.end());
protocol_.reserve(distance(url_s.begin(), prot_i));
transform(url_s.begin(), prot_i,
back_inserter(protocol_),
ptr_fun<int,int>(tolower)); // protocol is icase
if( prot_i == url_s.end() )
return;
advance(prot_i, prot_end.length());
string::const_iterator path_i = find(prot_i, url_s.end(), '/');
host_.reserve(distance(prot_i, path_i));
transform(prot_i, path_i,
back_inserter(host_),
ptr_fun<int,int>(tolower)); // host is icase
string::const_iterator query_i = find(path_i, url_s.end(), '?');
path_.assign(path_i, query_i);
if( query_i != url_s.end() )
++query_i;
query_.assign(query_i, url_s.end());
}
std::string protocol_;
std::string host_;
std::string path_;
std::string query_;
};*/
//---------------------------------------------------------------------------
namespace http
{
inline const std::regex rexp_match_gzip{"gzip|(deflate)", std::regex::icase};
inline const std::regex rexp_match_close{"^\\s*close", std::regex::icase};
inline const std::regex rexp_match_multipart_type{
R"re(^\s*multipart/[\w-]+; boundary=(?:"(.*?)"|\\"(.*?)\\"|([^\s;]*)))re", std::regex::icase};
template<typename net_client_type>
class http_simple_client_template : public i_target_handler, public abstract_http_client
{
private:
enum reciev_machine_state
{
reciev_machine_state_header,
reciev_machine_state_body_content_len,
reciev_machine_state_body_connection_close,
reciev_machine_state_body_chunked,
reciev_machine_state_done,
reciev_machine_state_error
};
enum chunked_state{
http_chunked_state_chunk_head,
http_chunked_state_chunk_body,
http_chunked_state_done,
http_chunked_state_undefined
};
net_client_type m_net_client;
std::string m_host_buff;
std::string m_port;
http_client_auth m_auth;
std::string m_header_cache;
http_response_info m_response_info;
size_t m_len_in_summary;
size_t m_len_in_remain;
//std::string* m_ptarget_buffer;
std::shared_ptr<i_sub_handler> m_pcontent_encoding_handler;
reciev_machine_state m_state;
chunked_state m_chunked_state;
std::string m_chunked_cache;
bool m_auto_connect;
mutable std::recursive_mutex m_lock;
public:
explicit http_simple_client_template()
: i_target_handler(), abstract_http_client()
, m_net_client()
, m_host_buff()
, m_port()
, m_auth()
, m_header_cache()
, m_response_info()
, m_len_in_summary(0)
, m_len_in_remain(0)
, m_pcontent_encoding_handler(nullptr)
, m_state()
, m_chunked_state()
, m_chunked_cache()
, m_auto_connect(true)
, m_lock()
{}
const std::string &get_host() const { return m_host_buff; };
const std::string &get_port() const { return m_port; };
using abstract_http_client::set_server;
void set_server(std::string host, std::string port, std::optional<login> user, ssl_options_t ssl_options = ssl_support_t::e_ssl_support_autodetect) override
{
std::lock_guard lock{m_lock};
disconnect();
m_host_buff = std::move(host);
m_port = std::move(port);
m_auth = user ? http_client_auth{std::move(*user)} : http_client_auth{};
m_net_client.set_ssl(std::move(ssl_options));
}
void set_auto_connect(bool auto_connect) override
{
m_auto_connect = auto_connect;
}
template<typename F>
void set_connector(F connector)
{
std::lock_guard lock{m_lock};
m_net_client.set_connector(std::move(connector));
}
bool connect(std::chrono::milliseconds timeout) override
{
std::lock_guard lock{m_lock};
return m_net_client.connect(m_host_buff, m_port, timeout);
}
//---------------------------------------------------------------------------
bool disconnect() override
{
std::lock_guard lock{m_lock};
return m_net_client.disconnect();
}
//---------------------------------------------------------------------------
bool is_connected(bool *ssl = NULL) override
{
std::lock_guard lock{m_lock};
return m_net_client.is_connected(ssl);
}
//---------------------------------------------------------------------------
virtual bool handle_target_data(std::string& piece_of_transfer) override
{
std::lock_guard lock{m_lock};
m_response_info.m_body += piece_of_transfer;
piece_of_transfer.clear();
return true;
}
//---------------------------------------------------------------------------
virtual bool on_header(const http_response_info &headers)
{
return true;
}
//---------------------------------------------------------------------------
inline
bool invoke_get(std::string_view uri, std::chrono::milliseconds timeout, const std::string& body = std::string(), const http_response_info** ppresponse_info = NULL, const fields_list& additional_params = fields_list()) override
{
std::lock_guard lock{m_lock};
return invoke(uri, "GET", body, timeout, ppresponse_info, additional_params);
}
//---------------------------------------------------------------------------
inline bool invoke(std::string_view uri, std::string_view method, const std::string& body, std::chrono::milliseconds timeout, const http_response_info** ppresponse_info = NULL, const fields_list& additional_params = fields_list()) override
{
std::lock_guard lock{m_lock};
if(!is_connected())
{
if (!m_auto_connect)
{
MWARNING("Auto connect attempt to " << m_host_buff << ":" << m_port << " disabled");
return false;
}
MDEBUG("Reconnecting...");
if(!connect(timeout))
{
MDEBUG("Failed to connect to " << m_host_buff << ":" << m_port);
return false;
}
}
std::string req_buff{};
req_buff.reserve(2048);
req_buff.append(method.data(), method.size()).append(" ").append(uri.data(), uri.size()).append(" HTTP/1.1\r\n");
add_field(req_buff, "Host", m_host_buff);
add_field(req_buff, "Content-Length", std::to_string(body.size()));
//handle "additional_params"
for(const auto& field : additional_params)
add_field(req_buff, field);
for (unsigned sends = 0; sends < 2; ++sends)
{
const std::size_t initial_size = req_buff.size();
const auto auth = m_auth.get_auth_field(method, uri);
if (auth)
add_field(req_buff, *auth);
req_buff += "\r\n";
//--
bool res = m_net_client.send(req_buff, timeout);
CHECK_AND_ASSERT_MES(res, false, "HTTP_CLIENT: Failed to SEND");
if(body.size())
res = m_net_client.send(body, timeout);
CHECK_AND_ASSERT_MES(res, false, "HTTP_CLIENT: Failed to SEND");
m_response_info.clear();
m_state = reciev_machine_state_header;
if (!handle_reciev(timeout))
return false;
if (m_response_info.m_response_code != 401)
{
if(ppresponse_info)
*ppresponse_info = std::addressof(m_response_info);
return true;
}
switch (m_auth.handle_401(m_response_info))
{
case http_client_auth::kSuccess:
break;
case http_client_auth::kBadPassword:
sends = 2;
break;
default:
case http_client_auth::kParseFailure:
LOG_ERROR("Bad server response for authentication");
return false;
}
req_buff.resize(initial_size); // rollback for new auth generation
}
LOG_ERROR("Client has incorrect username/password for server requiring authentication");
return false;
}
//---------------------------------------------------------------------------
inline bool invoke_post(std::string_view uri, const std::string& body, std::chrono::milliseconds timeout, const http_response_info** ppresponse_info = NULL, const fields_list& additional_params = fields_list()) override
{
std::lock_guard lock{m_lock};
return invoke(uri, "POST", body, timeout, ppresponse_info, additional_params);
}
//---------------------------------------------------------------------------
bool test(const std::string &s, std::chrono::milliseconds timeout) // TEST FUNC ONLY
{
std::lock_guard lock{m_lock};
m_net_client.set_test_data(s);
m_state = reciev_machine_state_header;
return handle_reciev(timeout);
}
//---------------------------------------------------------------------------
uint64_t get_bytes_sent() const override
{
return m_net_client.get_bytes_sent();
}
//---------------------------------------------------------------------------
uint64_t get_bytes_received() const override
{
return m_net_client.get_bytes_received();
}
//---------------------------------------------------------------------------
void wipe_response()
{
m_response_info.wipe();
}
//---------------------------------------------------------------------------
private:
//---------------------------------------------------------------------------
inline bool handle_reciev(std::chrono::milliseconds timeout)
{
std::lock_guard lock{m_lock};
bool keep_handling = true;
bool need_more_data = true;
std::string recv_buffer;
while(keep_handling)
{
if(need_more_data)
{
if(!m_net_client.recv(recv_buffer, timeout))
{
MERROR("Unexpected recv fail");
m_state = reciev_machine_state_error;
}
if(!recv_buffer.size())
{
//connection is going to be closed
if(reciev_machine_state_body_connection_close != m_state)
{
m_state = reciev_machine_state_error;
}
}
need_more_data = false;
}
switch(m_state)
{
case reciev_machine_state_header:
keep_handling = handle_header(recv_buffer, need_more_data);
break;
case reciev_machine_state_body_content_len:
keep_handling = handle_body_content_len(recv_buffer, need_more_data);
break;
case reciev_machine_state_body_connection_close:
keep_handling = handle_body_connection_close(recv_buffer, need_more_data);
break;
case reciev_machine_state_body_chunked:
keep_handling = handle_body_body_chunked(recv_buffer, need_more_data);
break;
case reciev_machine_state_done:
keep_handling = false;
break;
case reciev_machine_state_error:
keep_handling = false;
break;
}
}
m_header_cache.clear();
if(m_state != reciev_machine_state_error)
{
if(m_response_info.m_header_info.m_connection.size() && !string_tools::compare_no_case("close", m_response_info.m_header_info.m_connection))
disconnect();
return true;
}
else
{
LOG_PRINT_L3("Returning false because of wrong state machine. state: " << m_state);
return false;
}
}
//---------------------------------------------------------------------------
inline
bool handle_header(std::string& recv_buff, bool& need_more_data)
{
std::lock_guard lock{m_lock};
if(!recv_buff.size())
{
LOG_ERROR("Connection closed at handle_header");
m_state = reciev_machine_state_error;
return false;
}
m_header_cache += recv_buff;
recv_buff.clear();
std::string::size_type pos = m_header_cache.find("\r\n\r\n");
if(pos != std::string::npos)
{
recv_buff.assign(m_header_cache.begin()+pos+4, m_header_cache.end());
m_header_cache.erase(m_header_cache.begin()+pos+4, m_header_cache.end());
analyze_cached_header_and_invoke_state();
if (!on_header(m_response_info))
{
MDEBUG("Connection cancelled by on_header");
m_state = reciev_machine_state_done;
return false;
}
m_header_cache.clear();
if(!recv_buff.size() && (m_state != reciev_machine_state_error && m_state != reciev_machine_state_done))
need_more_data = true;
return true;
}else
need_more_data = true;
return true;
}
//---------------------------------------------------------------------------
inline
bool handle_body_content_len(std::string& recv_buff, bool& need_more_data)
{
std::lock_guard lock{m_lock};
if(!recv_buff.size())
{
MERROR("Warning: Content-Len mode, but connection unexpectedly closed");
m_state = reciev_machine_state_done;
return true;
}
CHECK_AND_ASSERT_MES(m_len_in_remain >= recv_buff.size(), false, "m_len_in_remain >= recv_buff.size()");
m_len_in_remain -= recv_buff.size();
if (!m_pcontent_encoding_handler->update_in(recv_buff))
{
m_state = reciev_machine_state_done;
return false;
}
if(m_len_in_remain == 0)
m_state = reciev_machine_state_done;
else
need_more_data = true;
return true;
}
//---------------------------------------------------------------------------
inline
bool handle_body_connection_close(std::string& recv_buff, bool& need_more_data)
{
std::lock_guard lock{m_lock};
if(!recv_buff.size())
{
m_state = reciev_machine_state_done;
return true;
}
need_more_data = true;
m_pcontent_encoding_handler->update_in(recv_buff);
return true;
}
//---------------------------------------------------------------------------
inline bool is_hex_symbol(char ch)
{
if( (ch >= '0' && ch <='9')||(ch >= 'A' && ch <='F')||(ch >= 'a' && ch <='f'))
return true;
else
return false;
}
//---------------------------------------------------------------------------
inline
bool get_len_from_chunk_head(const std::string &chunk_head, size_t& result_size)
{
std::stringstream str_stream;
str_stream << std::hex;
if(!(str_stream << chunk_head && str_stream >> result_size))
return false;
return true;
}
//---------------------------------------------------------------------------
inline
bool get_chunk_head(std::string& buff, size_t& chunk_size, bool& is_matched)
{
is_matched = false;
size_t offset = 0;
for(std::string::iterator it = buff.begin(); it!= buff.end(); it++, offset++)
{
if(!is_hex_symbol(*it))
{
if(*it == '\r' || *it == ' ' )
{
offset--;
continue;
}
else if(*it == '\n')
{
std::string chunk_head = buff.substr(0, offset);
if(!get_len_from_chunk_head(chunk_head, chunk_size))
return false;
if(0 == chunk_size)
{
//Here is a small confusion
//In brief - if the chunk is the last one we need to get terminating sequence
//along with the cipher, generally in the "ddd\r\n\r\n" form
for(it++;it != buff.end(); it++)
{
if('\r' == *it)
continue;
else if('\n' == *it)
break;
else
{
LOG_ERROR("http_stream_filter: Wrong last chunk terminator");
return false;
}
}
if(it == buff.end())
return true;
}
buff.erase(buff.begin(), ++it);
is_matched = true;
return true;
}
else
return false;
}
}
return true;
}
//---------------------------------------------------------------------------
inline
bool handle_body_body_chunked(std::string& recv_buff, bool& need_more_data)
{
std::lock_guard lock{m_lock};
if(!recv_buff.size())
{
MERROR("Warning: CHUNKED mode, but connection unexpectedly closed");
m_state = reciev_machine_state_done;
return true;
}
m_chunked_cache += recv_buff;
recv_buff.clear();
bool is_matched = false;
while(true)
{
if(!m_chunked_cache.size())
{
need_more_data = true;
break;
}
switch(m_chunked_state)
{
case http_chunked_state_chunk_head:
if(m_chunked_cache[0] == '\n' || m_chunked_cache[0] == '\r')
{
//optimize a bit
if(m_chunked_cache[0] == '\r' && m_chunked_cache.size()>1 && m_chunked_cache[1] == '\n')
m_chunked_cache.erase(0, 2);
else
m_chunked_cache.erase(0, 1);
break;
}
if(!get_chunk_head(m_chunked_cache, m_len_in_remain, is_matched))
{
LOG_ERROR("http_stream_filter::handle_chunked(*) Failed to get length from chunked head:" << m_chunked_cache);
m_state = reciev_machine_state_error;
return false;
}
if(!is_matched)
{
need_more_data = true;
return true;
}else
{
m_chunked_state = http_chunked_state_chunk_body;
if(m_len_in_remain == 0)
{//last chunk, let stop the stream and fix the chunk queue.
m_state = reciev_machine_state_done;
return true;
}
m_chunked_state = http_chunked_state_chunk_body;
break;
}
break;
case http_chunked_state_chunk_body:
{
std::string chunk_body;
if(m_len_in_remain >= m_chunked_cache.size())
{
m_len_in_remain -= m_chunked_cache.size();
chunk_body.swap(m_chunked_cache);
}else
{
chunk_body.assign(m_chunked_cache, 0, m_len_in_remain);
m_chunked_cache.erase(0, m_len_in_remain);
m_len_in_remain = 0;
}
if (!m_pcontent_encoding_handler->update_in(chunk_body))
{
m_state = reciev_machine_state_error;
return false;
}
if(!m_len_in_remain)
m_chunked_state = http_chunked_state_chunk_head;
}
break;
case http_chunked_state_done:
m_state = reciev_machine_state_done;
return true;
case http_chunked_state_undefined:
default:
LOG_ERROR("http_stream_filter::handle_chunked(): Wrong state" << m_chunked_state);
return false;
}
}
return true;
}
//---------------------------------------------------------------------------
inline bool parse_header(http_header_info& body_info, const std::string& m_cache_to_process)
{
MTRACE("http_stream_filter::parse_cached_header(*)");
const char *ptr = m_cache_to_process.c_str();
while (ptr[0] != '\r' || ptr[1] != '\n')
{
// optional \n
if (*ptr == '\n')
++ptr;
// an identifier composed of letters or -
const char *key_pos = ptr;
while (isalnum(*ptr) || *ptr == '_' || *ptr == '-')
++ptr;
const char *key_end = ptr;
// optional space (not in RFC, but in previous code)
if (*ptr == ' ')
++ptr;
CHECK_AND_ASSERT_MES(*ptr == ':', true, "http_stream_filter::parse_cached_header() invalid header in: " << m_cache_to_process);
++ptr;
// optional whitespace, but not newlines - line folding is obsolete, let's ignore it
while (isblank(*ptr))
++ptr;
const char *value_pos = ptr;
while (*ptr != '\r' && *ptr != '\n')
++ptr;
const char *value_end = ptr;
// optional trailing whitespace
while (value_end > value_pos && isblank(*(value_end-1)))
--value_end;
if (*ptr == '\r')
++ptr;
CHECK_AND_ASSERT_MES(*ptr == '\n', true, "http_stream_filter::parse_cached_header() invalid header in: " << m_cache_to_process);
++ptr;
const std::string key = std::string(key_pos, key_end - key_pos);
const std::string value = std::string(value_pos, value_end - value_pos);
if (!key.empty())
{
if (!string_tools::compare_no_case(key, "Connection"))
body_info.m_connection = value;
else if(!string_tools::compare_no_case(key, "Referrer"))
body_info.m_referer = value;
else if(!string_tools::compare_no_case(key, "Content-Length"))
body_info.m_content_length = value;
else if(!string_tools::compare_no_case(key, "Content-Type"))
body_info.m_content_type = value;
else if(!string_tools::compare_no_case(key, "Transfer-Encoding"))
body_info.m_transfer_encoding = value;
else if(!string_tools::compare_no_case(key, "Content-Encoding"))
body_info.m_content_encoding = value;
else if(!string_tools::compare_no_case(key, "Host"))
body_info.m_host = value;
else if(!string_tools::compare_no_case(key, "Cookie"))
body_info.m_cookie = value;
else if(!string_tools::compare_no_case(key, "User-Agent"))
body_info.m_user_agent = value;
else if(!string_tools::compare_no_case(key, "Origin"))
body_info.m_origin = value;
else
body_info.m_etc_fields.emplace_back(key, value);
}
}
return true;
}
//---------------------------------------------------------------------------
inline bool analyze_first_response_line()
{
//First line response, look like this: "HTTP/1.1 200 OK"
const char *ptr = m_header_cache.c_str();
CHECK_AND_ASSERT_MES(!memcmp(ptr, "HTTP/", 5), false, "Invalid first response line: " + m_header_cache);
ptr += 5;
CHECK_AND_ASSERT_MES(epee::misc_utils::parse::isdigit(*ptr), false, "Invalid first response line: " + m_header_cache);
unsigned long ul;
char *end;
ul = strtoul(ptr, &end, 10);
CHECK_AND_ASSERT_MES(ul <= INT_MAX && *end =='.', false, "Invalid first response line: " + m_header_cache);
m_response_info.m_http_ver_hi = ul;
ptr = end + 1;
CHECK_AND_ASSERT_MES(epee::misc_utils::parse::isdigit(*ptr), false, "Invalid first response line: " + m_header_cache + ", ptr: " << ptr);
ul = strtoul(ptr, &end, 10);
CHECK_AND_ASSERT_MES(ul <= INT_MAX && isblank(*end), false, "Invalid first response line: " + m_header_cache + ", ptr: " << ptr);
m_response_info.m_http_ver_lo = ul;
ptr = end + 1;
while (isblank(*ptr))
++ptr;
CHECK_AND_ASSERT_MES(epee::misc_utils::parse::isdigit(*ptr), false, "Invalid first response line: " + m_header_cache);
ul = strtoul(ptr, &end, 10);
CHECK_AND_ASSERT_MES(ul >= 100 && ul <= 999 && isspace(*end), false, "Invalid first response line: " + m_header_cache);
m_response_info.m_response_code = ul;
ptr = end;
// ignore the optional text, till the end
while (*ptr != '\r' && *ptr != '\n')
++ptr;
if (*ptr == '\r')
++ptr;
CHECK_AND_ASSERT_MES(*ptr == '\n', false, "Invalid first response line: " << m_header_cache);
++ptr;
m_header_cache.erase(0, ptr - m_header_cache.c_str());
return true;
}
inline
bool set_reply_content_encoder()
{
std::smatch result;
if(std::regex_search(m_response_info.m_header_info.m_content_encoding, result, rexp_match_gzip))
{
#ifdef HTTP_ENABLE_GZIP
m_pcontent_encoding_handler.reset(new content_encoding_gzip(this, result[1].matched));
#else
m_pcontent_encoding_handler.reset(new do_nothing_sub_handler(this));
LOG_ERROR("GZIP encoding not supported in this build, please add zlib to your project and define HTTP_ENABLE_GZIP");
return false;
#endif
}
else
{
m_pcontent_encoding_handler.reset(new do_nothing_sub_handler(this));
}
return true;
}
inline
bool analyze_cached_header_and_invoke_state()
{
m_response_info.clear();
analyze_first_response_line();
std::string fake_str; //gcc error workaround
bool res = parse_header(m_response_info.m_header_info, m_header_cache);
CHECK_AND_ASSERT_MES(res, false, "http_stream_filter::analyze_cached_reply_header_and_invoke_state(): failed to analyze reply header: " << m_header_cache);
set_reply_content_encoder();
m_len_in_summary = 0;
bool content_len_valid = false;
if(m_response_info.m_header_info.m_content_length.size())
content_len_valid = string_tools::get_xtype_from_string(m_len_in_summary, m_response_info.m_header_info.m_content_length);
if(!m_len_in_summary && ((m_response_info.m_response_code>=100&&m_response_info.m_response_code<200)
|| 204 == m_response_info.m_response_code
|| 304 == m_response_info.m_response_code) )
{//There will be no response body, server will display the local page with error
m_state = reciev_machine_state_done;
return true;
}else if(m_response_info.m_header_info.m_transfer_encoding.size())
{
string_tools::trim(m_response_info.m_header_info.m_transfer_encoding);
if(string_tools::compare_no_case(m_response_info.m_header_info.m_transfer_encoding, "chunked"))
{
LOG_ERROR("Wrong Transfer-Encoding:" << m_response_info.m_header_info.m_transfer_encoding);
m_state = reciev_machine_state_error;
return false;
}
m_state = reciev_machine_state_body_chunked;
m_chunked_state = http_chunked_state_chunk_head;
return true;
}
else if(!m_response_info.m_header_info.m_content_length.empty())
{
//In the response header the length was specified
if(!content_len_valid)
{
LOG_ERROR("http_stream_filter::analyze_cached_reply_header_and_invoke_state(): Failed to get_len_from_content_length();, m_query_info.m_content_length="<<m_response_info.m_header_info.m_content_length);
m_state = reciev_machine_state_error;
return false;
}
if(!m_len_in_summary)
{
m_state = reciev_machine_state_done;
return true;
}
else
{
m_len_in_remain = m_len_in_summary;
m_state = reciev_machine_state_body_content_len;
return true;
}
}else if(!m_response_info.m_header_info.m_connection.empty() && is_connection_close_field(m_response_info.m_header_info.m_connection))
{ //By indirect signs we suspect that data transfer will end with a connection break
m_state = reciev_machine_state_body_connection_close;
}else if(is_multipart_body(m_response_info.m_header_info, fake_str))
{
m_state = reciev_machine_state_error;
LOG_ERROR("Unsupported MULTIPART BODY.");
return false;
}else
{ //Apparently there are no signs of the form of transfer, will receive data until the connection is closed
m_state = reciev_machine_state_error;
MERROR("Undefined transfer type, consider http_body_transfer_connection_close method. header: " << m_header_cache);
return false;
}
return false;
}
inline
bool is_connection_close_field(const std::string& str)
{
std::smatch result;
return std::regex_search(str, result, rexp_match_close);
}
inline
bool is_multipart_body(const http_header_info& head_info, std::string& boundary)
{
//Check whether this is multi part - if yes, capture boundary immediately
std::smatch result;
if(std::regex_search(head_info.m_content_type, result, rexp_match_multipart_type))
{
for (size_t i : {1, 2, 3})
if(result[i].matched)
{
boundary = result[i];
return true;
}
LOG_ERROR("Failed to match boundary in content-type=" << head_info.m_content_type);
}
return false;
}
};
typedef http_simple_client_template<blocked_mode_client> http_simple_client;
class http_simple_client_factory : public http_client_factory
{
public:
std::unique_ptr<abstract_http_client> create() override {
return std::unique_ptr<epee::net_utils::http::abstract_http_client>(new epee::net_utils::http::http_simple_client());
}
};
}
}
}

76
contrib/epee/include/net/http_client_base.h
View File

@ -1,76 +0,0 @@
// Copyright (c) 2006-2013, Andrey N. Sabelnikov, www.sabelnikov.net
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of the Andrey N. Sabelnikov nor the
// names of its contributors may be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
// WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER BE LIABLE FOR ANY
// DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
// (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
// LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
// ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
// SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
#pragma once
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "net.http"
namespace epee
{
namespace net_utils
{
struct i_sub_handler
{
virtual ~i_sub_handler(){}
virtual bool update_in( std::string& piece_of_transfer)=0;
virtual void stop(std::string& collect_remains)=0;
virtual bool update_and_stop(std::string& collect_remains, bool& is_changed)
{
is_changed = true;
bool res = this->update_in(collect_remains);
if(res)
this->stop(collect_remains);
return res;
}
};
struct i_target_handler
{
virtual ~i_target_handler(){}
virtual bool handle_target_data( std::string& piece_of_transfer)=0;
};
class do_nothing_sub_handler: public i_sub_handler
{
public:
do_nothing_sub_handler(i_target_handler* powner_filter):m_powner_filter(powner_filter)
{}
virtual bool update_in( std::string& piece_of_transfer)
{
return m_powner_filter->handle_target_data(piece_of_transfer);
}
virtual void stop(std::string& collect_remains)
{
}
i_target_handler* m_powner_filter;
};
}
}

5
contrib/epee/include/net/http_server_impl_base.h
View File

@ -57,8 +57,7 @@ namespace epee
bool init(std::function<void(size_t, uint8_t*)> rng, const std::string& bind_port = "0", const std::string& bind_ip = "0.0.0.0",
const std::string& bind_ipv6_address = "::", bool use_ipv6 = false, bool require_ipv4 = true,
std::vector<std::string> access_control_origins = std::vector<std::string>(),
std::optional<net_utils::http::login> user = std::nullopt,
net_utils::ssl_options_t ssl_options = net_utils::ssl_support_t::e_ssl_support_autodetect)
std::optional<net_utils::http::login> user = std::nullopt)
{
//set self as callback handler
@ -79,7 +78,7 @@ namespace epee
{
MGINFO("Binding on " << bind_ipv6_address << " (IPv6):" << bind_port);
}
bool res = m_net_server.init_server(bind_port, bind_ip, bind_port, bind_ipv6_address, use_ipv6, require_ipv4, std::move(ssl_options));
bool res = m_net_server.init_server(bind_port, bind_ip, bind_port, bind_ipv6_address, use_ipv6, require_ipv4);
if(!res)
{
LOG_ERROR("Failed to bind server");

38
contrib/epee/include/net/net_fwd.h
View File

@ -1,38 +0,0 @@
// Copyright (c) 2019, The Monero Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#pragma once
namespace epee
{
namespace net_utils
{
struct ssl_authentication_t;
class ssl_options_t;
}
}

730
contrib/epee/include/net/net_helper.h
View File

@ -1,730 +0,0 @@
// Copyright (c) 2006-2013, Andrey N. Sabelnikov, www.sabelnikov.net
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of the Andrey N. Sabelnikov nor the
// names of its contributors may be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
// WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER BE LIABLE FOR ANY
// DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
// (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
// LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
// ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
// SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
#pragma once
//#include <Winsock2.h>
//#include <Ws2tcpip.h>
#include <atomic>
#include <string>
#include <boost/asio/io_service.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/read.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/asio/steady_timer.hpp>
#include <boost/system/error_code.hpp>
#include <future>
#include <functional>
#include "net/net_utils_base.h"
#include "net/net_ssl.h"
#include "misc_language.h"
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "net"
#ifndef MAKE_IP
#define MAKE_IP( a1, a2, a3, a4 ) (a1|(a2<<8)|(a3<<16)|(a4<<24))
#endif
namespace epee
{
namespace net_utils
{
struct direct_connect
{
std::future<boost::asio::ip::tcp::socket>
operator()(const std::string& addr, const std::string& port, boost::asio::steady_timer&) const;
};
class blocked_mode_client
{
enum try_connect_result_t
{
CONNECT_SUCCESS,
CONNECT_FAILURE,
CONNECT_NO_SSL,
};
struct handler_obj
{
handler_obj(boost::system::error_code& error, size_t& bytes_transferred):ref_error(error), ref_bytes_transferred(bytes_transferred)
{}
handler_obj(const handler_obj& other_obj):ref_error(other_obj.ref_error), ref_bytes_transferred(other_obj.ref_bytes_transferred)
{}
boost::system::error_code& ref_error;
size_t& ref_bytes_transferred;
void operator()(const boost::system::error_code& error, // Result of operation.
std::size_t bytes_transferred // Number of bytes read.
)
{
ref_error = error;
ref_bytes_transferred = bytes_transferred;
}
};
public:
inline
blocked_mode_client() :
m_io_service(),
m_ctx(boost::asio::ssl::context::tlsv12),
m_ssl_socket(new boost::asio::ssl::stream<boost::asio::ip::tcp::socket>(m_io_service, m_ctx)),
m_connector(direct_connect{}),
m_ssl_options(epee::net_utils::ssl_support_t::e_ssl_support_autodetect),
m_initialized(true),
m_connected(false),
m_deadline(m_io_service, std::chrono::steady_clock::time_point::max()),
m_shutdown(false),
m_bytes_sent(0),
m_bytes_received(0)
{
check_deadline();
}
/*! The first/second parameters are host/port respectively. The third
parameter is for setting the timeout callback - the timer is
already set by the caller, the callee only needs to set the
behavior.
Additional asynchronous operations should be queued using the
`io_service` from the timer. The implementation should assume
multi-threaded I/O processing.
If the callee cannot start an asynchronous operation, an exception
should be thrown to signal an immediate failure.
The return value is a future to a connected socket. Asynchronous
failures should use the `set_exception` method. */
using connect_func = std::function<std::future<boost::asio::ip::tcp::socket>(const std::string&, const std::string&, boost::asio::steady_timer&)>;
inline
~blocked_mode_client()
{
//profile_tools::local_coast lc("~blocked_mode_client()", 3);
try { shutdown(); }
catch(...) { /* ignore */ }
}
inline void set_ssl(ssl_options_t ssl_options)
{
if (ssl_options)
m_ctx = ssl_options.create_context();
else
m_ctx = boost::asio::ssl::context(boost::asio::ssl::context::tlsv12);
m_ssl_options = std::move(ssl_options);
}
inline
bool connect(const std::string& addr, int port, std::chrono::milliseconds timeout)
{
return connect(addr, std::to_string(port), timeout);
}
inline
try_connect_result_t try_connect(const std::string& addr, const std::string& port, std::chrono::milliseconds timeout)
{
m_deadline.expires_from_now(timeout);
auto connection = m_connector(addr, port, m_deadline);
do {
m_io_service.reset();
m_io_service.run_one();
} while (connection.wait_for(0s) != std::future_status::ready);
m_ssl_socket->next_layer() = connection.get();
m_deadline.cancel();
if (m_ssl_socket->next_layer().is_open())
{
m_connected = true;
m_deadline.expires_at(std::chrono::steady_clock::time_point::max());
// SSL Options
if (m_ssl_options.support == epee::net_utils::ssl_support_t::e_ssl_support_enabled || m_ssl_options.support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect)
{
if (!m_ssl_options.handshake(*m_ssl_socket, boost::asio::ssl::stream_base::client, addr, timeout))
{
if (m_ssl_options.support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect)
{
boost::system::error_code ignored_ec;
m_ssl_socket->next_layer().shutdown(boost::asio::ip::tcp::socket::shutdown_both, ignored_ec);
m_ssl_socket->next_layer().close();
m_connected = false;
return CONNECT_NO_SSL;
}
else
{
MWARNING("Failed to establish SSL connection");
m_connected = false;
return CONNECT_FAILURE;
}
}
}
return CONNECT_SUCCESS;
}else
{
MWARNING("Some problems at connect, expected open socket");
return CONNECT_FAILURE;
}
}
inline
bool connect(const std::string& addr, const std::string& port, std::chrono::milliseconds timeout)
{
m_connected = false;
try
{
m_ssl_socket->next_layer().close();
// Set SSL options
// disable sslv2
m_ssl_socket.reset(new boost::asio::ssl::stream<boost::asio::ip::tcp::socket>(m_io_service, m_ctx));
// Get a list of endpoints corresponding to the server name.
try_connect_result_t try_connect_result = try_connect(addr, port, timeout);
if (try_connect_result == CONNECT_FAILURE)
return false;
if (m_ssl_options.support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect)
{
if (try_connect_result == CONNECT_NO_SSL)
{
MERROR("SSL handshake failed on an autodetect connection, reconnecting without SSL");
m_ssl_options.support = epee::net_utils::ssl_support_t::e_ssl_support_disabled;
if (try_connect(addr, port, timeout) != CONNECT_SUCCESS)
return false;
}
}
}
catch(const boost::system::system_error& er)
{
MERROR("Some problems at connect, message: " << er.what());
return false;
}
catch(...)
{
MERROR("Some fatal problems.");
return false;
}
return true;
}
//! Change the connection routine (proxy, etc.)
void set_connector(connect_func connector)
{
m_connector = std::move(connector);
}
inline
bool disconnect()
{
try
{
if(m_connected)
{
m_connected = false;
if(m_ssl_options)
shutdown_ssl();
m_ssl_socket->next_layer().shutdown(boost::asio::ip::tcp::socket::shutdown_both);
}
}
catch(const boost::system::system_error& /*er*/)
{
//LOG_ERROR("Some problems at disconnect, message: " << er.what());
return false;
}
catch(...)
{
//LOG_ERROR("Some fatal problems.");
return false;
}
return true;
}
inline
bool send(const std::string& buff, std::chrono::milliseconds timeout)
{
try
{
m_deadline.expires_from_now(timeout);
// Set up the variable that receives the result of the asynchronous
// operation. The error code is set to would_block to signal that the
// operation is incomplete. Asio guarantees that its asynchronous
// operations will never fail with would_block, so any other value in
// ec indicates completion.
boost::system::error_code ec = boost::asio::error::would_block;
async_write(buff.c_str(), buff.size(), ec);
// Block until the asynchronous operation has completed.
while (ec == boost::asio::error::would_block)
{
m_io_service.reset();
m_io_service.run_one();
}
if (ec)
{
LOG_PRINT_L3("Problems at write: " << ec.message());
m_connected = false;
return false;
}else
{
m_deadline.expires_at(std::chrono::steady_clock::time_point::max());
m_bytes_sent += buff.size();
}
}
catch(const boost::system::system_error& er)
{
LOG_ERROR("Some problems at connect, message: " << er.what());
return false;
}
catch(...)
{
LOG_ERROR("Some fatal problems.");
return false;
}
return true;
}
inline
bool send(const void* data, size_t sz)
{
try
{
boost::system::error_code ec;
size_t writen = write(data, sz, ec);
if (!writen || ec)
{
LOG_PRINT_L3("Problems at write: " << ec.message());
m_connected = false;
return false;
}else
{
m_deadline.expires_at(std::chrono::steady_clock::time_point::max());
m_bytes_sent += sz;
}
}
catch(const boost::system::system_error& er)
{
LOG_ERROR("Some problems at send, message: " << er.what());
m_connected = false;
return false;
}
catch(...)
{
LOG_ERROR("Some fatal problems.");
return false;
}
return true;
}
bool is_connected(bool *ssl = NULL) const
{
if (!m_connected || !m_ssl_socket->next_layer().is_open())
return false;
if (ssl)
*ssl = m_ssl_options.support != ssl_support_t::e_ssl_support_disabled;
return true;
}
inline
bool recv(std::string& buff, std::chrono::milliseconds timeout)
{
try
{
// Set a deadline for the asynchronous operation. Since this function uses
// a composed operation (async_read_until), the deadline applies to the
// entire operation, rather than individual reads from the socket.
m_deadline.expires_from_now(timeout);
// Set up the variable that receives the result of the asynchronous
// operation. The error code is set to would_block to signal that the
// operation is incomplete. Asio guarantees that its asynchronous
// operations will never fail with would_block, so any other value in
// ec indicates completion.
//boost::system::error_code ec = boost::asio::error::would_block;
boost::system::error_code ec = boost::asio::error::would_block;
size_t bytes_transfered = 0;
handler_obj hndlr(ec, bytes_transfered);
static const size_t max_size = 16384;
buff.resize(max_size);
async_read(&buff[0], max_size, boost::asio::transfer_at_least(1), hndlr);
// Block until the asynchronous operation has completed.
while (ec == boost::asio::error::would_block && !m_shutdown)
{
m_io_service.reset();
m_io_service.run_one();
}
if (ec)
{
MTRACE("READ ENDS: Connection err_code " << ec.value());
if(ec == boost::asio::error::eof)
{
MTRACE("Connection err_code eof.");
//connection closed there, empty
buff.clear();
return true;
}
MDEBUG("Problems at read: " << ec.message());
m_connected = false;
return false;
}else
{
MTRACE("READ ENDS: Success. bytes_tr: " << bytes_transfered);
m_deadline.expires_at(std::chrono::steady_clock::time_point::max());
}
/*if(!bytes_transfered)
return false;*/
m_bytes_received += bytes_transfered;
buff.resize(bytes_transfered);
return true;
}
catch(const boost::system::system_error& er)
{
LOG_ERROR("Some problems at read, message: " << er.what());
m_connected = false;
return false;
}
catch(...)
{
LOG_ERROR("Some fatal problems at read.");
return false;
}
return false;
}
inline bool recv_n(std::string& buff, int64_t sz, std::chrono::milliseconds timeout)
{
try
{
// Set a deadline for the asynchronous operation. Since this function uses
// a composed operation (async_read_until), the deadline applies to the
// entire operation, rather than individual reads from the socket.
m_deadline.expires_from_now(timeout);
buff.resize(static_cast<size_t>(sz));
boost::system::error_code ec = boost::asio::error::would_block;
size_t bytes_transfered = 0;
handler_obj hndlr(ec, bytes_transfered);
async_read((char*)buff.data(), buff.size(), boost::asio::transfer_at_least(buff.size()), hndlr);
// Block until the asynchronous operation has completed.
while (ec == boost::asio::error::would_block && !m_shutdown)
{
m_io_service.run_one();
}
if (ec)
{
LOG_PRINT_L3("Problems at read: " << ec.message());
m_connected = false;
return false;
}else
{
m_deadline.expires_at(std::chrono::steady_clock::time_point::max());
}
m_bytes_received += bytes_transfered;
if(bytes_transfered != buff.size())
{
LOG_ERROR("Transferred mismatch with transfer_at_least value: m_bytes_transferred=" << bytes_transfered << " at_least value=" << buff.size());
return false;
}
return true;
}
catch(const boost::system::system_error& er)
{
LOG_ERROR("Some problems at read, message: " << er.what());
m_connected = false;
return false;
}
catch(...)
{
LOG_ERROR("Some fatal problems at read.");
return false;
}
return false;
}
bool shutdown()
{
m_deadline.cancel();
boost::system::error_code ec;
if(m_ssl_options)
shutdown_ssl();
m_ssl_socket->next_layer().cancel(ec);
if(ec)
MDEBUG("Problems at cancel: " << ec.message());
m_ssl_socket->next_layer().shutdown(boost::asio::ip::tcp::socket::shutdown_both, ec);
if(ec)
MDEBUG("Problems at shutdown: " << ec.message());
m_ssl_socket->next_layer().close(ec);
if(ec)
MDEBUG("Problems at close: " << ec.message());
m_shutdown = true;
m_connected = false;
return true;
}
boost::asio::io_service& get_io_service()
{
return m_io_service;
}
boost::asio::ip::tcp::socket& get_socket()
{
return m_ssl_socket->next_layer();
}
uint64_t get_bytes_sent() const
{
return m_bytes_sent;
}
uint64_t get_bytes_received() const
{
return m_bytes_received;
}
private:
void check_deadline()
{
// Check whether the deadline has passed. We compare the deadline against
// the current time since a new asynchronous operation may have moved the
// deadline before this actor had a chance to run.
if (m_deadline.expires_at() <= std::chrono::steady_clock::now())
{
// The deadline has passed. The socket is closed so that any outstanding
// asynchronous operations are cancelled. This allows the blocked
// connect(), read_line() or write_line() functions to return.
LOG_PRINT_L3("Timed out socket");
m_connected = false;
m_ssl_socket->next_layer().close();
// There is no longer an active deadline. The expiry is set to positive
// infinity so that the actor takes no action until a new deadline is set.
m_deadline.expires_at(std::chrono::steady_clock::time_point::max());
}
// Put the actor back to sleep.
m_deadline.async_wait([this] (const boost::system::error_code&) { check_deadline(); });
}
void shutdown_ssl() {
// ssl socket shutdown blocks if server doesn't respond. We close after 2 secs
boost::system::error_code ec = boost::asio::error::would_block;
m_deadline.expires_from_now(2s);
m_ssl_socket->async_shutdown([&ec](const boost::system::error_code& e) { ec = e; });
while (ec == boost::asio::error::would_block)
{
m_io_service.reset();
m_io_service.run_one();
}
// Ignore "short read" error
if (ec.category() == boost::asio::error::get_ssl_category() &&
ec.value() !=
boost::asio::ssl::error::stream_truncated
)
MDEBUG("Problems at ssl shutdown: " << ec.message());
}
protected:
bool write(const void* data, size_t sz, boost::system::error_code& ec)
{
bool success;
if(m_ssl_options.support != ssl_support_t::e_ssl_support_disabled)
success = boost::asio::write(*m_ssl_socket, boost::asio::buffer(data, sz), ec);
else
success = boost::asio::write(m_ssl_socket->next_layer(), boost::asio::buffer(data, sz), ec);
return success;
}
void async_write(const void* data, size_t sz, boost::system::error_code& ec)
{
auto handler = [&ec](const boost::system::error_code& e, size_t) { ec = e; };
if(m_ssl_options.support != ssl_support_t::e_ssl_support_disabled)
boost::asio::async_write(*m_ssl_socket, boost::asio::buffer(data, sz), std::move(handler));
else
boost::asio::async_write(m_ssl_socket->next_layer(), boost::asio::buffer(data, sz), std::move(handler));
}
void async_read(char* buff, size_t sz, boost::asio::detail::transfer_at_least_t transfer_at_least, handler_obj& hndlr)
{
if(m_ssl_options.support == ssl_support_t::e_ssl_support_disabled)
boost::asio::async_read(m_ssl_socket->next_layer(), boost::asio::buffer(buff, sz), transfer_at_least, hndlr);
else
boost::asio::async_read(*m_ssl_socket, boost::asio::buffer(buff, sz), transfer_at_least, hndlr);
}
protected:
boost::asio::io_service m_io_service;
boost::asio::ssl::context m_ctx;
std::shared_ptr<boost::asio::ssl::stream<boost::asio::ip::tcp::socket>> m_ssl_socket;
connect_func m_connector;
ssl_options_t m_ssl_options;
bool m_initialized;
bool m_connected;
boost::asio::steady_timer m_deadline;
std::atomic<bool> m_shutdown;
std::atomic<uint64_t> m_bytes_sent;
std::atomic<uint64_t> m_bytes_received;
};
/************************************************************************/
/* */
/************************************************************************/
class async_blocked_mode_client: public blocked_mode_client
{
public:
async_blocked_mode_client():m_send_deadline(blocked_mode_client::m_io_service)
{
// No deadline is required until the first socket operation is started. We
// set the deadline to positive infinity so that the actor takes no action
// until a specific deadline is set.
m_send_deadline.expires_at(std::chrono::steady_clock::time_point::max());
// Start the persistent actor that checks for deadline expiry.
check_send_deadline();
}
~async_blocked_mode_client()
{
m_send_deadline.cancel();
}
bool shutdown()
{
blocked_mode_client::shutdown();
m_send_deadline.cancel();
return true;
}
inline
bool send(const void* data, size_t sz)
{
try
{
boost::system::error_code ec;
size_t writen = write(data, sz, ec);
if (!writen || ec)
{
LOG_PRINT_L3("Problems at write: " << ec.message());
return false;
}else
{
m_send_deadline.expires_at(std::chrono::steady_clock::time_point::max());
}
}
catch(const boost::system::system_error& er)
{
LOG_ERROR("Some problems at connect, message: " << er.what());
return false;
}
catch(...)
{
LOG_ERROR("Some fatal problems.");
return false;
}
return true;
}
private:
boost::asio::steady_timer m_send_deadline;
void check_send_deadline()
{
// Check whether the deadline has passed. We compare the deadline against
// the current time since a new asynchronous operation may have moved the
// deadline before this actor had a chance to run.
if (m_send_deadline.expires_at() <= std::chrono::steady_clock::now())
{
// The deadline has passed. The socket is closed so that any outstanding
// asynchronous operations are cancelled. This allows the blocked
// connect(), read_line() or write_line() functions to return.
LOG_PRINT_L3("Timed out socket");
m_ssl_socket->next_layer().close();
// There is no longer an active deadline. The expiry is set to positive
// infinity so that the actor takes no action until a new deadline is set.
m_send_deadline.expires_at(std::chrono::steady_clock::time_point::max());
}
// Put the actor back to sleep.
m_send_deadline.async_wait([this] (const boost::system::error_code&) { check_send_deadline(); });
}
};
}
}

151
contrib/epee/include/net/net_ssl.h
View File

@ -1,151 +0,0 @@
// Copyright (c) 2006-2013, Andrey N. Sabelnikov, www.sabelnikov.net
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of the Andrey N. Sabelnikov nor the
// names of its contributors may be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
// WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER BE LIABLE FOR ANY
// DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
// (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
// LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
// ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
// SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
#ifndef _NET_SSL_H
#define _NET_SSL_H
#include <chrono>
#include <cstdint>
#include <string>
#include <vector>
#include <string_view>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/system/error_code.hpp>
#define SSL_FINGERPRINT_SIZE 32
namespace epee
{
using namespace std::literals;
namespace net_utils
{
enum class ssl_support_t: uint8_t {
e_ssl_support_disabled,
e_ssl_support_enabled,
e_ssl_support_autodetect,
};
enum class ssl_verification_t : uint8_t
{
none = 0, //!< Do not verify peer.
system_ca, //!< Verify peer via system ca only (do not inspect user certificates)
user_certificates,//!< Verify peer via specific (non-chain) certificate(s) only.
user_ca //!< Verify peer via specific (possibly chain) certificate(s) only.
};
struct ssl_authentication_t
{
std::string private_key_path; //!< Private key used for authentication
std::string certificate_path; //!< Certificate used for authentication to peer.
//! Load `private_key_path` and `certificate_path` into `ssl_context`.
void use_ssl_certificate(boost::asio::ssl::context &ssl_context) const;
};
/*!
\note `verification != disabled && support == disabled` is currently
"allowed" via public interface but obviously invalid configuation.
*/
class ssl_options_t
{
// force sorted behavior in private
std::vector<std::vector<std::uint8_t>> fingerprints_;
public:
std::string ca_path;
ssl_authentication_t auth;
ssl_support_t support;
ssl_verification_t verification;
//! Verification is set to system ca unless SSL is disabled.
ssl_options_t(ssl_support_t support)
: fingerprints_(),
ca_path(),
auth(),
support(support),
verification(support == ssl_support_t::e_ssl_support_disabled ? ssl_verification_t::none : ssl_verification_t::system_ca)
{}
//! Provide user fingerprints and/or ca path. Enables SSL and user_certificate verification
ssl_options_t(std::vector<std::vector<std::uint8_t>> fingerprints, std::string ca_path);
ssl_options_t(const ssl_options_t&) = default;
ssl_options_t(ssl_options_t&&) = default;
ssl_options_t& operator=(const ssl_options_t&) = default;
ssl_options_t& operator=(ssl_options_t&&) = default;
//! \return False iff ssl is disabled, otherwise true.
explicit operator bool() const noexcept { return support != ssl_support_t::e_ssl_support_disabled; }
//! \retrurn True if `host` can be verified using `this` configuration WITHOUT system "root" CAs.
bool has_strong_verification(std::string_view host) const noexcept;
//! Search against internal fingerprints. Always false if `behavior() != user_certificate_check`.
bool has_fingerprint(boost::asio::ssl::verify_context &ctx) const;
boost::asio::ssl::context create_context() const;
/*! \note If `this->support == autodetect && this->verification != none`,
then the handshake will not fail when peer verification fails. The
assumption is that a re-connect will be attempted, so a warning is
logged instead of failure.
\note It is strongly encouraged that clients using `system_ca`
verification provide a non-empty `host` for rfc2818 verification.
\param socket Used in SSL handshake and verification
\param type Client or server
\param host This parameter is only used when
`type == client && !host.empty()`. The value is sent to the server for
situations where multiple hostnames are being handled by a server. If
`verification == system_ca` the client also does a rfc2818 check to
ensure that the server certificate is to the provided hostname.
\return True if the SSL handshake completes with peer verification
settings. */
bool handshake(
boost::asio::ssl::stream<boost::asio::ip::tcp::socket> &socket,
boost::asio::ssl::stream_base::handshake_type type,
const std::string& host = {},
std::chrono::milliseconds timeout = std::chrono::seconds(15)) const;
};
// https://security.stackexchange.com/questions/34780/checking-client-hello-for-https-classification
constexpr size_t get_ssl_magic_size() { return 9; }
bool is_ssl(std::string_view data);
bool ssl_support_from_string(ssl_support_t &ssl, std::string_view s);
bool create_ec_ssl_certificate(EVP_PKEY *&pkey, X509 *&cert);
bool create_rsa_ssl_certificate(EVP_PKEY *&pkey, X509 *&cert);
}
}
#endif //_NET_SSL_H

13
contrib/epee/include/net/net_utils_base.h
View File

@ -350,7 +350,6 @@ namespace net_utils
const network_address m_remote_address;
const bool m_is_income;
std::chrono::steady_clock::time_point m_started;
const bool m_ssl;
std::chrono::steady_clock::time_point m_last_recv;
std::chrono::steady_clock::time_point m_last_send;
uint64_t m_recv_cnt;
@ -361,7 +360,7 @@ namespace net_utils
double m_max_speed_up;
connection_context_base(boost::uuids::uuid connection_id,
const network_address &remote_address, bool is_income, bool ssl,
const network_address &remote_address, bool is_income,
std::chrono::steady_clock::time_point last_recv = std::chrono::steady_clock::time_point::min(),
std::chrono::steady_clock::time_point last_send = std::chrono::steady_clock::time_point::min(),
uint64_t recv_cnt = 0, uint64_t send_cnt = 0):
@ -369,7 +368,6 @@ namespace net_utils
m_remote_address(remote_address),
m_is_income(is_income),
m_started(std::chrono::steady_clock::now()),
m_ssl(ssl),
m_last_recv(last_recv),
m_last_send(last_send),
m_recv_cnt(recv_cnt),
@ -384,7 +382,6 @@ namespace net_utils
m_remote_address(),
m_is_income(false),
m_started(std::chrono::steady_clock::now()),
m_ssl(false),
m_last_recv(std::chrono::steady_clock::time_point::min()),
m_last_send(std::chrono::steady_clock::time_point::min()),
m_recv_cnt(0),
@ -397,22 +394,22 @@ namespace net_utils
connection_context_base(const connection_context_base& a): connection_context_base()
{
set_details(a.m_connection_id, a.m_remote_address, a.m_is_income, a.m_ssl);
set_details(a.m_connection_id, a.m_remote_address, a.m_is_income);
}
connection_context_base& operator=(const connection_context_base& a)
{
set_details(a.m_connection_id, a.m_remote_address, a.m_is_income, a.m_ssl);
set_details(a.m_connection_id, a.m_remote_address, a.m_is_income);
return *this;
}
private:
template<class t_protocol_handler>
friend class connection;
void set_details(boost::uuids::uuid connection_id, const network_address &remote_address, bool is_income, bool ssl)
void set_details(boost::uuids::uuid connection_id, const network_address &remote_address, bool is_income)
{
this->~connection_context_base();
new(this) connection_context_base(connection_id, remote_address, is_income, ssl);
new(this) connection_context_base(connection_id, remote_address, is_income);
}
};

22
contrib/epee/src/CMakeLists.txt
View File

@ -26,9 +26,23 @@
# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
# THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
add_library(epee hex.cpp abstract_http_client.cpp http_auth.cpp mlog.cpp net_helper.cpp net_utils_base.cpp string_tools.cpp wipeable_string.cpp
levin_base.cpp memwipe.c connection_basic.cpp network_throttle.cpp network_throttle-detail.cpp mlocker.cpp buffer.cpp net_ssl.cpp portable_storage.cpp
time_helper.cpp)
add_library(epee
buffer.cpp
connection_basic.cpp
hex.cpp
http_auth.cpp
levin_base.cpp
memwipe.c
mlocker.cpp
mlog.cpp
net_utils_base.cpp
network_throttle.cpp
network_throttle-detail.cpp
portable_storage.cpp
string_tools.cpp
time_helper.cpp
wipeable_string.cpp
)
if (TARGET readline)
target_sources(epee PRIVATE readline_buffer.cpp)
@ -54,8 +68,6 @@ target_link_libraries(epee
PRIVATE
Boost::filesystem
Boost::thread
OpenSSL::SSL
OpenSSL::Crypto
extra)
target_include_directories(epee PUBLIC ../include)

131
contrib/epee/src/abstract_http_client.cpp
View File

@ -1,131 +0,0 @@
#include "net/abstract_http_client.h"
#include "net/http_base.h"
#include "net/net_parse_helpers.h"
#undef MONERO_DEFAULT_LOG_CATEGORY
#define MONERO_DEFAULT_LOG_CATEGORY "net.http"
namespace epee
{
namespace net_utils
{
//----------------------------------------------------------------------------------------------------
bool is_unsafe(unsigned char compare_char)
{
if(compare_char <= 32 || compare_char >= 123)
return true;
const char* punsave = get_unsave_chars();
for(int ichar_pos = 0; 0!=punsave[ichar_pos] ;ichar_pos++)
if(compare_char == punsave[ichar_pos])
return true;
return false;
}
//----------------------------------------------------------------------------------------------------
std::string dec_to_hex(char num, int radix)
{
int temp=0;
std::string csTmp;
int num_char;
num_char = (int) num;
if (num_char < 0)
num_char = 256 + num_char;
while (num_char >= radix)
{
temp = num_char % radix;
num_char = (int)floor((float)num_char / (float)radix);
csTmp = get_hex_vals()[temp];
}
csTmp += get_hex_vals()[num_char];
if(csTmp.size() < 2)
{
csTmp += '0';
}
std::reverse(csTmp.begin(), csTmp.end());
//_mbsrev((unsigned char*)csTmp.data());
return csTmp;
}
//----------------------------------------------------------------------------------------------------
int get_index(const char *s, char c)
{
const char *ptr = (const char*)memchr(s, c, 16);
return ptr ? ptr-s : -1;
}
//----------------------------------------------------------------------------------------------------
std::string hex_to_dec_2bytes(const char *s)
{
const char *hex = get_hex_vals();
int i0 = get_index(hex, toupper(s[0]));
int i1 = get_index(hex, toupper(s[1]));
if (i0 < 0 || i1 < 0)
return std::string("%") + std::string(1, s[0]) + std::string(1, s[1]);
return std::string(1, i0 * 16 | i1);
}
//----------------------------------------------------------------------------------------------------
std::string convert(char val)
{
std::string csRet;
csRet += "%";
csRet += dec_to_hex(val, 16);
return csRet;
}
//----------------------------------------------------------------------------------------------------
std::string convert_to_url_format(std::string_view uri)
{
std::string result;
for(size_t i = 0; i!= uri.size(); i++)
{
if(is_unsafe(uri[i]))
result += convert(uri[i]);
else
result += uri[i];
}
return result;
}
//----------------------------------------------------------------------------------------------------
std::string convert_from_url_format(std::string_view uri)
{
std::string result;
for(size_t i = 0; i!= uri.size(); i++)
{
if(uri[i] == '%' && i + 2 < uri.size())
{
result += hex_to_dec_2bytes(uri.data() + i + 1);
i += 2;
}
else
result += uri[i];
}
return result;
}
namespace http
{
//----------------------------------------------------------------------------------------------------
bool epee::net_utils::http::abstract_http_client::set_server(const std::string& address, std::optional<login> user, ssl_options_t ssl_options)
{
http::url_content parsed{};
const bool r = parse_url(address, parsed);
CHECK_AND_ASSERT_MES(r, false, "failed to parse url: " << address);
set_server(std::move(parsed.host), std::to_string(parsed.port), std::move(user), std::move(ssl_options));
return true;
}
}
}
}

27
contrib/epee/src/connection_basic.cpp
View File

@ -65,15 +65,6 @@ namespace epee
namespace net_utils
{
namespace
{
boost::asio::ssl::context& get_context(connection_basic_shared_state* state)
{
CHECK_AND_ASSERT_THROW_MES(state != nullptr, "state shared_ptr cannot be null");
return state->ssl_context;
}
}
std::string to_string(t_connection_type type)
{
if (type == e_connection_type_NET)
@ -127,41 +118,39 @@ connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_thro
int connection_basic_pimpl::m_default_tos;
// methods:
connection_basic::connection_basic(boost::asio::ip::tcp::socket&& sock, std::shared_ptr<connection_basic_shared_state> state, ssl_support_t ssl_support)
connection_basic::connection_basic(boost::asio::ip::tcp::socket&& sock, std::shared_ptr<connection_basic_shared_state> state)
:
m_state(std::move(state)),
mI( new connection_basic_pimpl("peer") ),
strand_(GET_IO_SERVICE(sock)),
socket_(GET_IO_SERVICE(sock), get_context(m_state.get())),
socket_(GET_IO_SERVICE(sock)),
m_want_close_connection(false),
m_was_shutdown(false),
m_is_multithreaded(false),
m_ssl_support(ssl_support)
m_is_multithreaded(false)
{
// add nullptr checks if removed
assert(m_state != nullptr); // release runtime check in get_context
socket_.next_layer() = std::move(sock);
socket_ = std::move(sock);
++(m_state->sock_count); // increase the global counter
mI->m_peer_number = m_state->sock_number.fetch_add(1); // use, and increase the generated number
std::string remote_addr_str = "?";
try { boost::system::error_code e; remote_addr_str = socket().remote_endpoint(e).address().to_string(); } catch(...){} ;
try { boost::system::error_code e; remote_addr_str = socket_.remote_endpoint(e).address().to_string(); } catch(...){} ;
MDEBUG("Spawned connection #"<<mI->m_peer_number<<" to " << remote_addr_str << " currently we have sockets count:" << m_state->sock_count);
}
connection_basic::connection_basic(boost::asio::io_service &io_service, std::shared_ptr<connection_basic_shared_state> state, ssl_support_t ssl_support)
connection_basic::connection_basic(boost::asio::io_service &io_service, std::shared_ptr<connection_basic_shared_state> state)
:
m_state(std::move(state)),
mI( new connection_basic_pimpl("peer") ),
strand_(io_service),
socket_(io_service, get_context(m_state.get())),
socket_(io_service),
m_want_close_connection(false),
m_was_shutdown(false),
m_is_multithreaded(false),
m_ssl_support(ssl_support)
m_is_multithreaded(false)
{
// add nullptr checks if removed
assert(m_state != nullptr); // release runtime check in get_context

2
contrib/epee/src/mlog.cpp
View File

@ -100,7 +100,7 @@ static const char *get_default_categories(int level)
switch (level)
{
case 0:
categories = "*:WARNING,net:FATAL,net.http:FATAL,net.ssl:FATAL,net.p2p:FATAL,net.cn:FATAL,global:INFO,verify:FATAL,serialization:FATAL,logging:INFO,msgwriter:INFO";
categories = "*:WARNING,net:FATAL,net.http:FATAL,net.p2p:FATAL,net.cn:FATAL,global:INFO,verify:FATAL,serialization:FATAL,logging:INFO,msgwriter:INFO";
break;
case 1:
categories = "*:INFO,global:INFO,stacktrace:INFO,logging:INFO,msgwriter:INFO,perf.*:DEBUG";

86
contrib/epee/src/net_helper.cpp
View File

@ -1,86 +0,0 @@
#include "net/net_helper.h"
namespace epee
{
namespace net_utils
{
std::future<boost::asio::ip::tcp::socket>
direct_connect::operator()(const std::string& addr, const std::string& port, boost::asio::steady_timer& timeout) const
{
// Get a list of endpoints corresponding to the server name.
//////////////////////////////////////////////////////////////////////////
boost::asio::ip::tcp::resolver resolver(GET_IO_SERVICE(timeout));
boost::asio::ip::tcp::resolver::query query(boost::asio::ip::tcp::v4(), addr, port, boost::asio::ip::tcp::resolver::query::canonical_name);
bool try_ipv6 = false;
boost::asio::ip::tcp::resolver::iterator iterator;
boost::asio::ip::tcp::resolver::iterator end;
boost::system::error_code resolve_error;
try
{
iterator = resolver.resolve(query, resolve_error);
if(iterator == end) // Documentation states that successful call is guaranteed to be non-empty
{
// if IPv4 resolution fails, try IPv6. Unintentional outgoing IPv6 connections should only
// be possible if for some reason a hostname was given and that hostname fails IPv4 resolution,
// so at least for now there should not be a need for a flag "using ipv6 is ok"
try_ipv6 = true;
}
}
catch (const boost::system::system_error& e)
{
if (resolve_error != boost::asio::error::host_not_found &&
resolve_error != boost::asio::error::host_not_found_try_again)
{
throw;
}
try_ipv6 = true;
}
if (try_ipv6)
{
boost::asio::ip::tcp::resolver::query query6(boost::asio::ip::tcp::v6(), addr, port, boost::asio::ip::tcp::resolver::query::canonical_name);
iterator = resolver.resolve(query6);
if (iterator == end)
throw boost::system::system_error{boost::asio::error::fault, "Failed to resolve " + addr};
}
//////////////////////////////////////////////////////////////////////////
struct new_connection
{
std::promise<boost::asio::ip::tcp::socket> result_;
boost::asio::ip::tcp::socket socket_;
explicit new_connection(boost::asio::io_service& io_service)
: result_(), socket_(io_service)
{}
};
const auto shared = std::make_shared<new_connection>(GET_IO_SERVICE(timeout));
timeout.async_wait([shared] (boost::system::error_code error)
{
if (error != boost::system::errc::operation_canceled && shared && shared->socket_.is_open())
{
shared->socket_.shutdown(boost::asio::ip::tcp::socket::shutdown_both);
shared->socket_.close();
}
});
shared->socket_.async_connect(*iterator, [shared] (boost::system::error_code error)
{
if (shared)
{
if (error)
{
try { throw boost::system::system_error{error}; }
catch (...) { shared->result_.set_exception(std::current_exception()); }
}
else
shared->result_.set_value(std::move(shared->socket_));
}
});
return shared->result_.get_future();
}
}
}

607
contrib/epee/src/net_ssl.cpp
View File

@ -1,607 +0,0 @@
// Copyright (c) 2018, The Monero Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include <thread>
#include <cstring>
#include <boost/asio/ssl.hpp>
extern "C" {
#include <openssl/ssl.h>
#include <openssl/pem.h>
}
#include "misc_log_ex.h"
#include "net/net_helper.h"
#include "net/net_ssl.h"
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "net.ssl"
// openssl genrsa -out /tmp/KEY 4096
// openssl req -new -key /tmp/KEY -out /tmp/REQ
// openssl x509 -req -days 999999 -sha256 -in /tmp/REQ -signkey /tmp/KEY -out /tmp/CERT
#ifdef _WIN32
static void add_windows_root_certs(SSL_CTX *ctx) noexcept;
#endif
namespace
{
struct openssl_bio_free
{
void operator()(BIO* ptr) const noexcept
{
BIO_free(ptr);
}
};
using openssl_bio = std::unique_ptr<BIO, openssl_bio_free>;
struct openssl_pkey_free
{
void operator()(EVP_PKEY* ptr) const noexcept
{
EVP_PKEY_free(ptr);
}
};
using openssl_pkey = std::unique_ptr<EVP_PKEY, openssl_pkey_free>;
struct openssl_rsa_free
{
void operator()(RSA* ptr) const noexcept
{
RSA_free(ptr);
}
};
using openssl_rsa = std::unique_ptr<RSA, openssl_rsa_free>;
struct openssl_bignum_free
{
void operator()(BIGNUM* ptr) const noexcept
{
BN_free(ptr);
}
};
using openssl_bignum = std::unique_ptr<BIGNUM, openssl_bignum_free>;
struct openssl_ec_key_free
{
void operator()(EC_KEY* ptr) const noexcept
{
EC_KEY_free(ptr);
}
};
using openssl_ec_key = std::unique_ptr<EC_KEY, openssl_ec_key_free>;
struct openssl_group_free
{
void operator()(EC_GROUP* ptr) const noexcept
{
EC_GROUP_free(ptr);
}
};
using openssl_group = std::unique_ptr<EC_GROUP, openssl_group_free>;
boost::system::error_code load_ca_file(boost::asio::ssl::context& ctx, const std::string& path)
{
SSL_CTX* const ssl_ctx = ctx.native_handle(); // could be moved from context
if (ssl_ctx == nullptr)
return {boost::asio::error::invalid_argument};
if (!SSL_CTX_load_verify_locations(ssl_ctx, path.c_str(), nullptr))
{
return boost::system::error_code{
int(::ERR_get_error()), boost::asio::error::get_ssl_category()
};
}
return boost::system::error_code{};
}
}
namespace epee
{
namespace net_utils
{
// https://stackoverflow.com/questions/256405/programmatically-create-x509-certificate-using-openssl
bool create_rsa_ssl_certificate(EVP_PKEY *&pkey, X509 *&cert)
{
MINFO("Generating SSL certificate");
pkey = EVP_PKEY_new();
if (!pkey)
{
MERROR("Failed to create new private key");
return false;
}
openssl_pkey pkey_deleter{pkey};
openssl_rsa rsa{RSA_new()};
if (!rsa)
{
MERROR("Error allocating RSA private key");
return false;
}
openssl_bignum exponent{BN_new()};
if (!exponent)
{
MERROR("Error allocating exponent");
return false;
}
BN_set_word(exponent.get(), RSA_F4);
if (RSA_generate_key_ex(rsa.get(), 4096, exponent.get(), nullptr) != 1)
{
MERROR("Error generating RSA private key");
return false;
}
if (EVP_PKEY_assign_RSA(pkey, rsa.get()) <= 0)
{
MERROR("Error assigning RSA private key");
return false;
}
// the RSA key is now managed by the EVP_PKEY structure
(void)rsa.release();
cert = X509_new();
if (!cert)
{
MERROR("Failed to create new X509 certificate");
return false;
}
ASN1_INTEGER_set(X509_get_serialNumber(cert), 1);
X509_gmtime_adj(X509_get_notBefore(cert), 0);
X509_gmtime_adj(X509_get_notAfter(cert), 3600 * 24 * 182); // half a year
if (!X509_set_pubkey(cert, pkey))
{
MERROR("Error setting pubkey on certificate");
X509_free(cert);
return false;
}
X509_NAME *name = X509_get_subject_name(cert);
X509_set_issuer_name(cert, name);
if (X509_sign(cert, pkey, EVP_sha256()) == 0)
{
MERROR("Error signing certificate");
X509_free(cert);
return false;
}
(void)pkey_deleter.release();
return true;
}
bool create_ec_ssl_certificate(EVP_PKEY *&pkey, X509 *&cert, int type)
{
MINFO("Generating SSL certificate");
pkey = EVP_PKEY_new();
if (!pkey)
{
MERROR("Failed to create new private key");
return false;
}
openssl_pkey pkey_deleter{pkey};
openssl_ec_key ec_key{EC_KEY_new()};
if (!ec_key)
{
MERROR("Error allocating EC private key");
return false;
}
EC_GROUP *group = EC_GROUP_new_by_curve_name(type);
if (!group)
{
MERROR("Error getting EC group " << type);
return false;
}
openssl_group group_deleter{group};
EC_GROUP_set_asn1_flag(group, OPENSSL_EC_NAMED_CURVE);
EC_GROUP_set_point_conversion_form(group, POINT_CONVERSION_UNCOMPRESSED);
if (!EC_GROUP_check(group, NULL))
{
MERROR("Group failed check: " << ERR_reason_error_string(ERR_get_error()));
return false;
}
if (EC_KEY_set_group(ec_key.get(), group) != 1)
{
MERROR("Error setting EC group");
return false;
}
if (EC_KEY_generate_key(ec_key.get()) != 1)
{
MERROR("Error generating EC private key");
return false;
}
if (EVP_PKEY_assign_EC_KEY(pkey, ec_key.get()) <= 0)
{
MERROR("Error assigning EC private key");
return false;
}
// the key is now managed by the EVP_PKEY structure
(void)ec_key.release();
cert = X509_new();
if (!cert)
{
MERROR("Failed to create new X509 certificate");
return false;
}
ASN1_INTEGER_set(X509_get_serialNumber(cert), 1);
X509_gmtime_adj(X509_get_notBefore(cert), 0);
X509_gmtime_adj(X509_get_notAfter(cert), 3600 * 24 * 182); // half a year
if (!X509_set_pubkey(cert, pkey))
{
MERROR("Error setting pubkey on certificate");
X509_free(cert);
return false;
}
X509_NAME *name = X509_get_subject_name(cert);
X509_set_issuer_name(cert, name);
if (X509_sign(cert, pkey, EVP_sha256()) == 0)
{
MERROR("Error signing certificate");
X509_free(cert);
return false;
}
(void)pkey_deleter.release();
return true;
}
ssl_options_t::ssl_options_t(std::vector<std::vector<std::uint8_t>> fingerprints, std::string ca_path)
: fingerprints_(std::move(fingerprints)),
ca_path(std::move(ca_path)),
auth(),
support(ssl_support_t::e_ssl_support_enabled),
verification(ssl_verification_t::user_certificates)
{
std::sort(fingerprints_.begin(), fingerprints_.end());
}
boost::asio::ssl::context ssl_options_t::create_context() const
{
boost::asio::ssl::context ssl_context{boost::asio::ssl::context::tls};
if (!bool(*this))
return ssl_context;
// only allow tls v1.2 and up
ssl_context.set_options(boost::asio::ssl::context::default_workarounds);
ssl_context.set_options(boost::asio::ssl::context::no_sslv2);
ssl_context.set_options(boost::asio::ssl::context::no_sslv3);
ssl_context.set_options(boost::asio::ssl::context::no_tlsv1);
ssl_context.set_options(boost::asio::ssl::context::no_tlsv1_1);
// only allow a select handful of tls v1.3 and v1.2 ciphers to be used
SSL_CTX_set_cipher_list(ssl_context.native_handle(), "ECDHE-ECDSA-CHACHA20-POLY1305-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256");
// set options on the SSL context for added security
SSL_CTX *ctx = ssl_context.native_handle();
CHECK_AND_ASSERT_THROW_MES(ctx, "Failed to get SSL context");
SSL_CTX_clear_options(ctx, SSL_OP_LEGACY_SERVER_CONNECT); // SSL_CTX_SET_OPTIONS(3)
SSL_CTX_set_session_cache_mode(ctx, SSL_SESS_CACHE_OFF); // https://stackoverflow.com/questions/22378442
#ifdef SSL_OP_NO_TICKET
SSL_CTX_set_options(ctx, SSL_OP_NO_TICKET); // https://stackoverflow.com/questions/22378442
#endif
#ifdef SSL_OP_NO_RENEGOTIATION
SSL_CTX_set_options(ctx, SSL_OP_NO_RENEGOTIATION);
#endif
#ifdef SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION
SSL_CTX_set_options(ctx, SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION);
#endif
#ifdef SSL_OP_NO_COMPRESSION
SSL_CTX_set_options(ctx, SSL_OP_NO_COMPRESSION);
#endif
#ifdef SSL_OP_CIPHER_SERVER_PREFERENCE
SSL_CTX_set_options(ctx, SSL_OP_CIPHER_SERVER_PREFERENCE);
#endif
(void)SSL_CTX_set_ecdh_auto(ctx, 1);
switch (verification)
{
case ssl_verification_t::system_ca:
#ifdef _WIN32
try { add_windows_root_certs(ssl_context.native_handle()); }
catch (const std::exception &e) { ssl_context.set_default_verify_paths(); }
#else
ssl_context.set_default_verify_paths();
#endif
break;
case ssl_verification_t::user_certificates:
ssl_context.set_verify_depth(0);
/* fallthrough */
case ssl_verification_t::user_ca:
if (!ca_path.empty())
{
const boost::system::error_code err = load_ca_file(ssl_context, ca_path);
if (err)
throw boost::system::system_error{err, "Failed to load user CA file at " + ca_path};
}
break;
default:
break;
}
CHECK_AND_ASSERT_THROW_MES(auth.private_key_path.empty() == auth.certificate_path.empty(), "private key and certificate must be either both given or both empty");
if (auth.private_key_path.empty())
{
EVP_PKEY *pkey;
X509 *cert;
bool ok = false;
#ifdef USE_EXTRA_EC_CERT
CHECK_AND_ASSERT_THROW_MES(create_ec_ssl_certificate(pkey, cert, NID_secp256k1), "Failed to create certificate");
CHECK_AND_ASSERT_THROW_MES(SSL_CTX_use_certificate(ctx, cert), "Failed to use generated certificate");
if (!SSL_CTX_use_PrivateKey(ctx, pkey))
MERROR("Failed to use generated EC private key for " << NID_secp256k1);
else
ok = true;
X509_free(cert);
EVP_PKEY_free(pkey);
#endif
CHECK_AND_ASSERT_THROW_MES(create_rsa_ssl_certificate(pkey, cert), "Failed to create certificate");
CHECK_AND_ASSERT_THROW_MES(SSL_CTX_use_certificate(ctx, cert), "Failed to use generated certificate");
if (!SSL_CTX_use_PrivateKey(ctx, pkey))
MERROR("Failed to use generated RSA private key for RSA");
else
ok = true;
X509_free(cert);
EVP_PKEY_free(pkey);
CHECK_AND_ASSERT_THROW_MES(ok, "Failed to use any generated certificate");
}
else
auth.use_ssl_certificate(ssl_context);
return ssl_context;
}
void ssl_authentication_t::use_ssl_certificate(boost::asio::ssl::context &ssl_context) const
{
ssl_context.use_private_key_file(private_key_path, boost::asio::ssl::context::pem);
ssl_context.use_certificate_chain_file(certificate_path);
}
bool is_ssl(std::string_view data)
{
if (data.size() < get_ssl_magic_size())
return false;
// https://security.stackexchange.com/questions/34780/checking-client-hello-for-https-classification
MDEBUG("SSL detection buffer, " << data.size() << " bytes: "
<< (unsigned)(unsigned char)data[0] << " " << (unsigned)(unsigned char)data[1] << " "
<< (unsigned)(unsigned char)data[2] << " " << (unsigned)(unsigned char)data[3] << " "
<< (unsigned)(unsigned char)data[4] << " " << (unsigned)(unsigned char)data[5] << " "
<< (unsigned)(unsigned char)data[6] << " " << (unsigned)(unsigned char)data[7] << " "
<< (unsigned)(unsigned char)data[8]);
if (data[0] == 0x16) // record
if (data[1] == 3) // major version
if (data[5] == 1) // ClientHello
if (data[6] == 0 && data[3]*256 + data[4] == data[7]*256 + data[8] + 4) // length check
return true;
return false;
}
static bool ends_with(std::string_view str, std::string_view suffix) {
return str.size() >= suffix.size() && str.substr(str.size() - suffix.size()) == suffix;
}
bool ssl_options_t::has_strong_verification(std::string_view host) const noexcept
{
// onion and i2p addresses contain information about the server cert
// which both authenticates and encrypts
if (ends_with(host, ".onion"sv) || ends_with(host, ".i2p"sv))
return true;
switch (verification)
{
default:
case ssl_verification_t::none:
case ssl_verification_t::system_ca:
return false;
case ssl_verification_t::user_certificates:
case ssl_verification_t::user_ca:
break;
}
return true;
}
bool ssl_options_t::has_fingerprint(boost::asio::ssl::verify_context &ctx) const
{
// can we check the certificate against a list of fingerprints?
if (!fingerprints_.empty()) {
X509_STORE_CTX *sctx = ctx.native_handle();
if (!sctx)
{
MERROR("Error getting verify_context handle");
return false;
}
X509* cert = nullptr;
const STACK_OF(X509)* chain = X509_STORE_CTX_get_chain(sctx);
if (!chain || sk_X509_num(chain) < 1 || !(cert = sk_X509_value(chain, 0)))
{
MERROR("No certificate found in verify_context");
return false;
}
// buffer for the certificate digest and the size of the result
std::vector<uint8_t> digest(EVP_MAX_MD_SIZE);
unsigned int size{ 0 };
// create the digest from the certificate
if (!X509_digest(cert, EVP_sha256(), digest.data(), &size)) {
MERROR("Failed to create certificate fingerprint");
return false;
}
// strip unnecessary bytes from the digest
digest.resize(size);
return std::binary_search(fingerprints_.begin(), fingerprints_.end(), digest);
}
return false;
}
bool ssl_options_t::handshake(
boost::asio::ssl::stream<boost::asio::ip::tcp::socket> &socket,
boost::asio::ssl::stream_base::handshake_type type,
const std::string& host,
std::chrono::milliseconds timeout) const
{
socket.next_layer().set_option(boost::asio::ip::tcp::no_delay(true));
/* Using system-wide CA store for client verification is funky - there is
no expected hostname for server to verify against. If server doesn't have
specific whitelisted certificates for client, don't require client to
send certificate at all. */
const bool no_verification = verification == ssl_verification_t::none ||
(type == boost::asio::ssl::stream_base::server && fingerprints_.empty() && ca_path.empty());
/* According to OpenSSL documentation (and SSL specifications), server must
always send certificate unless "anonymous" cipher mode is used which are
disabled by default. Either way, the certificate is never inspected. */
if (no_verification)
socket.set_verify_mode(boost::asio::ssl::verify_none);
else
{
socket.set_verify_mode(boost::asio::ssl::verify_peer | boost::asio::ssl::verify_fail_if_no_peer_cert);
// in case server is doing "virtual" domains, set hostname
SSL* const ssl_ctx = socket.native_handle();
if (type == boost::asio::ssl::stream_base::client && !host.empty() && ssl_ctx)
SSL_set_tlsext_host_name(ssl_ctx, host.c_str());
socket.set_verify_callback([&](const bool preverified, boost::asio::ssl::verify_context &ctx)
{
// preverified means it passed system or user CA check. System CA is never loaded
// when fingerprints are whitelisted.
const bool verified = preverified &&
(verification != ssl_verification_t::system_ca || host.empty() || boost::asio::ssl::rfc2818_verification(host)(preverified, ctx));
if (!verified && !has_fingerprint(ctx))
{
// autodetect will reconnect without SSL - warn and keep connection encrypted
if (support != ssl_support_t::e_ssl_support_autodetect)
{
MERROR("SSL certificate is not in the allowed list, connection droppped");
return false;
}
MWARNING("SSL peer has not been verified");
}
return true;
});
}
auto& io_service = GET_IO_SERVICE(socket);
boost::asio::steady_timer deadline(io_service, timeout);
deadline.async_wait([&socket](const boost::system::error_code& error) {
if (error != boost::asio::error::operation_aborted)
{
socket.next_layer().close();
}
});
boost::system::error_code ec = boost::asio::error::would_block;
socket.async_handshake(type, [&ec](const auto& val) { ec = val; });
if (io_service.stopped())
{
io_service.reset();
}
while (ec == boost::asio::error::would_block && !io_service.stopped())
{
// should poll_one(), can't run_one() because it can block if there is
// another worker thread executing io_service's tasks
// TODO: once we get Boost 1.66+, replace with run_one_for/run_until
std::this_thread::sleep_for(std::chrono::milliseconds(30));
io_service.poll_one();
}
if (ec)
{
MERROR("SSL handshake failed, connection dropped: " << ec.message());
return false;
}
MDEBUG("SSL handshake success");
return true;
}
bool ssl_support_from_string(ssl_support_t &ssl, std::string_view s)
{
if (s == "enabled")
ssl = epee::net_utils::ssl_support_t::e_ssl_support_enabled;
else if (s == "disabled")
ssl = epee::net_utils::ssl_support_t::e_ssl_support_disabled;
else if (s == "autodetect")
ssl = epee::net_utils::ssl_support_t::e_ssl_support_autodetect;
else
return false;
return true;
}
} // namespace
} // namespace
#ifdef _WIN32
// https://stackoverflow.com/questions/40307541
// Because Windows always has to do things wonkily
#include <wincrypt.h>
static void add_windows_root_certs(SSL_CTX *ctx) noexcept
{
HCERTSTORE hStore = CertOpenSystemStore(0, "ROOT");
if (hStore == NULL) {
return;
}
X509_STORE *store = X509_STORE_new();
PCCERT_CONTEXT pContext = NULL;
while ((pContext = CertEnumCertificatesInStore(hStore, pContext)) != NULL) {
// convert from DER to internal format
X509 *x509 = d2i_X509(NULL,
(const unsigned char **)&pContext->pbCertEncoded,
pContext->cbCertEncoded);
if(x509 != NULL) {
X509_STORE_add_cert(store, x509);
X509_free(x509);
}
}
CertFreeCertificateContext(pContext);
CertCloseStore(hStore, 0);
// attach X509_STORE to boost ssl context
SSL_CTX_set_cert_store(ctx, store);
}
#endif

14
external/CMakeLists.txt vendored
View File

@ -121,3 +121,17 @@ add_library(uWebSockets INTERFACE)
target_include_directories(uWebSockets INTERFACE ${CMAKE_CURRENT_BINARY_DIR}/uWebSockets)
target_link_libraries(uWebSockets INTERFACE uSockets)
target_compile_definitions(uWebSockets INTERFACE UWS_HTTPRESPONSE_NO_WRITEMARK UWS_NO_ZLIB)
# cpr configuration. Ideally we'd just do this via add_subdirectory, but cpr's cmake requires
# 3.15+, and we target lower than that (and this is fairly simple to build).
find_package(CURL REQUIRED COMPONENTS PROTOCOLS HTTP HTTPS FEATURES SSL)
file(GLOB cpr_sources ${conf_depends} cpr/cpr/*.cpp)
add_library(cpr STATIC EXCLUDE_FROM_ALL ${cpr_sources})
target_link_libraries(cpr PUBLIC CURL::libcurl)
target_include_directories(cpr PUBLIC cpr/include)
target_compile_definitions(cpr PUBLIC CPR_CURL_NOSIGNAL)
add_library(cpr::cpr ALIAS cpr)

1
external/cpr vendored Submodule

@ -0,0 +1 @@
Subproject commit 5410d5503237c175df899f76310037b064879599

2
src/common/CMakeLists.txt
View File

@ -33,7 +33,6 @@ add_library(common
combinator.cpp
command_line.cpp
dns_utils.cpp
download.cpp
error.cpp
expect.cpp
file.cpp
@ -51,7 +50,6 @@ add_library(common
string_util.cpp
threadpool.cpp
timings.cc
updates.cpp
util.cpp
${PROJECT_BINARY_DIR}/translations/translation_files.cpp
)

315
src/common/download.cpp
View File

@ -1,315 +0,0 @@
// Copyright (c) 2017-2019, The Monero Project
// Copyright (c) 2018, The Loki Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include <string>
#include <atomic>
#include <thread>
#include <boost/filesystem.hpp>
#include "file_io_utils.h"
#include "net/http_client.h"
#include "download.h"
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "net.dl"
namespace tools
{
struct download_thread_control
{
const std::string path;
const std::string uri;
std::function<void(const std::string&, const std::string&, bool)> result_cb;
std::function<bool(const std::string&, const std::string&, size_t, ssize_t)> progress_cb;
bool stop;
bool stopped;
bool success;
std::thread thread;
std::mutex mutex;
download_thread_control(const std::string &path, const std::string &uri, std::function<void(const std::string&, const std::string&, bool)> result_cb, std::function<bool(const std::string&, const std::string&, size_t, ssize_t)> progress_cb):
path(path), uri(uri), result_cb(result_cb), progress_cb(progress_cb), stop(false), stopped(false), success(false) {}
~download_thread_control() { if (thread.joinable()) thread.detach(); }
};
static void download_thread(download_async_handle control)
{
static std::atomic<unsigned int> thread_id(0);
MLOG_SET_THREAD_NAME("DL" + std::to_string(thread_id++));
struct stopped_setter
{
stopped_setter(const download_async_handle &control): control(control) {}
~stopped_setter() { control->stopped = true; }
download_async_handle control;
} stopped_setter(control);
try
{
std::unique_lock lock{control->mutex};
std::ios_base::openmode mode = std::ios_base::out | std::ios_base::binary;
uint64_t existing_size = 0;
if (epee::file_io_utils::get_file_size(control->path, existing_size) && existing_size > 0)
{
MINFO("Resuming downloading " << control->uri << " to " << control->path << " from " << existing_size);
mode |= std::ios_base::app;
}
else
{
MINFO("Downloading " << control->uri << " to " << control->path);
mode |= std::ios_base::trunc;
}
std::ofstream f;
f.open(control->path, mode);
if (!f.good()) {
MERROR("Failed to open file " << control->path);
control->result_cb(control->path, control->uri, control->success);
return;
}
class download_client: public epee::net_utils::http::http_simple_client
{
public:
download_client(download_async_handle control, std::ofstream &f, uint64_t offset = 0):
control(control), f(f), content_length(-1), total(0), offset(offset) {}
virtual ~download_client() { f.close(); }
virtual bool on_header(const epee::net_utils::http::http_response_info &headers)
{
for (const auto &kv: headers.m_header_info.m_etc_fields)
MDEBUG("Header: " << kv.first << ": " << kv.second);
ssize_t length;
if (epee::string_tools::get_xtype_from_string(length, headers.m_header_info.m_content_length) && length >= 0)
{
MINFO("Content-Length: " << length);
content_length = length;
boost::filesystem::path path(control->path);
try
{
boost::filesystem::space_info si = boost::filesystem::space(path);
if (si.available < (size_t)content_length)
{
const uint64_t avail = (si.available + 1023) / 1024, needed = (content_length + 1023) / 1024;
MERROR("Not enough space to download " << needed << " kB to " << path << " (" << avail << " kB available)");
return false;
}
}
catch (const std::exception &e) { MWARNING("Failed to check for free space: " << e.what()); }
}
if (offset > 0)
{
// we requested a range, so check if we're getting it, otherwise truncate
bool got_range = false;
const std::string prefix = "bytes=" + std::to_string(offset) + "-";
for (const auto &kv: headers.m_header_info.m_etc_fields)
{
if (kv.first == "Content-Range" && strncmp(kv.second.c_str(), prefix.c_str(), prefix.size()))
{
got_range = true;
break;
}
}
if (!got_range)
{
MWARNING("We did not get the requested range, downloading from start");
f.close();
f.open(control->path, std::ios_base::out | std::ios_base::binary | std::ios_base::trunc);
}
}
return true;
}
virtual bool handle_target_data(std::string &piece_of_transfer)
{
try
{
std::lock_guard lock{control->mutex};
if (control->stop)
return false;
f << piece_of_transfer;
total += piece_of_transfer.size();
if (control->progress_cb && !control->progress_cb(control->path, control->uri, total, content_length))
return false;
return f.good();
}
catch (const std::exception &e)
{
MERROR("Error writing data: " << e.what());
return false;
}
}
private:
download_async_handle control;
std::ofstream &f;
ssize_t content_length;
size_t total;
uint64_t offset;
} client(control, f, existing_size);
epee::net_utils::http::url_content u_c;
if (!epee::net_utils::parse_url(control->uri, u_c))
{
MERROR("Failed to parse URL " << control->uri);
control->result_cb(control->path, control->uri, control->success);
return;
}
if (u_c.host.empty())
{
MERROR("Failed to determine address from URL " << control->uri);
control->result_cb(control->path, control->uri, control->success);
return;
}
lock.unlock();
epee::net_utils::ssl_support_t ssl = u_c.schema == "https" ? epee::net_utils::ssl_support_t::e_ssl_support_enabled : epee::net_utils::ssl_support_t::e_ssl_support_disabled;
uint16_t port = u_c.port ? u_c.port : ssl == epee::net_utils::ssl_support_t::e_ssl_support_enabled ? 443 : 80;
MDEBUG("Connecting to " << u_c.host << ":" << port);
client.set_server(u_c.host, std::to_string(port), std::nullopt, ssl);
if (!client.connect(std::chrono::seconds(30)))
{
std::lock_guard lock{control->mutex};
MERROR("Failed to connect to " << control->uri);
control->result_cb(control->path, control->uri, control->success);
return;
}
MDEBUG("GETting " << u_c.uri);
const epee::net_utils::http::http_response_info *info = NULL;
epee::net_utils::http::fields_list fields;
if (existing_size > 0)
{
const std::string range = "bytes=" + std::to_string(existing_size) + "-";
MDEBUG("Asking for range: " << range);
fields.push_back(std::make_pair("Range", range));
}
if (!client.invoke_get(u_c.uri, std::chrono::seconds(30), "", &info, fields))
{
std::lock_guard lock{control->mutex};
MERROR("Failed to connect to " << control->uri);
client.disconnect();
control->result_cb(control->path, control->uri, control->success);
return;
}
if (control->stop)
{
std::lock_guard lock{control->mutex};
MDEBUG("Download cancelled");
client.disconnect();
control->result_cb(control->path, control->uri, control->success);
return;
}
if (!info)
{
std::lock_guard lock{control->mutex};
MERROR("Failed invoking GET command to " << control->uri << ", no status info returned");
client.disconnect();
control->result_cb(control->path, control->uri, control->success);
return;
}
MDEBUG("response code: " << info->m_response_code);
MDEBUG("response length: " << info->m_header_info.m_content_length);
MDEBUG("response comment: " << info->m_response_comment);
MDEBUG("response body: " << info->m_body);
for (const auto &f: info->m_additional_fields)
MDEBUG("additional field: " << f.first << ": " << f.second);
if (info->m_response_code != 200 && info->m_response_code != 206)
{
std::lock_guard lock{control->mutex};
MERROR("Status code " << info->m_response_code);
client.disconnect();
control->result_cb(control->path, control->uri, control->success);
return;
}
client.disconnect();
f.close();
MDEBUG("Download complete");
lock.lock();
control->success = true;
control->result_cb(control->path, control->uri, control->success);
return;
}
catch (const std::exception &e)
{
MERROR("Exception in download thread: " << e.what());
// fall through and call result_cb not from the catch block to avoid another exception
}
std::lock_guard lock{control->mutex};
control->result_cb(control->path, control->uri, control->success);
}
bool download(const std::string &path, const std::string &url, std::function<bool(const std::string&, const std::string&, size_t, ssize_t)> cb)
{
bool success = false;
download_async_handle handle = download_async(path, url, [&success](const std::string&, const std::string&, bool result) {success = result;}, cb);
download_wait(handle);
return success;
}
download_async_handle download_async(const std::string &path, const std::string &url, std::function<void(const std::string&, const std::string&, bool)> result, std::function<bool(const std::string&, const std::string&, size_t, ssize_t)> progress)
{
download_async_handle control = std::make_shared<download_thread_control>(path, url, result, progress);
control->thread = std::thread([control](){ download_thread(control); });
return control;
}
bool download_finished(const download_async_handle &control)
{
CHECK_AND_ASSERT_MES(control != 0, false, "NULL async download handle");
std::lock_guard lock{control->mutex};
return control->stopped;
}
bool download_error(const download_async_handle &control)
{
CHECK_AND_ASSERT_MES(control != 0, false, "NULL async download handle");
std::lock_guard lock{control->mutex};
return !control->success;
}
bool download_wait(const download_async_handle &control)
{
CHECK_AND_ASSERT_MES(control != 0, false, "NULL async download handle");
{
std::lock_guard lock{control->mutex};
if (control->stopped)
return true;
}
control->thread.join();
return true;
}
bool download_cancel(const download_async_handle &control)
{
CHECK_AND_ASSERT_MES(control != 0, false, "NULL async download handle");
{
std::lock_guard lock{control->mutex};
if (control->stopped)
return true;
control->stop = true;
}
control->thread.join();
return true;
}
}

44
src/common/download.h
View File

@ -1,44 +0,0 @@
// Copyright (c) 2017-2019, The Monero Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#pragma once
#include <string>
namespace tools
{
struct download_thread_control;
typedef std::shared_ptr<download_thread_control> download_async_handle;
bool download(const std::string &path, const std::string &url, std::function<bool(const std::string&, const std::string&, size_t, ssize_t)> progress = NULL);
download_async_handle download_async(const std::string &path, const std::string &url, std::function<void(const std::string&, const std::string&, bool)> result, std::function<bool(const std::string&, const std::string&, size_t, ssize_t)> progress = NULL);
bool download_error(const download_async_handle &h);
bool download_finished(const download_async_handle &h);
bool download_wait(const download_async_handle &h);
bool download_cancel(const download_async_handle &h);
}

69
src/common/http_connection.h
View File

@ -1,69 +0,0 @@
// Copyright (c) 2014-2019, The Monero Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#pragma once
#include <chrono>
#include "string_tools.h"
#include "net/http_client.h"
namespace tools {
class t_http_connection {
private:
epee::net_utils::http::http_simple_client * mp_http_client;
bool m_ok;
public:
static constexpr std::chrono::seconds TIMEOUT()
{
return std::chrono::minutes(3) + std::chrono::seconds(30);
}
t_http_connection(epee::net_utils::http::http_simple_client* p_http_client)
: mp_http_client(p_http_client)
, m_ok(false)
{
m_ok = mp_http_client->connect(TIMEOUT());
}
~t_http_connection()
{
if (m_ok)
{
try { mp_http_client->disconnect(); }
catch (...) { /* do not propagate through dtor */ }
}
}
bool is_open() const
{
return m_ok;
}
}; // class t_http_connection
} // namespace tools

5
src/common/password.cpp
View File

@ -250,11 +250,6 @@ namespace tools
{
}
password_container::~password_container() noexcept
{
m_password.clear();
}
std::atomic<bool> password_container::is_prompting(false);
std::optional<password_container> password_container::prompt(const bool verify, const char *message, bool hide_input)

3
src/common/password.h
View File

@ -58,9 +58,6 @@ namespace tools
password_container(const password_container&) = delete;
password_container(password_container&& rhs) = default;
//! Wipes internal password
~password_container() noexcept;
password_container& operator=(const password_container&) = delete;
password_container& operator=(password_container&&) = default;

151
src/common/rpc_client.h
View File

@ -1,151 +0,0 @@
// Copyright (c) 2014-2019, The Monero Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#pragma once
#include <optional>
#include "common/http_connection.h"
#include "common/scoped_message_writer.h"
#include "rpc/core_rpc_server_commands_defs.h"
#include "storages/http_abstract_invoke.h"
#include "net/http_auth.h"
#include "net/http_client.h"
#include "net/net_ssl.h"
#include "string_tools.h"
namespace tools
{
class t_rpc_client final
{
private:
epee::net_utils::http::http_simple_client m_http_client;
public:
t_rpc_client(
uint32_t ip
, uint16_t port
, std::optional<epee::net_utils::http::login> user
, epee::net_utils::ssl_options_t ssl_options
)
: m_http_client{}
{
m_http_client.set_server(
epee::string_tools::get_ip_string_from_int32(ip), std::to_string(port), std::move(user), std::move(ssl_options)
);
}
template <typename T_req, typename T_res>
bool basic_json_rpc_request(
T_req & req
, T_res & res
, std::string const & method_name
)
{
t_http_connection connection(&m_http_client);
bool ok = connection.is_open();
if (!ok)
{
fail_msg_writer() << "Couldn't connect to daemon: " << m_http_client.get_host() << ":" << m_http_client.get_port();
return false;
}
ok = epee::net_utils::invoke_http_json_rpc("/json_rpc", method_name, req, res, m_http_client, t_http_connection::TIMEOUT());
if (!ok)
{
fail_msg_writer() << "basic_json_rpc_request: Daemon request failed";
return false;
}
else
{
return true;
}
}
template <typename T_req, typename T_res>
bool json_rpc_request(
T_req & req
, T_res & res
, std::string const & method_name
, std::string const & fail_msg
)
{
t_http_connection connection(&m_http_client);
bool ok = connection.is_open();
if (!ok)
{
fail_msg_writer() << "Couldn't connect to daemon: " << m_http_client.get_host() << ":" << m_http_client.get_port();
return false;
}
ok = epee::net_utils::invoke_http_json_rpc("/json_rpc", method_name, req, res, m_http_client, t_http_connection::TIMEOUT());
if (!ok || res.status != cryptonote::rpc::STATUS_OK) // TODO - handle cryptonote::rpc::STATUS_BUSY ?
{
fail_msg_writer() << fail_msg << " -- json_rpc_request: " << res.status;
return false;
}
else
{
return true;
}
}
template <typename T_req, typename T_res>
bool rpc_request(
T_req & req
, T_res & res
, std::string const & relative_url
, std::string const & fail_msg
)
{
t_http_connection connection(&m_http_client);
bool ok = connection.is_open();
if (!ok)
{
fail_msg_writer() << "Couldn't connect to daemon: " << m_http_client.get_host() << ":" << m_http_client.get_port();
return false;
}
ok = epee::net_utils::invoke_http_json(relative_url, req, res, m_http_client, t_http_connection::TIMEOUT());
if (!ok || res.status != cryptonote::rpc::STATUS_OK) // TODO - handle cryptonote::rpc::STATUS_BUSY ?
{
fail_msg_writer() << fail_msg << "-- rpc_request: " << res.status;
return false;
}
else
{
return true;
}
}
bool check_connection()
{
t_http_connection connection(&m_http_client);
return connection.is_open();
}
};
}

112
src/common/updates.cpp
View File

@ -1,112 +0,0 @@
// Copyright (c) 2017-2019, The Monero Project
// Copyright (c) 2018, The Loki Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "common/string_util.h"
#include "misc_log_ex.h"
#include "util.h"
#include "dns_utils.h"
#include "updates.h"
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "updates"
namespace tools
{
bool check_updates(const std::string &software, const std::string &buildtag, std::string &version, std::string &hash)
{
std::vector<std::string> records;
bool found = false;
MDEBUG("Checking updates for " << buildtag << " " << software);
// All four MoneroPulse domains have DNSSEC on and valid
static const std::vector<std::string> dns_urls = {};
if (!tools::dns_utils::load_txt_records_from_dns(records, dns_urls))
return false;
for (const auto& record : records)
{
auto fields = tools::split(record, ":");
if (fields.size() != 4)
{
MWARNING("Updates record does not have 4 fields: " << record);
continue;
}
if (software != fields[0] || buildtag != fields[1])
continue;
bool alnum = true;
for (auto c: fields[3])
if (!isalnum(c))
alnum = false;
if (fields[3].size() != 64 && !alnum)
{
MWARNING("Invalid hash: " << fields[3]);
continue;
}
// use highest version
if (found)
{
int cmp = vercmp(version, fields[2]);
if (cmp > 0)
continue;
if (cmp == 0 && hash != fields[3])
MWARNING("Two matches found for " << software << " version " << version << " on " << buildtag);
}
version = fields[2];
hash = fields[3];
MINFO("Found new version " << version << " with hash " << hash);
found = true;
}
return found;
}
std::string get_update_url(const std::string &software, const std::string &subdir, const std::string &buildtag, const std::string &version, bool user)
{
const char *base = user ? "https://downloads.getmonero.org/" : "https://updates.getmonero.org/";
#ifdef _WIN32
static const char *extension = strncmp(buildtag.c_str(), "source", 6) ? (strncmp(buildtag.c_str(), "install-", 8) ? ".zip" : ".exe") : ".tar.bz2";
#else
static const char extension[] = ".tar.bz2";
#endif
std::string url;
url = base;
if (!subdir.empty())
url += subdir + "/";
url = url + software + "-" + buildtag + "-v" + version + extension;
return url;
}
}

37
src/common/updates.h
View File

@ -1,37 +0,0 @@
// Copyright (c) 2017-2019, The Monero Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#pragma once
#include <string>
namespace tools
{
bool check_updates(const std::string &software, const std::string &buildtag, std::string &version, std::string &hash);
std::string get_update_url(const std::string &software, const std::string &subdir, const std::string &buildtag, const std::string &version, bool user);
}

6
src/common/util.cpp
View File

@ -30,18 +30,22 @@
// Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers
#include <string>
#include <iomanip>
#include <openssl/ssl.h>
#include "unbound.h"
#include "include_base_utils.h"
#include "string_tools.h"
#include "wipeable_string.h"
#include "crypto/crypto.h"
#include "util.h"
#include "stack_trace.h"
#include "net/http_client.h" // epee::net_utils::...
#include "misc_os_dependent.h"
#include "readline_buffer.h"
#include "string_util.h"
#include <boost/filesystem/path.hpp>
#include "i18n.h"

15
src/cryptonote_basic/subaddress_index.h
View File

@ -42,23 +42,22 @@ namespace cryptonote
{
uint32_t major; // The account index, major index
uint32_t minor; // The subaddress index, minor index
bool operator==(const subaddress_index& rhs) const { return !memcmp(this, &rhs, sizeof(subaddress_index)); }
bool operator==(const subaddress_index& rhs) const { return major == rhs.major && minor == rhs.minor; }
bool operator!=(const subaddress_index& rhs) const { return !(*this == rhs); }
bool is_zero() const { return major == 0 && minor == 0; }
BEGIN_SERIALIZE_OBJECT()
FIELD(major)
FIELD(minor)
END_SERIALIZE()
BEGIN_KV_SERIALIZE_MAP()
KV_SERIALIZE(major)
KV_SERIALIZE(minor)
END_KV_SERIALIZE_MAP()
};
}
namespace cryptonote {
template <class Archive>
void serialize_value(Archive& ar, subaddress_index& x) {
field(ar, "major", x.major);
field(ar, "minor", x.minor);
}
inline std::ostream& operator<<(std::ostream& out, const cryptonote::subaddress_index& subaddr_index)
{
return out << subaddr_index.major << '/' << subaddr_index.minor;

165
src/cryptonote_core/cryptonote_core.cpp
View File

@ -51,8 +51,6 @@ extern "C" {
#include "cryptonote_core.h"
#include "common/file.h"
#include "common/sha256sum.h"
#include "common/updates.h"
#include "common/download.h"
#include "common/threadpool.h"
#include "common/command_line.h"
#include "warnings.h"
@ -170,11 +168,6 @@ namespace cryptonote
, "How many blocks to sync at once during chain synchronization (0 = adaptive)."
, 0
};
static const command_line::arg_descriptor<std::string> arg_check_updates = {
"check-updates"
, "Check for new versions of loki: [disabled|notify|download|update]"
, "notify"
};
static const command_line::arg_descriptor<bool> arg_pad_transactions = {
"pad-transactions"
, "Pad relayed transactions to help defend against traffic volume analysis"
@ -313,9 +306,7 @@ namespace cryptonote
, m_target_blockchain_height(0)
, m_checkpoints_path("")
, m_last_json_checkpoints_update(0)
, m_update_download(0)
, m_nettype(UNDEFINED)
, m_update_available(false)
, m_last_storage_server_ping(0)
, m_last_lokinet_ping(0)
, m_pad_transactions(false)
@ -355,15 +346,6 @@ namespace cryptonote
{
m_miner.stop();
m_blockchain_storage.cancel();
tools::download_async_handle handle;
{
std::lock_guard lock{m_update_mutex};
handle = m_update_download;
m_update_download = 0;
}
if (handle)
tools::download_cancel(handle);
}
//-----------------------------------------------------------------------------------
void core::init_options(boost::program_options::options_description& desc)
@ -383,7 +365,6 @@ namespace cryptonote
command_line::add_arg(desc, arg_fast_block_sync);
command_line::add_arg(desc, arg_show_time_stats);
command_line::add_arg(desc, arg_block_sync_size);
command_line::add_arg(desc, arg_check_updates);
command_line::add_arg(desc, arg_offline);
command_line::add_arg(desc, arg_block_download_max_size);
command_line::add_arg(desc, arg_max_txpool_weight);
@ -653,7 +634,6 @@ namespace cryptonote
bool db_salvage = command_line::get_arg(vm, cryptonote::arg_db_salvage) != 0;
bool fast_sync = command_line::get_arg(vm, arg_fast_block_sync) != 0;
uint64_t blocks_threads = command_line::get_arg(vm, arg_prep_blocks_threads);
std::string check_updates_string = command_line::get_arg(vm, arg_check_updates);
size_t max_txpool_weight = command_line::get_arg(vm, arg_max_txpool_weight);
bool const prune_blockchain = false; /* command_line::get_arg(vm, arg_prune_blockchain); */
bool keep_alt_blocks = command_line::get_arg(vm, arg_keep_alt_blocks);
@ -892,20 +872,6 @@ namespace cryptonote
MGINFO("Loading checkpoints");
CHECK_AND_ASSERT_MES(update_checkpoints_from_json_file(), false, "One or more checkpoints loaded from json conflicted with existing checkpoints.");
// DNS versions checking
if (check_updates_string == "disabled")
check_updates_level = UPDATES_DISABLED;
else if (check_updates_string == "notify")
check_updates_level = UPDATES_NOTIFY;
else if (check_updates_string == "download")
check_updates_level = UPDATES_DOWNLOAD;
else if (check_updates_string == "update")
check_updates_level = UPDATES_UPDATE;
else {
MERROR("Invalid argument to --dns-versions-check: " << check_updates_string);
return false;
}
r = m_miner.init(vm, m_nettype);
CHECK_AND_ASSERT_MES(r, false, "Failed to initialize miner instance");
@ -2216,7 +2182,6 @@ namespace cryptonote
m_fork_moaner.do_call([this] { return check_fork_time(); });
m_txpool_auto_relayer.do_call([this] { return relay_txpool_transactions(); });
m_service_node_vote_relayer.do_call([this] { return relay_service_node_votes(); });
// m_check_updates_interval.do_call([this] { return check_updates(); });
m_check_disk_space_interval.do_call([this] { return check_disk_space(); });
m_block_rate_interval.do_call([this] { return check_block_rate(); });
m_sn_proof_cleanup_interval.do_call([&snl=m_service_node_list] { snl.cleanup_proofs(); return true; });
@ -2285,136 +2250,6 @@ namespace cryptonote
return get_blockchain_storage().get_earliest_ideal_height_for_version(version);
}
//-----------------------------------------------------------------------------------------------
bool core::check_updates()
{
static const char software[] = "loki";
#ifdef BUILD_TAG
static const char buildtag[] = BOOST_PP_STRINGIZE(BUILD_TAG);
static const char subdir[] = "cli"; // because it can never be simple
#else
static const char buildtag[] = "source";
static const char subdir[] = "source"; // because it can never be simple
#endif
if (m_offline)
return true;
if (check_updates_level == UPDATES_DISABLED)
return true;
std::string version, hash;
MCDEBUG("updates", "Checking for a new " << software << " version for " << buildtag);
if (!tools::check_updates(software, buildtag, version, hash))
return false;
if (tools::vercmp(version.c_str(), LOKI_VERSION_STR) <= 0)
{
m_update_available = false;
return true;
}
std::string url = tools::get_update_url(software, subdir, buildtag, version, true);
MCLOG_CYAN(el::Level::Info, "global", "Version " << version << " of " << software << " for " << buildtag << " is available: " << url << ", SHA256 hash " << hash);
m_update_available = true;
if (check_updates_level == UPDATES_NOTIFY)
return true;
url = tools::get_update_url(software, subdir, buildtag, version, false);
std::string filename;
const char *slash = strrchr(url.c_str(), '/');
if (slash)
filename = slash + 1;
else
filename = std::string(software) + "-update-" + version;
boost::filesystem::path path(epee::string_tools::get_current_module_folder());
path /= filename;
std::unique_lock lock{m_update_mutex};
if (m_update_download != 0)
{
MCDEBUG("updates", "Already downloading update");
return true;
}
crypto::hash file_hash;
if (!tools::sha256sum_file(path.string(), file_hash) || (hash != epee::string_tools::pod_to_hex(file_hash)))
{
MCDEBUG("updates", "We don't have that file already, downloading");
const std::string tmppath = path.string() + ".tmp";
if (epee::file_io_utils::is_file_exist(tmppath))
{
MCDEBUG("updates", "We have part of the file already, resuming download");
}
m_last_update_length = 0;
m_update_download = tools::download_async(tmppath, url, [this, hash, path](const std::string &tmppath, const std::string &uri, bool success) {
bool remove = false, good = true;
if (success)
{
crypto::hash file_hash;
if (!tools::sha256sum_file(tmppath, file_hash))
{
MCERROR("updates", "Failed to hash " << tmppath);
remove = true;
good = false;
}
else if (hash != epee::string_tools::pod_to_hex(file_hash))
{
MCERROR("updates", "Download from " << uri << " does not match the expected hash");
remove = true;
good = false;
}
}
else
{
MCERROR("updates", "Failed to download " << uri);
good = false;
}
std::unique_lock lock{m_update_mutex};
m_update_download = 0;
if (success && !remove)
{
std::error_code e = tools::replace_file(tmppath, path.string());
if (e)
{
MCERROR("updates", "Failed to rename downloaded file");
good = false;
}
}
else if (remove)
{
if (!boost::filesystem::remove(tmppath))
{
MCERROR("updates", "Failed to remove invalid downloaded file");
good = false;
}
}
if (good)
MCLOG_CYAN(el::Level::Info, "updates", "New version downloaded to " << path.string());
}, [this](const std::string &path, const std::string &uri, size_t length, ssize_t content_length) {
if (length >= m_last_update_length + 1024 * 1024 * 10)
{
m_last_update_length = length;
MCDEBUG("updates", "Downloaded " << length << "/" << (content_length ? std::to_string(content_length) : "unknown"));
}
return true;
});
}
else
{
MCDEBUG("updates", "We already have " << path << " with expected hash");
}
lock.unlock();
if (check_updates_level == UPDATES_DOWNLOAD)
return true;
MCERROR("updates", "Download/update not implemented yet");
return true;
}
//-----------------------------------------------------------------------------------------------
bool core::check_disk_space()
{
uint64_t free_space = get_free_space();

32
src/cryptonote_core/cryptonote_core.h
View File

@ -41,7 +41,6 @@
#include "cryptonote_protocol/cryptonote_protocol_handler_common.h"
#include "storages/portable_storage_template_helper.h"
#include "common/download.h"
#include "common/command_line.h"
#include "tx_pool.h"
#include "blockchain.h"
@ -807,16 +806,6 @@ namespace cryptonote
*/
network_type get_nettype() const { return m_nettype; };
/**
* @brief check whether an update is known to be available or not
*
* This does not actually trigger a check, but returns the result
* of the last check
*
* @return whether an update is known to be available or not
*/
bool is_update_available() const { return m_update_available; }
/**
* @brief get whether transaction relay should be padded
*
@ -1082,13 +1071,6 @@ namespace cryptonote
*/
bool check_fork_time();
/**
* @brief checks DNS versions
*
* @return true on success, false otherwise
*/
bool check_updates();
/**
* @brief checks free disk space
*
@ -1180,7 +1162,6 @@ namespace cryptonote
tools::periodic_task m_store_blockchain_interval{12h, false}; //!< interval for manual storing of Blockchain, if enabled
tools::periodic_task m_fork_moaner{2h}; //!< interval for checking HardFork status
tools::periodic_task m_txpool_auto_relayer{2min, false}; //!< interval for checking re-relaying txpool transactions
tools::periodic_task m_check_updates_interval{12h}; //!< interval for checking for new versions
tools::periodic_task m_check_disk_space_interval{10min}; //!< interval for checking for disk space
tools::periodic_task m_check_uptime_proof_interval{std::chrono::seconds{UPTIME_PROOF_TIMER_SECONDS}}; //!< interval for checking our own uptime proof
tools::periodic_task m_block_rate_interval{90s, false}; //!< interval for checking block rate
@ -1195,8 +1176,6 @@ namespace cryptonote
network_type m_nettype; //!< which network are we on?
std::atomic<bool> m_update_available;
std::string m_checkpoints_path; //!< path to json checkpoints file
time_t m_last_json_checkpoints_update; //!< time when json checkpoints were last updated
@ -1228,17 +1207,6 @@ namespace cryptonote
std::unordered_set<crypto::hash> bad_semantics_txes[2];
std::mutex bad_semantics_txes_lock;
enum {
UPDATES_DISABLED,
UPDATES_NOTIFY,
UPDATES_DOWNLOAD,
UPDATES_UPDATE,
} check_updates_level;
tools::download_async_handle m_update_download;
size_t m_last_update_length;
std::mutex m_update_mutex;
bool m_offline;
bool m_pad_transactions;

1
src/cryptonote_protocol/cryptonote_protocol_handler.inl
View File

@ -324,7 +324,6 @@ namespace cryptonote
cnx.current_upload = cntxt.m_current_speed_up / 1024;
cnx.connection_id = epee::string_tools::pod_to_hex(cntxt.m_connection_id);
cnx.ssl = cntxt.m_ssl;
cnx.height = cntxt.m_remote_blockchain_height;
cnx.pruning_seed = cntxt.m_pruning_seed;

1
src/daemon/CMakeLists.txt
View File

@ -37,6 +37,7 @@ loki_add_executable(daemon "lokid"
target_link_libraries(daemon
PRIVATE
rpc
rpc_http_client
blockchain_db
cryptonote_core
p2p

19
src/daemon/command_parser_executor.cpp
View File

@ -40,12 +40,8 @@
namespace daemonize {
command_parser_executor::command_parser_executor(
uint32_t ip
, uint16_t port
, const std::optional<tools::login>& login
)
: m_executor{ip, port, login}
command_parser_executor::command_parser_executor(std::string daemon_url, const std::optional<tools::login>& login)
: m_executor{std::move(daemon_url), login}
{}
command_parser_executor::command_parser_executor(cryptonote::rpc::core_rpc_server& rpc_server)
@ -871,17 +867,6 @@ bool command_parser_executor::print_blockchain_dynamic_stats(const std::vector<s
return m_executor.print_blockchain_dynamic_stats(nblocks);
}
bool command_parser_executor::update(const std::vector<std::string>& args)
{
if(args.size() != 1)
{
std::cout << "Exactly one parameter is needed: check, download, or update" << std::endl;
return false;
}
return m_executor.update(args.front());
}
bool command_parser_executor::relay_tx(const std::vector<std::string>& args)
{
if (args.size() != 1) return false;

9
src/daemon/command_parser_executor.h
View File

@ -33,7 +33,6 @@
#include "daemon/rpc_command_executor.h"
#include "common/common_fwd.h"
#include "net/net_fwd.h"
#include "rpc/core_rpc_server.h"
namespace daemonize {
@ -44,11 +43,7 @@ private:
rpc_command_executor m_executor;
public:
/// Invokes via remote RPC
command_parser_executor(
uint32_t ip
, uint16_t port
, const std::optional<tools::login>& login
);
command_parser_executor(std::string daemon_url, const std::optional<tools::login>& login);
/// Invokes via local daemon
command_parser_executor(cryptonote::rpc::core_rpc_server& rpc_server);
@ -143,8 +138,6 @@ public:
bool print_blockchain_dynamic_stats(const std::vector<std::string>& args);
bool update(const std::vector<std::string>& args);
bool relay_tx(const std::vector<std::string>& args);
bool sync_info(const std::vector<std::string>& args);

8
src/daemon/command_server.cpp
View File

@ -44,12 +44,8 @@
namespace daemonize {
command_server::command_server(
uint32_t ip
, uint16_t port
, const std::optional<tools::login>& login
)
: m_parser{ip, port, login}
command_server::command_server(std::string daemon_url, const std::optional<tools::login>& login)
: m_parser{std::move(daemon_url), login}
{
init_commands();
}

7
src/daemon/command_server.h
View File

@ -33,7 +33,6 @@
#include "common/common_fwd.h"
#include "console_handler.h"
#include "daemon/command_parser_executor.h"
#include "net/net_fwd.h"
namespace daemonize {
@ -45,11 +44,7 @@ private:
public:
/// Remote HTTP RPC constructor
command_server(
uint32_t ip
, uint16_t port
, const std::optional<tools::login>& login
);
command_server(std::string daemon_url, const std::optional<tools::login>& login);
/// Non-remote constructor
command_server(cryptonote::rpc::core_rpc_server& rpc_server);

1
src/daemon/daemon.cpp
View File

@ -49,7 +49,6 @@
#include "daemon/command_server.h"
#include "daemon/command_line_args.h"
#include "net/parse.h"
#include "net/net_ssl.h"
#include "version.h"
#include "command_server.h"

7
src/daemon/main.cpp
View File

@ -56,6 +56,8 @@
namespace po = boost::program_options;
namespace bf = boost::filesystem;
using namespace std::literals;
int main(int argc, char const * argv[])
{
try {
@ -220,8 +222,7 @@ int main(int argc, char const * argv[])
auto rpc_ip_str = command_line::get_arg(vm, arg.rpc_bind_ip);
auto rpc_port = command_line::get_arg(vm, cryptonote::rpc::http_server::arg_rpc_bind_port);
uint32_t rpc_ip;
if (!epee::string_tools::get_ip_int32_from_string(rpc_ip, rpc_ip_str))
if (uint32_t rpc_ip; !epee::string_tools::get_ip_int32_from_string(rpc_ip, rpc_ip_str))
{
std::cerr << "Invalid IP: " << rpc_ip_str << std::endl;
return 1;
@ -246,7 +247,7 @@ int main(int argc, char const * argv[])
}
}
daemonize::command_server rpc_commands{rpc_ip, rpc_port, std::move(login)};
daemonize::command_server rpc_commands{"http://"s + rpc_ip_str + ":" + std::to_string(rpc_port), std::move(login)};
return rpc_commands.process_command_and_log(command) ? 0 : 1;
}
}

50
src/daemon/rpc_command_executor.cpp
View File

@ -216,16 +216,13 @@ namespace {
}
rpc_command_executor::rpc_command_executor(
uint32_t ip
, uint16_t port
, const std::optional<tools::login>& login
std::string remote_url,
const std::optional<tools::login>& login
)
{
std::optional<epee::net_utils::http::login> http_login{};
m_rpc_client.emplace(remote_url);
if (login)
http_login.emplace(login->username, login->password.password());
// FIXME: make ssl argument here optional (and default to disabled)
m_rpc_client = std::make_unique<tools::t_rpc_client>(ip, port, std::move(http_login), epee::net_utils::ssl_support_t::e_ssl_support_disabled);
m_rpc_client->set_auth(login->username, std::string{login->password.password().view()});
}
bool rpc_command_executor::print_checkpoints(uint64_t start_height, uint64_t end_height, bool print_json)
@ -1147,13 +1144,12 @@ bool rpc_command_executor::print_status()
return false;
}
bool daemon_is_alive = m_rpc_client->check_connection();
if(daemon_is_alive) {
tools::success_msg_writer() << "lokid is running";
// Make a request to get_height because it is public and relatively simple
GET_HEIGHT::response res;
if (invoke<GET_HEIGHT>({}, res, "lokid is NOT running")) {
tools::success_msg_writer() << "lokid is running (height: " << res.height << ")";
return true;
}
tools::fail_msg_writer() << "lokid is NOT running";
return false;
}
@ -1496,34 +1492,6 @@ bool rpc_command_executor::print_blockchain_dynamic_stats(uint64_t nblocks)
return true;
}
bool rpc_command_executor::update(const std::string &command)
{
UPDATE::response res{};
if (!invoke<UPDATE>({command}, res, "Failed to fetch update info"))
return false;
if (!res.update)
{
tools::msg_writer() << "No update available";
return true;
}
tools::msg_writer() << "Update available: v" << res.version << ": " << res.user_uri << ", hash " << res.hash;
if (command == "check")
return true;
if (!res.path.empty())
tools::msg_writer() << "Update downloaded to: " << res.path;
else
tools::msg_writer() << "Update download failed: " << res.status;
if (command == "download")
return true;
tools::msg_writer() << "'" << command << "' not implemented yet";
return true;
}
bool rpc_command_executor::relay_tx(const std::string &txid)
{
RELAY_TX::response res{};
@ -2367,7 +2335,7 @@ bool rpc_command_executor::prepare_registration()
const uint64_t amount_left = staking_requirement - state.total_reserved_contributions;
std::cout << "Summary:" << std::endl;
std::cout << "Operating costs as % of reward: " << (state.operator_fee_portions * 100.0 / STAKING_PORTIONS) << "%" << std::endl;
std::cout << "Operating costs as % of reward: " << (state.operator_fee_portions * 100.0 / static_cast<double>(STAKING_PORTIONS)) << "%" << std::endl;
printf("%-16s%-9s%-19s%-s\n", "Contributor", "Address", "Contribution", "Contribution(%)");
printf("%-16s%-9s%-19s%-s\n", "___________", "_______", "____________", "_______________");

18
src/daemon/rpc_command_executor.h
View File

@ -34,9 +34,9 @@
#include <optional>
#include "common/common_fwd.h"
#include "common/rpc_client.h"
#include "common/scoped_message_writer.h"
#include "rpc/http_client.h"
#include "cryptonote_basic/cryptonote_basic.h"
#include "net/net_fwd.h"
#include "rpc/core_rpc_server.h"
#undef LOKI_DEFAULT_LOG_CATEGORY
@ -46,16 +46,15 @@ namespace daemonize {
class rpc_command_executor final {
private:
std::unique_ptr<tools::t_rpc_client> m_rpc_client;
std::optional<cryptonote::rpc::http_client> m_rpc_client;
cryptonote::rpc::core_rpc_server* m_rpc_server = nullptr;
const cryptonote::rpc::rpc_context m_server_context{true};
public:
/// Executor for remote connection RPC
rpc_command_executor(
uint32_t ip
, uint16_t port
, const std::optional<tools::login>& user
std::string remote_url,
const std::optional<tools::login>& user
);
/// Executor for local daemon RPC
rpc_command_executor(cryptonote::rpc::core_rpc_server& rpc_server)
@ -74,8 +73,7 @@ public:
{
try {
if (m_rpc_client) {
if (!m_rpc_client->json_rpc_request(req, res, std::string{RPC::names()[0]}, error))
return false;
res = m_rpc_client->json_rpc<RPC>(RPC::names()[0], req);
} else {
res = m_rpc_server->invoke(std::move(req), m_server_context);
}
@ -84,7 +82,7 @@ public:
} catch (const std::exception& e) {
if (!error.empty())
tools::fail_msg_writer() << error << ": " << e.what();
return true;
return false;
} catch (...) {}
if (!error.empty())
tools::fail_msg_writer() << error;
@ -177,8 +175,6 @@ public:
bool print_blockchain_dynamic_stats(uint64_t nblocks);
bool update(const std::string &command);
bool relay_tx(const std::string &txid);
bool sync_info();

1
src/debug_utilities/CMakeLists.txt
View File

@ -45,6 +45,7 @@ target_link_libraries(object_sizes
PRIVATE
lmdb
p2p
cpr::cpr
extra
)

1
src/device_trezor/CMakeLists.txt
View File

@ -58,6 +58,7 @@ if(DEVICE_TREZOR_READY)
sodium
libusb
protobuf
cpr::cpr
extra)
else()

103
src/device_trezor/trezor/transport.cpp
View File

@ -33,11 +33,14 @@
#include <algorithm>
#include <chrono>
#include <iomanip>
#include <sstream>
#include <boost/endian/conversion.hpp>
#include <boost/asio/io_service.hpp>
#include <boost/asio/ip/udp.hpp>
#include <boost/format.hpp>
#include <lokimq/hex.h>
#include "common/apply_permutation.h"
#include "common/string_util.h"
#include "transport.hpp"
#include "messages/messages-common.pb.h"
@ -335,26 +338,23 @@ namespace trezor{
BridgeTransport::BridgeTransport(
std::optional<std::string> device_path,
std::optional<std::string> bridge_host):
std::optional<std::string> bridge_url):
m_device_path(device_path),
m_bridge_host(bridge_host.value_or(DEFAULT_BRIDGE)),
m_response(std::nullopt),
m_session(std::nullopt),
m_device_info(std::nullopt)
{
m_bridge_url(bridge_url.value_or(DEFAULT_BRIDGE))
{
const char *env_bridge_port = nullptr;
if (!bridge_host && (env_bridge_port = getenv("TREZOR_BRIDGE_PORT")) != nullptr)
if (!bridge_url && (env_bridge_port = getenv("TREZOR_BRIDGE_PORT")) != nullptr)
{
uint16_t bridge_port;
CHECK_AND_ASSERT_THROW_MES(epee::string_tools::get_xtype_from_string(bridge_port, env_bridge_port), "Invalid bridge port: " << env_bridge_port);
CHECK_AND_ASSERT_THROW_MES(tools::parse_int(env_bridge_port, bridge_port), "Invalid bridge port: " << env_bridge_port);
assert_port_number(bridge_port);
m_bridge_host = "127.0.0.1:" + std::to_string(bridge_port);
MDEBUG("Bridge host: " << m_bridge_host);
m_bridge_url = "http://127.0.0.1:" + std::to_string(bridge_port);
}
m_http_client.set_server(m_bridge_host, std::nullopt, epee::net_utils::ssl_support_t::e_ssl_support_disabled);
}
else if (!tools::starts_with(m_bridge_url, "http://") && !tools::starts_with(m_bridge_url, "https://"))
m_bridge_url.insert(0, "http://");
MDEBUG("Bridge host: " << m_bridge_url);
}
std::string BridgeTransport::get_path() const {
if (!m_device_path){
@ -368,7 +368,7 @@ namespace trezor{
json bridge_res;
std::string req;
bool req_status = invoke_bridge_http("/enumerate", req, bridge_res, m_http_client);
bool req_status = invoke_bridge_http("/enumerate", req, bridge_res);
if (!req_status){
throw exc::CommunicationException("Bridge enumeration failed");
}
@ -412,7 +412,7 @@ namespace trezor{
std::string uri = "/acquire/" + *m_device_path + "/null";
std::string req;
json bridge_res;
bool req_status = invoke_bridge_http(uri, req, bridge_res, m_http_client);
bool req_status = invoke_bridge_http(uri, req, bridge_res);
if (!req_status){
throw exc::CommunicationException("Failed to acquire device");
}
@ -434,7 +434,7 @@ namespace trezor{
std::string uri = "/release/" + *m_session;
std::string req;
json bridge_res;
bool req_status = invoke_bridge_http(uri, req, bridge_res, m_http_client);
bool req_status = invoke_bridge_http(uri, req, bridge_res);
if (!req_status){
throw exc::CommunicationException("Failed to release device");
}
@ -456,9 +456,11 @@ namespace trezor{
std::string uri = "/call/" + *m_session;
epee::wipeable_string res_hex;
epee::wipeable_string req_hex = epee::to_hex::wipeable_string(epee::span<const std::uint8_t>(req_buff_raw, buff_size));
epee::wipeable_string req_hex;
req_hex.reserve(buff_size * 2);
lokimq::to_hex(req_buff_raw, req_buff_raw + buff_size, std::back_inserter(req_hex));
bool req_status = invoke_bridge_http(uri, req_hex, res_hex, m_http_client);
bool req_status = invoke_bridge_http(uri, req_hex, res_hex);
if (!req_status){
throw exc::CommunicationException("Call method failed");
}
@ -494,6 +496,48 @@ namespace trezor{
msg = msg_wrap;
}
namespace {
class WipingBody : public cpr::Body {
public:
using cpr::Body::Body;
~WipingBody() override { memwipe(str_.data(), str_.size()); }
};
}
std::string BridgeTransport::post_json(std::string_view uri, std::string json) {
std::string url = m_bridge_url;
if (!tools::ends_with(url, "/") && !tools::starts_with(uri, "/"))
url += '/';
url += uri;
WipingBody body{std::move(json)};
m_http_session.SetUrl(std::string{url});
m_http_session.SetTimeout(HTTP_TIMEOUT);
m_http_session.SetHeader({
{"Origin", "https://monero.trezor.io"}, // FIXME (loki) - does this matter to the bridge?
{"Content-Type", "application/json; charset=utf-8"}
});
m_http_session.SetBody(body);
cpr::Response res;
LOKI_DEFER {
if (!res.text.empty())
memwipe(res.text.data(), res.text.size());
};
res = m_http_session.Post();
if (res.error)
throw std::runtime_error{"Trezor bridge request failed: " + res.error.message};
if (res.status_code != 200)
throw std::runtime_error{"Trezor bridge request failed: received bad HTTP status " + res.status_line};
return std::move(res.text);
}
const std::optional<json> & BridgeTransport::device_info() const {
return m_device_info;
}
@ -508,13 +552,9 @@ namespace trezor{
//
// UdpTransport
//
const char * UdpTransport::PATH_PREFIX = "udp:";
const char * UdpTransport::DEFAULT_HOST = "127.0.0.1";
const int UdpTransport::DEFAULT_PORT = 21324;
static void parse_udp_path(std::string &host, int &port, std::string path)
{
if (boost::starts_with(path, UdpTransport::PATH_PREFIX))
if (tools::starts_with(path, UdpTransport::PATH_PREFIX))
{
path = path.substr(strlen(UdpTransport::PATH_PREFIX));
}
@ -538,7 +578,7 @@ namespace trezor{
if (device_path) {
parse_udp_path(m_device_host, m_device_port, *device_path);
} else if ((env_trezor_path = getenv("TREZOR_PATH")) != nullptr && boost::starts_with(env_trezor_path, UdpTransport::PATH_PREFIX)){
} else if ((env_trezor_path = getenv("TREZOR_PATH")) != nullptr && tools::starts_with(env_trezor_path, UdpTransport::PATH_PREFIX)){
parse_udp_path(m_device_host, m_device_port, std::string(env_trezor_path));
MDEBUG("Applied TREZOR_PATH: " << m_device_host << ":" << m_device_port);
} else {
@ -837,11 +877,10 @@ namespace trezor{
}
static std::string get_usb_path(uint8_t bus_id, const std::vector<uint8_t> &path){
std::stringstream ss;
ss << WebUsbTransport::PATH_PREFIX << (boost::format("%03d") % ((int)bus_id));
for(uint8_t port : path){
ss << ":" << ((int) port);
}
std::ostringstream ss;
ss << WebUsbTransport::PATH_PREFIX << std::setw(3) << std::setfill('0') << (int)bus_id;
for (int port : path)
ss << ':' << port;
return ss.str();
}
@ -1216,10 +1255,10 @@ namespace trezor{
}
std::shared_ptr<Transport> transport(const std::string & path){
if (boost::starts_with(path, BridgeTransport::PATH_PREFIX)){
if (tools::starts_with(path, BridgeTransport::PATH_PREFIX)){
return std::make_shared<BridgeTransport>(path.substr(strlen(BridgeTransport::PATH_PREFIX)));
} else if (boost::starts_with(path, UdpTransport::PATH_PREFIX)){
} else if (tools::starts_with(path, UdpTransport::PATH_PREFIX)){
return std::make_shared<UdpTransport>(path.substr(strlen(UdpTransport::PATH_PREFIX)));
} else {

86
src/device_trezor/trezor/transport.hpp
View File

@ -37,7 +37,11 @@
#include <string_view>
#include <type_traits>
#include <chrono>
#include "net/http_client.h"
#include <cpr/session.h>
#include "wipeable_string.h"
#include "misc_log_ex.h"
#include "rapidjson/document.h"
#include "rapidjson/writer.h"
@ -47,6 +51,8 @@
#include "trezor_defs.hpp"
#include "messages_map.hpp"
#include "common/loki.h"
#include "messages/messages.pb.h"
#include "messages/messages-common.pb.h"
#include "messages/messages-management.pb.h"
@ -58,9 +64,9 @@ namespace trezor {
using json = rapidjson::Document;
using json_val = rapidjson::Value;
namespace http = epee::net_utils::http;
const std::string DEFAULT_BRIDGE = "127.0.0.1:21325";
constexpr const char* DEFAULT_BRIDGE = "http://127.0.0.1:21325";
constexpr std::chrono::milliseconds HTTP_TIMEOUT = 180s;
uint64_t pack_version(uint32_t major, uint32_t minor=0, uint32_t patch=0);
@ -74,47 +80,6 @@ namespace trezor {
bool t_deserialize(std::string & in, epee::wipeable_string & out);
bool t_deserialize(const std::string & in, json & out);
// Flexible json serialization. HTTP client tailored for bridge API
template<class t_req, class t_res, class t_transport>
bool invoke_bridge_http(std::string_view uri, const t_req & out_struct, t_res & result_struct, t_transport& transport, std::string_view method = "POST"sv, std::chrono::milliseconds timeout = 180s)
{
std::string req_param;
t_serialize(out_struct, req_param);
http::fields_list additional_params;
additional_params.push_back(std::make_pair("Origin","https://monero.trezor.io"));
additional_params.push_back(std::make_pair("Content-Type","application/json; charset=utf-8"));
const http::http_response_info* pri = nullptr;
const auto data_cleaner = epee::misc_utils::create_scope_leave_handler([&]() {
if (!req_param.empty()) {
memwipe(&req_param[0], req_param.size());
}
transport.wipe_response();
});
if(!transport.invoke(uri, method, req_param, timeout, &pri, std::move(additional_params)))
{
MERROR("Failed to invoke http request to " << uri);
return false;
}
if(!pri)
{
MERROR("Failed to invoke http request to " << uri << ", internal error (null response ptr)");
return false;
}
if(pri->m_response_code != 200)
{
MERROR("Failed to invoke http request to " << uri << ", wrong response code: " << pri->m_response_code
<< " Response Body: " << pri->m_body);
return false;
}
return t_deserialize(const_cast<http::http_response_info*>(pri)->m_body, result_struct);
}
// Forward decl
class Transport;
class Protocol;
@ -190,9 +155,32 @@ namespace trezor {
const std::optional<json> & device_info() const;
std::ostream& dump(std::ostream& o) const override;
// posts a json request with the given body to the bridge. Returns the body on success, throws
// on error.
std::string post_json(std::string_view uri, std::string json);
// Flexible json serialization. HTTP client tailored for bridge API
template<class t_req, class t_res>
bool invoke_bridge_http(std::string_view uri, const t_req & request, t_res & result)
{
std::string req;
t_serialize(request, req);
std::string res;
LOKI_DEFER { if (!res.empty()) memwipe(res.data(), res.size()); };
try {
res = post_json(uri, std::move(req));
} catch (const std::exception& e) {
MERROR(e.what() << " while requesting " << uri);
}
return t_deserialize(res, result);
}
private:
epee::net_utils::http::http_simple_client m_http_client;
std::string m_bridge_host;
cpr::Session m_http_session;
std::string m_bridge_url;
std::optional<std::string> m_device_path;
std::optional<std::string> m_session;
std::optional<epee::wipeable_string> m_response;
@ -211,9 +199,9 @@ namespace trezor {
virtual ~UdpTransport() = default;
static const char * PATH_PREFIX;
static const char * DEFAULT_HOST;
static const int DEFAULT_PORT;
static constexpr const char* PATH_PREFIX = "udp:";
static constexpr const char* DEFAULT_HOST = "127.0.0.1";
static constexpr int DEFAULT_PORT = 21324;
bool ping() override;
std::string get_path() const override;

1
src/net/CMakeLists.txt
View File

@ -32,7 +32,6 @@ add_library(net
i2p_address.cpp
parse.cpp
socks.cpp
socks_connect.cpp
tor_address.cpp
epee_network_address_hack.cpp
)

93
src/net/socks_connect.cpp
View File

@ -1,93 +0,0 @@
// Copyright (c) 2019, The Monero Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "socks_connect.h"
#include <boost/system/error_code.hpp>
#include <boost/system/system_error.hpp>
#include <cstdint>
#include <memory>
#include <system_error>
#include "net/error.h"
#include "net/net_utils_base.h"
#include "net/socks.h"
#include "string_tools.h"
namespace net
{
namespace socks
{
std::future<boost::asio::ip::tcp::socket>
connector::operator()(const std::string& remote_host, const std::string& remote_port, boost::asio::steady_timer& timeout) const
{
struct future_socket
{
std::promise<boost::asio::ip::tcp::socket> result_;
void operator()(boost::system::error_code error, boost::asio::ip::tcp::socket&& socket)
{
if (error)
{
try { throw boost::system::system_error{error}; }
catch (...) { result_.set_exception(std::current_exception()); }
}
else
result_.set_value(std::move(socket));
}
};
std::future<boost::asio::ip::tcp::socket> out{};
{
std::uint16_t port = 0;
if (!epee::string_tools::get_xtype_from_string(port, remote_port))
throw std::system_error{net::error::invalid_port, "Remote port for socks proxy"};
bool is_set = false;
std::uint32_t ip_address = 0;
std::promise<boost::asio::ip::tcp::socket> result{};
out = result.get_future();
const auto proxy = net::socks::make_connect_client(
boost::asio::ip::tcp::socket{GET_IO_SERVICE(timeout)}, net::socks::version::v4a, future_socket{std::move(result)}
);
if (epee::string_tools::get_ip_int32_from_string(ip_address, remote_host))
is_set = proxy->set_connect_command(epee::net_utils::ipv4_network_address{ip_address, port});
else
is_set = proxy->set_connect_command(remote_host, port);
if (!is_set || !net::socks::client::connect_and_send(proxy, proxy_address))
throw std::system_error{net::error::invalid_host, "Address for socks proxy"};
timeout.async_wait(net::socks::client::async_close{std::move(proxy)});
}
return out;
}
} // socks
} // net

55
src/net/socks_connect.h
View File

@ -1,55 +0,0 @@
// Copyright (c) 2019, The Monero Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#pragma once
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/steady_timer.hpp>
#include <future>
#include <string>
namespace net
{
namespace socks
{
//! Primarily for use with `epee::net_utils::http_client`.
struct connector
{
boost::asio::ip::tcp::endpoint proxy_address;
/*! Creates a new socket, asynchronously connects to `proxy_address`,
and requests a connection to `remote_host` on `remote_port`. Sets
socket as closed if `timeout` is reached.
\return The socket if successful, and exception in the future with
error otherwise. */
std::future<boost::asio::ip::tcp::socket>
operator()(const std::string& remote_host, const std::string& remote_port, boost::asio::steady_timer& timeout) const;
};
} // socks
} // net

8
src/p2p/net_node.h
View File

@ -133,7 +133,7 @@ namespace nodetool
typedef epee::net_utils::boosted_tcp_server<epee::levin::async_protocol_handler<p2p_connection_context>> net_server;
struct network_zone;
using connect_func = std::optional<p2p_connection_context>(network_zone&, epee::net_utils::network_address const&, epee::net_utils::ssl_support_t);
using connect_func = std::optional<p2p_connection_context>(network_zone&, epee::net_utils::network_address const&);
struct config
{
@ -453,8 +453,8 @@ namespace nodetool
//keep connections to initiate some interactions
static std::optional<p2p_connection_context> public_connect(network_zone&, epee::net_utils::network_address const&, epee::net_utils::ssl_support_t);
static std::optional<p2p_connection_context> socks_connect(network_zone&, epee::net_utils::network_address const&, epee::net_utils::ssl_support_t);
static std::optional<p2p_connection_context> public_connect(network_zone&, epee::net_utils::network_address const&);
static std::optional<p2p_connection_context> socks_connect(network_zone&, epee::net_utils::network_address const&);
/* A `std::map` provides constant iterators and key/value pointers even with
@ -480,8 +480,6 @@ namespace nodetool
boost::uuids::uuid m_network_id;
cryptonote::network_type m_nettype;
epee::net_utils::ssl_support_t m_ssl_support;
};
const int64_t default_limit_up = P2P_DEFAULT_LIMIT_RATE_UP; // kB/s

16
src/p2p/net_node.inl
View File

@ -48,7 +48,6 @@
#include "common/dns_utils.h"
#include "common/pruning.h"
#include "net/error.h"
#include "net/net_helper.h"
#include "common/periodic_task.h"
#include "misc_log_ex.h"
#include "p2p_protocol_defs.h"
@ -768,7 +767,6 @@ namespace nodetool
return res;
//try to bind
m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_disabled;
for (auto& zone : m_network_zones)
{
zone.second.m_net_server.get_config_object().set_handler(this);
@ -786,7 +784,7 @@ namespace nodetool
ipv6_port = zone.second.m_port_ipv6;
MINFO("Binding (IPv6) on " << zone.second.m_bind_ipv6_address << ":" << zone.second.m_port_ipv6);
}
res = zone.second.m_net_server.init_server(zone.second.m_port, zone.second.m_bind_ip, ipv6_port, ipv6_addr, m_use_ipv6, m_require_ipv4, epee::net_utils::ssl_support_t::e_ssl_support_disabled);
res = zone.second.m_net_server.init_server(zone.second.m_port, zone.second.m_bind_ip, ipv6_port, ipv6_addr, m_use_ipv6, m_require_ipv4);
CHECK_AND_ASSERT_MES(res, false, "Failed to bind server");
}
}
@ -1193,7 +1191,7 @@ namespace nodetool
<< (last_seen_stamp ? epee::misc_utils::get_time_interval_string(time(NULL) - last_seen_stamp):"never")
<< ")...");
auto con = zone.m_connect(zone, na, m_ssl_support);
auto con = zone.m_connect(zone, na);
if(!con)
{
bool is_priority = is_priority_node(na);
@ -1258,7 +1256,7 @@ namespace nodetool
<< (last_seen_stamp ? epee::misc_utils::get_time_interval_string(time(NULL) - last_seen_stamp):"never")
<< ")...");
auto con = zone.m_connect(zone, na, m_ssl_support);
auto con = zone.m_connect(zone, na);
if (!con) {
bool is_priority = is_priority_node(na);
@ -2794,13 +2792,13 @@ namespace nodetool
template<typename t_payload_net_handler>
std::optional<p2p_connection_context_t<typename t_payload_net_handler::connection_context>>
node_server<t_payload_net_handler>::socks_connect(network_zone& zone, const epee::net_utils::network_address& remote, epee::net_utils::ssl_support_t ssl_support)
node_server<t_payload_net_handler>::socks_connect(network_zone& zone, const epee::net_utils::network_address& remote)
{
auto result = socks_connect_internal(zone.m_net_server.get_stop_signal(), zone.m_net_server.get_io_service(), zone.m_proxy_address, remote);
if (result) // if no error
{
p2p_connection_context context{};
if (zone.m_net_server.add_connection(context, std::move(*result), remote, ssl_support))
if (zone.m_net_server.add_connection(context, std::move(*result), remote))
return {std::move(context)};
}
return std::nullopt;
@ -2808,7 +2806,7 @@ namespace nodetool
template<typename t_payload_net_handler>
std::optional<p2p_connection_context_t<typename t_payload_net_handler::connection_context>>
node_server<t_payload_net_handler>::public_connect(network_zone& zone, epee::net_utils::network_address const& na, epee::net_utils::ssl_support_t ssl_support)
node_server<t_payload_net_handler>::public_connect(network_zone& zone, epee::net_utils::network_address const& na)
{
bool is_ipv4 = na.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id();
bool is_ipv6 = na.get_type_id() == epee::net_utils::ipv6_network_address::get_type_id();
@ -2839,7 +2837,7 @@ namespace nodetool
typename net_server::t_connection_context con{};
const bool res = zone.m_net_server.connect(address, port,
zone.m_config.m_net_config.connection_timeout,
con, zone.m_bind_ip, ssl_support);
con, zone.m_bind_ip);
if (res)
return {std::move(con)};

11
src/rpc/CMakeLists.txt
View File

@ -52,9 +52,13 @@ add_library(daemon_rpc_server
lmq_server.cpp
)
add_library(rpc_http_client
http_client.cpp
)
target_link_libraries(rpc_commands
PUBLIC
common
cpr::cpr
PRIVATE
cryptonote_protocol
extra)
@ -96,3 +100,10 @@ target_link_libraries(daemon_rpc_server
serialization
Boost::thread
extra)
target_link_libraries(rpc_http_client
PUBLIC
common
cpr::cpr
PRIVATE
extra)

46
src/rpc/bootstrap_daemon.cpp
View File

@ -2,6 +2,7 @@
#include <stdexcept>
#include "common/string_util.h"
#include "crypto/crypto.h"
#include "cryptonote_core/cryptonote_core.h"
#include "misc_log_ex.h"
@ -17,7 +18,7 @@ namespace cryptonote
{
}
bootstrap_daemon::bootstrap_daemon(const std::string &address, const std::optional<epee::net_utils::http::login> &credentials)
bootstrap_daemon::bootstrap_daemon(const std::string &address, const std::optional<std::pair<std::string_view, std::string_view>> &credentials)
: bootstrap_daemon(nullptr)
{
if (!set_server(address, credentials))
@ -28,20 +29,14 @@ namespace cryptonote
std::string bootstrap_daemon::address() const noexcept
{
const auto& host = m_http_client.get_host();
if (host.empty())
{
return std::string();
}
return host + ":" + m_http_client.get_port();
return m_http_client.get_base_url();
}
std::optional<uint64_t> bootstrap_daemon::get_height()
{
// query bootstrap daemon's height
cryptonote::rpc::GET_HEIGHT::request req{};
cryptonote::rpc::GET_HEIGHT::response res{};
if (!invoke_http_json("/getheight", req, res))
rpc::GET_HEIGHT::response res{};
if (!invoke<rpc::GET_HEIGHT>({}, res))
{
return std::nullopt;
}
@ -54,38 +49,29 @@ namespace cryptonote
return res.height;
}
bool bootstrap_daemon::handle_result(bool success)
bool bootstrap_daemon::set_server(std::string url, const std::optional<std::pair<std::string_view, std::string_view>> &credentials /* = std::nullopt */)
{
if (!success && m_get_next_public_node)
{
m_http_client.disconnect();
}
if (!tools::starts_with(url, "http://") && !tools::starts_with(url, "https://"))
url.insert(0, "http://");
m_http_client.set_base_url(std::move(url));
if (credentials)
m_http_client.set_auth(credentials->first, credentials->second);
else
m_http_client.set_auth();
return success;
}
bool bootstrap_daemon::set_server(const std::string &address, const std::optional<epee::net_utils::http::login> &credentials /* = std::nullopt */)
{
if (!m_http_client.set_server(address, credentials))
{
MERROR("Failed to set bootstrap daemon address " << address);
return false;
}
MINFO("Changed bootstrap daemon address to " << address);
MINFO("Changed bootstrap daemon address to " << url);
return true;
}
bool bootstrap_daemon::switch_server_if_needed()
{
if (!m_get_next_public_node || m_http_client.is_connected())
{
if (!m_failed || !m_get_next_public_node)
return true;
}
const std::optional<std::string> address = m_get_next_public_node();
if (address) {
m_failed = false;
return set_server(*address);
}

60
src/rpc/bootstrap_daemon.h
View File

@ -3,8 +3,8 @@
#include <functional>
#include <vector>
#include "net/http_client.h"
#include "storages/http_abstract_invoke.h"
#include "rpc/http_client.h"
#include "rpc/core_rpc_server_commands_defs.h"
namespace cryptonote
{
@ -13,52 +13,46 @@ namespace cryptonote
{
public:
bootstrap_daemon(std::function<std::optional<std::string>()> get_next_public_node);
bootstrap_daemon(const std::string &address, const std::optional<epee::net_utils::http::login> &credentials);
bootstrap_daemon(const std::string &address, const std::optional<std::pair<std::string_view, std::string_view>> &credentials);
std::string address() const noexcept;
std::optional<uint64_t> get_height();
bool handle_result(bool success);
// Called when a request has failed either internally or for some external reason; the next
// request will attempt to use a different bootstrap server (if configured).
void set_failed() { m_failed = true; }
template <class t_request, class t_response>
bool invoke_http_json(const std::string_view uri, const t_request &out_struct, t_response &result_struct)
template <class RPC, std::enable_if_t<std::is_base_of_v<rpc::RPC_COMMAND, RPC>, int> = 0>
bool invoke(const typename RPC::request& req, typename RPC::response& res)
{
if (!switch_server_if_needed())
{
return false;
try {
if constexpr (std::is_base_of_v<rpc::LEGACY, RPC>)
// TODO: post-8.x hard fork we can remove this one and let everything go through the
// non-binary json_rpc version instead (because all legacy json commands are callable via
// json_rpc as of daemon 8.x).
res = m_http_client.json<RPC>(RPC::names().front(), req);
else if constexpr (std::is_base_of_v<rpc::BINARY, RPC>)
res = m_http_client.binary<RPC>(RPC::names().front(), req);
else
res = m_http_client.json_rpc<RPC>(RPC::names().front(), req);
} catch (const std::exception& e) {
MWARNING("bootstrap daemon request failed: " << e.what());
set_failed();
return false;
}
return handle_result(epee::net_utils::invoke_http_json(uri, out_struct, result_struct, m_http_client));
}
template <class t_request, class t_response>
bool invoke_http_bin(const std::string_view uri, const t_request &out_struct, t_response &result_struct)
{
if (!switch_server_if_needed())
{
return false;
}
return handle_result(epee::net_utils::invoke_http_bin(uri, out_struct, result_struct, m_http_client));
}
template <class t_request, class t_response>
bool invoke_http_json_rpc(const std::string_view command_name, const t_request &out_struct, t_response &result_struct)
{
if (!switch_server_if_needed())
{
return false;
}
return handle_result(epee::net_utils::invoke_http_json_rpc("/json_rpc", std::string(command_name.begin(), command_name.end()), out_struct, result_struct, m_http_client));
return true;
}
private:
bool set_server(const std::string &address, const std::optional<epee::net_utils::http::login> &credentials = std::nullopt);
bool set_server(std::string address, const std::optional<std::pair<std::string_view, std::string_view>> &credentials = std::nullopt);
bool switch_server_if_needed();
private:
epee::net_utils::http::http_simple_client m_http_client;
rpc::http_client m_http_client;
std::function<std::optional<std::string>()> m_get_next_public_node;
bool m_failed = false;
};
}

199
src/rpc/core_rpc_server.cpp
View File

@ -35,13 +35,12 @@
#include <boost/endian/conversion.hpp>
#include <algorithm>
#include <cstring>
#include <type_traits>
#include <variant>
#include "include_base_utils.h"
#include "string_tools.h"
#include "core_rpc_server.h"
#include "common/command_line.h"
#include "common/updates.h"
#include "common/download.h"
#include "common/loki.h"
#include "common/sha256sum.h"
#include "common/perf_timer.h"
@ -142,7 +141,7 @@ namespace cryptonote { namespace rpc {
}
};
template <typename RPC>
template <typename RPC, std::enable_if_t<std::is_base_of_v<RPC_COMMAND, RPC>, int> = 0>
void register_rpc_command(std::unordered_map<std::string, std::shared_ptr<const rpc_command>>& regs)
{
using Request = typename RPC::request;
@ -214,16 +213,35 @@ namespace cryptonote { namespace rpc {
, m_p2p(p2p)
, m_was_bootstrap_ever_used(false)
{}
//------------------------------------------------------------------------------------------------------------------------------
bool core_rpc_server::set_bootstrap_daemon(const std::string &address, const std::string &username_password)
bool core_rpc_server::set_bootstrap_daemon(const std::string &address, std::string_view username_password)
{
std::optional<epee::net_utils::http::login> credentials;
const auto loc = username_password.find(':');
if (loc != std::string::npos)
std::string_view username, password;
if (auto loc = username_password.find(':'); loc != std::string::npos)
{
credentials = epee::net_utils::http::login(username_password.substr(0, loc), username_password.substr(loc + 1));
username = username_password.substr(0, loc);
password = username_password.substr(loc + 1);
}
return set_bootstrap_daemon(address, credentials);
return set_bootstrap_daemon(address, username, password);
}
//------------------------------------------------------------------------------------------------------------------------------
bool core_rpc_server::set_bootstrap_daemon(const std::string &address, std::string_view username, std::string_view password)
{
std::optional<std::pair<std::string_view, std::string_view>> credentials;
if (!username.empty() || !password.empty())
credentials.emplace(username, password);
std::unique_lock lock{m_bootstrap_daemon_mutex};
if (address.empty())
m_bootstrap_daemon.reset();
else if (address == "auto")
m_bootstrap_daemon = std::make_unique<bootstrap_daemon>([this]{ return get_random_public_node(); });
else
m_bootstrap_daemon = std::make_unique<bootstrap_daemon>(address, credentials);
m_should_use_bootstrap_daemon = (bool) m_bootstrap_daemon;
return true;
}
//------------------------------------------------------------------------------------------------------------------------------
std::optional<std::string> core_rpc_server::get_random_public_node()
@ -266,28 +284,6 @@ namespace cryptonote { namespace rpc {
return std::nullopt;
}
//------------------------------------------------------------------------------------------------------------------------------
bool core_rpc_server::set_bootstrap_daemon(const std::string &address, const std::optional<epee::net_utils::http::login> &credentials)
{
std::unique_lock lock{m_bootstrap_daemon_mutex};
if (address.empty())
{
m_bootstrap_daemon.reset(nullptr);
}
else if (address == "auto")
{
m_bootstrap_daemon.reset(new bootstrap_daemon([this]{ return get_random_public_node(); }));
}
else
{
m_bootstrap_daemon.reset(new bootstrap_daemon(address, credentials));
}
m_should_use_bootstrap_daemon = m_bootstrap_daemon.get() != nullptr;
return true;
}
//------------------------------------------------------------------------------------------------------------------------------
void core_rpc_server::init(const boost::program_options::variables_map& vm)
{
if (!set_bootstrap_daemon(command_line::get_arg(vm, arg_bootstrap_daemon_address),
@ -426,7 +422,6 @@ namespace cryptonote { namespace rpc {
res.database_size = m_core.get_blockchain_storage().get_db().get_database_size();
if (restricted)
res.database_size = round_up(res.database_size, 1'000'000'000);
res.update_available = restricted ? false : m_core.is_update_available();
res.version = restricted ? std::to_string(LOKI_VERSION[0]) : LOKI_VERSION_FULL;
res.status_line = !restricted ? m_core.get_status_string() :
"v" + std::to_string(LOKI_VERSION[0]) + "; Height: " + std::to_string(res.height);
@ -1368,13 +1363,7 @@ namespace cryptonote { namespace rpc {
{
PERF_TIMER(on_set_bootstrap_daemon);
std::optional<epee::net_utils::http::login> credentials;
if (!req.username.empty() || !req.password.empty())
{
credentials = epee::net_utils::http::login(req.username, req.password);
}
if (!set_bootstrap_daemon(req.address, credentials))
if (!set_bootstrap_daemon(req.address, req.username, req.password))
throw rpc_error{ERROR_WRONG_PARAM, "Failed to set bootstrap daemon to address = " + req.address};
SET_BOOTSTRAP_DAEMON::response res{};
@ -1713,7 +1702,7 @@ namespace cryptonote { namespace rpc {
{
MINFO("Bootstrap daemon is out of sync");
lock.unlock();
m_bootstrap_daemon->handle_result(false);
m_bootstrap_daemon->set_failed();
return lock;
}
@ -1748,26 +1737,7 @@ namespace cryptonote { namespace rpc {
std::string command_name{RPC::names().front()};
bool success;
if (std::is_base_of<BINARY, RPC>::value)
success = m_bootstrap_daemon->invoke_http_bin(command_name, req, res);
else
{
// FIXME: this type explosion of having to instantiate nested types is an epee pain point:
// epee is only incapable of nested serialization if you build nested C++ classes mimicing the
// JSON nesting. Ew.
epee::json_rpc::request<typename RPC::request> json_req{};
epee::json_rpc::response_with_error<typename RPC::response> json_resp{};
json_req.jsonrpc = "2.0";
json_req.id = epee::serialization::storage_entry(0);
json_req.method = command_name;
json_req.params = req;
success = m_bootstrap_daemon->invoke_http_json_rpc(command_name, json_req, json_resp);
if (success)
res = std::move(json_resp.result);
}
if (!success)
if (!m_bootstrap_daemon->invoke<RPC>(req, res))
throw std::runtime_error{"Bootstrap request failed"};
m_was_bootstrap_ever_used = true;
@ -2323,113 +2293,6 @@ namespace cryptonote { namespace rpc {
return res;
}
//------------------------------------------------------------------------------------------------------------------------------
UPDATE::response core_rpc_server::invoke(UPDATE::request&& req, rpc_context context)
{
UPDATE::response res{};
PERF_TIMER(on_update);
if (m_core.offline())
{
res.status = "Daemon is running offline";
return res;
}
static const char software[] = "loki";
#ifdef BUILD_TAG
static const char buildtag[] = BOOST_PP_STRINGIZE(BUILD_TAG);
static const char subdir[] = "cli";
#else
static const char buildtag[] = "source";
static const char subdir[] = "source";
#endif
if (req.command != "check" && req.command != "download" && req.command != "update")
{
res.status = "unknown command: '" + req.command + "'";
return res;
}
std::string version, hash;
if (!tools::check_updates(software, buildtag, version, hash))
{
res.status = "Error checking for updates";
return res;
}
if (tools::vercmp(version.c_str(), LOKI_VERSION_STR) <= 0)
{
res.update = false;
res.status = STATUS_OK;
return res;
}
res.update = true;
res.version = version;
res.user_uri = tools::get_update_url(software, subdir, buildtag, version, true);
res.auto_uri = tools::get_update_url(software, subdir, buildtag, version, false);
res.hash = hash;
if (req.command == "check")
{
res.status = STATUS_OK;
return res;
}
boost::filesystem::path path;
if (req.path.empty())
{
std::string filename;
const char *slash = strrchr(res.auto_uri.c_str(), '/');
if (slash)
filename = slash + 1;
else
filename = std::string(software) + "-update-" + version;
path = epee::string_tools::get_current_module_folder();
path /= filename;
}
else
{
path = req.path;
}
crypto::hash file_hash;
if (!tools::sha256sum_file(path.string(), file_hash) || (hash != epee::string_tools::pod_to_hex(file_hash)))
{
MDEBUG("We don't have that file already, downloading");
if (!tools::download(path.string(), res.auto_uri))
{
MERROR("Failed to download " << res.auto_uri);
res.status = "Failed to download";
return res;
}
if (!tools::sha256sum_file(path.string(), file_hash))
{
MERROR("Failed to hash " << path);
res.status = "Failed to hash";
return res;
}
if (hash != epee::string_tools::pod_to_hex(file_hash))
{
MERROR("Download from " << res.auto_uri << " does not match the expected hash");
res.status = "Failed: hash mismatch";
return res;
}
MINFO("New version downloaded to " << path);
}
else
{
MDEBUG("We already have " << path << " with expected hash");
}
res.path = path.string();
if (req.command == "download")
{
res.status = STATUS_OK;
return res;
}
res.status = "'update' not implemented yet";
return res;
}
//------------------------------------------------------------------------------------------------------------------------------
POP_BLOCKS::response core_rpc_server::invoke(POP_BLOCKS::request&& req, rpc_context context)
{
POP_BLOCKS::response res{};

7
src/rpc/core_rpc_server.h
View File

@ -38,8 +38,6 @@
#include <boost/program_options/variables_map.hpp>
#include "bootstrap_daemon.h"
#include "net/http_server_impl_base.h"
#include "net/http_client.h"
#include "core_rpc_server_commands_defs.h"
#include "cryptonote_core/cryptonote_core.h"
#include "p2p/net_node.h"
@ -215,7 +213,6 @@ namespace cryptonote { namespace rpc {
SET_LIMIT::response invoke(SET_LIMIT::request&& req, rpc_context context);
OUT_PEERS::response invoke(OUT_PEERS::request&& req, rpc_context context);
IN_PEERS::response invoke(IN_PEERS::request&& req, rpc_context context);
UPDATE::response invoke(UPDATE::request&& req, rpc_context context);
GET_OUTPUT_DISTRIBUTION::response invoke(GET_OUTPUT_DISTRIBUTION::request&& req, rpc_context context, bool binary = false);
GET_OUTPUT_DISTRIBUTION_BIN::response invoke(GET_OUTPUT_DISTRIBUTION_BIN::request&& req, rpc_context context);
POP_BLOCKS::response invoke(POP_BLOCKS::request&& req, rpc_context context);
@ -312,8 +309,8 @@ private:
//utils
uint64_t get_block_reward(const block& blk);
std::optional<std::string> get_random_public_node();
bool set_bootstrap_daemon(const std::string &address, const std::string &username_password);
bool set_bootstrap_daemon(const std::string &address, const std::optional<epee::net_utils::http::login> &credentials);
bool set_bootstrap_daemon(const std::string &address, std::string_view username_password);
bool set_bootstrap_daemon(const std::string &address, std::string_view username, std::string_view password);
void fill_block_header_response(const block& blk, bool orphan_status, uint64_t height, const crypto::hash& hash, block_header_response& response, bool fill_pow_hash);
std::unique_lock<std::shared_mutex> should_bootstrap_lock();

18
src/rpc/core_rpc_server_commands_defs.cpp
View File

@ -276,7 +276,6 @@ KV_SERIALIZE_MAP_CODE_BEGIN(GET_INFO::response)
KV_SERIALIZE(height_without_bootstrap)
KV_SERIALIZE(was_bootstrap_ever_used)
KV_SERIALIZE(database_size)
KV_SERIALIZE(update_available)
KV_SERIALIZE(version)
KV_SERIALIZE(status_line)
KV_SERIALIZE_MAP_CODE_END()
@ -794,23 +793,6 @@ KV_SERIALIZE_MAP_CODE_BEGIN(GET_ALTERNATE_CHAINS::response)
KV_SERIALIZE_MAP_CODE_END()
KV_SERIALIZE_MAP_CODE_BEGIN(UPDATE::request)
KV_SERIALIZE(command);
KV_SERIALIZE(path);
KV_SERIALIZE_MAP_CODE_END()
KV_SERIALIZE_MAP_CODE_BEGIN(UPDATE::response)
KV_SERIALIZE(status)
KV_SERIALIZE(update)
KV_SERIALIZE(version)
KV_SERIALIZE(user_uri)
KV_SERIALIZE(auto_uri)
KV_SERIALIZE(hash)
KV_SERIALIZE(path)
KV_SERIALIZE_MAP_CODE_END()
KV_SERIALIZE_MAP_CODE_BEGIN(RELAY_TX::request)
KV_SERIALIZE(txids)
KV_SERIALIZE_MAP_CODE_END()

36
src/rpc/core_rpc_server_commands_defs.h
View File

@ -579,7 +579,6 @@ namespace rpc {
uint64_t height_without_bootstrap; // Current length of the local chain of the daemon.
bool was_bootstrap_ever_used; // States if a bootstrap node has ever been used since the daemon started.
uint64_t database_size; // Current size of Blockchain data.
bool update_available; // States if a update is available ('true') and if one is not available ('false').
std::string version; // Current version of software running.
std::string status_line; // A short one-line summary status of the node (requires an admin/unrestricted connection for most details)
@ -831,7 +830,7 @@ namespace rpc {
LOKI_RPC_DOC_INTROSPECT
// Full block information can be retrieved by either block height or hash, like with the above block header calls.
// For full block information, both lookups use the same method, but with different input parameters.
struct GET_BLOCK
struct GET_BLOCK : RPC_COMMAND
{
static constexpr auto names() { return NAMES("get_block", "getblock"); }
@ -1113,7 +1112,7 @@ namespace rpc {
LOKI_RPC_DOC_INTROSPECT
// Get all transaction pool backlog.
struct GET_TRANSACTION_POOL_BACKLOG
struct GET_TRANSACTION_POOL_BACKLOG : RPC_COMMAND
{
static constexpr auto names() { return NAMES("get_txpool_backlog"); }
@ -1593,34 +1592,6 @@ namespace rpc {
};
};
LOKI_RPC_DOC_INTROSPECT
// Update daemon.
struct UPDATE : LEGACY
{
static constexpr auto names() { return NAMES("update"); }
struct request
{
std::string command; // Command to use, either check or download.
std::string path; // Optional, path where to download the update.
KV_MAP_SERIALIZABLE
};
struct response
{
std::string status; // General RPC error code. "OK" means everything looks good.
bool update; // States if an update is available to download (`true`) or not (`false`).
std::string version; // Version available for download.
std::string user_uri;
std::string auto_uri;
std::string hash;
std::string path; // Path to download the update.
KV_MAP_SERIALIZABLE
};
};
LOKI_RPC_DOC_INTROSPECT
// Relay a list of transaction IDs.
struct RELAY_TX : RPC_COMMAND
@ -2324,7 +2295,7 @@ namespace rpc {
LOKI_RPC_DOC_INTROSPECT
struct REPORT_PEER_SS_STATUS
struct REPORT_PEER_SS_STATUS : RPC_COMMAND
{
static constexpr auto names() { return NAMES("report_peer_storage_server_status"); }
@ -2516,7 +2487,6 @@ namespace rpc {
GET_COINBASE_TX_SUM,
GET_BASE_FEE_ESTIMATE,
GET_ALTERNATE_CHAINS,
UPDATE,
RELAY_TX,
SYNC_INFO,
GET_OUTPUT_DISTRIBUTION,

251
src/rpc/http_client.cpp Normal file
View File

@ -0,0 +1,251 @@
#include "http_client.h"
#include <chrono>
#include <regex>
#include <cpr/cpr.h>
#include "common/string_util.h"
#include "cpr/ssl_options.h"
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "rpc.http_client"
namespace cryptonote::rpc {
http_client_connect_error::http_client_connect_error(const cpr::Error& err, const std::string& prefix) :
http_client_error{prefix + err.message},
code{err.code}
{}
void http_client::set_base_url(std::string base_url_) {
std::lock_guard lock{params_mutex};
if (!base_url_.empty() && base_url_.back() != '/')
base_url_ += '/';
base_url = cpr::Url{std::move(base_url_)};
}
std::string http_client::get_base_url() const {
std::shared_lock lock{params_mutex};
return base_url.str();
}
void http_client::set_timeout(std::chrono::milliseconds timeout_) {
std::lock_guard lock{params_mutex};
if (timeout_ > 0s)
timeout = timeout_;
else
timeout.reset();
apply_timeout = true;
}
std::chrono::milliseconds http_client::get_timeout() const {
std::shared_lock lock{params_mutex};
return timeout ? timeout->ms : 0s;
}
void http_client::cancel() {
// We *don't* take a session lock here, which is a little dirty, but resetting the timeout seems
// small enough as to not cause issues when done from another thread.
session.SetTimeout(1ms);
}
http_client::WipedAuth::WipedAuth(std::string_view username, std::string_view password)
: Authentication("", "")
{
auth_string_.clear();
auth_string_.reserve(username.size() + 1 + password.size());
auth_string_ += username;
auth_string_ += ':';
auth_string_ += password;
}
http_client::WipedAuth::~WipedAuth() {
memwipe(auth_string_.data(), auth_string_.size());
}
void http_client::set_auth(std::string_view username, std::string_view password) {
std::lock_guard lock{params_mutex};
if (username.empty() && password.empty()) {
if (auth) {
auth.reset();
apply_auth = true;
}
} else {
auth.emplace(username, password);
apply_auth = true;
}
}
void http_client::set_proxy(std::string proxy_) {
std::lock_guard lock{params_mutex};
if (proxy != proxy_) {
proxy = std::move(proxy_);
apply_proxy = true;
}
}
std::string http_client::get_proxy() const {
std::shared_lock lock{params_mutex};
return proxy;
}
void http_client::set_https_client_cert(std::string cert_path, std::string key_path) {
std::lock_guard lock{params_mutex};
if (cert_path.empty() || key_path.empty()) {
if (client_cert) {
client_cert.reset();
apply_ssl = true;
}
} else {
client_cert.emplace(std::move(cert_path), std::move(key_path));
apply_ssl = true;
}
}
void http_client::set_insecure_https(bool insecure) {
std::lock_guard lock{params_mutex};
if (insecure != !verify_https) {
verify_https = !insecure;
apply_ssl = true;
}
}
void http_client::set_https_cainfo(std::string cainfo_bundle_path) {
std::lock_guard lock{params_mutex};
if (cainfo_bundle_path.empty()) {
if (ca_info) {
ca_info.reset();
apply_ssl = true;
}
} else {
ca_info.emplace(std::move(cainfo_bundle_path));
}
}
void http_client::copy_params_from(const http_client& other) {
std::unique_lock lock{params_mutex, std::defer_lock};
std::shared_lock olock{other.params_mutex, std::defer_lock};
std::lock(lock, olock);
base_url = other.base_url;
timeout = other.timeout;
auth = other.auth;
}
cpr::Response http_client::post(const std::string& uri, cpr::Body body, cpr::Header header) {
if (base_url.str().empty())
throw http_client_error{"Cannot submit request: no base url has been set"};
cpr::Response res;
{
std::shared_lock plock{params_mutex};
std::chrono::steady_clock::time_point start;
if (LOG_ENABLED(Debug))
start = std::chrono::steady_clock::now();
auto url = base_url + uri;
// See if we need to update any of the session paramters; we do it here with only the parameter
// lock, then actually load the setting below once we have the session lock.
std::optional<cpr::Timeout> new_timeout;
if (apply_timeout) {
new_timeout = timeout ? *timeout : cpr::Timeout{0ms};
apply_timeout = false;
}
std::optional<cpr::Authentication> new_auth;
if (apply_auth) {
new_auth = auth ? *auth : cpr::Authentication{"", ""};
apply_auth = false;
}
std::optional<cpr::Proxies> new_proxy;
if (apply_proxy) {
if (proxy.empty())
new_proxy.emplace();
else
new_proxy = cpr::Proxies{{{"http", proxy}, {"https", proxy}}};
apply_proxy = false;
}
std::optional<cpr::SslOptions> new_ssl_opts;
if (apply_ssl) {
new_ssl_opts.emplace();
if (client_cert) {
new_ssl_opts->SetOption(client_cert->first);
new_ssl_opts->SetOption(client_cert->second);
}
if (!verify_https) {
MWARNING("HTTPS certificate verification disabled; this connection is not secure");
new_ssl_opts->SetOption(cpr::ssl::VerifyHost(false));
new_ssl_opts->SetOption(cpr::ssl::VerifyPeer(false));
new_ssl_opts->SetOption(cpr::ssl::VerifyStatus(false));
}
if (ca_info) {
new_ssl_opts->SetOption(*ca_info);
}
}
plock.unlock();
{
std::lock_guard slock{session_mutex};
if (new_auth) session.SetAuth(*new_auth);
if (new_timeout) session.SetTimeout(*new_timeout);
if (new_proxy) session.SetProxies(*std::move(new_proxy));
if (new_ssl_opts) session.SetSslOptions(*new_ssl_opts);
MDEBUG("Submitting post request to " << url);
session.SetUrl(url);
session.SetHeader(header);
session.SetBody(std::move(body));
res = session.Post();
}
MDEBUG(url << ": " <<
(res.error.code != cpr::ErrorCode::OK ? res.error.message : res.status_line) <<
", sent " << res.uploaded_bytes << " bytes, received " << res.downloaded_bytes << " bytes in " <<
tools::friendly_duration(std::chrono::steady_clock::now() - start));
bytes_sent += res.uploaded_bytes;
bytes_received += res.downloaded_bytes;
}
if (res.error.code != cpr::ErrorCode::OK)
throw http_client_connect_error(res.error, "HTTP request failed: ");
if (res.status_code != 200)
throw http_client_response_error(true, res.status_code, "HTTP request failed; server returned " +
(res.status_line.empty() ? std::to_string(res.status_code) : res.status_line));
return res;
}
static const std::regex rexp_match_url{R"(^(?:([a-zA-Z][a-zA-Z0-9.+-]*)://)?(?:(\[[0-9a-fA-F:.]*\])|([^\[\]/:?]*))(?::(\d+))?)", std::regex::optimize};
// proto ipv6 ipv4/host port
std::tuple<std::string, std::string, uint16_t, std::string> http_client::parse_url(const std::string& url) {
std::string proto, host, uri;
uint16_t port = 0;
std::smatch result;
if (!std::regex_search(url, result, rexp_match_url))
throw http_client_error{"Failed to parse URL: " + url};
if (result[1].matched)
proto = result[1];
host = result[result[2].matched ? 2 : 3];
if (result[4].matched) {
auto port_str = result[4].str();
if (!tools::parse_int(port_str, port))
throw http_client_error{"Failed to parse URL: invalid port '" + port_str + "'"}; // i.e. most likely some port value > 65535
}
uri = result.suffix();
return {std::move(proto), std::move(host), port, std::move(uri)};
}
std::tuple<std::string, std::string, uint16_t, std::string> http_client::parse_base_url() const {
std::shared_lock lock{params_mutex};
return parse_url(base_url.str());
}
}

288
src/rpc/http_client.h Normal file
View File

@ -0,0 +1,288 @@
#pragma once
#include <chrono>
#include <stdexcept>
#include <mutex>
#include <shared_mutex>
#include <string_view>
#include "storages/portable_storage_template_helper.h"
#include "net/jsonrpc_structs.h"
#include "memwipe.h"
#include "common/meta.h"
#include "version.h"
#include <cpr/session.h>
#include <cpr/cprtypes.h>
#include <cpr/error.h>
#include <cpr/auth.h>
namespace cryptonote::rpc {
using namespace std::literals;
/// base class for all exceptions thrown by http_client
class http_client_error : public std::runtime_error {
public:
http_client_error(const char* what) : std::runtime_error{what} {}
http_client_error(const std::string& what) : std::runtime_error{what} {}
};
/// Exception class thrown on HTTP communication errors (for example, failure to connect). `what()`
/// contains a description of the error, while `code` can be used to programmatically handle
/// different error conditions.
class http_client_connect_error : public http_client_error {
public:
http_client_connect_error(const cpr::Error& err, const std::string& prefix);
cpr::ErrorCode code; ///< The error code value as returned by cpr
};
/// Exception thrown if we fail to deserialize response data. It can also be thrown if we fail to
/// *serialize* the request, though that is not common and indicates a fundamentally broken request
/// class.
class http_client_serialization_error : public http_client_error {
public:
using http_client_error::http_client_error;
};
/// Exception class thrown for a request that receives an error response from the remote. This can
/// either be an HTTP error response (i.e. did not receive status 200) or a JSON-RPC error.
class http_client_response_error : public http_client_error {
public:
http_client_response_error(bool http_error, int64_t code, const std::string& what)
: http_client_error(what), http_error{http_error}, code{code} {}
bool http_error; // true for HTTP errors, false for JSON errors
int64_t code;
};
/// Class for accessing a remote node for binary, json, or json rpc requests
///
/// This class is thread-safe but not multithreaded (i.e. only one thread can be making a request at
/// a time). Parameters (such as base_url) have their own lock and can be set even while a request
/// is underway in another thread (the changed parameters will not take effect until the next
/// request).
class http_client
{
public:
/// Constructs an http client.
/// \param base_url - the base url for requests; see set_base_url(). If omitted here, it must be
/// specified via set_base_url before making requests.
explicit http_client(std::string base_url_ = "")
{
if (!base_url_.empty())
set_base_url(std::move(base_url_));
session.SetUserAgent("loki rpc client v"s + LOKI_VERSION_STR);
}
/// Sets the base_url to the given one. Will have / appended if it doesn't already end in /. The
/// URL must be a full URL, include protocol (http:// or https://). This call does not validate
/// the given URL: passing something invalid here will not be caught until you attempt to make a
/// request. (You can call parse_url if you want to perform some sanity checks first).
void set_base_url(std::string base_url);
/// Returns the current base URL
std::string get_base_url() const;
/// Parses the given url into [protocol, hostname, port, uri]. Throws http_client_error if the
/// URL is not parseable. For a literal IPv6 address (which must be surrounded in [ ]) the
/// hostname is returned with the surrounding [ ]. Port is optional and will be set to 0 if
/// not explicitly specified in the URL.
///
/// Note that this does non-exhaustive validation of the url; it will catch common errors, but
/// does not attempt to do complete URL validation.
///
/// Example Return value
/// https://example.com:1234/some/path?foo ["https", "example.com", 1234, "/some/path?foo"]
/// blah://example.com ["blah", "example.com", 0, ""]
/// example.com:1234/ ["", "example.com", 1234, "/"]
/// http://[a:b::1]:80/ ["http", "[a:b::1]", 1234, "/"]
static std::tuple<std::string, std::string, uint16_t, std::string> parse_url(const std::string& url);
/// Parses the current base url by passing it to parse_url(). Equivalent to
/// `parse_url(c.get_base_url())`, but slightly more efficient.
std::tuple<std::string, std::string, uint16_t, std::string> parse_base_url() const;
/// Replaces the timeout for future requests with the given value. 0 = no timeout. Default is
/// 15s.
void set_timeout(std::chrono::milliseconds timeout);
/// Gets the timeout. 0 = no timeout.
std::chrono::milliseconds get_timeout() const;
/// Attempts to cancel an existing request by resetting the active timeout to 1ms. Note that any
/// subsequent request will reset the active timeout to the current value configured via
/// set_timeout().
void cancel();
/// Sets a username and password to use for authentication. If both are empty then authentication
/// is disabled.
void set_auth(std::string_view username = ""sv, std::string_view password = ""sv);
/// Sets a proxy server for http/https requests. If empty, clears the current proxy.
void set_proxy(std::string proxy = "");
/// Returns the proxy server, if any.
std::string get_proxy() const;
/// Sets up HTTPS client certificate and key for connecting to a remote node that requires HTTPS
/// client authentication (this is relatively uncommon). Both values are the path to a
/// PEM-encoded file. If either are empty then HTTPS client certificates are disabled.
void set_https_client_cert(std::string cert_path, std::string key_path);
/// Specifies a CA certificate bundle file to use to verify the server's certificate instead of
/// using the operating system's CA certificates. This is typically used to verify self-signed
/// certificates. If the path is empty then the OS CA certificates are used.
void set_https_cainfo(std::string cainfo_bundle_path);
/// Disable HTTPS certificate validation. This is a bad idea: it effectively makes HTTPS
/// insecure.
void set_insecure_https(bool insecure);
/// Copies parameters (base url, timeout, authentication) from another http_client.
void copy_params_from(const http_client& other);
/// Makes a JSON-RPC request; that is, a POST request to /json_rpc with a proper JSON-RPC wrapper
/// around the serialized json data as the body. On a successful response the response is
/// deserialized into a RPC::response which is returned.
///
/// \tparam RPC - the RPC base class; subtypes RPC::request and RPC::response are used for the
/// request and response, respectively.
///
/// \param method - the end-point to be passed as the "method" parameter of the JSON-RPC request.
/// \param req - the request to be serialized and sent as the JSON-RPC "params" value.
///
/// \returns RPC::request, deserialized from the response.
///
/// \throws rpc::http_client_error on connection-related failure
/// \throws rpc::http_client_serialization_error on a serialization failure
/// \throws rpc::http_client_response_error on a successful HTTP request that returns a json_rpc
/// error, or on an HTTP request that returns an HTTP error code.
template <typename RPC>
typename RPC::response json_rpc(std::string_view method, const typename RPC::request& req)
{
epee::json_rpc::request<const typename RPC::request&> jsonrpc_req{"2.0", std::string{method}, json_rpc_id++, req};
std::string req_serialized;
if(!epee::serialization::store_t_to_json(jsonrpc_req, req_serialized))
throw http_client_serialization_error{"Failed to serialize " + tools::type_name(typeid(typename RPC::request))
+ " for json_rpc request for " + std::string{method}};
cpr::Response res = post("json_rpc", std::move(req_serialized), {{"Content-Type", "application/json; charset=utf-8"}});
epee::json_rpc::response_with_error<typename RPC::response> resp{};
if (!epee::serialization::load_t_from_json(resp, res.text))
throw http_client_serialization_error{"Failed to deserialize response for json_rpc request for " + std::string{method}};
if(resp.error.code || resp.error.message.size())
throw http_client_response_error{false, resp.error.code,
"JSON RPC returned an error response: " + (resp.error.message.empty() ? "(no message)" : resp.error.message)};
return std::move(resp.result);
}
/// Makes a binary request; that is, a POST request to /target with the binary-serialized request
/// as the body. The response is binary-deserialized into a RPC::response which is returned.
///
/// \tparam RPC - the RPC base class; subtypes RPC::request and RPC::response are used for the
/// request and response, respectively.
///
/// \param target - the end-point, without a leading /, to be concatenated with the base_url given
/// construction. For example "foo" used with a base url of "https://example.com" will post to
/// https://example.com/foo.
/// \param req - the request to be serialized and sent as JSON.
///
/// \returns RPC::request, deserialized from the response.
///
/// \throws rpc::http_client_error on connection-related failure
/// \throws rpc::http_client_serialization_error on a serialization failure
template <typename RPC>
typename RPC::response binary(std::string_view target_, const typename RPC::request& req)
{
std::string target{target_};
std::string req_serialized;
if(!epee::serialization::store_t_to_binary(req, req_serialized))
throw http_client_serialization_error{"Failed to serialize " + tools::type_name(typeid(typename RPC::request))
+ " for binary request /" + target};
cpr::Response res = post(target, std::move(req_serialized), {{"Content-Type", "application/octet-stream"}});
typename RPC::response result;
if (!epee::serialization::load_t_from_binary(result, res.text))
throw http_client_serialization_error{"Failed to deserialize response for binary request for /" + target};
return result;
}
/// Makes a "legacy" JSON request; that is, a POST request to /target with the serialized json
/// as the body. The response is deserialized into a RPC::response which is returned.
///
/// Note: this is *not* a JSON-RPC request, but rather a plain JSON request where the body is the
/// direct dump of the given request value. See json_rpc if you want to do a JSON-RPC request
/// instead.
///
/// \tparam RPC - the RPC base class; subtypes RPC::request and RPC::response are used for the
/// request and response, respectively.
///
/// \param target - the end-point, without a leading /, to be concatenated with the base_url given
/// construction. For example "foo" used with a base url of "https://example.com" will post to
/// https://example.com/foo.
/// \param req - the request to be serialized and sent as JSON.
///
/// \returns RPC::request, deserialized from the response.
///
/// \throws rpc::http_client_error on connection-related failure
/// \throws rpc::http_client_serialization_error on a serialization failure
template <typename RPC>
typename RPC::response json(std::string_view target_, const typename RPC::request& req)
{
std::string target{target_};
std::string req_serialized;
if(!epee::serialization::store_t_to_json(req, req_serialized))
throw http_client_serialization_error{"Failed to serialize " + tools::type_name(typeid(typename RPC::request))
+ " for json request /" + target};
cpr::Response res = post(target, std::move(req_serialized), {{"Content-Type", "application/json; charset=utf-8"}});
typename RPC::response result;
if (!epee::serialization::load_t_from_json(result, res.text))
throw http_client_serialization_error{"Failed to deserialize response for json request for /" + target};
return result;
}
// Makes a post request.
cpr::Response post(const std::string& uri, cpr::Body body, cpr::Header header);
uint64_t get_bytes_sent() const { return bytes_sent; }
uint64_t get_bytes_received() const { return bytes_received; }
private:
class WipedAuth : public cpr::Authentication {
public:
WipedAuth(std::string_view username, std::string_view password);
~WipedAuth() override;
};
cpr::Session session;
cpr::Url base_url;
std::optional<cpr::Timeout> timeout{15s};
std::optional<WipedAuth> auth;
std::string proxy;
std::optional<std::pair<cpr::ssl::CertFile, cpr::ssl::KeyFile>> client_cert;
bool verify_https = true;
std::optional<cpr::ssl::CaInfo> ca_info;
// Whether we need to apply the above to the session when making the next request
bool apply_timeout = true, apply_auth = false, apply_proxy = false, apply_ssl = false;
mutable std::shared_mutex params_mutex;
mutable std::mutex session_mutex;
std::atomic<int> json_rpc_id = 0;
std::atomic<uint64_t> bytes_sent = 0;
std::atomic<uint64_t> bytes_received = 0;
};
}

137
src/simplewallet/simplewallet.cpp
View File

@ -3195,7 +3195,9 @@ Pending or Failed: "failed"|"pending", "out", Lock, Checkpointed, Time, Amount*
simple_wallet::~simple_wallet()
{
if (m_wallet) m_wallet->m_long_poll_disabled = true;
if (m_wallet) {
m_wallet->cancel_long_poll();
}
if (m_long_poll_thread.joinable())
m_long_poll_thread.join();
}
@ -3486,6 +3488,12 @@ bool simple_wallet::init(const boost::program_options::variables_map& vm)
{
LOKI_DEFER { m_electrum_seed.wipe(); };
if (auto deprecations = tools::wallet2::has_deprecated_options(vm); !deprecations.empty())
{
for (auto msg : deprecations)
message_writer(epee::console_color_red, true) << tr("Warning: option is deprecated and will be removed in the future: ") << msg;
}
const bool testnet = tools::wallet2::has_testnet_option(vm);
const bool stagenet = tools::wallet2::has_stagenet_option(vm);
if (testnet && stagenet)
@ -4151,12 +4159,22 @@ bool simple_wallet::try_connect_to_daemon(bool silent, rpc::version_t* version)
rpc::version_t version_{};
if (!version)
version = &version_;
if (!m_wallet->check_connection(version))
{
bool good = false;
bool threw = false;
constexpr const char* bad_msg = "Check the port and daemon address; if incorrect you can use the 'set_daemon' command or '--daemon-address' option to change them.";
try {
good = m_wallet->check_connection(version, nullptr, !silent);
} catch (const std::exception& e) {
threw = true;
if (!silent)
fail_msg_writer() << tr("wallet failed to connect to daemon: ") << m_wallet->get_daemon_address() << ". " <<
tr("Daemon either is not started or wrong port was passed. "
"Please make sure daemon is running or change the daemon address using the 'set_daemon' command.");
fail_msg_writer() << tr("wallet failed to connect to daemon at ") << m_wallet->get_daemon_address() << ": " << e.what() << ".\n"
<< tr(bad_msg);
}
if (!good)
{
if (!silent && !threw)
// If we get here, the above didn't throw, which means we connected and got a response but the daemon returned a non-okay status
fail_msg_writer() << tr("wallet got bad status from daemon at ") << m_wallet->get_daemon_address() << ".\n" << tr(bad_msg);
return false;
}
if (!m_allow_mismatched_daemon_version && version->first != rpc::VERSION.first)
@ -4732,51 +4750,41 @@ bool simple_wallet::set_daemon(const std::vector<std::string>& args)
return true;
}
std::regex rgx{R"((?:.*://)?(?:[A-Za-z0-9.-]+|\[[0-9a-fA-F:.]+\])(:[0-9]+)?)"};
// proto hostname/ipv4 [ipv6] :port
std::smatch match;
if (std::regex_match(args[0], match, rgx))
{
daemon_url = args[0];
// If no port has been provided append the default from config
if (!match[1].matched)
{
int daemon_port = get_config(m_wallet->nettype()).RPC_DEFAULT_PORT;
daemon_url += ':';
daemon_url += std::to_string(daemon_port);
}
LOCK_IDLE_SCOPE();
m_wallet->init(daemon_url);
bool is_local = false;
try {
auto [proto, host, port, uri] = rpc::http_client::parse_url(args[0]);
if (proto.empty())
proto = "http";
if (port == 0)
port = get_config(m_wallet->nettype()).RPC_DEFAULT_PORT;
daemon_url = std::move(proto) + "://" + host + ":" + std::to_string(port) + uri;
is_local = tools::is_local_address(host);
} catch (const std::exception& e) {
fail_msg_writer() << tr("This does not seem to be a valid daemon URL; enter a URL such as: http://example.com:1234");
return false;
}
if (args.size() == 2)
{
if (args[1] == "trusted")
m_wallet->set_trusted_daemon(true);
else if (args[1] == "untrusted")
m_wallet->set_trusted_daemon(false);
else
{
fail_msg_writer() << tr("Expected trusted or untrusted, got ") << args[1] << ": assuming untrusted";
m_wallet->set_trusted_daemon(false);
}
}
LOCK_IDLE_SCOPE();
m_wallet->init(daemon_url);
if (args.size() == 2)
{
if (args[1] == "trusted")
m_wallet->set_trusted_daemon(true);
else if (args[1] == "untrusted")
m_wallet->set_trusted_daemon(false);
else
{
fail_msg_writer() << tr("Expected trusted or untrusted, got ") << args[1] << ": assuming untrusted";
m_wallet->set_trusted_daemon(false);
try
{
if (tools::is_local_address(m_wallet->get_daemon_address()))
{
MINFO(tr("Daemon is local, assuming trusted"));
m_wallet->set_trusted_daemon(true);
}
}
catch (const std::exception &e) { }
}
success_msg_writer() << boost::format("Daemon set to %s, %s") % daemon_url % (m_wallet->is_trusted_daemon() ? tr("trusted") : tr("untrusted"));
} else {
fail_msg_writer() << tr("This does not seem to be a valid daemon URL.");
}
else if (is_local)
{
MINFO(tr("Daemon is local, assuming trusted"));
m_wallet->set_trusted_daemon(true);
}
success_msg_writer() << "Daemon set to " << daemon_url << ", " << tr(m_wallet->is_trusted_daemon() ? "trusted" : "untrusted");
return true;
}
//----------------------------------------------------------------------------------------------------
@ -6258,11 +6266,10 @@ bool simple_wallet::query_locked_stakes(bool print_result)
std::string msg_buf;
{
using namespace cryptonote;
std::optional<std::string> failed;
const std::vector<rpc::GET_SERVICE_NODES::response::entry> response = m_wallet->get_all_service_nodes(failed);
if (failed)
auto [success, response] = m_wallet->get_all_service_nodes();
if (!success)
{
fail_msg_writer() << *failed;
fail_msg_writer() << "Connection to daemon failed when requesting full service node list";
return has_locked_stakes;
}
@ -6331,12 +6338,10 @@ bool simple_wallet::query_locked_stakes(bool print_result)
}
{
using namespace cryptonote;
std::optional<std::string> failed;
const std::vector<rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::entry> response = m_wallet->get_service_node_blacklisted_key_images(failed);
if (failed)
auto [success, response] = m_wallet->get_service_node_blacklisted_key_images();
if (!success)
{
fail_msg_writer() << *failed;
fail_msg_writer() << "Connection to daemon failed when retrieving blacklisted key images";
return has_locked_stakes;
}
@ -6692,11 +6697,10 @@ bool simple_wallet::lns_print_name_to_owners(const std::vector<std::string>& arg
rpc::LNS_NAMES_TO_OWNERS::request_entry &entry = request.entries.back();
if (entry.types.empty()) entry.types.push_back(static_cast<uint16_t>(lns::mapping_type::session));
std::optional<std::string> failed;
std::vector<rpc::LNS_NAMES_TO_OWNERS::response_entry> response = m_wallet->lns_names_to_owners(request, failed);
if (failed)
auto [success, response] = m_wallet->lns_names_to_owners(request);
if (!success)
{
fail_msg_writer() << *failed;
fail_msg_writer() << "Connection to daemon failed when requesting LNS owners";
return false;
}
@ -6732,7 +6736,6 @@ bool simple_wallet::lns_print_owners_to_names(const std::vector<std::string>& ar
if (!try_connect_to_daemon())
return false;
std::optional<std::string> failed;
std::vector<std::vector<cryptonote::rpc::LNS_OWNERS_TO_NAMES::response_entry>> rpc_results;
std::vector<cryptonote::rpc::LNS_OWNERS_TO_NAMES::request> requests(1);
@ -6770,10 +6773,10 @@ bool simple_wallet::lns_print_owners_to_names(const std::vector<std::string>& ar
rpc_results.reserve(requests.size());
for (auto const &request : requests)
{
std::vector<cryptonote::rpc::LNS_OWNERS_TO_NAMES::response_entry> result = m_wallet->lns_owners_to_names(request, failed);
if (failed)
auto [success, result] = m_wallet->lns_owners_to_names(request);
if (!success)
{
fail_msg_writer() << *failed;
fail_msg_writer() << "Connection to daemon failed when requesting LNS names";
return false;
}
rpc_results.emplace_back(std::move(result));
@ -8670,7 +8673,7 @@ std::string simple_wallet::get_prompt() const
return std::string("[") + tr("locked due to inactivity") + "]";
std::string addr_start = m_wallet->get_subaddress_as_str({m_current_subaddress_account, 0}).substr(0, 6);
std::string prompt = std::string("[") + tr("wallet") + " " + addr_start;
if (!m_wallet->check_connection(NULL))
if (!m_wallet->check_connection())
prompt += tr(" (no daemon)");
else
{
@ -9922,7 +9925,8 @@ int main(int argc, char* argv[])
auto opt_size = command_line::boost_option_sizes();
po::options_description desc_params(wallet_args::tr("Wallet options"), opt_size.first, opt_size.second);
tools::wallet2::init_options(desc_params);
po::options_description hidden_params("Hidden");
tools::wallet2::init_options(desc_params, hidden_params);
command_line::add_arg(desc_params, arg_wallet_file);
command_line::add_arg(desc_params, arg_generate_new_wallet);
command_line::add_arg(desc_params, arg_generate_from_device);
@ -9932,7 +9936,8 @@ int main(int argc, char* argv[])
command_line::add_arg(desc_params, arg_generate_from_multisig_keys);
command_line::add_arg(desc_params, arg_generate_from_json);
command_line::add_arg(desc_params, arg_mnemonic_language);
command_line::add_arg(desc_params, arg_command);
// Positional argument
command_line::add_arg(hidden_params, arg_command);
command_line::add_arg(desc_params, arg_restore_deterministic_wallet );
command_line::add_arg(desc_params, arg_restore_multisig_wallet );
@ -9954,7 +9959,7 @@ int main(int argc, char* argv[])
"loki-wallet-cli [--wallet-file=<filename>|--generate-new-wallet=<filename>] [<COMMAND>]",
sw::tr("This is the command line Loki wallet. It needs to connect to a Loki\ndaemon to work correctly.\n\nWARNING: Do not reuse your Loki keys on a contentious fork, doing so will harm your privacy.\n Only consider reusing your key on a contentious fork if the fork has key reuse mitigations built in."),
desc_params,
po::options_description{},
hidden_params,
positional_options,
[](const std::string &s, bool emphasis){ tools::scoped_message_writer(emphasis ? epee::console_color_white : epee::console_color_default, true) << s; },
"loki-wallet-cli.log"

1
src/wallet/CMakeLists.txt
View File

@ -47,6 +47,7 @@ target_link_libraries(wallet
device_trezor
net
lmdb
rpc_http_client
Boost::serialization
Boost::filesystem
Boost::thread

3
src/wallet/message_store.cpp
View File

@ -34,7 +34,6 @@
#include <fstream>
#include <sstream>
#include "file_io_utils.h"
#include "storages/http_abstract_invoke.h"
#include "wallet_errors.h"
#include "serialization/binary_utils.h"
#include "common/base58.h"
@ -48,7 +47,7 @@
namespace mms
{
message_store::message_store(std::unique_ptr<epee::net_utils::http::abstract_http_client> http_client) : m_transporter(std::move(http_client))
message_store::message_store()
{
m_active = false;
m_auto_send = false;

4
src/wallet/message_store.h
View File

@ -42,8 +42,6 @@
#include "cryptonote_basic/cryptonote_basic.h"
#include "common/i18n.h"
#include "common/command_line.h"
#include "wipeable_string.h"
#include "net/abstract_http_client.h"
#include "message_transporter.h"
#undef LOKI_DEFAULT_LOG_CATEGORY
@ -203,7 +201,7 @@ namespace mms
class message_store
{
public:
message_store(std::unique_ptr<epee::net_utils::http::abstract_http_client> http_client);
message_store();
// Initialize and start to use the MMS, set the first signer, this wallet itself
// Filename, if not null and not empty, is used to create the ".mms" file

73
src/wallet/message_transporter.cpp
View File

@ -30,14 +30,14 @@
#include "string_coding.h"
#include <boost/format.hpp>
#include "wallet_errors.h"
#include "net/http_client.h"
#include "net/net_parse_helpers.h"
#include <algorithm>
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "wallet.mms"
#define PYBITMESSAGE_DEFAULT_API_PORT 8442
using namespace std::literals;
namespace mms
{
@ -78,23 +78,18 @@ namespace bitmessage_rpc
}
message_transporter::message_transporter(std::unique_ptr<epee::net_utils::http::abstract_http_client> http_client) : m_http_client(std::move(http_client))
{
m_run = true;
}
void message_transporter::set_options(const std::string &bitmessage_address, const epee::wipeable_string &bitmessage_login)
{
m_bitmessage_url = bitmessage_address;
epee::net_utils::http::url_content address_parts{};
epee::net_utils::parse_url(m_bitmessage_url, address_parts);
if (address_parts.port == 0)
{
address_parts.port = PYBITMESSAGE_DEFAULT_API_PORT;
}
m_bitmessage_login = bitmessage_login;
auto [proto, host, port, uri] = m_http_client.parse_url(bitmessage_address);
if (port == 0) port = PYBITMESSAGE_DEFAULT_API_PORT;
m_http_client->set_server(address_parts.host, std::to_string(address_parts.port), std::nullopt);
m_http_client.set_base_url(proto + "://" + host + ":" + std::to_string(port) + uri);
if (!bitmessage_login.empty()) {
auto login = bitmessage_login.view();
auto colon = login.find(':');
m_http_client.set_auth(login.substr(0, colon), login.substr(colon == std::string_view::npos ? login.size() : colon + 1));
}
m_http_client.set_timeout(15s);
}
bool message_transporter::receive_messages(const std::vector<std::string> &destination_transport_addresses,
@ -161,7 +156,7 @@ bool message_transporter::receive_messages(const std::vector<std::string> &desti
return true;
}
bool message_transporter::send_message(const transport_message &message)
void message_transporter::send_message(const transport_message &message)
{
// <toAddress> <fromAddress> <subject> <message> [encodingType [TTL]]
std::string request;
@ -176,10 +171,9 @@ bool message_transporter::send_message(const transport_message &message)
end_xml_rpc_cmd(request);
std::string answer;
post_request(request, answer);
return true;
}
bool message_transporter::delete_message(const std::string &transport_id)
void message_transporter::delete_message(const std::string &transport_id)
{
std::string request;
start_xml_rpc_cmd(request, "trashMessage");
@ -187,7 +181,6 @@ bool message_transporter::delete_message(const std::string &transport_id)
end_xml_rpc_cmd(request);
std::string answer;
post_request(request, answer);
return true;
}
// Deterministically derive a new transport address from 'seed' (the 10-hex-digits auto-config
@ -227,44 +220,26 @@ std::string message_transporter::derive_transport_address(const std::string &see
return address;
}
bool message_transporter::delete_transport_address(const std::string &transport_address)
void message_transporter::delete_transport_address(const std::string &transport_address)
{
std::string request;
start_xml_rpc_cmd(request, "leaveChan");
add_xml_rpc_string_param(request, transport_address);
end_xml_rpc_cmd(request);
std::string answer;
return post_request(request, answer);
post_request(request, answer);
}
bool message_transporter::post_request(const std::string &request, std::string &answer)
void message_transporter::post_request(const std::string &request, std::string &answer)
{
// Somehow things do not work out if one tries to connect "m_http_client" to Bitmessage
// and keep it connected over the course of several calls. But with a new connection per
// call and disconnecting after the call there is no problem (despite perhaps a small
// slowdown)
epee::net_utils::http::fields_list additional_params;
// Basic access authentication according to RFC 7617 (which the epee HTTP classes do not seem to support?)
// "m_bitmessage_login" just contains what is needed here, "user:password"
std::string auth_string = epee::string_encoding::base64_encode((const unsigned char*)m_bitmessage_login.data(), m_bitmessage_login.size());
auth_string.insert(0, "Basic ");
additional_params.push_back(std::make_pair("Authorization", auth_string));
additional_params.push_back(std::make_pair("Content-Type", "application/xml; charset=utf-8"));
const epee::net_utils::http::http_response_info* response = NULL;
std::chrono::milliseconds timeout = std::chrono::seconds(15);
bool r = m_http_client->invoke("/", "POST", request, timeout, std::addressof(response), std::move(additional_params));
if (r)
{
answer = response->m_body;
try {
auto res = m_http_client.post("", request, {{"Content-Type", "application/xml; charset=utf-8"}});
answer = res.text;
} catch (const std::exception& e) {
LOG_ERROR("POST request to Bitmessage failed: " << e.what());
THROW_WALLET_EXCEPTION(tools::error::no_connection_to_bitmessage, m_http_client.get_base_url());
}
else
{
LOG_ERROR("POST request to Bitmessage failed: " << request.substr(0, 300));
THROW_WALLET_EXCEPTION(tools::error::no_connection_to_bitmessage, m_bitmessage_url);
}
m_http_client->disconnect(); // see comment above
std::string string_value = get_str_between_tags(answer, "<string>", "</string>");
if ((string_value.find("API Error") == 0) || (string_value.find("RPC ") == 0))
{
@ -285,8 +260,6 @@ bool message_transporter::post_request(const std::string &request, std::string &
THROW_WALLET_EXCEPTION(tools::error::bitmessage_api_error, string_value);
}
}
return r;
}
// Pick some string between two delimiters

20
src/wallet/message_transporter.h
View File

@ -32,9 +32,7 @@
#include "cryptonote_basic/cryptonote_boost_serialization.h"
#include "cryptonote_basic/account_boost_serialization.h"
#include "cryptonote_basic/cryptonote_basic.h"
#include "net/http_server_impl_base.h"
#include "net/http_client.h"
#include "net/abstract_http_client.h"
#include "rpc/http_client.h"
#include "common/util.h"
#include "wipeable_string.h"
#include <vector>
@ -82,23 +80,21 @@ struct transport_message
class message_transporter
{
public:
message_transporter(std::unique_ptr<epee::net_utils::http::abstract_http_client> http_client);
message_transporter() = default;
void set_options(const std::string &bitmessage_address, const epee::wipeable_string &bitmessage_login);
bool send_message(const transport_message &message);
void send_message(const transport_message &message);
bool receive_messages(const std::vector<std::string> &destination_transport_addresses,
std::vector<transport_message> &messages);
bool delete_message(const std::string &transport_id);
void delete_message(const std::string &transport_id);
void stop() { m_run.store(false, std::memory_order_relaxed); }
std::string derive_transport_address(const std::string &seed);
bool delete_transport_address(const std::string &transport_address);
void delete_transport_address(const std::string &transport_address);
private:
const std::unique_ptr<epee::net_utils::http::abstract_http_client> m_http_client;
std::string m_bitmessage_url;
epee::wipeable_string m_bitmessage_login;
std::atomic<bool> m_run;
cryptonote::rpc::http_client m_http_client;
std::atomic<bool> m_run{true};
bool post_request(const std::string &request, std::string &answer);
void post_request(const std::string &request, std::string &answer);
static std::string get_str_between_tags(const std::string &s, const std::string &start_delim, const std::string &stop_delim);
static void start_xml_rpc_cmd(std::string &xml, const std::string &method_name);

371
src/wallet/node_rpc_proxy.cpp
View File

@ -27,18 +27,21 @@
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "node_rpc_proxy.h"
#include "storages/http_abstract_invoke.h"
#include "rpc/core_rpc_server_commands_defs.h"
#include <chrono>
#include <cpr/cpr.h>
namespace rpc = cryptonote::rpc;
using namespace std::literals;
namespace tools
{
static constexpr std::chrono::seconds rpc_timeout{30};
NodeRPCProxy::NodeRPCProxy(epee::net_utils::http::abstract_http_client &http_client, std::recursive_mutex &mutex)
: m_http_client(http_client)
, m_daemon_rpc_mutex(mutex)
NodeRPCProxy::NodeRPCProxy(rpc::http_client& http_client)
: m_http_client{http_client}
, m_offline(false)
{
invalidate();
@ -67,28 +70,22 @@ void NodeRPCProxy::invalidate()
m_rpc_version = {0, 0};
m_target_height = 0;
m_block_weight_limit = 0;
m_get_info_time = 0;
m_height_time = 0;
m_get_info_time = std::chrono::steady_clock::time_point::min();
m_height_time = std::chrono::steady_clock::time_point::min();
}
std::optional<std::string> NodeRPCProxy::get_rpc_version(rpc::version_t &rpc_version) const
bool NodeRPCProxy::get_rpc_version(rpc::version_t &rpc_version) const
{
if (m_offline)
return std::optional<std::string>("offline");
if (m_offline) return false;
if (m_rpc_version == rpc::version_t{0, 0})
{
cryptonote::rpc::GET_VERSION::request req_t{};
cryptonote::rpc::GET_VERSION::response resp_t{};
m_daemon_rpc_mutex.lock();
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "get_version", req_t, resp_t, m_http_client, rpc_timeout);
m_daemon_rpc_mutex.unlock();
CHECK_AND_ASSERT_MES(r, std::string("Failed to connect to daemon"), "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp_t.status != rpc::STATUS_BUSY, resp_t.status, "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp_t.status == rpc::STATUS_OK, resp_t.status, "Failed to get daemon RPC version");
m_rpc_version = rpc::make_version(resp_t.version);
try {
auto res = invoke_json_rpc<rpc::GET_VERSION>({});
m_rpc_version = rpc::make_version(res.version);
} catch (...) { return false; }
}
rpc_version = m_rpc_version;
return std::nullopt;
return true;
}
void NodeRPCProxy::set_height(uint64_t h)
@ -96,100 +93,80 @@ void NodeRPCProxy::set_height(uint64_t h)
m_height = h;
if (h < m_immutable_height)
m_immutable_height = 0;
m_height_time = time(NULL);
m_height_time = std::chrono::steady_clock::now();
}
std::optional<std::string> NodeRPCProxy::get_info() const
bool NodeRPCProxy::get_info() const
{
if (m_offline)
return std::optional<std::string>("offline");
const time_t now = time(NULL);
if (now >= m_get_info_time + 30) // re-cache every 30 seconds
if (m_offline) return false;
auto now = std::chrono::steady_clock::now();
if (now >= m_get_info_time + 30s) // re-cache every 30 seconds
{
cryptonote::rpc::GET_INFO::request req_t{};
cryptonote::rpc::GET_INFO::response resp_t{};
try {
auto resp_t = invoke_json_rpc<rpc::GET_INFO>({});
m_daemon_rpc_mutex.lock();
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "get_info", req_t, resp_t, m_http_client, rpc_timeout);
m_daemon_rpc_mutex.unlock();
CHECK_AND_ASSERT_MES(r, std::string("Failed to connect to daemon"), "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp_t.status != rpc::STATUS_BUSY, resp_t.status, "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp_t.status == rpc::STATUS_OK, resp_t.status, "Failed to get target blockchain height");
m_height = resp_t.height;
m_target_height = resp_t.target_height;
m_block_weight_limit = resp_t.block_weight_limit ? resp_t.block_weight_limit : resp_t.block_size_limit;
m_immutable_height = resp_t.immutable_height;
m_get_info_time = now;
m_height_time = now;
m_height = resp_t.height;
m_target_height = resp_t.target_height;
m_block_weight_limit = resp_t.block_weight_limit ? resp_t.block_weight_limit : resp_t.block_size_limit;
m_immutable_height = resp_t.immutable_height;
m_get_info_time = now;
m_height_time = now;
} catch (...) { return false; }
}
return std::nullopt;
return true;
}
std::optional<std::string> NodeRPCProxy::get_height(uint64_t &height) const
bool NodeRPCProxy::get_height(uint64_t &height) const
{
const time_t now = time(NULL);
if (now < m_height_time + 30) // re-cache every 30 seconds
{
height = m_height;
return std::nullopt;
}
auto now = std::chrono::steady_clock::now();
if (now >= m_height_time + 30s) // re-cache every 30 seconds
if (!get_info())
return false;
auto res = get_info();
if (res)
return res;
height = m_height;
return std::nullopt;
return true;
}
std::optional<std::string> NodeRPCProxy::get_target_height(uint64_t &height) const
bool NodeRPCProxy::get_target_height(uint64_t &height) const
{
auto res = get_info();
if (res)
return res;
if (!get_info())
return false;
height = m_target_height;
return std::nullopt;
return true;
}
std::optional<std::string> NodeRPCProxy::get_immutable_height(uint64_t &height) const
bool NodeRPCProxy::get_immutable_height(uint64_t &height) const
{
auto res = get_info();
if (res)
return res;
if (!get_info())
return false;
height = m_immutable_height;
return std::nullopt;
return false;
}
std::optional<std::string> NodeRPCProxy::get_block_weight_limit(uint64_t &block_weight_limit) const
bool NodeRPCProxy::get_block_weight_limit(uint64_t &block_weight_limit) const
{
auto res = get_info();
if (res)
return res;
if (!get_info())
return false;
block_weight_limit = m_block_weight_limit;
return std::nullopt;
return true;
}
std::optional<std::string> NodeRPCProxy::get_earliest_height(uint8_t version, uint64_t &earliest_height) const
bool NodeRPCProxy::get_earliest_height(uint8_t version, uint64_t &earliest_height) const
{
if (m_offline)
return std::optional<std::string>("offline");
return false;
if (m_earliest_height[version] == 0)
{
cryptonote::rpc::HARD_FORK_INFO::request req_t{};
cryptonote::rpc::HARD_FORK_INFO::response resp_t{};
m_daemon_rpc_mutex.lock();
rpc::HARD_FORK_INFO::request req_t{};
req_t.version = version;
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "hard_fork_info", req_t, resp_t, m_http_client, rpc_timeout);
m_daemon_rpc_mutex.unlock();
CHECK_AND_ASSERT_MES(r, std::string("Failed to connect to daemon"), "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp_t.status != rpc::STATUS_BUSY, resp_t.status, "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp_t.status == rpc::STATUS_OK, resp_t.status, "Failed to get hard fork status");
m_earliest_height[version] = resp_t.earliest_height;
try {
auto resp_t = invoke_json_rpc<rpc::HARD_FORK_INFO>(req_t);
m_earliest_height[version] = resp_t.earliest_height;
} catch (...) { return false; }
}
earliest_height = m_earliest_height[version];
return std::nullopt;
return true;
}
std::optional<uint8_t> NodeRPCProxy::get_hardfork_version() const
@ -197,61 +174,46 @@ std::optional<uint8_t> NodeRPCProxy::get_hardfork_version() const
if (m_offline)
return std::nullopt;
cryptonote::rpc::HARD_FORK_INFO::request req{};
cryptonote::rpc::HARD_FORK_INFO::response resp{};
try {
return invoke_json_rpc<rpc::HARD_FORK_INFO>({}).version;
} catch (...) {}
m_daemon_rpc_mutex.lock();
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "hard_fork_info", req, resp, m_http_client, rpc_timeout);
m_daemon_rpc_mutex.unlock();
CHECK_AND_ASSERT_MES(r, {}, "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp.status != rpc::STATUS_BUSY, {}, "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp.status == rpc::STATUS_OK, {}, "Failed to get hard fork status");
return resp.version;
return std::nullopt;
}
std::optional<std::string> NodeRPCProxy::refresh_dynamic_base_fee_cache(uint64_t grace_blocks) const
bool NodeRPCProxy::refresh_dynamic_base_fee_cache(uint64_t grace_blocks) const
{
uint64_t height;
if (auto error = get_height(height))
return error;
if (m_offline)
return std::string{"offline"};
if (m_offline || !get_height(height))
return false;
if (m_dynamic_base_fee_estimate_cached_height != height || m_dynamic_base_fee_estimate_grace_blocks != grace_blocks)
{
cryptonote::rpc::GET_BASE_FEE_ESTIMATE::request req_t{};
cryptonote::rpc::GET_BASE_FEE_ESTIMATE::response resp_t{};
m_daemon_rpc_mutex.lock();
rpc::GET_BASE_FEE_ESTIMATE::request req_t{};
req_t.grace_blocks = grace_blocks;
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "get_fee_estimate", req_t, resp_t, m_http_client, rpc_timeout);
m_daemon_rpc_mutex.unlock();
CHECK_AND_ASSERT_MES(r, std::string("Failed to connect to daemon"), "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp_t.status != rpc::STATUS_BUSY, resp_t.status, "Failed to connect to daemon");
CHECK_AND_ASSERT_MES(resp_t.status == rpc::STATUS_OK, resp_t.status, "Failed to get fee estimate");
m_dynamic_base_fee_estimate = {resp_t.fee_per_byte, resp_t.fee_per_output};
m_dynamic_base_fee_estimate_cached_height = height;
m_dynamic_base_fee_estimate_grace_blocks = grace_blocks;
m_fee_quantization_mask = resp_t.quantization_mask;
try {
auto resp_t = invoke_json_rpc<rpc::GET_BASE_FEE_ESTIMATE>(req_t);
m_dynamic_base_fee_estimate = {resp_t.fee_per_byte, resp_t.fee_per_output};
m_dynamic_base_fee_estimate_cached_height = height;
m_dynamic_base_fee_estimate_grace_blocks = grace_blocks;
m_fee_quantization_mask = resp_t.quantization_mask;
} catch (...) { return false; }
}
return {};
return true;
}
std::optional<std::string> NodeRPCProxy::get_dynamic_base_fee_estimate(uint64_t grace_blocks, cryptonote::byte_and_output_fees &fees) const
bool NodeRPCProxy::get_dynamic_base_fee_estimate(uint64_t grace_blocks, cryptonote::byte_and_output_fees &fees) const
{
if (auto error = refresh_dynamic_base_fee_cache(grace_blocks))
return error;
if (!refresh_dynamic_base_fee_cache(grace_blocks))
return false;
fees = m_dynamic_base_fee_estimate;
return {};
return true;
}
std::optional<std::string> NodeRPCProxy::get_fee_quantization_mask(uint64_t &fee_quantization_mask) const
bool NodeRPCProxy::get_fee_quantization_mask(uint64_t &fee_quantization_mask) const
{
if (auto error = refresh_dynamic_base_fee_cache(m_dynamic_base_fee_estimate_grace_blocks))
return error;
if (!refresh_dynamic_base_fee_cache(m_dynamic_base_fee_estimate_grace_blocks))
return false;
fee_quantization_mask = m_fee_quantization_mask;
if (fee_quantization_mask == 0)
@ -259,197 +221,128 @@ std::optional<std::string> NodeRPCProxy::get_fee_quantization_mask(uint64_t &fee
MERROR("Fee quantization mask is 0, forcing to 1");
fee_quantization_mask = 1;
}
return {};
}
template <typename T>
static bool check_invoke(bool r, T &response, std::optional<std::string> &failed)
{
if (!r)
{
failed = std::string("Failed to connect to daemon");
return false;
}
if (response.status != rpc::STATUS_OK)
{
failed = response.status;
return false;
}
return true;
}
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> NodeRPCProxy::get_service_nodes(std::vector<std::string> const &pubkeys, std::optional<std::string> &failed) const
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry>> NodeRPCProxy::get_service_nodes(std::vector<std::string> pubkeys) const
{
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> result;
if (m_offline)
{
failed = std::string("offline");
return result;
}
cryptonote::rpc::GET_SERVICE_NODES::request req = {};
cryptonote::rpc::GET_SERVICE_NODES::response res = {};
req.service_node_pubkeys = pubkeys;
m_daemon_rpc_mutex.lock();
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "get_service_nodes", req, res, m_http_client, rpc_timeout);
m_daemon_rpc_mutex.unlock();
if (!check_invoke(r, res, failed))
return result;
result = std::move(res.service_node_states);
return result;
rpc::GET_SERVICE_NODES::request req{};
req.service_node_pubkeys = std::move(pubkeys);
return get_result_pair<rpc::GET_SERVICE_NODES>(req, [](auto&& res) { return std::move(res.service_node_states); });
}
// Updates the cache of all service nodes; the mutex lock must be already held
bool NodeRPCProxy::update_all_service_nodes_cache(uint64_t height, std::optional<std::string> &failed) const {
bool NodeRPCProxy::update_all_service_nodes_cache(uint64_t height) const {
if (m_offline)
{
failed = std::string("offline");
return false;
}
cryptonote::rpc::GET_SERVICE_NODES::request req = {};
cryptonote::rpc::GET_SERVICE_NODES::response res = {};
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "get_all_service_nodes", req, res, m_http_client, rpc_timeout);
if (!check_invoke(r, res, failed))
return false;
m_all_service_nodes_cached_height = height;
m_all_service_nodes = std::move(res.service_node_states);
try {
auto res = invoke_json_rpc<rpc::GET_SERVICE_NODES>({});
m_all_service_nodes_cached_height = height;
m_all_service_nodes = std::move(res.service_node_states);
} catch (...) { return false; }
return true;
}
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> NodeRPCProxy::get_all_service_nodes(std::optional<std::string> &failed) const
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry>> NodeRPCProxy::get_all_service_nodes() const
{
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> result{};
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry>> result;
auto& [success, sns] = result;
success = false;
uint64_t height{0};
failed = get_height(height);
if (failed)
if (!get_height(height))
return result;
{
std::lock_guard<std::recursive_mutex> lock(m_daemon_rpc_mutex);
if (m_all_service_nodes_cached_height != height && !update_all_service_nodes_cache(height, failed))
std::lock_guard lock{m_sn_cache_mutex};
if (m_all_service_nodes_cached_height != height && !update_all_service_nodes_cache(height))
return result;
result = m_all_service_nodes;
sns = m_all_service_nodes;
}
success = true;
return result;
}
// Filtered version of the above that caches the filtered result as long as used on the same
// contributor at the same height (which is very common, for example, for wallet balance lookups).
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> NodeRPCProxy::get_contributed_service_nodes(const std::string &contributor, std::optional<std::string> &failed) const
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry>> NodeRPCProxy::get_contributed_service_nodes(const std::string &contributor) const
{
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> result{};
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry>> result;
auto& [success, sns] = result;
success = false;
uint64_t height;
failed = get_height(height);
if (failed)
if (m_offline || !get_height(height))
return result;
{
std::lock_guard<std::recursive_mutex> lock(m_daemon_rpc_mutex);
std::lock_guard lock{m_sn_cache_mutex};
if (m_contributed_service_nodes_cached_height != height || m_contributed_service_nodes_cached_address != contributor) {
if (m_all_service_nodes_cached_height != height && !update_all_service_nodes_cache(height, failed))
if (m_all_service_nodes_cached_height != height && !update_all_service_nodes_cache(height))
return result;
m_contributed_service_nodes.clear();
std::copy_if(m_all_service_nodes.begin(), m_all_service_nodes.end(), std::back_inserter(m_contributed_service_nodes),
[&contributor](const cryptonote::rpc::GET_SERVICE_NODES::response::entry &e)
[&contributor](const auto& sn)
{
return std::any_of(e.contributors.begin(), e.contributors.end(),
[&contributor](const cryptonote::rpc::service_node_contributor &c) { return contributor == c.address; });
return std::any_of(sn.contributors.begin(), sn.contributors.end(),
[&contributor](const auto& c) { return contributor == c.address; });
}
);
m_contributed_service_nodes_cached_height = height;
m_contributed_service_nodes_cached_address = contributor;
}
result = m_contributed_service_nodes;
sns = m_contributed_service_nodes;
}
success = true;
return result;
}
std::vector<cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::entry> NodeRPCProxy::get_service_node_blacklisted_key_images(std::optional<std::string> &failed) const
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::entry>> NodeRPCProxy::get_service_node_blacklisted_key_images() const
{
std::vector<cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::entry> result{};
if (m_offline)
{
failed = std::string("offline");
return result;
}
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::entry>> result;
auto& [success, sns] = result;
success = false;
uint64_t height;
failed = get_height(height);
if (failed)
if (m_offline || !get_height(height))
return result;
{
std::lock_guard<std::recursive_mutex> lock(m_daemon_rpc_mutex);
std::lock_guard lock{m_sn_cache_mutex};
if (m_service_node_blacklisted_key_images_cached_height != height)
{
cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::request req = {};
cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::response res = {};
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "get_service_node_blacklisted_key_images", req, res, m_http_client, rpc_timeout);
if (!check_invoke(r, res, failed))
return {};
m_service_node_blacklisted_key_images_cached_height = height;
m_service_node_blacklisted_key_images = std::move(res.blacklist);
try {
auto res = invoke_json_rpc<rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES>({});
m_service_node_blacklisted_key_images_cached_height = height;
m_service_node_blacklisted_key_images = std::move(res.blacklist);
} catch (...) {
return result;
}
}
result = m_service_node_blacklisted_key_images;
sns = m_service_node_blacklisted_key_images;
}
success = true;
return result;
}
std::vector<cryptonote::rpc::LNS_OWNERS_TO_NAMES::response_entry> NodeRPCProxy::lns_owners_to_names(cryptonote::rpc::LNS_OWNERS_TO_NAMES::request const &request, std::optional<std::string> &failed) const
std::pair<bool, std::vector<cryptonote::rpc::LNS_OWNERS_TO_NAMES::response_entry>> NodeRPCProxy::lns_owners_to_names(cryptonote::rpc::LNS_OWNERS_TO_NAMES::request const &request) const
{
if (m_offline)
{
failed = std::string("offline");
return {};
}
cryptonote::rpc::LNS_OWNERS_TO_NAMES::response res = {};
{
std::lock_guard<std::recursive_mutex> lock(m_daemon_rpc_mutex);
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "lns_owners_to_names", request, res, m_http_client, rpc_timeout);
if (!check_invoke(r, res, failed))
return {};
}
return res.entries;
return get_result_pair<rpc::LNS_OWNERS_TO_NAMES>(request, [](auto&& res) { return std::move(res.entries); });
}
std::vector<cryptonote::rpc::LNS_NAMES_TO_OWNERS::response_entry> NodeRPCProxy::lns_names_to_owners(cryptonote::rpc::LNS_NAMES_TO_OWNERS::request const &request, std::optional<std::string> &failed) const
std::pair<bool, std::vector<cryptonote::rpc::LNS_NAMES_TO_OWNERS::response_entry>> NodeRPCProxy::lns_names_to_owners(cryptonote::rpc::LNS_NAMES_TO_OWNERS::request const &request) const
{
if (m_offline)
{
failed = std::string("offline");
return {};
}
cryptonote::rpc::LNS_NAMES_TO_OWNERS::response res = {};
{
std::lock_guard<std::recursive_mutex> lock(m_daemon_rpc_mutex);
bool r = epee::net_utils::invoke_http_json_rpc("/json_rpc", "lns_names_to_owners", request, res, m_http_client, rpc_timeout);
if (!check_invoke(r, res, failed))
return {};
}
return res.entries;
return get_result_pair<rpc::LNS_NAMES_TO_OWNERS>(request, [](auto&& res) { return std::move(res.entries); });
}
}

97
src/wallet/node_rpc_proxy.h
View File

@ -28,10 +28,12 @@
#pragma once
#include <chrono>
#include <string>
#include <mutex>
#include <type_traits>
#include "include_base_utils.h"
#include "net/abstract_http_client.h"
#include "rpc/http_client.h"
#include "rpc/core_rpc_server_commands_defs.h"
namespace tools
@ -40,41 +42,86 @@ namespace tools
class NodeRPCProxy
{
public:
NodeRPCProxy(epee::net_utils::http::abstract_http_client &http_client, std::recursive_mutex &mutex);
explicit NodeRPCProxy(cryptonote::rpc::http_client& http_client);
void invalidate();
void set_offline(bool offline) { m_offline = offline; }
std::optional<std::string> get_rpc_version(cryptonote::rpc::version_t &version) const;
std::optional<std::string> get_height(uint64_t &height) const;
bool get_rpc_version(cryptonote::rpc::version_t &version) const;
bool get_height(uint64_t &height) const;
void set_height(uint64_t h);
std::optional<std::string> get_target_height(uint64_t &height) const;
std::optional<std::string> get_immutable_height(uint64_t &height) const;
std::optional<std::string> get_block_weight_limit(uint64_t &block_weight_limit) const;
std::optional<std::string> get_earliest_height(uint8_t version, uint64_t &earliest_height) const;
std::optional<std::string> get_dynamic_base_fee_estimate(uint64_t grace_blocks, cryptonote::byte_and_output_fees &fees) const;
std::optional<std::string> get_fee_quantization_mask(uint64_t &fee_quantization_mask) const;
std::optional<uint8_t> get_hardfork_version() const;
bool get_target_height(uint64_t &height) const;
bool get_immutable_height(uint64_t &height) const;
bool get_block_weight_limit(uint64_t &block_weight_limit) const;
bool get_earliest_height(uint8_t version, uint64_t &earliest_height) const;
bool get_dynamic_base_fee_estimate(uint64_t grace_blocks, cryptonote::byte_and_output_fees &fees) const;
bool get_fee_quantization_mask(uint64_t &fee_quantization_mask) const;
std::optional<uint8_t> get_hardfork_version() const;
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> get_service_nodes(std::vector<std::string> const &pubkeys, std::optional<std::string> &failed) const;
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> get_all_service_nodes(std::optional<std::string> &failed) const;
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> get_contributed_service_nodes(const std::string &contributor, std::optional<std::string> &failed) const;
std::vector<cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::entry> get_service_node_blacklisted_key_images(std::optional<std::string> &failed) const;
std::vector<cryptonote::rpc::LNS_OWNERS_TO_NAMES::response_entry> lns_owners_to_names(cryptonote::rpc::LNS_OWNERS_TO_NAMES::request const &request, std::optional<std::string> &failed) const;
std::vector<cryptonote::rpc::LNS_NAMES_TO_OWNERS::response_entry> lns_names_to_owners(cryptonote::rpc::LNS_NAMES_TO_OWNERS::request const &request, std::optional<std::string> &failed) const;
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry>> get_service_nodes(std::vector<std::string> pubkeys) const;
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry>> get_all_service_nodes() const;
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry>> get_contributed_service_nodes(const std::string& contributor) const;
std::pair<bool, std::vector<cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::entry>> get_service_node_blacklisted_key_images() const;
std::pair<bool, std::vector<cryptonote::rpc::LNS_OWNERS_TO_NAMES::response_entry>> lns_owners_to_names(cryptonote::rpc::LNS_OWNERS_TO_NAMES::request const &request) const;
std::pair<bool, std::vector<cryptonote::rpc::LNS_NAMES_TO_OWNERS::response_entry>> lns_names_to_owners(cryptonote::rpc::LNS_NAMES_TO_OWNERS::request const &request) const;
private:
std::optional<std::string> get_info() const;
bool get_info() const;
epee::net_utils::http::abstract_http_client &m_http_client;
std::recursive_mutex &m_daemon_rpc_mutex;
// Invokes an JSON RPC request and checks it for errors, include a check that the response
// `.status` value is equal to rpc::STATUS_OK. Returns the response on success, logs and throws
// on error.
template <typename RPC>
typename RPC::response invoke_json_rpc(const typename RPC::request& req) const
{
typename RPC::response result;
try {
result = m_http_client.json_rpc<RPC>(RPC::names().front(), req);
} catch (const std::exception& e) {
MERROR(e.what());
throw;
}
if (result.status != cryptonote::rpc::STATUS_OK) {
std::string error = "Request for " + std::string{RPC::names().front()} + " failed: " + (result.status == cryptonote::rpc::STATUS_BUSY ? "daemon is busy" : result.status);
MERROR(error);
throw std::runtime_error{error};
}
return result;
}
// Makes a json rpc request with the given request value and (if successful) returns a
// std::pair<bool, Value>. Takes two arguments: the request, and a lambda that takes an rvalue
// response and returns an value (typically moved via something like `return
// std::move(response.whatever)`).
template <typename RPC, typename GetValue,
typename Value = decltype(std::declval<GetValue>()(typename RPC::response{}))>
std::pair<bool, Value> get_result_pair(const typename RPC::request& req, GetValue get_value, uint64_t* check_height = nullptr) const
{
std::pair<bool, Value> result;
auto& [success, value] = result;
success = false;
if (m_offline)
return result;
try {
value = get_value(invoke_json_rpc<RPC>(req));
success = true;
} catch (...) {}
return result;
}
cryptonote::rpc::http_client& m_http_client;
bool m_offline;
mutable uint64_t m_service_node_blacklisted_key_images_cached_height;
mutable std::vector<cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::entry> m_service_node_blacklisted_key_images;
bool update_all_service_nodes_cache(uint64_t height, std::optional<std::string> &failed) const;
bool update_all_service_nodes_cache(uint64_t height) const;
mutable std::mutex m_sn_cache_mutex;
mutable uint64_t m_all_service_nodes_cached_height;
mutable std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> m_all_service_nodes;
@ -84,17 +131,17 @@ private:
mutable uint64_t m_height;
mutable uint64_t m_immutable_height;
mutable uint64_t m_earliest_height[256];
mutable std::array<uint64_t, 256> m_earliest_height;
mutable cryptonote::byte_and_output_fees m_dynamic_base_fee_estimate;
mutable uint64_t m_dynamic_base_fee_estimate_cached_height;
mutable uint64_t m_dynamic_base_fee_estimate_grace_blocks;
mutable uint64_t m_fee_quantization_mask;
std::optional<std::string> refresh_dynamic_base_fee_cache(uint64_t grace_blocks) const;
bool refresh_dynamic_base_fee_cache(uint64_t grace_blocks) const;
mutable cryptonote::rpc::version_t m_rpc_version;
mutable uint64_t m_target_height;
mutable uint64_t m_block_weight_limit;
mutable time_t m_get_info_time;
mutable time_t m_height_time;
mutable std::chrono::steady_clock::time_point m_get_info_time;
mutable std::chrono::steady_clock::time_point m_height_time;
};
}

691
src/wallet/wallet2.cpp
View File

File diff suppressed because it is too large Load Diff

110
src/wallet/wallet2.h
View File

@ -45,8 +45,6 @@
#include "cryptonote_basic/account.h"
#include "cryptonote_basic/account_boost_serialization.h"
#include "cryptonote_basic/cryptonote_basic_impl.h"
#include "net/http_client.h"
#include "storages/http_abstract_invoke.h"
#include "rpc/core_rpc_server_commands_defs.h"
#include "cryptonote_basic/cryptonote_format_utils.h"
#include "cryptonote_core/cryptonote_tx_utils.h"
@ -78,6 +76,8 @@
#include "common/loki_integration_test_hooks.h"
#include "wipeable_string.h"
#include "rpc/http_client.h"
#undef LOKI_DEFAULT_LOG_CATEGORY
#define LOKI_DEFAULT_LOG_CATEGORY "wallet.wallet2"
@ -263,10 +263,11 @@ private:
static bool has_testnet_option(const boost::program_options::variables_map& vm);
static bool has_stagenet_option(const boost::program_options::variables_map& vm);
static std::vector<std::string> has_deprecated_options(const boost::program_options::variables_map& vm);
static bool has_disable_rpc_long_poll(const boost::program_options::variables_map& vm);
static std::string device_name_option(const boost::program_options::variables_map& vm);
static std::string device_derivation_path_option(const boost::program_options::variables_map &vm);
static void init_options(boost::program_options::options_description& desc_params);
static void init_options(boost::program_options::options_description& desc_params, boost::program_options::options_description& hidden_params);
//! Uses stdin and stdout. Returns a wallet2 if no errors.
static std::pair<std::unique_ptr<wallet2>, password_container> make_from_json(const boost::program_options::variables_map& vm, bool unattended, const std::string& json_file, const std::function<std::optional<password_container>(const char *, bool)> &password_prompter);
@ -284,7 +285,7 @@ private:
static bool verify_password(const std::string& keys_file_name, const epee::wipeable_string& password, bool no_spend_key, hw::device &hwdev, uint64_t kdf_rounds);
static bool query_device(hw::device::device_type& device_type, const std::string& keys_file_name, const epee::wipeable_string& password, uint64_t kdf_rounds = 1);
wallet2(cryptonote::network_type nettype = cryptonote::MAINNET, uint64_t kdf_rounds = 1, bool unattended = false, std::unique_ptr<epee::net_utils::http::http_client_factory> http_client_factory = std::unique_ptr<epee::net_utils::http::http_simple_client_factory>(new epee::net_utils::http::http_simple_client_factory()));
wallet2(cryptonote::network_type nettype = cryptonote::MAINNET, uint64_t kdf_rounds = 1, bool unattended = false);
~wallet2();
struct tx_scan_info_t
@ -618,15 +619,17 @@ private:
bool explicit_refresh_from_block_height() const {return m_explicit_refresh_from_block_height;}
bool deinit();
bool init(std::string daemon_address = "http://localhost:8080",
std::optional<epee::net_utils::http::login> daemon_login = std::nullopt,
boost::asio::ip::tcp::endpoint proxy = {},
uint64_t upper_transaction_weight_limit = 0,
bool trusted_daemon = true,
epee::net_utils::ssl_options_t ssl_options = epee::net_utils::ssl_support_t::e_ssl_support_autodetect);
bool set_daemon(std::string daemon_address = "http://localhost:8080",
std::optional<epee::net_utils::http::login> daemon_login = std::nullopt, bool trusted_daemon = true,
epee::net_utils::ssl_options_t ssl_options = epee::net_utils::ssl_support_t::e_ssl_support_autodetect);
bool init(
std::string daemon_address,
std::optional<tools::login> daemon_login = std::nullopt,
std::string proxy = "",
uint64_t upper_transaction_weight_limit = 0,
bool trusted_daemon = true);
bool set_daemon(
std::string daemon_address,
std::optional<tools::login> daemon_login = std::nullopt,
std::string proxy = "",
bool trusted_daemon = true);
void stop() { m_run.store(false, std::memory_order_relaxed); m_message_store.stop(); }
@ -754,8 +757,7 @@ private:
bool sign_multisig_tx_to_file(multisig_tx_set &exported_txs, const std::string &filename, std::vector<crypto::hash> &txids);
std::vector<pending_tx> create_unmixable_sweep_transactions();
void discard_unmixable_outputs();
bool is_connected() const;
bool check_connection(cryptonote::rpc::version_t *version = NULL, bool *ssl = NULL, uint32_t timeout = 200000);
bool check_connection(cryptonote::rpc::version_t *version = nullptr, bool *ssl = nullptr, bool throw_on_http_error = false);
wallet::transfer_view make_transfer_view(const crypto::hash &txid, const crypto::hash &payment_id, const wallet2::payment_details &pd) const;
wallet::transfer_view make_transfer_view(const crypto::hash &txid, const tools::wallet2::confirmed_transfer_details &pd) const;
wallet::transfer_view make_transfer_view(const crypto::hash &txid, const tools::wallet2::unconfirmed_transfer_details &pd) const;
@ -787,12 +789,15 @@ private:
void get_unconfirmed_payments_out(std::list<std::pair<crypto::hash,wallet2::unconfirmed_transfer_details>>& unconfirmed_payments, const std::optional<uint32_t>& subaddr_account = std::nullopt, const std::set<uint32_t>& subaddr_indices = {}) const;
void get_unconfirmed_payments(std::list<std::pair<crypto::hash,wallet2::pool_payment_details>>& unconfirmed_payments, const std::optional<uint32_t>& subaddr_account = std::nullopt, const std::set<uint32_t>& subaddr_indices = {}) const;
// These return pairs where .first == true if the request was successful, and .second is a
// vector of the requested entries.
//
// NOTE(loki): get_all_service_node caches the result, get_service_nodes doesn't
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> get_all_service_nodes(std::optional<std::string> &failed) const { return m_node_rpc_proxy.get_all_service_nodes(failed); }
std::vector<cryptonote::rpc::GET_SERVICE_NODES::response::entry> get_service_nodes (std::vector<std::string> const &pubkeys, std::optional<std::string> &failed) const { return m_node_rpc_proxy.get_service_nodes(pubkeys, failed); }
std::vector<cryptonote::rpc::GET_SERVICE_NODE_BLACKLISTED_KEY_IMAGES::entry> get_service_node_blacklisted_key_images(std::optional<std::string> &failed) const { return m_node_rpc_proxy.get_service_node_blacklisted_key_images(failed); }
std::vector<cryptonote::rpc::LNS_OWNERS_TO_NAMES::response_entry> lns_owners_to_names(cryptonote::rpc::LNS_OWNERS_TO_NAMES::request const &request, std::optional<std::string> &failed) const { return m_node_rpc_proxy.lns_owners_to_names(request, failed); }
std::vector<cryptonote::rpc::LNS_NAMES_TO_OWNERS::response_entry> lns_names_to_owners(cryptonote::rpc::LNS_NAMES_TO_OWNERS::request const &request, std::optional<std::string> &failed) const { return m_node_rpc_proxy.lns_names_to_owners(request, failed); }
auto get_all_service_nodes() const { return m_node_rpc_proxy.get_all_service_nodes(); }
auto get_service_nodes(std::vector<std::string> const &pubkeys) const { return m_node_rpc_proxy.get_service_nodes(pubkeys); }
auto get_service_node_blacklisted_key_images() const { return m_node_rpc_proxy.get_service_node_blacklisted_key_images(); }
auto lns_owners_to_names(cryptonote::rpc::LNS_OWNERS_TO_NAMES::request const &request) const { return m_node_rpc_proxy.lns_owners_to_names(request); }
auto lns_names_to_owners(cryptonote::rpc::LNS_NAMES_TO_OWNERS::request const &request) const { return m_node_rpc_proxy.lns_names_to_owners(request); }
uint64_t get_blockchain_current_height() const { return m_light_wallet_blockchain_height ? m_light_wallet_blockchain_height : m_blockchain.size(); }
void rescan_spent();
@ -1038,7 +1043,6 @@ private:
std::string get_wallet_file() const;
std::string get_keys_file() const;
std::string get_daemon_address() const;
const std::optional<epee::net_utils::http::login>& get_daemon_login() const { return m_daemon_login; }
uint64_t get_daemon_blockchain_height(std::string& err) const;
uint64_t get_daemon_blockchain_target_height(std::string& err);
/*!
@ -1135,6 +1139,9 @@ private:
// if a network error.
bool long_poll_pool_state();
// Attempts to cancel an existing long poll request (by resetting the timeout).
void cancel_long_poll();
struct get_pool_state_tx
{
cryptonote::transaction tx;
@ -1213,25 +1220,38 @@ private:
crypto::public_key get_multisig_signing_public_key(const crypto::secret_key &skey) const;
template <typename RPC>
bool invoke_http(const typename RPC::request& req, typename RPC::response& res)
bool invoke_http(const typename RPC::request& req, typename RPC::response& res, bool throw_on_error = false)
{
using namespace cryptonote::rpc;
static_assert(std::is_base_of_v<RPC_COMMAND, RPC> || std::is_base_of_v<tools::light_rpc::LIGHT_RPC_COMMAND, RPC>);
if (m_offline) return false;
std::lock_guard lock{m_daemon_rpc_mutex};
if constexpr (std::is_base_of_v<LEGACY, RPC>)
// TODO: post-8.x hard fork we can remove this one and let everything go through the
// non-binary json_rpc version instead (because all legacy json commands are callable via
// json_rpc as of daemon 8.x).
return epee::net_utils::invoke_http_json("/" + std::string{RPC::names().front()}, req, res, *m_http_client, rpc_timeout, "POST");
else if constexpr (std::is_base_of_v<BINARY, RPC>)
return epee::net_utils::invoke_http_bin("/" + std::string{RPC::names().front()}, req, res, *m_http_client, rpc_timeout, "POST");
else if constexpr (std::is_base_of_v<RPC_COMMAND, RPC>)
return epee::net_utils::invoke_http_json_rpc("/json_rpc", std::string{RPC::names().front()}, req, res, *m_http_client, rpc_timeout, "POST", "0");
else // light RPC:
return epee::net_utils::invoke_http_json("/" + std::string{RPC::name}, req, res, *m_http_client, rpc_timeout, "POST");
try {
if constexpr (std::is_base_of_v<LEGACY, RPC>)
// TODO: post-8.x hard fork we can remove this one and let everything go through the
// non-binary json_rpc version instead (because all legacy json commands are callable via
// json_rpc as of daemon 8.x).
res = m_http_client.json<RPC>(RPC::names().front(), req);
else if constexpr (std::is_base_of_v<BINARY, RPC>)
res = m_http_client.binary<RPC>(RPC::names().front(), req);
else if constexpr (std::is_base_of_v<RPC_COMMAND, RPC>)
res = m_http_client.json_rpc<RPC>(RPC::names().front(), req);
else // light RPC:
res = m_http_client.json<RPC>(RPC::name, req);
return true;
} catch (const std::exception& e) {
if (throw_on_error)
throw;
else
MERROR("HTTP request failed: " << e.what());
} catch (...) {
if (throw_on_error)
throw;
else
MERROR("HTTP request failed: unknown error");
}
return false;
}
bool set_ring_database(const std::string &filename);
@ -1355,7 +1375,7 @@ private:
void change_password(const std::string &filename, const epee::wipeable_string &original_password, const epee::wipeable_string &new_password);
void set_tx_notify(const std::shared_ptr<tools::Notify> &notify) { m_tx_notify = notify; }
void set_tx_notify(std::shared_ptr<tools::Notify> notify) { m_tx_notify = std::move(notify); }
bool is_tx_spendtime_unlocked(uint64_t unlock_time, uint64_t block_height) const;
void hash_m_transfer(const transfer_details & transfer, crypto::hash &hash) const;
@ -1365,7 +1385,7 @@ private:
std::atomic<bool> m_long_poll_disabled;
static std::string get_default_daemon_address() { std::lock_guard lock{default_daemon_address_mutex}; return default_daemon_address; }
static std::string get_default_daemon_address();
/// Requests transactions from daemon given hex strings of the tx ids; throws a wallet exception
/// on error, otherwise returns the response.
@ -1378,6 +1398,9 @@ private:
/// Same as above, but for a single transaction.
cryptonote::rpc::GET_TRANSACTIONS::response request_transaction(const crypto::hash& txid) { return request_transactions(std::vector<crypto::hash>{{txid}}); }
// The wallet's RPC client; public for advanced configuration purposes.
cryptonote::rpc::http_client m_http_client;
private:
/*!
* \brief Stores wallet information to wallet file.
@ -1484,17 +1507,13 @@ private:
void on_device_progress(const hw::device_progress& event);
std::string get_rpc_status(const std::string &s) const;
void throw_on_rpc_response_error(const std::optional<std::string> &status, const char *method) const;
bool should_expand(const cryptonote::subaddress_index &index) const;
cryptonote::account_base m_account;
std::optional<epee::net_utils::http::login> m_daemon_login;
std::string m_daemon_address;
std::string m_wallet_file;
std::string m_keys_file;
std::string m_mms_file;
const std::unique_ptr<epee::net_utils::http::abstract_http_client> m_http_client;
hashchain m_blockchain;
std::unordered_map<crypto::hash, unconfirmed_transfer_details> m_unconfirmed_txs;
std::unordered_map<crypto::hash, confirmed_transfer_details> m_confirmed_txs;
@ -1502,11 +1521,10 @@ private:
std::unordered_map<crypto::hash, crypto::secret_key> m_tx_keys;
std::unordered_map<crypto::hash, std::vector<crypto::secret_key>> m_additional_tx_keys;
std::recursive_mutex m_long_poll_mutex;
epee::net_utils::http::http_simple_client m_long_poll_client;
cryptonote::rpc::http_client m_long_poll_client;
bool m_long_poll_local;
mutable std::mutex m_long_poll_tx_pool_checksum_mutex;
crypto::hash m_long_poll_tx_pool_checksum = {};
epee::net_utils::ssl_options_t m_long_poll_ssl_options = epee::net_utils::ssl_support_t::e_ssl_support_autodetect;
transfer_container m_transfers;
payment_container m_payments;
@ -1526,8 +1544,6 @@ private:
std::atomic<bool> m_run;
std::recursive_mutex m_daemon_rpc_mutex;
bool m_trusted_daemon;
i_wallet2_callback* m_callback;
hw::device::device_type m_key_device_type;
@ -1622,8 +1638,8 @@ private:
ExportFormat m_export_format;
static std::mutex default_daemon_address_mutex;
static std::string default_daemon_address;
inline static std::mutex default_daemon_address_mutex;
inline static std::string default_daemon_address;
};
// TODO(loki): Hmm. We need this here because we make register_service_node do

49
src/wallet/wallet_rpc_server.cpp
View File

@ -4014,51 +4014,18 @@ namespace tools
er.message = "Command unavailable in restricted mode.";
return false;
}
std::vector<std::vector<uint8_t>> ssl_allowed_fingerprints;
ssl_allowed_fingerprints.reserve(req.ssl_allowed_fingerprints.size());
for (const std::string &fp: req.ssl_allowed_fingerprints)
{
ssl_allowed_fingerprints.push_back({});
std::vector<uint8_t> &v = ssl_allowed_fingerprints.back();
for (auto c: fp)
v.push_back(c);
}
epee::net_utils::ssl_options_t ssl_options = epee::net_utils::ssl_support_t::e_ssl_support_enabled;
if (req.ssl_allow_any_cert)
ssl_options.verification = epee::net_utils::ssl_verification_t::none;
else if (!ssl_allowed_fingerprints.empty() || !req.ssl_ca_file.empty())
ssl_options = epee::net_utils::ssl_options_t{std::move(ssl_allowed_fingerprints), std::move(req.ssl_ca_file)};
if (!epee::net_utils::ssl_support_from_string(ssl_options.support, req.ssl_support))
{
er.code = WALLET_RPC_ERROR_CODE_NO_DAEMON_CONNECTION;
er.message = std::string("Invalid ssl support mode");
return false;
}
ssl_options.auth = epee::net_utils::ssl_authentication_t{
std::move(req.ssl_private_key_path), std::move(req.ssl_certificate_path)
};
const bool verification_required =
ssl_options.verification != epee::net_utils::ssl_verification_t::none &&
ssl_options.support == epee::net_utils::ssl_support_t::e_ssl_support_enabled;
if (verification_required && !ssl_options.has_strong_verification(""sv))
{
er.code = WALLET_RPC_ERROR_CODE_NO_DAEMON_CONNECTION;
er.message = "SSL is enabled but no user certificate or fingerprints were provided";
return false;
}
if (!m_wallet->set_daemon(req.address, std::nullopt, req.trusted, std::move(ssl_options)))
if (!m_wallet->set_daemon(req.address, std::nullopt, req.proxy, req.trusted))
{
er.code = WALLET_RPC_ERROR_CODE_NO_DAEMON_CONNECTION;
er.message = std::string("Unable to set daemon");
return false;
}
m_wallet->m_http_client.set_https_client_cert(req.ssl_certificate_path, req.ssl_private_key_path);
m_wallet->m_http_client.set_insecure_https(req.ssl_allow_any_cert);
m_wallet->m_http_client.set_https_cainfo(req.ssl_ca_file);
return true;
}
//------------------------------------------------------------------------------------------------------------------------------
@ -4633,7 +4600,8 @@ int main(int argc, char **argv)
auto opt_size = command_line::boost_option_sizes();
po::options_description desc_params(wallet_args::tr("Wallet options"), opt_size.first, opt_size.second);
tools::wallet2::init_options(desc_params);
po::options_description hidden_params("Hidden");
tools::wallet2::init_options(desc_params, hidden_params);
command_line::add_arg(desc_params, arg_rpc_bind_port);
command_line::add_arg(desc_params, arg_disable_rpc_login);
command_line::add_arg(desc_params, arg_restricted);
@ -4643,7 +4611,6 @@ int main(int argc, char **argv)
command_line::add_arg(desc_params, arg_wallet_dir);
command_line::add_arg(desc_params, arg_prompt_for_password);
po::options_description hidden_params("Hidden");
daemonizer::init_options(hidden_params, desc_params);
auto [vm, should_terminate] = wallet_args::main(

16
src/wallet/wallet_rpc_server_commands_defs.h
View File

@ -2655,23 +2655,21 @@ namespace wallet_rpc
{
struct request
{
std::string address; // The remote address of the daemon
std::string address; // The remote url of the daemon.
std::string proxy; // Optional proxy to use for connection. E.g. socks4a://hostname:port for a SOCKS proxy.
bool trusted; // When true, allow the usage of commands that may compromise privacy
std::string ssl_support; // disabled, enabled, autodetect
std::string ssl_private_key_path;
std::string ssl_certificate_path;
std::string ssl_ca_file;
std::vector<std::string> ssl_allowed_fingerprints;
bool ssl_allow_any_cert;
std::string ssl_private_key_path; // HTTPS client authentication: path to private key. Must use an address starting with https://
std::string ssl_certificate_path; // HTTPS client authentication: path to certificate. Must use an address starting with https://
std::string ssl_ca_file; // Path to CA bundle to use for HTTPS server certificate verification instead of system CA. Requires an https:// address.
bool ssl_allow_any_cert; // Make HTTPS insecure: disable HTTPS certificate verification when using an https:// address.
BEGIN_KV_SERIALIZE_MAP()
KV_SERIALIZE(address)
KV_SERIALIZE(proxy)
KV_SERIALIZE_OPT(trusted, false)
KV_SERIALIZE_OPT(ssl_support, (std::string)"autodetect")
KV_SERIALIZE(ssl_private_key_path)
KV_SERIALIZE(ssl_certificate_path)
KV_SERIALIZE(ssl_ca_file)
KV_SERIALIZE(ssl_allowed_fingerprints)
KV_SERIALIZE_OPT(ssl_allow_any_cert, false)
END_KV_SERIALIZE_MAP()
};

13
tests/fuzz/CMakeLists.txt
View File

@ -134,19 +134,6 @@ set_property(TARGET parse-url_fuzz_tests
PROPERTY
FOLDER "tests")
add_executable(http-client_fuzz_tests http-client.cpp fuzzer.cpp)
target_link_libraries(http-client_fuzz_tests
PRIVATE
common
epee
Boost::thread
Boost::program_options
Boost::system
extra)
set_property(TARGET http-client_fuzz_tests
PROPERTY
FOLDER "tests")
add_executable(levin_fuzz_tests levin.cpp fuzzer.cpp)
target_link_libraries(levin_fuzz_tests
PRIVATE

2
tests/fuzz/cold-outputs.cpp
View File

@ -54,7 +54,7 @@ int ColdOutputsFuzzer::init()
try
{
wallet.init("", std::nullopt, boost::asio::ip::tcp::endpoint{}, 0, true, epee::net_utils::ssl_support_t::e_ssl_support_disabled);
wallet.init("");
wallet.set_subaddress_lookahead(1, 1);
wallet.generate("", "", spendkey, true, false);
}

2
tests/fuzz/cold-transaction.cpp
View File

@ -55,7 +55,7 @@ int ColdTransactionFuzzer::init()
try
{
wallet.init("", std::nullopt, boost::asio::ip::tcp::endpoint{}, 0, true, epee::net_utils::ssl_support_t::e_ssl_support_disabled);
wallet.init("");
wallet.set_subaddress_lookahead(1, 1);
wallet.generate("", "", spendkey, true, false);
}

105
tests/fuzz/http-client.cpp
View File

@ -1,105 +0,0 @@
// Copyright (c) 2017-2018, The Monero Project
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification, are
// permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this list of
// conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice, this list
// of conditions and the following disclaimer in the documentation and/or other
// materials provided with the distribution.
//
// 3. Neither the name of the copyright holder nor the names of its contributors may be
// used to endorse or promote products derived from this software without specific
// prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "include_base_utils.h"
#include "file_io_utils.h"
#include "net/http_client.h"
#include "net/net_ssl.h"
#include "fuzzer.h"
class dummy_client
{
public:
bool connect(const std::string& addr, int port, std::chrono::milliseconds timeout, bool ssl = false, const std::string& bind_ip = "0.0.0.0") { return true; }
bool connect(const std::string& addr, const std::string& port, std::chrono::milliseconds timeout, bool ssl = false, const std::string& bind_ip = "0.0.0.0") { return true; }
bool disconnect() { return true; }
bool send(const std::string& buff, std::chrono::milliseconds timeout) { return true; }
bool send(const void* data, size_t sz) { return true; }
bool is_connected() { return true; }
bool recv(std::string& buff, std::chrono::milliseconds timeout)
{
buff = data;
data.clear();
return true;
}
void set_ssl(epee::net_utils::ssl_options_t ssl_options) { }
bool is_connected(bool *ssl = NULL) { return true; }
uint64_t get_bytes_sent() const { return 1; }
uint64_t get_bytes_received() const { return 1; }
void set_test_data(const std::string &s) { data = s; }
private:
std::string data;
};
class HTTPClientFuzzer: public Fuzzer
{
public:
HTTPClientFuzzer() {}
virtual int init();
virtual int run(const std::string &filename);
private:
epee::net_utils::http::http_simple_client_template<dummy_client> client;
};
int HTTPClientFuzzer::init()
{
return 0;
}
int HTTPClientFuzzer::run(const std::string &filename)
{
std::string s;
if (!epee::file_io_utils::load_file_to_string(filename, s))
{
std::cout << "Error: failed to load file " << filename << std::endl;
return 1;
}
try
{
client.test(s, std::chrono::milliseconds(1000));
}
catch (const std::exception &e)
{
std::cerr << "Failed to test http client: " << e.what() << std::endl;
return 1;
}
return 0;
}
int main(int argc, const char **argv)
{
TRY_ENTRY();
HTTPClientFuzzer fuzzer;
return run_fuzzer(argc, argv, fuzzer);
CATCH_ENTRY_L0("main", 1);
}

2
tests/fuzz/signature.cpp
View File

@ -54,7 +54,7 @@ int SignatureFuzzer::init()
try
{
wallet.init("", std::nullopt, boost::asio::ip::tcp::endpoint{}, 0, true, epee::net_utils::ssl_support_t::e_ssl_support_disabled);
wallet.init("");
wallet.set_subaddress_lookahead(1, 1);
wallet.generate("", "", spendkey, true, false);

2
tests/unit_tests/levin.cpp
View File

@ -126,7 +126,7 @@ namespace
handler_(std::addressof(endpoint_), connections, context_)
{
using base_type = epee::net_utils::connection_context_base;
static_cast<base_type&>(context_) = base_type{random_generator(), {}, is_incoming, false};
static_cast<base_type&>(context_) = base_type{random_generator(), {}, is_incoming};
handler_.after_init_connection();
}

2
tests/unit_tests/multisig.cpp
View File

@ -71,7 +71,7 @@ static void make_wallet(unsigned int idx, tools::wallet2 &wallet)
try
{
wallet.init("", std::nullopt, boost::asio::ip::tcp::endpoint{}, 0, true, epee::net_utils::ssl_support_t::e_ssl_support_disabled);
wallet.init("");
wallet.set_subaddress_lookahead(1, 1);
wallet.generate("", "", spendkey, true, false);
ASSERT_TRUE(test_addresses[idx].address == wallet.get_account().get_public_address_str(cryptonote::TESTNET));

317
tests/unit_tests/net.cpp
View File

@ -56,8 +56,6 @@
#include "net/error.h"
#include "net/i2p_address.h"
#include "net/net_utils_base.h"
#include "net/socks.h"
#include "net/socks_connect.h"
#include "net/parse.h"
#include "net/tor_address.h"
#include "p2p/net_peerlist_boost_serialization.h"
@ -934,321 +932,6 @@ TEST(get_network_address, ipv4subnet)
EXPECT_STREQ("12.34.0.0/16", address->str().c_str());
}
namespace
{
using stream_type = boost::asio::ip::tcp;
struct io_thread
{
boost::asio::io_service io_service;
boost::asio::io_service::work work;
stream_type::socket server;
stream_type::acceptor acceptor;
std::thread io;
std::atomic<bool> connected;
io_thread()
: io_service(),
work(io_service),
server(io_service),
acceptor(io_service),
io([this] () { try { this->io_service.run(); } catch (const std::exception& e) { MERROR(e.what()); }}),
connected(false)
{
acceptor.open(boost::asio::ip::tcp::v4());
acceptor.bind(stream_type::endpoint{boost::asio::ip::address_v4::loopback(), 0});
acceptor.listen();
acceptor.async_accept(server, [this] (boost::system::error_code error) {
this->connected = true;
if (error)
throw boost::system::system_error{error};
});
}
~io_thread() noexcept
{
io_service.stop();
if (io.joinable())
io.join();
}
};
struct checked_client
{
std::atomic<bool>* called_;
bool expected_;
void operator()(boost::system::error_code error, net::socks::client::stream_type::socket&&) const
{
EXPECT_EQ(expected_, bool(error)) << "Socks server: " << error.message();
ASSERT_TRUE(called_ != nullptr);
(*called_) = true;
}
};
}
TEST(socks_client, unsupported_command)
{
boost::asio::io_service io_service{};
stream_type::socket client{io_service};
auto test_client = net::socks::make_connect_client(
std::move(client), net::socks::version::v4, std::bind( [] {} )
);
ASSERT_TRUE(bool(test_client));
EXPECT_TRUE(test_client->buffer().empty());
EXPECT_FALSE(test_client->set_connect_command("example.com", 8080));
EXPECT_TRUE(test_client->buffer().empty());
EXPECT_FALSE(test_client->set_resolve_command("example.com"));
EXPECT_TRUE(test_client->buffer().empty());
}
TEST(socks_client, no_command)
{
boost::asio::io_service io_service{};
stream_type::socket client{io_service};
auto test_client = net::socks::make_connect_client(
std::move(client), net::socks::version::v4a, std::bind( [] {} )
);
ASSERT_TRUE(bool(test_client));
EXPECT_FALSE(net::socks::client::send(std::move(test_client)));
}
TEST(socks_client, connect_command)
{
io_thread io{};
stream_type::socket client{io.io_service};
std::atomic<bool> called{false};
auto test_client = net::socks::make_connect_client(
std::move(client), net::socks::version::v4a, checked_client{std::addressof(called), false}
);
ASSERT_TRUE(bool(test_client));
ASSERT_TRUE(test_client->set_connect_command("example.com", 8080));
EXPECT_FALSE(test_client->buffer().empty());
ASSERT_TRUE(net::socks::client::connect_and_send(std::move(test_client), io.acceptor.local_endpoint()));
while (!io.connected)
ASSERT_FALSE(called);
const std::uint8_t expected_bytes[] = {
4, 1, 0x1f, 0x90, 0x00, 0x00, 0x00, 0x01, 0x00,
'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', 0x00
};
std::uint8_t actual_bytes[sizeof(expected_bytes)];
boost::asio::read(io.server, boost::asio::buffer(actual_bytes));
EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0);
const std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0, 0, 0, 0};
boost::asio::write(io.server, boost::asio::buffer(reply_bytes));
// yikes!
while (!called);
}
TEST(socks_client, connect_command_failed)
{
io_thread io{};
stream_type::socket client{io.io_service};
std::atomic<bool> called{false};
auto test_client = net::socks::make_connect_client(
std::move(client), net::socks::version::v4, checked_client{std::addressof(called), true}
);
ASSERT_TRUE(bool(test_client));
ASSERT_TRUE(
test_client->set_connect_command(
epee::net_utils::ipv4_network_address{boost::endian::native_to_big(std::uint32_t(5000)), 3000}
)
);
EXPECT_FALSE(test_client->buffer().empty());
ASSERT_TRUE(net::socks::client::connect_and_send(std::move(test_client), io.acceptor.local_endpoint()));
while (!io.connected)
ASSERT_FALSE(called);
const std::uint8_t expected_bytes[] = {
4, 1, 0x0b, 0xb8, 0x00, 0x00, 0x13, 0x88, 0x00
};
std::uint8_t actual_bytes[sizeof(expected_bytes)];
boost::asio::read(io.server, boost::asio::buffer(actual_bytes));
EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0);
const std::uint8_t reply_bytes[] = {0, 91, 0, 0, 0, 0, 0, 0};
boost::asio::write(io.server, boost::asio::buffer(reply_bytes));
// yikes!
while (!called);
}
TEST(socks_client, resolve_command)
{
static std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0xff, 0, 0xad, 0};
struct resolve_client : net::socks::client
{
std::atomic<unsigned> called_;
bool expected_;
resolve_client(stream_type::socket&& proxy)
: net::socks::client(std::move(proxy), net::socks::version::v4a_tor)
, called_(0)
, expected_(false)
{};
virtual void done(boost::system::error_code error, std::shared_ptr<client> self) override
{
EXPECT_EQ(this, self.get());
EXPECT_EQ(expected_, bool(error)) << "Resolve failure: " << error.message();
if (!error)
{
ASSERT_EQ(sizeof(reply_bytes), buffer().size());
EXPECT_EQ(0u, std::memcmp(buffer().data(), reply_bytes, sizeof(reply_bytes)));
}
++called_;
}
};
io_thread io{};
stream_type::socket client{io.io_service};
auto test_client = std::make_shared<resolve_client>(std::move(client));
ASSERT_TRUE(bool(test_client));
ASSERT_TRUE(test_client->set_resolve_command("example.com"));
EXPECT_FALSE(test_client->buffer().empty());
ASSERT_TRUE(net::socks::client::connect_and_send(test_client, io.acceptor.local_endpoint()));
while (!io.connected)
ASSERT_EQ(0u, test_client->called_);
const std::uint8_t expected_bytes[] = {
4, 0xf0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00,
'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', 0x00
};
std::uint8_t actual_bytes[sizeof(expected_bytes)];
boost::asio::read(io.server, boost::asio::buffer(actual_bytes));
EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0);
boost::asio::write(io.server, boost::asio::buffer(reply_bytes));
// yikes!
while (test_client->called_ == 0);
test_client->expected_ = true;
ASSERT_TRUE(test_client->set_resolve_command("example.com"));
EXPECT_FALSE(test_client->buffer().empty());
ASSERT_TRUE(net::socks::client::send(test_client));
boost::asio::read(io.server, boost::asio::buffer(actual_bytes));
EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0);
reply_bytes[1] = 91;
boost::asio::write(io.server, boost::asio::buffer(reply_bytes));
// yikes!
while (test_client->called_ == 1);
}
TEST(socks_connector, host)
{
io_thread io{};
boost::asio::steady_timer timeout{io.io_service};
timeout.expires_from_now(std::chrono::seconds{5});
std::future<boost::asio::ip::tcp::socket> sock =
net::socks::connector{io.acceptor.local_endpoint()}("example.com", "8080", timeout);
while (!io.connected)
ASSERT_NE(sock.wait_for(0s), std::future_status::ready);
const std::uint8_t expected_bytes[] = {
4, 1, 0x1f, 0x90, 0x00, 0x00, 0x00, 0x01, 0x00,
'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', 0x00
};
std::uint8_t actual_bytes[sizeof(expected_bytes)];
boost::asio::read(io.server, boost::asio::buffer(actual_bytes));
EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0);
const std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0, 0, 0, 0};
boost::asio::write(io.server, boost::asio::buffer(reply_bytes));
ASSERT_EQ(std::future_status::ready, sock.wait_for(3s));
EXPECT_TRUE(sock.get().is_open());
}
TEST(socks_connector, ipv4)
{
io_thread io{};
boost::asio::steady_timer timeout{io.io_service};
timeout.expires_from_now(std::chrono::seconds{5});
std::future<boost::asio::ip::tcp::socket> sock =
net::socks::connector{io.acceptor.local_endpoint()}("250.88.125.99", "8080", timeout);
while (!io.connected)
ASSERT_NE(sock.wait_for(0s), std::future_status::ready);
const std::uint8_t expected_bytes[] = {
4, 1, 0x1f, 0x90, 0xfa, 0x58, 0x7d, 0x63, 0x00
};
std::uint8_t actual_bytes[sizeof(expected_bytes)];
boost::asio::read(io.server, boost::asio::buffer(actual_bytes));
EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0);
const std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0, 0, 0, 0};
boost::asio::write(io.server, boost::asio::buffer(reply_bytes));
ASSERT_EQ(std::future_status::ready, sock.wait_for(3s));
EXPECT_TRUE(sock.get().is_open());
}
TEST(socks_connector, error)
{
io_thread io{};
boost::asio::steady_timer timeout{io.io_service};
timeout.expires_from_now(std::chrono::seconds{5});
std::future<boost::asio::ip::tcp::socket> sock =
net::socks::connector{io.acceptor.local_endpoint()}("250.88.125.99", "8080", timeout);
while (!io.connected)
ASSERT_NE(sock.wait_for(0s), std::future_status::ready);
const std::uint8_t expected_bytes[] = {
4, 1, 0x1f, 0x90, 0xfa, 0x58, 0x7d, 0x63, 0x00
};
std::uint8_t actual_bytes[sizeof(expected_bytes)];
boost::asio::read(io.server, boost::asio::buffer(actual_bytes));
EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0);
const std::uint8_t reply_bytes[] = {0, 91, 0, 0, 0, 0, 0, 0};
boost::asio::write(io.server, boost::asio::buffer(reply_bytes));
ASSERT_EQ(std::future_status::ready, sock.wait_for(3s));
EXPECT_THROW(sock.get().is_open(), boost::system::system_error);
}
TEST(socks_connector, timeout)
{
io_thread io{};
boost::asio::steady_timer timeout{io.io_service};
timeout.expires_from_now(std::chrono::milliseconds{10});
std::future<boost::asio::ip::tcp::socket> sock =
net::socks::connector{io.acceptor.local_endpoint()}("250.88.125.99", "8080", timeout);
ASSERT_EQ(std::future_status::ready, sock.wait_for(3s));
EXPECT_THROW(sock.get().is_open(), boost::system::system_error);
}
TEST(dandelionpp_map, traits)
{
EXPECT_TRUE(std::is_default_constructible<net::dandelionpp::connection_map>());