oxen-io
/
oxen-core
mirror of https://github.com/oxen-io/oxen-core.git
2
1
Fork 0
Code Issues Projects Releases Wiki Activity
oxen-core/.drone.jsonnet

230 lines
11 KiB
Plaintext
Raw Normal View History

Remove boost filesystem/date-time from deps
2020-10-22 23:44:11 +02:00
local default_deps_base='libsystemd-dev libboost-thread-dev libgtest-dev ' +
Remove boost chrono and regex from readme/drone
2020-06-22 05:12:08 +02:00
'libboost-serialization-dev libboost-program-options-dev libunbound-dev nettle-dev libevent-dev libminiupnpc-dev ' +
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
'libunwind8-dev libsodium-dev libssl-dev libreadline-dev libhidapi-dev libusb-1.0-0-dev libprotobuf-dev protobuf-compiler python3 ' +
Replace epee http client with curl-based client In short: epee's http client is garbage, standard violating, and unreliable. This completely removes the epee http client support and replaces it with cpr, a curl-based C++ wrapper. rpc/http_client.h wraps cpr for RPC requests specifically, but it is also usable directly. This replacement has a number of advantages: - requests are considerably more reliable. The epee http client code assumes that a connection will be kept alive forever, and returns a failure if a connection is ever closed. This results in some very annoying things: for example, preparing a transaction and then waiting a long tim before confirming it will usually result in an error communication with the daemon. This is just terribly behaviour: the right thing to do on a connection failure is to resubmit the request. - epee's http client is broken in lots of other ways: for example, it tries throwing SSL at the port to see if it is HTTPS, but this is protocol violating and just breaks (with a several second timeout) on anything that *isn't* epee http server (for example, when lokid is behind a proxying server). - even when it isn't doing the above, the client breaks in other ways: for example, there is a comment (replaced in this PR) in the Trezor PR code that forces a connection close after every request because epee's http client doesn't do proper keep-alive request handling. - it seems noticeably faster to me in practical use in this PR; both simple requests (for example, when running `lokid status`) and wallet<->daemon connections are faster, probably because of crappy code in epee. (I think this is also related to the throw-ssl-at-it junk above: the epee client always generates an ssl certificate during static initialization because it might need one at some point). - significantly reduces the amount of code we have to maintain. - removes all the epee ssl option code: curl can handle all of that just fine. - removes the epee socks proxy code; curl can handle that just fine. (And can do more: it also supports using HTTP/HTTPS proxies). - When a cli wallet connection fails we know show why it failed (which now is an error message from curl), which could have all sorts of reasons like hostname resolution failure, bad ssl certificate, etc. Previously you just got a useless generic error that tells you nothing. Other related changes in this PR: - Drops the check-for-update and download-update code. To the best of my knowledge these have never been supported in loki-core and so it didn't seem worth the trouble to convert them to use cpr for the requests. - Cleaned up node_rpc_proxy return values: there was an inconsistent mix of ways to return errors and how the returned strings were handled. Instead this cleans it up to return a pair<bool, val>, which (with C++17) can be transparently captured as: auto [success, val] = node.whatever(req); This drops the failure message string, but it was almost always set to something fairly useless (if we want to resurrect it we could easily change the first element to be a custom type with a bool operator for success, and a `.error` attribute containing some error string, but for the most part the current code wasn't doing much useful with the failure string). - changed local detection (for automatic trusted daemon determination) to just look for localhost, and to not try to resolve anything. Trusting non-public IPs does not work well (e.g. with lokinet where all .loki addresses resolve to a local IP). - ssl fingerprint option is removed; this isn't supported by curl (because it is essentially just duplicating what a custom cainfo bundle does) - --daemon-ssl-allow-chained is removed; it wasn't a useful option (if you don't want chaining, don't specify a cainfo chain). - --daemon-address is now a URL instead of just host:port. (If you omit the protocol, http:// is prepended). - --daemon-host and --daemon-port are now deprecated and produce a warning (in simplewallet) if used; the replacement is to use --daemon-address. - --daemon-ssl is deprecated; specify --daemon-address=https://whatever instead. - the above three are now hidden from --help - reordered the wallet connection options to make more logical sense.
2020-07-26 22:29:49 +02:00
'pkg-config libsqlite3-dev qttools5-dev libcurl4-openssl-dev';
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
local default_deps='g++ ' + default_deps_base; // g++ sometimes needs replacement
Small tweaks to build system - make tagged and stable branch builds not add git tag to the tar.xz filename - copy apt-get tweaks (to make it less noisy) from deb branches - do a faster shallow submodule clone - remove exclusion for removed is_hdd test - remove deb_builder (deb building drone code is in its own branches) - fix upload dir for a tagged build
2020-10-04 03:55:02 +02:00
local gtest_filter='-AddressFromURL.Failure:DNSResolver.DNSSEC*';
Disable DNSSEC, is_hdd tests in CI The DNSSEC tests fail on me quite often; the is_hdd tests *always* fails inside a container.
2020-06-10 03:33:15 +02:00
Clean up submodule commands
2020-10-28 17:46:04 +01:00
local submodules_commands = ['git fetch --tags', 'git submodule update --init --recursive --depth=1'];
Drone: Add tags/submodules as separate step This makes it a little clearer what failed when something fails. Ideally we'd also be able to distinguish between build and test, but currently that's not practical to do with drone: between each stage of the build the docker image gets removed and replaced with the next stage's image, which means we'd have to build up a new docker image with dependencies before running the tests. (Being able to have multiple stages in one docker image is a drone upstream wishlist item).
2020-06-12 18:56:27 +02:00
local submodules = {
name: 'submodules',
image: 'drone/git',
Clean up submodule commands
2020-10-28 17:46:04 +01:00
commands: submodules_commands
Drone: Add tags/submodules as separate step This makes it a little clearer what failed when something fails. Ideally we'd also be able to distinguish between build and test, but currently that's not practical to do with drone: between each stage of the build the docker image gets removed and replaced with the next stage's image, which means we'd have to build up a new docker image with dependencies before running the tests. (Being able to have multiple stages in one docker image is a drone upstream wishlist item).
2020-06-12 18:56:27 +02:00
};
Small tweaks to build system - make tagged and stable branch builds not add git tag to the tar.xz filename - copy apt-get tweaks (to make it less noisy) from deb branches - do a faster shallow submodule clone - remove exclusion for removed is_hdd test - remove deb_builder (deb building drone code is in its own branches) - fix upload dir for a tagged build
2020-10-04 03:55:02 +02:00
local apt_get_quiet = 'apt-get -o=Dpkg::Use-Pty=0 -q';
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
// Regular build on a debian-like system:
local debian_pipeline(name, image,
arch='amd64',
deps=default_deps,
build_type='Release',
lto=false,
Disable -Werror for now Many of the warnings being generated (which *are* legitimate problems) are better addressed in the C++17 PR.
2020-06-10 00:15:59 +02:00
werror=false, // FIXME
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
build_tests=true,
Drone: add static builds for mac/linux/windows Adds static drone builds for linux (built on bionic), mac, and Windows (built with mingw32). The builds get uploaded to https://builds.lokinet.dev The linux and mac builds use LTO (which takes longer, but significantly reduces binary size). The mingw32 build can probably also get there, but currently fails with LTO when unbound tries linking against openssl (it probably just needs a small patch to add magic -lwsock2 dep in the right place in unbound). The Mac binaries are built using a 10.13 deployment target. (I'm not 100% sure that this is sufficient -- it's possible we might have to also push the magic mac deployment flag to the built dependencies).
2020-06-15 04:26:02 +02:00
test_lokid=true, # Simple lokid offline startup test
run_tests=false, # Runs full test suite
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
cmake_extra='',
extra_cmds=[],
Lower parallel jobs to 6
2020-06-15 19:51:41 +02:00
jobs=6,
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
allow_fail=false) = {
kind: 'pipeline',
type: 'docker',
name: name,
platform: { arch: arch },
Small tweaks to build system - make tagged and stable branch builds not add git tag to the tar.xz filename - copy apt-get tweaks (to make it less noisy) from deb branches - do a faster shallow submodule clone - remove exclusion for removed is_hdd test - remove deb_builder (deb building drone code is in its own branches) - fix upload dir for a tagged build
2020-10-04 03:55:02 +02:00
steps: [
submodules,
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
{
name: 'build',
image: image,
[if allow_fail then "failure"]: "ignore",
Disable DNSSEC, is_hdd tests in CI The DNSSEC tests fail on me quite often; the is_hdd tests *always* fails inside a container.
2020-06-10 03:33:15 +02:00
environment: { SSH_KEY: { from_secret: "SSH_KEY" }, GTEST_FILTER: gtest_filter },
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
commands: [
Print CI machine at beginning of build
2020-11-12 21:51:43 +01:00
'echo "Building on ${DRONE_STAGE_MACHINE}"',
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
'echo "man-db man-db/auto-update boolean false" | debconf-set-selections',
Small tweaks to build system - make tagged and stable branch builds not add git tag to the tar.xz filename - copy apt-get tweaks (to make it less noisy) from deb branches - do a faster shallow submodule clone - remove exclusion for removed is_hdd test - remove deb_builder (deb building drone code is in its own branches) - fix upload dir for a tagged build
2020-10-04 03:55:02 +02:00
apt_get_quiet + ' update',
apt_get_quiet + ' install -y eatmydata',
'eatmydata ' + apt_get_quiet + ' dist-upgrade -y',
Run lokid test under gdb Scripts for running under gdb + dumping a stack trace on problems from lokinet.
2020-11-06 16:57:00 +01:00
'eatmydata ' + apt_get_quiet + ' install -y --no-install-recommends cmake git ca-certificates ninja-build ccache '
+ deps + (if test_lokid then ' gdb' else ''),
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
'mkdir build',
'cd build',
'cmake .. -G Ninja -DCMAKE_CXX_FLAGS=-fdiagnostics-color=always -DCMAKE_BUILD_TYPE='+build_type+' ' +
Add local static source mirror I've uploaded the various deps to https://builds.lokinet.dev/deps so as not to hit the upstream mirrors so much (also reduces the reliance on github, which is sometimes flakey). This is prefixed onto the existing URLs so that if it isn't found we fall back to the upstream URL.
2020-06-12 17:20:26 +02:00
'-DLOCAL_MIRROR=https://builds.lokinet.dev/deps -DUSE_LTO=' + (if lto then 'ON ' else 'OFF ') +
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
(if werror then '-DWARNINGS_AS_ERRORS=ON ' else '') +
(if build_tests || run_tests then '-DBUILD_TESTS=ON ' else '') +
Build less stuff on ARM ci Turn off the tests, and copy the -j1/-j2 split from the travis config.
2020-06-10 03:03:34 +02:00
cmake_extra
] + (if arch == 'arm64' && jobs > 1 then
// The wallet code is too bloated to be compiled at -j2 with only 4GB ram, so do
// the huge bloated jobs at -j1 and the rest at -j2
['ninja -j1 rpc wallet -v', 'ninja -j2 daemon device_trezor -v', 'ninja -j1 wallet_rpc_server -v', 'ninja -j2 -v']
else
['ninja -j' + jobs + ' -v']
Drone: add static builds for mac/linux/windows Adds static drone builds for linux (built on bionic), mac, and Windows (built with mingw32). The builds get uploaded to https://builds.lokinet.dev The linux and mac builds use LTO (which takes longer, but significantly reduces binary size). The mingw32 build can probably also get there, but currently fails with LTO when unbound tries linking against openssl (it probably just needs a small patch to add magic -lwsock2 dep in the right place in unbound). The Mac binaries are built using a 10.13 deployment target. (I'm not 100% sure that this is sufficient -- it's possible we might have to also push the magic mac deployment flag to the built dependencies).
2020-06-15 04:26:02 +02:00
) + (
if test_lokid then [
Run lokid test under gdb Scripts for running under gdb + dumping a stack trace on problems from lokinet.
2020-11-06 16:57:00 +01:00
'(sleep 3; echo "status\ndiff\nexit") | TERM=xterm ../utils/build_scripts/drone-gdb.sh ./bin/lokid --offline --data-dir=startuptest'
Drone: add static builds for mac/linux/windows Adds static drone builds for linux (built on bionic), mac, and Windows (built with mingw32). The builds get uploaded to https://builds.lokinet.dev The linux and mac builds use LTO (which takes longer, but significantly reduces binary size). The mingw32 build can probably also get there, but currently fails with LTO when unbound tries linking against openssl (it probably just needs a small patch to add magic -lwsock2 dep in the right place in unbound). The Mac binaries are built using a 10.13 deployment target. (I'm not 100% sure that this is sufficient -- it's possible we might have to also push the magic mac deployment flag to the built dependencies).
2020-06-15 04:26:02 +02:00
] else []
) + (
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
if run_tests then [
'mkdir -v -p $$HOME/.loki',
'GTEST_COLOR=1 ctest --output-on-failure -j'+jobs
] else []
) + extra_cmds,
}
],
Add android build to drone steps
2020-10-24 20:35:08 +02:00
};
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
// Macos build
local mac_builder(name,
build_type='Release',
lto=false,
Disable -Werror for now Many of the warnings being generated (which *are* legitimate problems) are better addressed in the C++17 PR.
2020-06-10 00:15:59 +02:00
werror=false, // FIXME
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
build_tests=true,
run_tests=false,
cmake_extra='',
extra_cmds=[],
jobs=6,
allow_fail=false) = {
kind: 'pipeline',
type: 'exec',
name: name,
platform: { os: 'darwin', arch: 'amd64' },
steps: [
Clean up submodule commands
2020-10-28 17:46:04 +01:00
{ name: 'submodules', commands: submodules_commands },
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
{
name: 'build',
Disable DNSSEC, is_hdd tests in CI The DNSSEC tests fail on me quite often; the is_hdd tests *always* fails inside a container.
2020-06-10 03:33:15 +02:00
environment: { SSH_KEY: { from_secret: "SSH_KEY" }, GTEST_FILTER: gtest_filter },
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
commands: [
// If you don't do this then the C compiler doesn't have an include path containing
// basic system headers. WTF apple:
'export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"',
'mkdir build',
'cd build',
'cmake .. -G Ninja -DCMAKE_CXX_FLAGS=-fcolor-diagnostics -DCMAKE_BUILD_TYPE='+build_type+' ' +
Add local static source mirror I've uploaded the various deps to https://builds.lokinet.dev/deps so as not to hit the upstream mirrors so much (also reduces the reliance on github, which is sometimes flakey). This is prefixed onto the existing URLs so that if it isn't found we fall back to the upstream URL.
2020-06-12 17:20:26 +02:00
'-DLOCAL_MIRROR=https://builds.lokinet.dev/deps -DUSE_LTO=' + (if lto then 'ON ' else 'OFF ') +
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
(if werror then '-DWARNINGS_AS_ERRORS=ON ' else '') +
(if build_tests || run_tests then '-DBUILD_TESTS=ON ' else '') +
cmake_extra,
'ninja -j' + jobs + ' -v'
] + (
if run_tests then [
'mkdir -v -p $$HOME/.loki',
'GTEST_COLOR=1 ctest --output-on-failure -j'+jobs
] else []
) + extra_cmds,
}
]
};
Drone: add static builds for mac/linux/windows Adds static drone builds for linux (built on bionic), mac, and Windows (built with mingw32). The builds get uploaded to https://builds.lokinet.dev The linux and mac builds use LTO (which takes longer, but significantly reduces binary size). The mingw32 build can probably also get there, but currently fails with LTO when unbound tries linking against openssl (it probably just needs a small patch to add magic -lwsock2 dep in the right place in unbound). The Mac binaries are built using a 10.13 deployment target. (I'm not 100% sure that this is sufficient -- it's possible we might have to also push the magic mac deployment flag to the built dependencies).
2020-06-15 04:26:02 +02:00
local static_check_and_upload = [
'../utils/build_scripts/drone-check-static-libs.sh',
'ninja strip_binaries',
'ninja create_tarxz',
'../utils/build_scripts/drone-static-upload.sh'
];
local static_build_deps='autoconf automake make qttools5-dev file libtool gperf pkg-config patch openssh-client';
Fix compiler over-optimization
2020-10-27 03:16:49 +01:00
local android_build_steps(android_abi, android_platform=21, jobs=6, cmake_extra='') = [
Add android build to drone steps
2020-10-24 20:35:08 +02:00
'mkdir build-' + android_abi,
'cd build-' + android_abi,
'cmake .. -DCMAKE_CXX_FLAGS=-fdiagnostics-color=always -DCMAKE_C_FLAGS=-fdiagnostics-color=always ' +
Package merged wallet_api rather than plain wallet_api
2020-10-26 20:51:38 +01:00
'-DCMAKE_BUILD_TYPE=Release ' +
Add android build to drone steps
2020-10-24 20:35:08 +02:00
'-DCMAKE_TOOLCHAIN_FILE=/usr/lib/android-sdk/ndk-bundle/build/cmake/android.toolchain.cmake ' +
'-DANDROID_PLATFORM=' + android_platform + ' -DANDROID_ABI=' + android_abi + ' ' +
Only compile cn_monero_hash on Android This causes a ton of fake positive malware detections if it is build into the binary, so just disable it on everything except Android.
2020-11-18 03:05:31 +01:00
'-DMONERO_SLOW_HASH=ON ' +
Fix compiler over-optimization
2020-10-27 03:16:49 +01:00
'-DBUILD_STATIC_DEPS=ON -DSTATIC=ON -G Ninja ' + cmake_extra,
Package merged wallet_api rather than plain wallet_api
2020-10-26 20:51:38 +01:00
'ninja -j' + jobs + ' -v wallet_merged',
Add android build to drone steps
2020-10-24 20:35:08 +02:00
'cd ..',
];
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
[
// Various debian builds
Use sid more; crank clang version to 11 Switch several builds to sid, and updates the linux clang build to 11.
2020-10-23 23:09:55 +02:00
debian_pipeline("Debian (w/ tests) (amd64)", "debian:sid", lto=true, run_tests=true),
debian_pipeline("Debian Debug (amd64)", "debian:sid", build_type='Debug'),
debian_pipeline("Debian clang-11 (amd64)", "debian:sid", deps='clang-11 '+default_deps_base,
cmake_extra='-DCMAKE_C_COMPILER=clang-11 -DCMAKE_CXX_COMPILER=clang++-11 ', lto=true),
Switch debian builds to testing Stuff breaks more often on sid (like qt5 packages right now) and testing is not far behind.
2020-06-24 21:45:51 +02:00
debian_pipeline("Debian gcc-10 (amd64)", "debian:testing", deps='g++-10 '+default_deps_base,
Make debug_utilities available to build even without -DBUILD_DEBUG_UTILS (#1183) This uses EXCLUDE_FROM_ALL when BUILD_DEBUG_UTILS is not on so that they don't get built but they *can* be built with a `make cn_deserialize` (or whatever target you want) from a build dir. Also adds them to a couple drone builds to make sure they build.
2020-06-25 05:10:09 +02:00
cmake_extra='-DCMAKE_C_COMPILER=gcc-10 -DCMAKE_CXX_COMPILER=g++-10 -DBUILD_DEBUG_UTILS=ON'),
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
debian_pipeline("Debian buster (i386)", "i386/debian:buster", cmake_extra='-DDOWNLOAD_SODIUM=ON -DARCH_ID=i386'),
debian_pipeline("Ubuntu focal (amd64)", "ubuntu:focal"),
// ARM builds (ARM64 and armhf)
Use sid more; crank clang version to 11 Switch several builds to sid, and updates the linux clang build to 11.
2020-10-23 23:09:55 +02:00
debian_pipeline("Debian (ARM64)", "debian:sid", arch="arm64", build_tests=false),
Build less stuff on ARM ci Turn off the tests, and copy the -j1/-j2 split from the travis config.
2020-06-10 03:03:34 +02:00
debian_pipeline("Debian buster (armhf)", "arm32v7/debian:buster", arch="arm64", build_tests=false, cmake_extra='-DDOWNLOAD_SODIUM=ON -DARCH_ID=armhf'),
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
// Static build (on bionic) which gets uploaded to builds.lokinet.dev:
Drone: add static builds for mac/linux/windows Adds static drone builds for linux (built on bionic), mac, and Windows (built with mingw32). The builds get uploaded to https://builds.lokinet.dev The linux and mac builds use LTO (which takes longer, but significantly reduces binary size). The mingw32 build can probably also get there, but currently fails with LTO when unbound tries linking against openssl (it probably just needs a small patch to add magic -lwsock2 dep in the right place in unbound). The Mac binaries are built using a 10.13 deployment target. (I'm not 100% sure that this is sufficient -- it's possible we might have to also push the magic mac deployment flag to the built dependencies).
2020-06-15 04:26:02 +02:00
debian_pipeline("Static (bionic amd64)", "ubuntu:bionic", deps='g++-8 '+static_build_deps,
Make static build compile for deprecit hardware (#1287)
2020-09-29 04:16:16 +02:00
cmake_extra='-DBUILD_STATIC_DEPS=ON -DCMAKE_C_COMPILER=gcc-8 -DCMAKE_CXX_COMPILER=g++-8 -DARCH=x86-64',
Drone: add static builds for mac/linux/windows Adds static drone builds for linux (built on bionic), mac, and Windows (built with mingw32). The builds get uploaded to https://builds.lokinet.dev The linux and mac builds use LTO (which takes longer, but significantly reduces binary size). The mingw32 build can probably also get there, but currently fails with LTO when unbound tries linking against openssl (it probably just needs a small patch to add magic -lwsock2 dep in the right place in unbound). The Mac binaries are built using a 10.13 deployment target. (I'm not 100% sure that this is sufficient -- it's possible we might have to also push the magic mac deployment flag to the built dependencies).
2020-06-15 04:26:02 +02:00
build_tests=false, lto=true, extra_cmds=static_check_and_upload),
// Static mingw build (on focal) which gets uploaded to builds.lokinet.dev:
debian_pipeline("Static (win64)", "ubuntu:focal", deps='g++ g++-mingw-w64-x86-64 '+static_build_deps,
Windows build: do a generic build The default uses -march=cpu which turns on avx2 and fma and other CPU features that crash on older CPUs.
2020-10-12 19:25:44 +02:00
cmake_extra='-DCMAKE_TOOLCHAIN_FILE=../cmake/64-bit-toolchain.cmake -DBUILD_STATIC_DEPS=ON -DARCH=x86-64',
Drone: add static builds for mac/linux/windows Adds static drone builds for linux (built on bionic), mac, and Windows (built with mingw32). The builds get uploaded to https://builds.lokinet.dev The linux and mac builds use LTO (which takes longer, but significantly reduces binary size). The mingw32 build can probably also get there, but currently fails with LTO when unbound tries linking against openssl (it probably just needs a small patch to add magic -lwsock2 dep in the right place in unbound). The Mac binaries are built using a 10.13 deployment target. (I'm not 100% sure that this is sufficient -- it's possible we might have to also push the magic mac deployment flag to the built dependencies).
2020-06-15 04:26:02 +02:00
build_tests=false, lto=false, test_lokid=false, extra_cmds=[
'ninja strip_binaries', 'ninja create_zip', '../utils/build_scripts/drone-static-upload.sh']),
Small tweaks to build system - make tagged and stable branch builds not add git tag to the tar.xz filename - copy apt-get tweaks (to make it less noisy) from deb branches - do a faster shallow submodule clone - remove exclusion for removed is_hdd test - remove deb_builder (deb building drone code is in its own branches) - fix upload dir for a tagged build
2020-10-04 03:55:02 +02:00
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
// Macos builds:
Target macos 10.12 When targetting macos <10.14 macos won't allow use of anything from C++17 that throws, such as: - std::get on a variant - std::visit - std::optional::value() - std::any_cast This avoids all of these. For std::get, we either replace with std::get_if (where appropriate), or else use a `var::get` implementation of std::get added to lokimq (also updated here). (This `var` namespace is just an `std` alias everywhere *except* old target macos). For std::visit, likewise lokimq adds an var::visit implementation for old macos that we use. std::optional::value() uses weren't useful anyway as everywhere it calls them we've already checked that the option has a value, in which case we can use `*opt` (which doesn't check for contents and throw). std::any just has to be avoided as far as I can tell, but the one place we used it is only ever a block, so I just replaced it with a `const block*`.
2020-10-16 00:44:36 +02:00
mac_builder('macOS (Static)', cmake_extra='-DBUILD_STATIC_DEPS=ON -DARCH=core2 -DARCH_ID=amd64',
Fix boost flags, enable LTO
2020-10-15 04:04:53 +02:00
build_tests=false, extra_cmds=static_check_and_upload, lto=true),
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
mac_builder('macOS (Release)', run_tests=true),
Make debug_utilities available to build even without -DBUILD_DEBUG_UTILS (#1183) This uses EXCLUDE_FROM_ALL when BUILD_DEBUG_UTILS is not on so that they don't get built but they *can* be built with a `make cn_deserialize` (or whatever target you want) from a build dir. Also adds them to a couple drone builds to make sure they build.
2020-06-25 05:10:09 +02:00
mac_builder('macOS (Debug)', build_type='Debug', cmake_extra='-DBUILD_DEBUG_UTILS=ON'),
Add android build to drone steps
2020-10-24 20:35:08 +02:00
// Android builds; we do them all in one image because the android NDK is huge
Retitle build pipeline
2020-10-26 19:49:43 +01:00
{ name: 'Android wallet_api', kind: 'pipeline', type: 'docker', platform: { arch: 'amd64' },
Add android build to drone steps
2020-10-24 20:35:08 +02:00
steps: [submodules, {
name: 'build',
image: 'debian:sid',
environment: { SSH_KEY: { from_secret: "SSH_KEY" } },
commands: [
'echo "man-db man-db/auto-update boolean false" | debconf-set-selections',
'echo deb http://deb.debian.org/debian sid contrib >/etc/apt/sources.list.d/sid-contrib.list',
apt_get_quiet + ' update',
apt_get_quiet + ' install -y eatmydata',
'eatmydata ' + apt_get_quiet + ' dist-upgrade -y',
// Keep cached copies of the android NDK around because it is huge:
'if [ -d /cache ]; then if ! [ -d /cache/google-android-ndk-installer ]; then mkdir /cache/google-android-ndk-installer; fi; ln -s /cache/google-android-ndk-installer /var/cache/; fi',
'eatmydata ' + apt_get_quiet + ' install -y --no-install-recommends cmake g++ git ninja-build ccache tar xz-utils google-android-ndk-installer ' + static_build_deps,
]
Fix compiler over-optimization
2020-10-27 03:16:49 +01:00
+ android_build_steps('armeabi-v7a', cmake_extra='-DARCH=armv7-a -DARCH_ID=arm32')
+ android_build_steps('arm64-v8a', cmake_extra='-DARCH=armv8-a -DARCH_ID=arm64')
+ android_build_steps('x86_64', cmake_extra='-DARCH="x86-64 -msse4.2 -mpopcnt" -DARCH_ID=x86-64')
+ android_build_steps('x86', cmake_extra='-DARCH="i686 -mssse3 -mfpmath=sse" -DARCH_ID=i386')
Add android build to drone steps
2020-10-24 20:35:08 +02:00
+ [
Add ios build & upload Also renames the android upload script to drone-android-blahblah to be more consistent.
2020-10-30 17:55:25 +01:00
'./utils/build_scripts/drone-android-static-upload.sh armeabi-v7a arm64-v8a x86_64 x86'
Add android build to drone steps
2020-10-24 20:35:08 +02:00
]
}
]
Add ios build & upload Also renames the android upload script to drone-android-blahblah to be more consistent.
2020-10-30 17:55:25 +01:00
},
// iOS build
{ name: 'iOS wallet_api', kind: 'pipeline', type: 'exec', platform: { os: 'darwin', arch: 'amd64' },
steps: [{
name: 'build',
environment: { SSH_KEY: { from_secret: "SSH_KEY" } },
commands: submodules_commands + [
Merge arm/sim libs and collect headers in ios deps package
2020-11-04 01:38:03 +01:00
'mkdir -p build/{arm64,sim64}',
'cd build/arm64',
'cmake ../.. -G Ninja ' +
'-DCMAKE_TOOLCHAIN_FILE=../../cmake/ios.toolchain.cmake -DPLATFORM=OS -DDEPLOYMENT_TARGET=11 -DENABLE_VISIBILITY=ON -DENABLE_BITCODE=OFF ' +
Add ios build & upload Also renames the android upload script to drone-android-blahblah to be more consistent.
2020-10-30 17:55:25 +01:00
'-DSTATIC=ON -DBUILD_STATIC_DEPS=ON -DUSE_LTO=OFF -DCMAKE_BUILD_TYPE=Release ' +
Merge arm/sim libs and collect headers in ios deps package
2020-11-04 01:38:03 +01:00
'-DRANDOMX_ENABLE_JIT=OFF -DCMAKE_CXX_FLAGS=-fcolor-diagnostics',
Add ios build & upload Also renames the android upload script to drone-android-blahblah to be more consistent.
2020-10-30 17:55:25 +01:00
'ninja -j6 -v wallet_merged',
Merge arm/sim libs and collect headers in ios deps package
2020-11-04 01:38:03 +01:00
'cd ../sim64',
'cmake ../.. -G Ninja ' +
'-DCMAKE_TOOLCHAIN_FILE=../../cmake/ios.toolchain.cmake -DPLATFORM=SIMULATOR64 -DDEPLOYMENT_TARGET=11 -DENABLE_VISIBILITY=ON -DENABLE_BITCODE=OFF ' +
Also build simulator library
2020-11-04 00:01:15 +01:00
'-DSTATIC=ON -DBUILD_STATIC_DEPS=ON -DUSE_LTO=OFF -DCMAKE_BUILD_TYPE=Release ' +
Merge arm/sim libs and collect headers in ios deps package
2020-11-04 01:38:03 +01:00
'-DRANDOMX_ENABLE_JIT=OFF -DCMAKE_CXX_FLAGS=-fcolor-diagnostics',
Also build simulator library
2020-11-04 00:01:15 +01:00
'ninja -j6 -v wallet_merged',
Merge arm/sim libs and collect headers in ios deps package
2020-11-04 01:38:03 +01:00
'cd ../..',
Add ios build & upload Also renames the android upload script to drone-android-blahblah to be more consistent.
2020-10-30 17:55:25 +01:00
'./utils/build_scripts/drone-ios-static-upload.sh'
]
}]
},
Add drone CI config Adds various builds and does a basic offline startup test on all, plus runs the full test suite on a debian and a mac build. This has the ability to do an all-static build and to build debs, and upload both to builds.lokinet.dev, but it is not yet enabled and will likely take some additional work.
2020-06-09 15:58:46 +02:00
]