v1.7.1
What's Changed
Remove stray comment and added missing header
Initialize help hook before actually parsing the command line
remove unnecessary generated include directory
Fixed build issues on window
Explicitly cast int literals to int8_t in tests to silence implicit-conversion warnings
Ensure we don't need benchmark installed to pass c++ feature checks
Avoid deprecation warning in NVHPC
run ClearRegisteredBenchmarks at exit
FIx typo in benchmark.h
add debug option for enabling more output for failed cxxfeaturechecks
Fix link error when use as static library on windows.
use CMAKE_INSTALL_FULL in pkg-config file (fixes nix package)
Clean up test documentation formatting
Enable aarch64 Linux wheel builds, use cibuildwheel action directly
Eliminate usage of deprecated API in sysinfo.cc
Stop using pandas.Timedelta
Discuss sources of variance in the user guide
Bump pybind11 version to enable Python 3.11 wheel builds
bump cmake dep and docs
Support for QuRT OS (Hexagon RTOS)
Fix Solaris compilation
Do not depend on unversioned python binary
Partially revert "Do not depend on unversioned python binary"
Add information for supported Python versions to setup.py
[bazel] Build libpfm as a dependency to allow collection of perf counters
v1.15.0
Features
- Disable colorized output in case of TERM=dumb or NO_COLOR=1, see #542 and
#555 (@nabijaczleweli)
- Add new (experimental) --min-benchmarking-time <secs> option, see #527
(@sharkdp)
Bugfixes
- Fix user and kernel times on Windows, see #368 and #538 (@clemenswasser)
Other
- Improve --help texts of --export-* options, see #506 and #522
(@Engineer-of-Efficiency)
Small release to replace broken v1.6.2 release
What's Changed
Stop generating the export header and just check it in by @dominichamon in #1435
use target_compile_definitions by @dominichamon in #1440
simplified code by @maochongxin in #1439
New Contributors
@maochongxin made their first contribution in #1439
v1.6.2
Add docs for ThreadRange.
Add docs on Memory profiling
Suppress GoogleTest warnings on windows (MSVC) too.
Expand documentation for unpacking arbitrary arguments.
Refine docs on changing cpufreq governor
Refine the User Guide CPU Frequency Scaling section
Fix some errors in Custom Statistics document demo code.
Cache PerfCounters instance in PerfCountersMeasurement
Fix cross compilation for macOS ARM builds in cibuildwheel
bump numby, as per dependabot
Use Win32 API only for Win32 apps
Add mutex when reading counters_
Avoid potential truncation issues for the integral type parameterized tests.
Expose default display reporter creation in public API
explicitly export public symbols
Check for macro existence before using
simplify reference to internal path
Introduce the possibility to customize the help printer function
move bzl file out of tools
resolve case sensitivity issues with WORKSPACE and workspace
Make generate_export_header.bzl work for Windows.
@platforms is magical; remove it from WORKSPACE.
restore BENCHMARK_MAIN()
Allow setting the default time unit globally
Add long description and content type for proper PyPI presentation
Add SetBenchmarkFilter() to set --benchmark_filter flag value in user code
Appended additional BSD 3-Clause to LICENSE
Add PyPI upload job to wheel building workflow
Fix float comparaison and add float comparison warning
Update LICENSE file to clearly state which file needs BSD 3
Add BENCHMARK_STATIC_DEFINE to the Python bindings' cc_binary local…
Remove conditional trigger from PyPI upload job
Change artifact download name to dist to match upload name
Build //:benchmark as a static library only.
Fix Bazel build breakage caused by commit 6a894bd.
[nfc] Reformat doc-string in generate_export_header
Updates for inclusive language
getting sysinfo in line with Google style
Small optimization to counter map management
Shut down Bazel gracefully and revert wheel build strategy to job matrix
Fix wheel job name for PyPI uploads
Filter out benchmarks that start with "DISABLED_"
Add benchmark labels to the output of the comparison tool
Enable -Wconversion
Add installation and build instructions for Python bindings
fix some typos
Add option to get the verbosity provided by commandline flag -v
Add support to get clock for new architecture CSKY
Introduce warmup phase to BenchmarkRunner
Report large numbers in scientific notation in console reporter
add multiple OSes to bazel workflow
Add possibility to ask for libbenchmark version number
Fix DoNotOptimize() GCC copy overhead
Clarify that the cpu frequency is not used for benchmark timings.
Revert "Add possibility to ask for libbenchmark version number
Remove redundant formatting tags
Fix DoNotOptimize() GCC compile error with some types
Expose default help printer function
fix sanitizer builds by using clang 13
Suppress nvcc offsetof warning
Expose google_benchmark.State for python bindings.
A simple and light-weight API for memory profiling of R expressions.
The profiling is built on top of R's built-in memory profiler
('utils::Rprofmem()'), which records every memory allocation done by R
(also native code).
Revision 3.432
Adding NetBSD 64 bit.
Revision 3.434
Adding code from Alexey Skidanov. Pthread burst I/O.
Revision 3.434
Convert calls from atoi() into atoll() for those things that are actually long longs.
Revision 3.436
Code cleanup for casting consistency
Revision 3.442
Code cleanup for casting consistency and *&(^&*( compiler warnings !!!
Revision 3.443
Code cleanup for all of the un-needed void * maddness.
Revision 3.444
Cleanup 64 bit compiles.
Revision 3.445
Some BSD changes to avoid compiler complaints.
Revision 3.446
Some BSD changes to avoid compiler complaints.
Revision 3.447
Make sure we don't have missing prototypes.
Revision 3.448
Add support for building targets with NO_SIGNAL, NO_FORK, NO_SOCKET
Revision 3.449
Add support for building targets with NO_SIGNAL, NO_FORK, NO_SOCKET
Revision 3.450
Fix missing typedef in FreeBSD.
Revision 3.451
Fix typeoh in spelling of DragonFly.
Revision 3.452
Make the sighandler_t change effective for all BSD derivatives.
Revision 3.454
Make the my_sig_t change effective for all BSD derivatives.
Revision 3.455
Unify the usage of __FreeBSD__
Revision 3.456
Build on BSD 10.3 without warnings. Clang compilers.
Revision 3.457
Build on BSD 10.3 without warnings. Clang compilers.
Revision 3.458
Make more friendly to build on Solaris.
Revision 3.459
Make more friendly to build on Solaris.
Revision 3.460
Make more friendly to build on Solaris.
Revision 3.461
Make more friendly to build on Solaris.
Revision 3.462
Make more friendly to build on Solaris.
Revision 3.463
Make more friendly to build on Solaris.
Revision 3.464
Make more friendly to build on Solaris.
Revision 3.465
Make more friendly to build on Solaris.
Revision 3.466
Fix error messages to use the correct file name from dummyfile[xx] instead of
"filename" in the thread tests.
Revision 3.467
Fix compiler warning from Clang on MacOS for mythread_create(). Prototype
didn't match the formal function definition.
Revision 3.469
Switch to IOZ_macosx everywhere for compatibility with Mountain Lion.
Revision 3.470
Increase MAXSTREAMS to 1024. Systems have more power today than 20+ years ago.
Revision 3.471
Make -+W -+C -+y use a flag instead of its option value to indiate its usage.
Revision 3.472
Adding -+M (special dedupe layout)
Adding -+a (set compression in special layout)
Adding -+Q (set dedupe granule size in special layout)
Revision 3.473
no change.
Revision 3.474
Cleanup for Cygwin64 build.
Revision 3.475
Cleanup for Cygwin64 build warnings.
Revision 3.476
Cleanup compiler warnings.
Revision 3.477
Bug fix code in special_gen_new_buf..
Revision 3.478
Bug fix code in special_gen_new_buf.. More copies of the other bug.
Revision 3.479
Bug fix code in special_gen_new_buf.. needed (*op ==0) in one more line.
Revision 3.480
Add -+R option to specify the file names are in a file.
Revision 3.481
Adding new contributor Sudhir Kumar
Revision 3.482
Fix help menu for -+R option. It does not also use -F
Bug fix for case of -H where gcc seems to think the size of long long
is != size of off64_t
Bug fix for prototype mismatch in async_write_no_copy.
Revision 3.483
Fix nesting of #ifdefs so that Windows lack of support for O_DIRECT presents
splash indication.
Enable O_DIRECT for Windows builds.
Revision 3.485
Improvements for building under Android
Revision 3.486
Fix compiler warning on Android.
Revision 3.488
Improve the Yates/Knuth method for shuffle.
Revision 3.489
Fix gcc-10 warning about no-common.
Revision 3.490
Fix compiler warning when HAVE_ANSIC is not defined
Revision 3.491
Fix CFLAGS issue from Fedora maintainer.
Revision 3.492
Improve the -+M option's compression pattern so that the achieved
compressibility is closer to the value specifed in the -+a option.
Revision 3.493
Move memorey allocaiton of main buffer to after the getopt, else
the -S can cause issues later.
-Add a new --output={null,pipe,inherit,<FILE>} option to control
where the output of the benchmarked program is redirected (if at all),
see #377 and #509 (@tavianator, originally suggested by @BurntSushi)
-Add Emacs org-mode as a new export format, see #491 (@ppaulweber)
-Relax glibc requirements, see #508 and #512 (@sharkdp)
Features:
-Added a new --shell=none/-N option to disable the intermediate shell for
executing the benchmarked commands. Hyperfine normally measures and subtracts
the shell spawning time, but the intermediate shell always introduces a
certain level of measurement noise. Using --shell=none/-N allows users to
benchmark very fast commands (with a runtime on the order of a few
milliseconds). See #336, #429, and #487 (@cipriancraciun and @sharkdp)
-Added --setup/-s option that can be used to run make all or similar.
It runs once per set of tests, like --cleanup/-c (@avar)
-Added new plot_progression.py script to debug background interference effects.
Changes:
-Breaking change: the -s short option for --style is now used for the new
--setup option.
-The environment offset randomization is now also available on Windows, see #484
Other:
-Improved documentation and test coverage, cleaned up code base for future
improvements.
Fixes
Remove unused parameter from lambda. by @dominichamon in #1223
Optimized docs installation by @xvitaly in #1225
Fix mention of --benchmarks in comment by @oontvoo in #1229
cmake: eliminate redundant target_include_directories by @sergiud in #1242
Cmake: options for controlling werror, disable werror for PGI compilers by @PhilipDeegan in #1246
Fix -Wdeprecated-declarations warning triggered by clang-cl. by @bc-lee in #1245
cmake: make package config relocatable by @sergiud in #1244
cmake: allow to use package config from build directory by @sergiud in #1240
Fix -Wdeprecated-declarations warning once more. by @bc-lee in #1256
Fix un-initted error in test and fix change the API previously proposed to use std::string instead of raw char* by @oontvoo in #1266
[cleanup] Change == "" to .empty() on string to avoid clang-tidy warnings by @oontvoo in #1271
Fix errorWshorten-64-to-32 with clang 12.0 by @bensuperpc in #1273
Fix error with Fix Werror=old-style-cast by @bensuperpc in #1272
Fixed typo in doc: s/marcro/macro by @oontvoo in #1274
Fix warning with MacOS by @bensuperpc in #1276
clang-format Google on {src/,include/} by @dominichamon in #1280
format tests with clang-format by @dominichamon in #1282
check clang format on pull requests and merges by @dominichamon in #1281
Fix dependency typo and unpin cibuildwheel version in wheel building … by @nicholasjng in #1263
disable lint check where we know it'd fail by @oontvoo in #1286
Disable clang-tidy (unused-using-decls) by @oontvoo in #1287
Add clang-tidy check by @dominc8 in #1290
Fix broken link to Setup/Teardown section by @Krzmbrzl in #1291
Update user_guide.md: thread_index should be thread_index() by @ShawnZhong in #1296
clang-tidy: readability-redundant and performance by @dominc8 in #1298
update googletest to latest release tag 1.11.0 by @dominichamon in #1301
Avoid errors due to "default label in switch which covers all enumeration values" in Windows codepath by @mstorsjo in #1302
Fix -DBENCHMARK_ENABLE_INSTALL=OFF build (Fixes#1275) by @LebedevRI in #1305
Address c4267 warning on MSVC by @staffantj in #1315
Destructor not returning is expected in some cases by @staffantj in #1316
Features
Added support of packaged GTest for running unit tests by @xvitaly in #1226
Introduce additional memory metrics by @oontvoo in #1238
Added Doxygen support by @xvitaly in #1228
Allow template arguments to be specified directly on the BENCHMARK macro by @oontvoo in #1262
[RFC] Adding API for setting/getting benchmark_filter flag? by @oontvoo in #1254
use docker container for ubuntu-16.04 builds by @dominichamon in #1265
Support for building with LLVM clang-10/clang-11 on Windows. by @alisenai in #1227
Add Setup/Teardown option on Benchmark. by @oontvoo in #1269
compare.py: compute and print 'OVERALL GEOMEAN' aggregate by @LebedevRI in #1289
Features:
--command-name can now take parameter names from --parameter-* options, see #351
and #391 (@silathdiir)
-Exit codes (or signals) are now printed in cases of command failures, see #342
(@KaindlJulian)
-Exit codes are now part of the JSON output, see #371 (@JordiChauzi)
-Colorized output should now be enabled on Windows by default, see #427
Changes:
-When --export-* commands are used, result files are created before benchmark
execution
-to fail early in case of, e.g., wrong permissions. See #306 (@s1ck).
-When --export-* options are used, result files are written after each
individual
-benchmark command instead of writing after all benchmarks have finished. See
#306 (@s1ck).
-Reduce number of shell startup time measurements from 200 to 50, generally
speeding up benchmarks. See #378
-User and system time are now in consistent time units, see #408 and #409
(@film42)
features
[breaking change] introduce accessorrs for public data members (#1208)
add support for percentage units in statistics (#1219)
introduce coefficient of variation aggregate (#1220)
format percentages in console reporter (#1221)
bugfixes
fix unreachable code warning (#1214)
replace #warning with #pragma message (#1216)
report PFM as found when it is
update u-test value expectations due to scipy upgrade
other stuff
refactored documentation to minimise README.md (#1211)
install docs when installing library (#1212)
features
helper methods to create integer lists (#1179)
default of --benchmark_filter is now rather than "." (#1207)
fixes
type warning (#1193)
returning a reference when callers want pointers (65dc63b)
cleanup
remove dead code from PredictNumItersNeeded (#1206)
fix clang-tidy warnings (#1195) and typos (#1194)
prefix macros (#1186) and flags (#1187, #1185) to avoid name clashes
downgrade warnings for googletest (twice) (#1203, 560b0834, ee726a7)
* Fixed macros in bon_csv2html.cpp that had lower case due to excessive
matching on a regex.
* Changed debian/compat to level 10
* Allow specifying the number of random seeks and the number of seeker
processes and store that in the CSV.
* Changed bon_csv2html and bon_csv2txt to take the new CSV format.
* Changed the text output to use KiB/MiB/GiB as units of measurement so we
can fit NVMe results on screen.
* Changed the HTML to be in MiB/s for the bulk IO tests.
* Changed the default size for getc_putc to work with faster CPUs and more
optimised libc.
-- Russell Coker <russell@coker.com.au> Wed, 26 Dec 2018 15:14:28 +1100
1.5.5
new features
Add support for new architecture loongarch (#1173)
Fixed version of random interleaving of benchmark repetitions (#1163, fixing #1051)
Easier comparison of results across families (#1168#1166#1165#1164)
fixes
Fix perf counter argument parsing (#1160)
internal cleanup
Drop warning to satisfy clang's -Wunused-but-set-variable diag (#1174)
Enable some sanitizer builds in github actions (#1167#1171)
Fix memory leak in test (#1169)
new features
better versioning in releases [#1047]
MSVC arm64 support [#1090]
add support for hardware performance counters [#1114, #1153]
add interface for custom context to be included [#1127, #1137]
random interleaving to reduce noise [#1105]
compiler cleanliness
support -Wsuggest-override [#1059]
builds correctly with gcc-11 [#1060]
fix some windows warnings [#1121]
fix -Wreserved-identifier failures [#1143]
fix pendantic warnings [#1156]
New features
Implement custom benchmark names (#1107)
Support for macro expansion in benchmark names (#1054)
Reduce ramp up repetitions when KeepRunningBatch is used (#1113)
Platform support
CycleTimer implemented for M68K architecture (#1050)
Support for DragonFly BSD (#1058)
Better support for z/OS (#1063, #1067)
Add MSVC ARM64 support for cycle clocks (#1052)
Add support for Elbrus 2000 (#1091)
Fix CPU frequency for AMD Ryzen (and probably other CPUs) (#1117)
Bug fixes
Fix range when starting at zero (#1073)
Tool improvements
Build tools with bazel (#982)
Support JSON dumps of benchmark diffs (#1042)
This also fixes build problems with bind-9.16.13. Detailed changes:
- Release 2.5.1
This release re-adds support for TYPEnnn and ANY in the datafile, this
was missed during the removal of the dependency on BINDs development
libraries in v2.4.0.
Also note that with v2.5.0, this software now depends on Concurrency
Kit (ck) for atomic operations.
- Release 2.5.0
This release adds re-connection support for TCP and DoT protocol,
new options to `resperf` and fixes a few bugs.
`dnsperf` and `resperf` will now try to re-connect when they lose a TCP
or DoT connection, and with that comes a few new statistics metrics.
For `dnsperf`, if a connection oriented protocol is used, it will now
show the total number of re-connections made and the connection latency.
For `resperf` it also shows the total number of re-connections made and
the gnuplot data now contains the total number of connections made and
the connection latency for each interval.
Beside re-connection support, improvements have been made when it comes
to tracking socket readiness while connections are established which
should generate less warnings about "socket not ready".
New `resperf` options:
- `-R`: Reopen the datafile if it runs out of data before the testing
is completed. This allows for long running tests on very small and
simple query datafile.
- `-F <fall_behind>`: Sets the maximum number of queries that can fall
behind being sent. `resperf` will stop when this many queries should
have been sent and it can be relative easy to hit if `-m <max_qps>`
is set too high.
The default is 1000 and setting it to zero (0) disables the check.
Bugfixes:
- Fixed port handling for host/network format when setting client side
port with `-x`
- Fix support for quoted characters, `\000` and `\.`, in domain names,
this was lost when removing BIND's internal development libraries
- Fix issue in `dnsperf`, it would loop forever if no connection could
be established
- Fix potential buffer overrun in `resperf` when using response id
for `queries[]`
- DoT: Fix bug when sending from buffer
Other changes:
- Always use `IPV6_V6ONLY` socket option for IPv6
- Add man-page on `-W` option added in v2.4.0
- Reformat man-pages
- `resperf`:
- Try and process more request each run to hopefully not hit max
outstanding so easy when high QPS
- Add default value to `-C` so it shows in help
- Release 2.4.2
This release fixes a few issues with reading of the datafile which
could lead to "ran out of data" errors.
The problem was that reading from the datafile was done before finding
a socket to send it on, or socket readyness, and that lead to progressing
the queries without really doing anything.
Another issues that's been fixed was that if the read lines perfectly
aligned with the buffer, it would be treated like EOF and caused an exit.
- Release 2.4.1
This release fixes an issue with the socket readiness function that
could cause a buffer overflow (`-T 10 -c 2000`) due to `select()` being
limited to check 1023 sockets. `poll()` is now used which has no limit.
There has also been a few fixes to the contrib script `queryparse` that
has to do with python v2 and v3 compatibility and better exception
handling.
- Release 2.4.0
This release removes the dependency on BIND's internal development
libraries! This make building and packaging a lot easier and less
troublesome in the future.
This software now depends only on OpenSSL (for TSIG feature) with an
optional depend on LDNS (for dynamic updates feature).
New option:
- Transport mode option `-m`/`-M` now recognizes `dot` alongside `tls`
for encrypted DNS
- Added `-W` for outputting warnings and errors to stdout
Other changes / bugfixes:
- Fix potential memory leak of query descriptions when using verbose
- Only use TLS v1.2 and above for DoT/TLS
- Add a lot of tests
- Add coverage testing
A command-line benchmarking tool.
Features:
-Statistical analysis across multiple runs.
-Support for arbitrary shell commands.
-Constant feedback about the benchmark progress and current estimates.
-Warmup runs can be executed before the actual benchmark.
-Cache-clearing commands can be set up before each timing run.
-Statistical outlier detection to detect interference from other programs and
caching effects.
-Export results to various formats: CSV, JSON, Markdown, AsciiDoc.
-Parameterized benchmarks (e.g. vary the number of threads).
