Getting Started with nanobench and Reading Its Output

Updated

To start with nanobench, add its single header to your project (direct inclusion or CMake integration), include nanobench.h, create an ankerl::nanobench::Bench, and call run() with a name and a lambda containing the code you want to measure. The default output is a table whose key columns are timePerUnit (time per iteration), unitPerSecond (throughput), error (relative measurement error), and optional hardware counters such as instructions, cycles, ipc, branches, and branchMisses. You can hide, show, or add columns, and attach context values that appear as extra columns.

Installation

nanobench is distributed as a header-only library, so there is no compiled library to link.

Direct inclusion

Copy the nanobench header into your project and include it:

#include "nanobench.h"

CMake integration

If you use CMake, add nanobench as a dependency and link the nanobench target to your benchmark executable. The documentation lists CMake Integration as the supported path; follow the target name it provides rather than inventing include paths.

Basic usage

The main entry point is ankerl::nanobench::Bench. A minimal benchmark looks like this:

#include "nanobench.h"

int main() {
    ankerl::nanobench::Bench b;
    b.title("my benchmarks")
     .unit("op")
     .warmup(100)
     .minEpochIterations(1000);

    b.run("some work", [] {
        // code under test
    });
}

What each part does:

  • title() sets the table heading.
  • unit() names what one iteration produces (for example op or byte), which affects how unitPerSecond is labeled.
  • warmup() runs the code before measuring so caches and branch predictors are primed.
  • minEpochIterations() sets a floor on how many iterations each measurement epoch performs.
  • run(name, lambda) executes the lambda repeatedly, measures it, and appends a row to the results.

run() can be called multiple times on the same Bench to build a table of several benchmarks.

Reading the output

The default table reports one row per run() call. The columns you will use most:

Column Meaning
relative Time relative to the fastest benchmark in the table
timePerUnit Measured time divided by iterations, in the chosen time unit
unitPerSecond Throughput: iterations (or units) completed per second
error Estimated relative error of the measurement
instructions Retired instructions per iteration (if performance counters are enabled)
cycles CPU cycles per iteration
ipc Instructions per cycle
branches Branch instructions per iteration
branchMisses Mispredicted branches per iteration

Two practical points when interpreting these:

  • error is a stability signal, not a correctness guarantee. A large error means the measurement varied a lot between epochs; treat the row as unreliable and investigate (see the documentation's sections on unstable benchmarks and serial correlation).
  • Hardware counters are optional. instructions, cycles, ipc, branches, and branchMisses depend on performanceCounters() being enabled and on platform support; do not assume they are populated.

Choosing, hiding, and showing columns

Use showColumn() and hideColumn() to control which columns appear, and isColumnVisible() to query the current state. This is useful when you only care about timing, or when you want counters without the default set.

Adding context as columns

context() attaches a named value to the current benchmark, and contextColumn() promotes a context key into its own table column. clearContext() and clearContextColumns() reset them. This lets you compare variants side by side, for example a context key recording which implementation or input size a row used.

Comparing alternatives

To compare implementations rather than just list them, use compare() on the Bench. It returns a CompareResult whose isSignificant() tells you whether the difference between the fastest and the others is statistically meaningful, and whose fastest() identifies the winner. Use this instead of eyeballing timePerUnit when the numbers are close.

Practical tips

  • Start with a small minEpochIterations() and let nanobench's adaptive epoch logic settle; the library adjusts epoch length to hit a target measurement interval.
  • Keep the lambda free of setup work you do not intend to measure, or move setup outside run().
  • If a benchmark looks unstable, check the documentation's guidance on warmup and serial correlation before increasing iteration counts blindly.
  • For machine-readable results, nanobench can render CSV, JSON, HTML box plots, and pyperf output through its render() templates, so you can feed results into other tooling instead of parsing the console table.