Skip to main content
Routr’s benchmarks run a released version on dedicated DigitalOcean servers with perfkit: one server for Routr, a DigitalOcean Kubernetes cluster for the Connect chart, and one load generator that drives both. Each run publishes a page at routr.io/benchmarks. Maintainers run them after significant code changes, not on every release. The definitions live in perf/ in the repository.

The benchmarks

Three config files define them. Only perf/perfkit.yaml publishes to routr.io:
  • perf/perfkit.yaml: the full run, all six benchmarks. It takes about 3.5 hours: the capacity benchmarks one after another, then both soaks at once.
  • perf/perfkit.calibration.yaml: the calibration config. The EdgePort ramp with a 1-hour soak at 5 calls/s, for tuning the benchmarks.
  • perf/perfkit.smoke.yaml: the smoke config. The six benchmarks, shortened, on a small cluster (two c-4 nodes): short ramps and 3-minute soaks. It takes under an hour and can be watched live like a full run. Run it first to check the harness before a full run.
A smoke or calibration run from GitHub runs everything except the publish job. Its report stays in the run’s perfkit-report-<run id> artifact.

Before you start

  • Who can run it. Running the workflow needs write access to fonoster/routr. Running from a laptop needs only the DigitalOcean token.
  • Cost. Each run creates two dedicated-CPU droplets (c-8 for Routr, c-16 for the load generator) and a Kubernetes cluster of three c-8 nodes with a load balancer, about $6 for a full run, billed until deleted. perfkit deletes everything at the end of every run, even a failed or cancelled one. perfkit sweep is the fallback: it deletes droplets and clusters left behind. The scheduled sweep runs every six hours from the default branch; until that branch pins perfkit v0.2.0, it removes droplets only, so after a run that failed early, run perfkit sweep by hand.
  • One-time setup. An admin adds two repository secrets, under Settings, then Secrets and variables, then Actions:
    • DIGITALOCEAN_TOKEN: a DigitalOcean API token with the custom scopes listed in perfkit’s README: create, read and delete for droplet, VPC, firewall, SSH key and tag; create, read, delete and access_cluster for kubernetes; read and delete for load_balancer and block_storage.
    • PERFKIT_SSH_KEY: the private half of an ed25519 key made for perfkit only:
      Paste the contents of perfkit (not perfkit.pub) into the secret, then delete both files.

Run from GitHub

1

Open the workflow

In fonoster/routr, open Actions, then Benchmarks, then Run workflow.
2

Fill in the inputs

3

Wait for the page

The run continues across several jobs because GitHub stops a job after six hours. When a perf/perfkit.yaml run ends, the publish job adds the report to the gh-pages branch. Other configs skip the publish job.
routr.io/benchmarks shows the new run. No commit lands on main, so no release starts.

Run from a laptop

You need Node.js 22 and npm 11. npx installs perfkit from its git tag, and npm 10, the npm that Node 22 ships with, fails on it. Check with npm --version, and update with npm install -g npm@11. From the repository root:
With npm 10, run the same command through npm 11:
Add --benchmarks edgeport-echo to run one benchmark, --config perf/perfkit.smoke.yaml for the short check, or --dry-run to print the plan without creating anything. The report is written to .perfkit/<run id>/report.json. A laptop run doesn’t publish to routr.io.

Where results appear

Every published run gets its own page:
  • https://routr.io/benchmarks/: the latest run.
  • https://routr.io/benchmarks/runs/<run id>/: one run, with its report.json.
  • https://routr.io/benchmarks/runs.json: every run, newest first.
The pages live only on the gh-pages branch. Nothing under site/ builds them, and the website deploy keeps them in place.

Delete a run

To remove a run, such as a smoke run you’ve reviewed:
1

Check out gh-pages

2

Remove the run

Delete benchmarks/runs/<run id>/, and delete its entry in benchmarks/runs.json.
3

Fix the latest page

If it was the latest run, benchmarks/index.html still shows it. Re-render it from the newest remaining run’s report, with the edited runs.json as history:
When no run remains, delete benchmarks/ instead.
4

Push

routr.io/benchmarks/runs.json no longer lists the run, and routr.io/benchmarks shows the next run, or nothing when it was the only one.

perfkit

The tool, its config format and its commands.

Run Routr from source

Build and test Routr on your machine.