Performance history¶
heapline must stay cheap enough to leave on in production runs. Each push to
main measures its overhead on five scenarios:
Scenario |
Command |
What it shows |
|---|---|---|
|
|
Cost per allocation, without contention |
|
|
Four threads that only allocate, in a small heap: the worst case |
|
|
The same churn in a process that holds 512 MiB, like a simulation with its arrays |
|
|
Blocks of 64 KiB to 1 MiB, published on every operation |
|
|
A real program: the Python interpreter |
malloc_churn keeps 64 blocks per thread and replaces them in turn, with a
pseudo-random size. In a small heap, the threads publish almost every change,
which is what keeps churn-4t slow; when the process holds more memory, as in
resident-4t, they publish in batches. large-4t also publishes on every
operation, but each of its allocations costs more, so the shared counter is a
smaller part of the time. Points measured before scenarios existed belong to
churn-4t. Points of churn-4t measured before 0.4.0 were timed by the
benchmark itself; later points time the whole command, which adds a few
milliseconds.
The figure is the relative overhead measured within one CI job: the median time of ten runs with heapline, divided by the median of ten runs without it, interleaved, minus one. Absolute times are not comparable between runs, because GitHub’s shared runners change machines. The ratio mostly is, but expect noise of a few tens of percentage points from one point to the next.
Release comparison¶
Measured on one machine, under WSL2, with
bench/run_overhead.sh SCENARIO LIB 10 -- COMMAND and ten runs per point.
Each figure is the median of three rounds that alternate the two libraries,
with the same benchmark binaries.
Scenario |
0.3.0 |
0.4.0 |
|---|---|---|
|
19% |
20% |
|
363% |
168% |
|
341% |
20% |
|
39% |
24% |
|
4% |
5% |
Measured on an Intel Core i7-10700F, 2.90 GHz. On churn-4t, 0.2.0 and 0.3.0
measured 409% and 402% in an earlier series. Between 0.3.0 and 0.4.0,
churn-4t went from 363% to 168%, resident-4t from 341% to 20% and
large-4t from 39% to 24%, while churn-1t and python changed by about one
point. churn-4t stays high because in a small heap the threads publish
almost every change.
History¶
The history lives in the bench-data branch of the repository, one line per
point. A push to main adds one point per scenario, and so does a manual run
on main; when pushes come in quick succession, some commits are skipped. The
table lists the last 25 points, newest first.
churn-1t¶
churn-4t¶
resident-4t¶
large-4t¶
python¶
Last points¶
Commit |
Date |
Scenario |
Overhead |
Allocations/s with heapline |
Runner CPU |
|---|---|---|---|---|---|
|
2026-10-20 |
python |
4.1% |
1.75e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-20 |
large-4t |
5.8% |
4.1e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-20 |
resident-4t |
22.5% |
3.16e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-20 |
churn-4t |
51.3% |
2.56e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-20 |
churn-1t |
18.4% |
1.17e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
python |
4.1% |
1.76e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
large-4t |
6.2% |
4.13e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
resident-4t |
22.8% |
3.16e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
churn-4t |
60.7% |
2.41e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
churn-1t |
18.3% |
1.17e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
python |
3.3% |
1.78e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
large-4t |
6.8% |
4.22e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
resident-4t |
22.7% |
3.16e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
churn-4t |
63.3% |
2.37e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
churn-1t |
18.5% |
1.17e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
python |
4.6% |
1.76e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
large-4t |
6.2% |
3.9e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
resident-4t |
22.8% |
3.15e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
churn-4t |
62.7% |
2.38e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
churn-1t |
18.1% |
1.17e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
python |
3.9% |
1.76e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
large-4t |
6.0% |
3.97e+06 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
resident-4t |
22.7% |
3.15e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
churn-4t |
64.7% |
2.34e+07 |
AMD EPYC 7763 64-Core Processor |
|
2026-10-17 |
churn-1t |
18.5% |
1.17e+07 |
AMD EPYC 7763 64-Core Processor |