Output format¶

HEAPLINE_OUTPUT receives one record per process. This page describes version 1.

Layout¶

A record is a small CSV file of three lines:

# heapline 1
rank,host,pid,job,elapsed_s,allocs,frees,peak_heap_bytes,peak_heap_time_s,live_heap_at_exit_bytes,vmhwm_bytes,vmrss_at_exit_bytes
3,node-a,51234,1655310,12.480,4153,4123,302268416,4.211,483532,331431936,9175040

The first line names the format and its version. Tools must check it and refuse versions they do not know: heapline-report does. Adding a column or changing the meaning of one raises the version.

Columns¶

Column	Unit	Meaning
`rank`		MPI rank from `OMPI_COMM_WORLD_RANK`, `PMIX_RANK`, `PMI_RANK`, `MV2_COMM_WORLD_RANK` or `SLURM_PROCID`, in this order; -1 outside MPI
`host`		Host name; commas, quotes and line breaks are replaced with `_`
`pid`		Process id
`job`		`SLURM_JOB_ID`, empty outside Slurm
`elapsed_s`	s	Time from the loading of the library to the end of the process; written with three decimals and a dot, whatever the locale of the program
`allocs`		Successful allocations; a `realloc` counts as one
`frees`		Released blocks; a `realloc` counts as one
`peak_heap_bytes`	bytes	Highest heap in use, in usable sizes; see the precision below
`peak_heap_time_s`	s	Time of that peak since the loading of the library, read from a coarse clock (one kernel tick, 1 to 10 ms); written with three decimals and a dot, whatever the locale of the program
`live_heap_at_exit_bytes`	bytes	Heap still in use at exit
`vmhwm_bytes`	bytes	Peak resident set size of the process (`VmHWM`), -1 if unknown
`vmrss_at_exit_bytes`	bytes	Resident set size at exit (`VmRSS`), -1 if unknown

The peak is exact as long as only one thread of the process has allocated or released memory. When several threads do, each thread counts in its own counters and publishes its changes in batches, so that threads do not wait for each other. The peak may then differ from the true peak by up to (ceil(log2 N) + 1)/8192 of it, N being the number of threads that have allocated or released memory: 0.037% with 4 threads, 0.061% with 16, 0.085% with 64, or 3.5 MiB for a 4 GiB peak with 64 threads. In both cases, as before, blocks that other threads are allocating or releasing at that instant may be missing. allocs, frees and live_heap_at_exit_bytes count every thread exactly, threads still running at exit included, except for operations in progress at that instant. The peak and its time are stored separately: under concurrent allocation, the time may belong to a slightly different peak than the one reported, and with several threads it may be the time of an earlier moment, before bytes held back by threads were published.

File names¶

In HEAPLINE_OUTPUT, %p is replaced with the process id, %r with the rank and %h with the host name. Any other % sequence is kept as is. The directory must exist; when the file cannot be opened, the record goes to stderr.