Limitations and alternatives

Limitations

  • Statically linked programs cannot be intercepted with LD_PRELOAD.

  • Programs that use another allocator, such as jemalloc or tcmalloc, bypass heapline.

  • Sizes are usable sizes, slightly above the requested sizes.

  • With several threads, the peak may differ from the true peak by a small fraction of it, 0.085% with 64 threads, see Output format.

  • heapline must be loaded at startup with LD_PRELOAD; loading it later with dlopen is not supported.

  • heapline_get_stats takes a mutex, so it cannot be called from a signal handler or from a fork handler.

  • A child process created with _Fork or a raw clone, which run no fork handlers, can hang at exit, or in heapline_get_stats, if another thread held the internal lock of heapline at the moment it was created: while starting, exiting or reading the totals. fork, vfork and posix_spawn are not affected.

  • Memory obtained with mmap directly, or allocated on a GPU, is not heap and is not counted.

  • A process killed by a signal writes no record. heapline-report lists it as a missing rank, unless it was the highest rank of the job, which it cannot tell apart from a smaller job.

  • The summary on stderr is lost when the program closes stderr before exiting, as GNU coreutils do. The record written to HEAPLINE_OUTPUT is not.

  • A wrapper process can overwrite the record of the program. A script run without exec, /usr/bin/time or any other process that stays the parent of the program, when preloaded with heapline, has the same rank and host as the program and exits last. With HEAPLINE_OUTPUT=heapline.%h.%r.csv, its record replaces the program’s without a warning: a run.sh that launched a 288 MiB Python program left the record of bash, with a peak of 158.7 KiB. Start the program with exec and preload it alone, or add %p to the file name. See Wrapper processes.

  • The cost is paid on every allocation. In a process that holds its data, as a simulation does, four threads that allocate continuously add about 20% to the run time, and so does a single thread. A small program whose threads do nothing but allocate adds the most, about 170%. See Performance history.

When to use something else

heapline answers “how much, where in the job, and when”. When you need to know which line of code allocates, use a tool that records call stacks:

  • heaptrack records every allocation with its stack and comes with a viewer. It records a call stack for every allocation, so each allocation costs more than with heapline.

  • Valgrind massif takes detailed heap snapshots, at the cost of running the program under Valgrind, many times slower.

  • jemalloc and tcmalloc have sampling heap profilers, if the program already uses, or can switch to, one of those allocators.