Repository navigation
PR #4613 — enabling jemalloc on arm64 (proxy memory never released on Graviton/aarch64) #15613
Description
What problem are you trying to solve?
The problem: linkerd2-proxy compiles jemalloc in as the global allocator only for x86_64-unknown-linux-gnu. On arm64 the proxy silently falls back to glibc malloc. glibc sizes its arena budget from the host's CPU count (not the container's cgroup limit — 16-64 cores on typical Graviton nodes) and, in practice, retains freed memory in fragmented arenas indefinitely. The result is that proxy RSS on arm64 ratchets up with peak load and never comes back down until the pod restarts. A second-order effect: any MALLOC_CONF tuning users apply per the usual guidance is a silent no-op on arm64, since there's no jemalloc in the binary to read it.
Measured impact (arm64 EKS cluster, meshed services, ~300 pooled HTTP/2 connections per pod): with the stock glibc proxy, closing all pooled connections — freeing their buffers — released zero RSS. With the same proxy version rebuilt with jemalloc enabled for aarch64, the identical experiment released 211MB → ~20MB (−91%) within minutes, reproduced three times. Same traffic, same version; only the allocator differs.
Happy to provide graphs, metric snapshots, or additional testing.
How should the problem be solved?
The change: the x86_64-only gate dates to a time when the jemallocator crate only listed x86_64 as fully tested. tikv-jemallocator 0.6 (already in Cargo.lock) supports aarch64-unknown-linux-gnu, so the PR widens the cfg gate and adds the matching target dependency — 8 lines, no lockfile changes, default behavior unchanged on every other platform. We've been running this build in a production-like environment on Graviton with the results above.
I'd like to flag a data-plane behavior that affects every arm64/aarch64 deployment, and a small PR that addresses it: linkerd/linkerd2-proxy#4613
One caveat worth maintainer input: cross-compiling jemalloc's autoconf build for aarch64-gnu from an amd64 host needs a working aarch64 C link path — building natively on arm64 works cleanly; CI may need a toolchain tweak for the cross path.
Any alternatives you've considered?
unable to solve without this change
How would users interact with this feature?
No response
Would you like to work on this feature?
None
Activity
ergdevops commentedon Sep 23, 2026
@anthoturc Thanks for picking this up. On the test-suite hesitancy — the aarch64-gnu disablement in tikv/jemallocator appears to be a QEMU-era leftover rather than a real failure: since tikv/jemallocator#149 that CI job runs on a native
ubuntu-24.04-armrunner, and the full jemalloc C suite passes when enabled there. I've opened tikv/jemallocator#186 flipping the flag; that PR's own CI run is pending maintainer approval, but the same change already ran green through the full matrix on my fork — https://github.com/ergdevops/jemallocator/actions/runs/35901181219 — with theaarch64-unknown-linux-gnujob passingmake checkin ~20 min: https://github.com/ergdevops/jemallocator/actions/runs/35901181219/job/107317494750Happy to adapt linkerd/linkerd2-proxy#4613 in whatever direction helps — e.g., gating jemalloc behind a cargo feature on this target if you'd prefer an opt-out.
ergdevops commentedon Sep 24, 2026
@anthoturc tikv/jemallocator#186 its merged now.