testing/libc/arch_libc: Add a throughput benchmark. by Fishwaldo · Pull Request #3706 · apache/nuttx-apps

Fishwaldo · 2026-08-07T11:33:51Z

Summary

Rebased onto #3712 as requested, and reduced to the part #3712 does not already
cover.

The correctness suite is gone. #3712 tests all sixteen functions this PR used to
test, plus stpcpy, strcat and strncpy that it did not, so there is nothing
left for it to add.

What remains is the throughput measurement. #3712 times one call per function at
one size, 128 bytes, with both operands aligned. This sweeps four sizes and four
source/destination alignment pairs across seventeen functions, which is #3712's
sixteen plus strlcpy.

It sits behind TESTING_ARCH_LIBC_BENCH, default n. Nothing changes when it is
off.

Why the alignment matrix

A machine implementation usually takes its wide path only when the pointers meet
some alignment condition, so a single aligned measurement reports the best case
and says nothing about the rest of the input space. On rv64:

  strcpy    32768 B  s+0/d+0     2938.0 MB/s
  strcpy    32768 B  s+1/d+1     2942.0 MB/s
  strcpy    32768 B  s+1/d+2      626.0 MB/s

Two pointers misaligned by the same amount run at the aligned rate. Misaligned by
different amounts they fall to a fifth of it. Neither number is visible from an
aligned measurement alone.

A function with no machine implementation reports the same rate at every
alignment, so the sweep also shows which functions a machine directory actually
covers.

In passing this caught a strlcpy regression in 931d5f50d4, where the misaligned
case runs 55x slower than the generic C it replaced. I will open a separate PR
for that.

What it reports

Each measurement gives MB/s, which compares across machines, and cycles per byte
where perf_gettime() is reachable from an application, both from one timed
loop.

Testing

EIC7700X (rv64, 1.4 GHz), CONFIG_BUILD_KERNEL, 248 measurements per run, in two
configurations: the generic C library, and the RISC-V assembly currently on
master. Both pass.

Note for kernel builds

perf_gettime() is not in syscall.csv, so an application cannot reach it
unless the C library builds its own copy
(CONFIG_ARCH_HAVE_PERF_EVENTS_USER_ACCESS) or the build is flat. This
benchmark guards for that and reports MB/s alone where the counter is out of
reach.

Fishwaldo · 2026-08-07T11:44:22Z

The RISC-V series this suite was written for is now open as
apache/nuttx#19735. Every correctness claim in that PR is this suite's output,
on rv64 hardware and on rv32 under QEMU.

This PR remains independent of it: the suite is arch-neutral, passes against
the generic C library when nothing is overridden, and is useful to any machine
directory that overrides these functions.

cederom

Thank you @Fishwaldo :-)

cederom · 2026-08-09T18:49:02Z

Looks like #3712 is a joint merge of several test suites and should be merged instead? :-)

xiaoxiang781216 · 2026-08-11T03:24:17Z

@Fishwaldo since #3712, could you try that change and close this pr? thanks.

hartmannathan · 2026-08-11T17:57:29Z

What's the status of this PR? Did #3712 include all the new tests that are implemented here?

xiaoxiang781216 · 2026-08-12T06:52:53Z

What's the status of this PR? Did #3712 include all the new tests that are implemented here?

@Fishwaldo could you check where #3712 contain all your test, if not, please apply your patch on top of #3712, thanks.

The existing speed checks time one call at one size, 128 bytes, with both operands aligned. A machine implementation usually takes its wide path only when the pointers satisfy some alignment condition, so that single point reports the best case and says nothing about the rest of the input space. Measure the same functions across a size sweep and every source and destination alignment pair instead, plus strlcpy. On rv64 the difference this exposes is not marginal: strcpy 32768 B s+0/d+0 2938.0 MB/s strcpy 32768 B s+1/d+1 2942.0 MB/s strcpy 32768 B s+1/d+2 626.0 MB/s memcmp 32768 B s+0/d+0 412.4 MB/s memcmp 32768 B s+1/d+2 41.0 MB/s Two pointers misaligned by the same amount run at the aligned rate; misaligned by different amounts they fall to a tenth of it. Neither number is visible from an aligned measurement alone. A function with no machine implementation reports the same rate at every alignment, so the sweep also shows which of them a machine directory actually covers. Each result reports MB/s, which compares across machines, and cycles per byte where perf_gettime() is reachable from an application, both from one timed loop. strcat starts from an empty destination on each turn, since appending to the last result would grow it without bound, so its figure includes that store. It sits behind TESTING_ARCH_LIBC_BENCH, default n, because a measurement runs for a fixed interval and a full sweep takes about a minute. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>

Fishwaldo · 2026-08-15T06:14:02Z

Updated. Dropped my correctness tests in favor of #3712 and updated the benchmarks to cover everything in #3712.

Note, on my box, perf_gettime isn't available for Kernel builds so also keep the original MB/s measurements.

strlcpy is the one function in this directory's reach that nothing here covers, and a machine directory may override it like any other. Sweep every source and destination alignment pair against sizes 1 to 64, and for each of those every capacity from zero to one past the length. Check the return value, which is the length of src whether or not the copy fit, the truncation point, the content, that a capacity of zero writes nothing at all, and that nothing lands past the terminator. The alignment pairs are the point. An implementation that walks one of the two pointers to a boundary and then copies a register at a time is correct whenever the two agree, so a test that only ever passes matching alignments says nothing about it. The timing half is guarded. perf_gettime() is not a system call, so an application reaches it only where the C library builds its own copy or where the application and the kernel are one image; calling it unconditionally leaves the test unbuildable on a kernel build, which is where the correctness half is still wanted. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>

Neither is covered here, and both are overridable, so a machine or libc implementation of either goes in unmeasured and unchecked. memccpy is checked with the search character present, where the copy stops just past it and the result points there, and absent, where the whole length is copied and the result is NULL. stpncpy is checked against every capacity from zero to four past the length, for the content, the zero padding beyond the terminator, and the returned pointer, which is the terminator when the string fits and one past the end when it does not. Both sweep all sixty four source and destination alignment pairs, and both are added to the benchmark, which now covers nineteen functions. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>

Fishwaldo · 2026-08-15T08:23:01Z

Sorry, found a few functions that were missing some correctness tests while working on the optimized fixes for riscv and the newlib variants. Added memccpy, stpncpy and strlcpy for completeness (including benchmarks)

Every measurement repeats until a stated interval has passed, so a clock that reads the same value twice does not slow the benchmark down, it stops it returning at all. CLOCK_MONOTONIC does not advance on every target. On qemu-intel64 it reports success and stays at zero, while CLOCK_REALTIME advances normally, and the benchmark spins in its first measurement with no output after the heading. Sample each candidate twice around a busy wait and take the first one whose reading changes. Where none does, say so and skip the timing rather than hang. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>

xiaoxiang781216 · 2026-08-15T10:36:12Z

      return 1;
    }

+  if (!bench_pick_clock())


move before line 438 and remove line 451-452

xiaoxiang781216 · 2026-08-15T10:39:19Z

+
+#include <nuttx/clock.h>
+
+#ifdef CONFIG_TESTING_ARCH_LIBC_BENCH


remove the check

fixed

Fishwaldo mentioned this pull request Aug 7, 2026

libc/machine/risc-v: Optimize string routines for misaligned pointers. apache/nuttx#19735

Closed

acassis requested review from cederom, jerpelea and xiaoxiang781216 August 8, 2026 19:40

github-actions Bot added Size: L Area: Testing labels Aug 8, 2026

Fishwaldo force-pushed the upstream-arch-libc-string branch from 542c585 to 555e43f Compare August 9, 2026 04:33

acassis previously approved these changes Aug 9, 2026

View reviewed changes

cederom previously approved these changes Aug 9, 2026

View reviewed changes

xiaoxiang781216 previously requested changes Aug 9, 2026

View reviewed changes

Comment thread testing/libc/arch_libc/arch_libc_test_main.c Outdated

Fishwaldo dismissed stale reviews from cederom and acassis via 31e7dc3 August 15, 2026 06:09

Fishwaldo force-pushed the upstream-arch-libc-string branch from 555e43f to 31e7dc3 Compare August 15, 2026 06:09

Fishwaldo changed the title ~~testing/libc/arch_libc: Test every overridable string function.~~ testing/libc/arch_libc: Add a throughput benchmark. Aug 15, 2026

github-actions Bot added Size: M and removed Size: L labels Aug 15, 2026

github-actions Bot added Size: L and removed Size: M labels Aug 15, 2026

Fishwaldo force-pushed the upstream-arch-libc-string branch from 1e02b99 to 30deac5 Compare August 15, 2026 07:44

This was referenced Aug 15, 2026

libs/libc/machine/risc-v: Correct RISC-V optimized functions. apache/nuttx#19856

Open

libs/libc/string: Optimize string routines for misaligned pointers. apache/nuttx#19857

Open

xiaoxiang781216 reviewed Aug 15, 2026

View reviewed changes

Navigation Menu

Uh oh!

testing/libc/arch_libc: Add a throughput benchmark. - #3706

testing/libc/arch_libc: Add a throughput benchmark.#3706
Fishwaldo wants to merge 4 commits into
apache:masterfrom
Fishwaldo:upstream-arch-libc-string

Fishwaldo commented Aug 7, 2026 •

edited

Loading

Uh oh!

Fishwaldo commented Aug 7, 2026

Uh oh!

cederom left a comment

Uh oh!

Uh oh!

cederom commented Aug 9, 2026

Uh oh!

xiaoxiang781216 commented Aug 11, 2026

Uh oh!

hartmannathan commented Aug 11, 2026

Uh oh!

xiaoxiang781216 commented Aug 12, 2026

Uh oh!

Fishwaldo commented Aug 15, 2026

Uh oh!

Fishwaldo commented Aug 15, 2026

Uh oh!

xiaoxiang781216 Aug 15, 2026

Uh oh!

xiaoxiang781216 Aug 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants


		#include <nuttx/clock.h>

		#ifdef CONFIG_TESTING_ARCH_LIBC_BENCH

Uh oh!

Conversation

Fishwaldo commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why the alignment matrix

What it reports

Testing

Note for kernel builds

Uh oh!

Fishwaldo commented Aug 7, 2026

Uh oh!

cederom left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cederom commented Aug 9, 2026

Uh oh!

xiaoxiang781216 commented Aug 11, 2026

Uh oh!

hartmannathan commented Aug 11, 2026

Uh oh!

xiaoxiang781216 commented Aug 12, 2026

Uh oh!

Fishwaldo commented Aug 15, 2026

Uh oh!

Fishwaldo commented Aug 15, 2026

Uh oh!

xiaoxiang781216 Aug 15, 2026

Choose a reason for hiding this comment

Uh oh!

xiaoxiang781216 Aug 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Fishwaldo commented Aug 7, 2026 •

edited

Loading