libs/libc/string: Optimize string routines for misaligned pointers.

[Fishwaldo]

Summary

Nine of the BSD string routines take their word path only when both pointers
are already on a long boundary:

#define UNALIGNED(x, y) \
  (((long)(uintptr_t)(x) & (sizeof(long) - 1)) | \
   ((long)(uintptr_t)(y) & (sizeof(long) - 1)))

That asks more than the loops need. They read and write at the same boundary
in both operands, so what matters is whether the two agree about where a
boundary falls, not whether either is already on one. A pair offset by the
same amount is walked up to the boundary a byte at a time and handled a word
at a time from there, because aligning one aligns the other.

The union also holds far less often than the difference. For arbitrary
pointers on a 64 bit target it is true about one time in 64 against one in
eight, and the case it rejects, two strings carved out of the same buffer or a
structure copied field by field, is a common one.

memccpy, memcmp, memcpy, stpcpy, stpncpy, strcmp, strcpy,
strncmp, strncpy. The five single pointer routines cannot benefit and are
untouched.

No unaligned access is introduced. Every word read and write is still on a
boundary, so this is safe where a misaligned access faults or is emulated.

Where this comes from

@xiaoxiang781216 asked for exactly this in #19735: keep the general
optimisation out of the arch specific code and put it in the BSD
implementation. #19856 corrects the RISC-V assembly that has the same problem;
this is the portable half, and it is the one that reaches every architecture.

Measured

EIC7700X, rv64 at 1.4 GHz, CONFIG_ALLOW_BSD_COMPONENTS=y with
CONFIG_LIBC_NEWLIB_OPTSPEED=y, 4 KB operands, taken with the benchmark in
apache/nuttx-apps#3706.

Both pointers offset by one, the case the union rejects:

             before      after
  memcpy        381       2777 MB/s
  memcmp         40        330
  strcpy        621       1836
  stpcpy        551       1835
  strcmp         41        273
  strncmp        28        203
  strncpy       376       1658
  stpncpy       276       1646
  memccpy       545       1439

Every one of those before figures is the byte loop. After the change each
lands within a few percent of the same function's aligned rate, which is what
says the prologue is doing what it should and nothing more.

Both pointers aligned, where the new test costs an extra branch and nothing
else:

             before      after
  memcpy       2885       2876 MB/s
  memcmp        354        335
  strcpy       1970       1957
  stpcpy       1906       1950
  strcmp        277        277
  strncmp       207        206
  strncpy      1776       1650
  stpncpy      1836       1644
  memccpy      1626       1434

strncpy, stpncpy and memccpy lose 7 to 12 per cent there. Those three
return from inside the prologue to handle padding or the stop character, which
adds a branch to the aligned path as well. The rest are unchanged.

Pointers that genuinely disagree are unaffected, as they must be: no single
boundary serves both.

This needs measuring on other hardware

I can only speak for one part. The gain depends on how expensive a byte loop
is relative to a word loop on a given core, and the loss on those three
functions depends on branch prediction, so both numbers will move.

Please do not take QEMU numbers for this. QEMU executes misaligned
accesses natively at full speed and does not model the alignment behaviour of
the part it emulates, so it flatters the code being replaced and hides what
this is worth. It is fine for correctness and useless for this measurement.
Every figure above is from silicon.

If you have a board to hand, apache/nuttx-apps#3706 sweeps sizes against every
source and destination alignment pair and reports MB/s and cycles per byte.

Testing

• Correctness: testing/libc/arch_libc under qemu rv64, all nineteen
  functions pass. memccpy and stpncpy had no coverage anywhere, so tests
  for both went into testing/libc/arch_libc: Add a throughput benchmark. nuttx-apps#3706 first; the padding, the stop
  character and the return values are what the prologue is easiest to get
  wrong on.
• Throughput: rv64 silicon as above. Also measured on rv32 under QEMU with
  -icount and on x86_64 under KVM, both of which agree in direction, though
  for the reason above I would not put weight on the QEMU figures.

[github-actions]

MemBrowse Memory Report

No memory changes detected for:

• arduino-mega2560
• esp32-devkitc
• hifive1-revb
• mirtoo
• qemu-armv8a
• qemu-intel64
• rx65n-rsk2mb
• s698pm-dkit
• stm32-nucleo-f103rb