translator: rich text is translated node-by-node, so word order and inline marks come out wrong

[SearheiParkhamchuk]

Problem

Rich text is expanded one chunk per Lexical text node, and each translation is written straight back into the node it came from. The node order in the tree therefore never changes, which means the translated sentence keeps the source language's word order, and every inline mark stays pinned to the position it had in the source.

For languages whose syntax differs from the source, the output is wrong in a way no prompt can fix — the model is never given the chance to reorder anything.

Example

a **red** car is stored as three text nodes:

{ "type": "paragraph", "children": [
  { "type": "text", "text": "a ",   "format": 0 },
  { "type": "text", "text": "red",  "format": 1 },
  { "type": "text", "text": " car", "format": 0 }
] }

French needs une voiture rouge — the adjective moves after the noun, and the bold has to move with the word.

What we produce instead: each node is translated in place, so the result reads une rouge voiture — source word order, and the emphasis is on the wrong word.

German subordinate clauses show the same failure: because we improve **the service** daily needs the verb at the end (weil wir den Service täglich verbessern), which per-node translation cannot produce. The further the target language is from the source in sentence structure, the worse it gets; for Japanese it is wrong nearly always.

Where this lives

• src/core/translation-pipeline/stages/text-expander/RichTextExpander.ts:17-34 — one chunk per text node
• src/core/translation-pipeline/stages/translation-applicator/TranslationMutator.ts:18-29 — chunk.nodeRef.text = translation, pinning each fragment to its original slot
• src/core/kernel/lexical/collectTextNodes.ts:12 — drops whitespace-only nodes, which are exactly the nodes holding the gaps between words

Proposal

Make the inline container (paragraph, heading, list item, quote) the unit of translation instead of the text node. Fragments inside it are wrapped in numbered marks; the model returns them in whatever order the target language requires; we rebuild the container's children in that order.

sent:     { 7: "<1>a </1><2>red</2><3> car</3>" }
returned: { 7: "<1>une </1><3>voiture </3><2>rouge</2>" }

A mark number is a pointer to the original inline node, not an address to write into. Applying a translation becomes:

1. walk the returned fragments left to right
2. for each mark, take its original node and write the fragment's text into its leaf
3. push it onto a fresh children array, then assign that array to the container

Nothing reads or writes format, style, detail, a link's fields, or any other mark representation — marks travel inside the nodes themselves. The only writes are text on a leaf and children on the container, so the layer stays within the type / text / children surface its types already declare.

Non-text inline nodes (line breaks, inline blocks) get self-closing marks so they survive the rebuild. Nodes whose mark disappears from the reply (fragment merged into a neighbour) simply drop out of the new array.

Cheaper paths kept

Case	Handling
Container holds a single unmarked text node	current path, no marks emitted at all
Marks return in their original order	current path — write into the original nodes
Mark order changed	rebuild children

Most content falls into the first two rows, so the new code runs only where it is needed.

Provider contract

Unchanged: Record<number, string> in, Record<number, string> out. Only the string contents gain marks, and mark parsing lives in the applicator, so no provider learns about Lexical. The response schema, key validation and job retries are untouched.

Compatibility risks to handle:

• a SystemPromptBuilder that ignores defaultPrompt will not carry the mark instruction, and the model will mangle the marks
• non-LLM providers (a machine-translation API behind CompletionFn) may translate or strip marks
• runDryRun.ts:40 transforms values and must pass marks through untouched
• gate the change behind a rich text granularity setting, defaulting to the current per-node behaviour for this major version

Open questions

1. Mark syntax and escaping — what to do when the text itself contains <.
2. Validation rules: which reply is considered corrupt (missing mark, duplicate, unknown number), and the fallback. Proposal: put the whole translation into a copy of the first node, losing inline marks within that container, and warn — alongside the existing partial-reply warning.
3. A new inline-level walk is needed; the current one drops whitespace-only nodes and does not record parents.
4. Confirm no provenance fingerprint is computed from the target side — leafSourceText (src/core/domain/content-projection/translatableLeaf.ts:41-49) joins source text nodes, and changing the target node count must not mark existing translations stale.

Acceptance

• A container whose mark order changes in the reply is rebuilt so both text and inline marks land on the right words; covered by a test with a French adjective and a German subordinate clause.
• A reply with a corrupt mark set does not damage the tree: text is applied, inline marks within that container are dropped, a warning is emitted.
• Whitespace-only nodes keep the gaps between words intact.
• With the setting left at its default, output is byte-identical to today's for the same input and reply.

[SearheiParkhamchuk]

Closed by #139. Rich text is now translated one container at a time, with inline formatting carried as numbered marks, so the translation can reorder pieces and emphasis follows the word it belongs to.

It ships behind experimental.inlineMarks, off by default, and needs a provider that declares capabilities.inlineMarks. The switch itself is deprecated on arrival — the next major removes it and the behaviour becomes the only mode. Tracked in docs/DEPRECATIONS.md#experimental-inline-marks.