Differentiate n times where n times was asked for (#1002)

[Rafael-SOWNet]

Closes #1002 — and the fix that issue proposes is wrong; see Why the two cannot simply be merged.

The defect

Differentiate(Variable) goes through the transformation, which ends at DifferentiateOnce and simplifies. Differentiate(Variable, int) called InnerDifferentiate straight in its loop, so nothing was ever simplified — and because each pass differentiated the unsimplified result of the last, every 0 * and * 1 the chain rule produces was still there to be differentiated again.

var x = MathS.Var("x");
"x ^ 3".ToEntity().Differentiate(x, 1)   // was: 3 * x ^ 2 * 1
                                         // now: 3 * x ^ 2
"x ^ 3".ToEntity().Differentiate(x, 2)   // was: (0 * x ^ 2 + 2 * x ^ 1 * 1 * 3) * 1 + 0 * 3 * x ^ 2
                                         // now: 2 * x * 3

I wrote in the issue that power = 1 was fine. It is not — one raw pass still leaks its * 1. At n = 3 it stops being untidy:

"x ^ 4".Differentiate(x, 3)
was: (0 * x ^ 3 + 3 * x ^ 2 * 1 * 0 + ((0 * x ^ 2 + 2 * x ^ 1 * 1 * 3) * 1 + 0 * 3 * x ^ 2) * 4
      + 0 * 3 * x ^ 2 * 1) * 1 + 0 * (0 * x ^ 3 + 3 * x ^ 2 * 1 * 4) + 0 * 4 * x ^ 3
      + (0 * x ^ 3 + 3 * x ^ 2 * 1 * 4) * 0
now: 2 * x * 3 * 4

The value was never wrong — the two routes agree numerically on both versions — so this is a change of form, and it breaks anyone matching on the shape. It also means the cost grew with the accumulated mess rather than with the derivative.

Why the two cannot simply be merged

#1002 says "use DifferentiateOnce in the loop". Doing exactly that overflows the stack:

$ probe 'innersimp::derivative(x!, x, 2)'
Stack overflow.
Repeated 3214 times: ...
   at AngouriMath.Entity.DifferentiateOnce(Variable)
   at AngouriMath.Entity.Differentiate(Variable, Int32)
   at AngouriMath.Entity+Derivativef+<>c__DisplayClass20_0.<InnerSimplify>b__0(...)

Derivativef's simplification decides whether a derivative can be taken by asking for it and keeping the node when a Derivativef comes back. That test is what terminates it. An overload that simplifies each pass makes it simplify the very node it is deciding about, which asks for the derivative again, forever.

So the raw loop stays, as an internal InnerDifferentiate(Variable, int) that Derivativef.InnerSimplify uses, and only the public overload simplifies. That is also the answer to the // TODO: should we call InnerSimplified here? sitting above that arm — no, and now there is a comment saying why.

And the raw variant must keep the negative-power branch

My first version of it did not, on the reasoning that "raw differentiation" has nothing to do with integration. Three tests disagreed:

DerivativeToIntegralSimplify   Expected: integral(apply(f, x), x)   Actual: apply(f, x)

A loop that never runs returns its input, and the input is not a Derivativef, so the caller took it for a resolved answer and the integral vanished. That is a wrong answer rather than a crash, so nothing but the suite would have caught it.

Measured

• Suite: 7378 passed, 0 failed, 14 skipped.
• Corpus: 116/119, 0 wrong, 0 error, 0 timeout, compared row by row — no verdict or answer changed.
• work/crashcheck: 1834 cases, 0 crashed, with seven cases added for this shape (both signs of the power, and a derivative that cannot be taken nested inside an expression).
• 12 new cases: the two routes agree at powers 0–3 over several expressions, the negative power still integrates, and a [Theory(Timeout = 30000)] guarding the overflow.
• BREAKING-CHANGES.md entry with both values measured on a build of each arm.

Against the other open PRs

Derived with git merge-tree --write-tree — this one is not doc-only:

against	conflicts
#990 fix/bound-name-must-be-symbolic	BREAKING-CHANGES.md + Evaluation.Continuous.Calculus.Classes.cs
#991 constant-node-984	BREAKING-CHANGES.md + Evaluation.Continuous.Calculus.Classes.cs
#997, #998, #1000, #1001	BREAKING-CHANGES.md only

Both source conflicts are one line each and additive — #990 adds a guard to the arm, #991 wraps the result in Core.Binding.Written, and this changes which method the arm calls. I resolved all three on a local integration branch of #1002 + #990 + #991 and ran it: 7467 passed, 0 failed. I will do the rebase whichever way they land.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KbKcbJP266A3EGyQ5kq7Ru