Repository navigation
Add pyfmi.checks: directional-derivative vs finite-difference check, and result comparison - #425
Draft
hubertus65 wants to merge 3 commits into
Draft
hubertus65 wants to merge 3 commits into
hubertus65 wants to merge 3 commits into
Conversation
… for ME FMUs
check_jacobian(model, times=...) evaluates PyFMI's coloured Jacobian three ways
at the current state and at further points on a CVode trajectory -- from the
FMU's directional derivatives, by forward differences and by central
differences -- and reports entries the directional derivatives leave out
(declared in the dependency structure, zero in the DD, clearly nonzero in the
central differences) or get wrong, the relative Frobenius norm of the
difference, and the worst entries by state name. Entries where forward and
central differences disagree (kinks next to the point) are excluded from the
verdict. Model time, states and force_finite_differences are restored.
python -m pyfmi.checks jacobian model.fmu --time 3.0
Motivation: an exported vehicle model (141 states) whose directional
derivatives return zero for 692 of the 7528 declared entries -- every
acceleration with respect to positions and velocities -- while forward and
central differences agree to all digits. Newton-based solvers still converge
on such a Jacobian, only slowly: with it Radau5ODE needed 1493 Jacobian
evaluations and 379 s for a 13 s simulation, with a finite-difference Jacobian
126 and 21 s; CVode 124 vs 62 Jacobians. The check finds this in seconds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…variable group
compare_results(ref, other) samples both results on the reference grid (last
stored value at duplicate times, i.e. the post-event value), scales each
variable by max(max|ref|, floor) and reports the deviation per variable and
per group -- by default controller-like names (control, regulat, PI, PID,
limiter, integrator, ...) against the rest, overridable by regex -- with
max, median, the count above a threshold and the worst variables with the
time of the worst deviation. Accepts PyFMI result objects, (t, {name: y})
pairs or (t, Y, names) triples; the CLI reads .mat/.txt result files:
python -m pyfmi.checks compare ref.mat other.mat --vars '^hrsg' --group plant='^hrsg|^turbine'
Motivation: on a 318-state combined-cycle plant the max-norm deviation from
an rtol 1e-8 reference was 6.7e-2 for CVode at rtol 1e-6 -- entirely in one
level-control integrator sitting at its limiter (a state with no feedback
to the plant, so its local error accumulates unchecked) -- while the 307
plant states agreed to 3e-3 (median 1e-5). A max-norm alone reads that as
"the solver is off by 7 %"; per group it reads as "the plant is resolved,
one controller integrator is not", which is what decides whether a solver
or a tolerance is acceptable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
OCT-generated FMUs carry the runtime parameters '_block_jacobian_check' and '_log_level'. With both set before initialization, every directional- derivative evaluation logs a <SolverInvocation> per nonlinear block and a <SingularJacobian ... dir_block="N"> warning when the linearized block system of the directional-derivative solve is singular -- the case in which the FMU returns fmi2OK with zero sensitivities for everything downstream of the block. enable_oct_block_jacobian_check(model) switches this on (no-op on other FMUs), count_oct_block_warnings(log) tallies the warnings per block, and 'python -m pyfmi.checks jacobian model.fmu --oct-block-check' prints them next to the DD-vs-FD verdict. On the vehicle export of the previous commit this names block 36 (45 iteration variables, 3417 solved variables) 141 times at t = 0, once per Jacobian column. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A new module
pyfmi.checkswith two diagnostics for ME FMUs, three commits, all new files(
src/pyfmi/checks.py,tests/test_checks.py) — nothing existing is touched.pyfmi.checks jacobianCompares an FMU's directional derivatives against forward and central finite differences along
a trajectory, not just at the initial point, and reports missing, wrong and kink entries by
variable name rather than by matrix index.
--oct-block-checkadditionally calls the OCTruntime's own
_block_jacobian_check.What it was written for, and what it found: an export whose directional derivatives returned
zeros for 692 declared entries because of a singular
dir_blocksolve inside the FMU. Nothingin the FMI interface reports that — the calls succeed — and the symptom at the solver level is a
Newton method that converges slowly for no visible reason.
pyfmi.checks compareDeviation between two result files per variable group. Written because a max-norm over all
variables is dominated by whichever controller integrator is sitting on a limiter, which hides
what the plant states are doing; grouping separates them.
Why it belongs in PyFMI
Both answer "is this FMU's Jacobian usable, and did this run actually agree with that one" —
questions a PyFMI user hits before they can sensibly file a bug against a solver or an exporting
tool. Today that means writing the comparison by hand each time.
Review notes
new submodule.
tests/test_checks.py, 10 tests, passing.🤖 Generated with Claude Code
https://claude.ai/code/session_01PyrxGeWuwCmAH9LishMW6p