Add reliable local test runner and repair workspace and LIMO fixtures

[arnodelorme]

Purpose

Make the installed local EEGLAB test suite reproducible without opening the MATLAB project, installing plugins, invoking nested runners or sending email. This is problem 1 from the local failure investigation. It also repairs the LIMO test fixtures needed to validate problem 5.

Changes

Add explicit test discovery, wildcard selection, an empty selection guard and a persistent manifest. Save MATLAB results, CSV, JUnit XML, individual completed results and the active test name. Preserve previous result directories. Restore the EEGLAB preference file, MATLAB path, working directory and default figure visibility.

Run the menu replay commands in a scoped base workspace fixture. Restore prior variables, global bindings, random state, warnings and existing EEGLAB windows, and write temporary menu datasets outside the source tree. Add assertions on the imported data and replayed state. Keep the canonical wrapper instead of separately discovering the same script.

Remove project only assumptions from the study and tutorial wrappers. Make the two LIMO wrappers discoverable. LIMO tests use explicit dataset roots and fresh output directories, resolve channel locations and model lists from returned references, preserve all outputs and full exceptions, and save first level preparation for independent second level reruns. Update the stale ANCOVA bootstrap filename. No source datasets or existing results are deleted.

Local validation

MATLAB R2025b with the user's installed plugins in disposable validation copies, with the related EEGLAB and LIMO patches applied only there.

The complete non LIMO run initially executed 809 unique cases. After correcting the project path assumptions, all 11 tutorial cases and the study test pass on targeted rerun. Eight new harness regressions also pass. The latest result per case is 816 passes and one failure across 817 non LIMO cases. This is a full run plus explicit targeted reruns, not a claim that one final uninterrupted run was entirely green.

The remaining failure is test_read_scan_files/test_read_scan_files, which requests the absent FieldTrip GSN-HydroCel-64_1.0.sfp template. The installed template has a different name. This is independent of the requested fixes and is reported rather than skipped.

The complete local manifest has 819 cases including the two LIMO workflows. Compared with the previous 812 result identifiers, the duplicate standalone menu script is consolidated, two supplemental LIMO calls become discoverable wrapper tests, and eight new harness tests are added. Legacy empty wrappers are still a coverage limitation.

Real LIMO OLS and WLS first level models and contrasts completed for 18 subjects, followed by all nine integration groups with the final input ordering correction. An independent artifact audit verifies 22 configurations, 48 observed and null TFCE map pairs with 101 bootstrap samples, and the expected input order for all 18 subjects in regression and repeated measures analysis.

The full preprocessing workflow also completed all stages for 18 subjects, including ASR, ICA, ICLabel, epoching, WLS, 1000 second level bootstrap samples, summaries, subject plots, contrasts, difference plots and result printing. Downstream stages were rerun from retained numerical outputs after the discovered popup fixes, so this is staged verification rather than one final uninterrupted green wrapper run. A final audit validates all model and contrast references, three null distributions with 1000 samples and saved summary artifacts. Printing completes with throwing dialog sentinels active and preserves an unrelated open figure. ICLabel used its built in MATLAB fallback.

The output writers use fprintf rather than the newer writelines, for compatibility with the older CI release. All three helper tests and the persistent progress writer pass after this change. The menu wrapper also passes independently with a populated base workspace preserved.

Full preprocessing validation additionally exposed a working directory assumption after limo_batch. The fixture now returns explicitly to its analysis directory before creating the remaining statistics and difference plots. Summary output filename capitalization also matches the actual files.

Related PRs

Event selection: sccn/eeglab#955

Colorbar handling: sccn/eeglab#956

LIMO implementation: LIMO-EEG-Toolbox/limo_tools#239

The EEGLAB submodule reference is deliberately unchanged. Full LIMO success requires the related implementation fixes. The original working checkout and its preexisting changes remain untouched.