Skip to content

Fixed control flow bugs : three bugs in how the ML pipeline finds and validates its input files - #36

Open
eboyer221 wants to merge 2 commits into
mainfrom
dev-run-ml-input-fixes
Open

Fixed control flow bugs : three bugs in how the ML pipeline finds and validates its input files#36
eboyer221 wants to merge 2 commits into
mainfrom
dev-run-ml-input-fixes

Conversation

@eboyer221

Copy link
Copy Markdown
Contributor

What

Three related bugs in run_ML.R:

  1. runMLmodels() used to check "did I find any files to work with?" before doing anything else, and stop with a clear message if not. That check was missing, so running it before any files exist just crashed with a confusing error.

  2. createMLinputList() figures out which files each model should train on. For one specific setup (leave-one-drug-out combined with cross-testing), it correctly built the right list of files but never handed that list back - it kept running and returned a different, wrong list instead, with no error.

  3. There used to be a check that immediately stopped with a clear error if someone asked for leave-one-out (LOO) modeling without also saying whether to group by year or country. That check had been turned off, so this mistake wasn't caught - the code just ran and quietly came back with nothing.

Why it matters

Bug 1 meant a first-time run crashed instead of explaining what went wrong. Bug 2 meant one specific setup silently used the wrong training data. Bug 3 meant a bad setup wasn't caught up front, and just failed quietly later, which is confusing to debug.

Fix, and one wrinkle

Bugs 1 and 2 were simple restores. Bug 3 needed a bit more thought: turning that check back on exactly as it was would have also blocked the setup bug 2 fixes (leave-one-drug-out + cross-testing), since that setup doesn't use year/country grouping at all - it's a different kind of "leave one out." So the check now only applies when someone isn't also cross-testing. It still catches the original mistake, but no longer blocks the leave-one-drug-out case.

Testing

Added tests/testthat/test-run-ml-models.R covering all three:

  • running with no files yet exits cleanly instead of crashing
  • the check still catches LOO without year/country when not cross-testing
  • leave-one-drug-out + cross-testing now correctly returns the right file list instead of an empty one

Full test suite passes (207/207).

Found while reviewing #32. Covers the remaining items from the "pipeline control flow" part of Issue #33 (the runModelingPipelineIntense() item was split into its own PR).

@AbhirupaGhosh

Copy link
Copy Markdown
Contributor

Some naming conventions need to be changed in run_ML.R to keep consistency with generating the ML matrices. I will work on that before merging this PR.

@AbhirupaGhosh

Copy link
Copy Markdown
Contributor

Some naming conventions need to be changed in run_ML.R to keep consistency with generating the ML matrices. I will work on that before merging this PR.

The name change is done in amRdata so it is not required here.

@AbhirupaGhosh AbhirupaGhosh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR does what it says. I checked for the correct message for the user if LOO + cross test without any stratification is chosen.

@eboyer221

Copy link
Copy Markdown
Contributor Author

@amcim reviewed this and added a second commit.

The underlying issue is that stratify_by = NULL means two different things — "don't stratify" and "leave one drug out" — which is what tripped up the code in all three places. A real "drug" option is the proper fix down the line; the TODO notes point at that.

The LOO check now only objects when someone passes a value that isn't "year" or "country". Typos still get caught, but leave-one-drug-out (passed as NULL) goes through.
The matrix path fix sends leave-one-drug-out to LOO_matrix_drug, where those files actually live. New tests confirm every combination of settings maps to a folder that exists.
Both test changes are improvements. The old test only passed because a drug was literally named "leaveout"; a normal drug name found nothing.
Full test suite passes locally (217/217).

Known follow-up: The leave-one-drug-out + cross-testing branch now errors out early on purpose; the pairing code after it is kept as a scaffold with notes for the follow-up work next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants