Fixed control flow bugs : three bugs in how the ML pipeline finds and validates its input files - #36
Fixed control flow bugs : three bugs in how the ML pipeline finds and validates its input files#36eboyer221 wants to merge 2 commits into
Conversation
|
Some naming conventions need to be changed in |
The name change is done in amRdata so it is not required here. |
AbhirupaGhosh
left a comment
There was a problem hiding this comment.
The PR does what it says. I checked for the correct message for the user if LOO + cross test without any stratification is chosen.
|
@amcim reviewed this and added a second commit. The underlying issue is that stratify_by = NULL means two different things — "don't stratify" and "leave one drug out" — which is what tripped up the code in all three places. A real "drug" option is the proper fix down the line; the TODO notes point at that. The LOO check now only objects when someone passes a value that isn't "year" or "country". Typos still get caught, but leave-one-drug-out (passed as NULL) goes through. Known follow-up: The leave-one-drug-out + cross-testing branch now errors out early on purpose; the pairing code after it is kept as a scaffold with notes for the follow-up work next week. |
What
Three related bugs in run_ML.R:
runMLmodels() used to check "did I find any files to work with?" before doing anything else, and stop with a clear message if not. That check was missing, so running it before any files exist just crashed with a confusing error.
createMLinputList() figures out which files each model should train on. For one specific setup (leave-one-drug-out combined with cross-testing), it correctly built the right list of files but never handed that list back - it kept running and returned a different, wrong list instead, with no error.
There used to be a check that immediately stopped with a clear error if someone asked for leave-one-out (LOO) modeling without also saying whether to group by year or country. That check had been turned off, so this mistake wasn't caught - the code just ran and quietly came back with nothing.
Why it matters
Bug 1 meant a first-time run crashed instead of explaining what went wrong. Bug 2 meant one specific setup silently used the wrong training data. Bug 3 meant a bad setup wasn't caught up front, and just failed quietly later, which is confusing to debug.
Fix, and one wrinkle
Bugs 1 and 2 were simple restores. Bug 3 needed a bit more thought: turning that check back on exactly as it was would have also blocked the setup bug 2 fixes (leave-one-drug-out + cross-testing), since that setup doesn't use year/country grouping at all - it's a different kind of "leave one out." So the check now only applies when someone isn't also cross-testing. It still catches the original mistake, but no longer blocks the leave-one-drug-out case.
Testing
Added tests/testthat/test-run-ml-models.R covering all three:
Full test suite passes (207/207).
Found while reviewing #32. Covers the remaining items from the "pipeline control flow" part of Issue #33 (the runModelingPipelineIntense() item was split into its own PR).