Skip to content

Migrate cuda_pathfinder/_dynamic_libs search modules from os.path to pathlib - #2489

Open
LeSingh1 wants to merge 1 commit into
NVIDIA:mainfrom
LeSingh1:pathlib/dynamic-libs
Open

Migrate cuda_pathfinder/_dynamic_libs search modules from os.path to pathlib#2489
LeSingh1 wants to merge 1 commit into
NVIDIA:mainfrom
LeSingh1:pathlib/dynamic-libs

Conversation

@LeSingh1

@LeSingh1 LeSingh1 commented Aug 4, 2026

Copy link
Copy Markdown

Part 1 of the series proposed in #2410: cuda_pathfinder/cuda/pathfinder/_dynamic_libs/search_platform.py and search_steps.py. Deliberately small so the conversion style and the compatibility approach can be reviewed before the remaining six parts follow the same template.

What changed

Path construction, joining, and filesystem predicates now go through Path/PurePath instead of os.path string manipulation. search_platform.py no longer imports os at all.

Compatibility

Treated as a hard constraint, per the discussion on the issue. Every entry point still accepts str, and every function that documents or returns str still returns strPath is used strictly as the internal representation and converted back with str() at each return. SearchPlatform, FindResult.abs_path, and the LoadedDL.abs_path that reaches users are unchanged in both type and value. No signature changes.

Deliberately not converted

Rather than folding these in silently, both are left alone with a comment saying why:

  • _find_lib_dir_using_anchor keeps os.path.normpath. pathlib has no equivalent because PurePath intentionally does not collapse .., so converting would change the path reported for an anchor such as a CUDA_PATH containing ...
  • _derive_ctk_root_windows keeps ntpath. ntpath.dirname slices its input and preserves the caller's separators; PureWindowsPath rewrites them to backslashes. derive_ctk_root also reaches this function on Linux, so converting would return an unusable root (\opt\foo) for a POSIX path whose parent directory happens to be named bin.

glob.glob is also retained where wildcard expansion is needed — Path.glob orders results differently, and the reverse-sorted newest-first policy from #1732 depends on the current string sort.

Happy to revisit any of these if you'd prefer them handled differently; that decision will set the pattern for the rest of the series.

Verification

No behavior change intended. Beyond the test suite, I fuzzed the old and new implementations against each other: 200k randomized path shapes for the derive_ctk_root helpers, and 3000 randomized real directory trees (including trailing-slash, // and /./ spellings) for the search helpers. The only string-level divergence is that a redundant . component in an input directory is now collapsed — the new string is the normalized form of the old and names the same file — and that input cannot occur, since lib_dir always arrives already normpath-ed.

pre-commit run passes every hook at the pinned versions; mypy is clean over cuda_pathfinder/cuda.

Since I don't have CUDA hardware, I verified the suite on Linux CI rather than locally. Against upstream/main the full pytest output is byte-identical apart from elapsed time — same 1188 passed, 4 skipped, and the same 88 Not found errors for absent NVIDIA libraries, in the same order.

os.path is soft-deprecated in favor of the type-safe pathlib API (NVIDIA#2410).
Convert the two filesystem-search modules under _dynamic_libs so path
construction, joining, and predicates go through Path/PurePath instead of
os.path string manipulation, which makes it explicit which values are paths
and which are the plain strings the search cascade accumulates for error
reporting.

Public compatibility is treated as a hard constraint: every entry point still
accepts str, and every function that documents or returns str still returns
str. Path is used strictly as the internal representation and converted back
with str() at each return, so SearchPlatform, FindResult.abs_path, and the
LoadedDL.abs_path that reaches users are all unchanged in type and value.

Two sites are deliberately left on the old API rather than converted silently,
each with a comment explaining why:

- _find_lib_dir_using_anchor keeps os.path.normpath. pathlib has no equivalent
  because PurePath intentionally does not collapse "..", so converting would
  change the path reported for an anchor such as a CUDA_PATH containing "..".

- _derive_ctk_root_windows keeps ntpath. ntpath.dirname slices its input and
  preserves the caller's separators, while PureWindowsPath rewrites them to
  backslashes. derive_ctk_root also reaches this function on Linux, so
  converting would return an unusable root for a POSIX path whose parent
  directory is named "bin".

glob.glob is also kept where wildcard expansion is needed; Path.glob differs in
result ordering, and the reverse-sorted newest-first policy from NVIDIA#1732 depends
on the current string sort.

No behavior change is intended. The only string-level difference is that
redundant "." components in an input directory are now collapsed, which cannot
occur for the values these functions actually receive.

Signed-off-by: LeSingh1 <sshaurya914@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the cuda.pathfinder Everything related to the cuda.pathfinder module label Aug 4, 2026
@LeSingh1

LeSingh1 commented Aug 4, 2026

Copy link
Copy Markdown
Author

@mdboom — this is the part 1 I offered on #2410, sent as a small sample so the conversion style and the compatibility approach can be checked before the remaining six parts follow the same template. Happy to hold the rest until you've had a look, or to drop this entirely if you'd rather assign it elsewhere.

Two notes for whoever picks it up:

pr-metadata-check needs a milestone and I can't set one as an outside contributor — the cuda.pathfinder label applied but the milestone didn't. cuda.pathfinder next looks like the right one.

On verification: I don't have CUDA hardware, so I ran the suite on Linux CI rather than locally. Rather than just matching the pass count, I diffed the full pytest output against upstream/main with timestamps stripped — it's byte-identical apart from elapsed time, including the same 88 Not found errors for absent NVIDIA libraries in the same order. I also fuzzed the old and new implementations against each other (200k randomized path shapes for the derive_ctk_root helpers, 3000 randomized directory trees for the search helpers), which is what surfaced the two sites I left on os.path/ntpath and documented in the description — the ntpath one is a genuine functional regression on Linux, not a style preference.

@mdboom

mdboom commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

A few implementation comments:

Treated as a hard constraint, per the discussion on the issue. Every entry point still accepts str, and every function that documents or returns str still returns str — Path is used strictly as the internal representation and converted back with str() at each return. SearchPlatform, FindResult.abs_path, and the LoadedDL.abs_path that reaches users are unchanged in both type and value. No signature changes.

This only needs to apply to functions in the public API. We are free to change the implementation details of internal function calls as we see fit, so we should prefer pathlib.Path wherever feasible.

glob.glob is also retained where wildcard expansion is needed — Path.glob orders results differently, and the reverse-sorted newest-first policy from #1732 depends on the current string sort.

All uses of glob.glob are passed to sorted anyway, so why does the sorting order matter? In the changes here we are paying a penalty of converting between Path and str objects, so it would be preferable to stay in the pathlib.Path space if possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.pathfinder Everything related to the cuda.pathfinder module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants