[skjerns]

Why

joblib.Memory keys its cache on the arguments a function was called with. For a function that takes a file path that is not enough: the path stays the same while the file behind it changes, and the stale result is returned forever.

What

file_signature(file) — the os.stat fingerprint of a file (size, mtime, ctime, inode), or of a list of files, without reading the contents. The same fields hash_file(use_cache=True) already used inline, which now builds its cache key from this instead.

cache_on_files(*params, memory=None) — decorator that computes that fingerprint on every call and folds it into the joblib cache key, where the body ignores it:

@cache_on_files('log_file', memory=config.cache_dir)
def log_to_triggers(log_file, condition):
    ...

A parameter may hold a list of files (order is part of the key), and the cache invalidates if any one of them changes.

Without parameter names, every argument that points at an existing file is picked up automatically. A path that does not exist yet is left alone and hashed as the plain string it is, so the result computed while it was missing is not reused once it appears:

@cache_on_files()
def check(recording, log_files, strict=True):
    ...

It can also be chained on top of a joblib cache, to take the Memory and its options from there:

@cache_on_files('filename')
@memory.cache
def func(filename): ...

This is not a chain of two caches. The fingerprint has to take part in the key that joblib computes, and joblib only hashes the arguments of the function it wraps itself — so the MemorizedFunc below is unwrapped and rebuilt with the same Memory rather than called through. ignore= on the inner cache is refused with an assertion instead of silently no longer ignoring anything.

.uncached (the original function) and .cached (the MemorizedFunc, e.g. for .clear()) are attached to the result.

Three details that are easy to get wrong

All three were found by the tests, not by reading:

1. @wraps on the cached function makes inspect follow __wrapped__ to the original signature, and joblib then rejects the extra argument. The identity is copied by hand instead — which is also why the function's source is hashed into the key explicitly, since joblib's own source-change invalidation would otherwise track the (never-changing) wrapper.
2. joblib cannot map keyword-only parameters of a *args function, so the fingerprint and source hash are leading positional parameters.
3. f(x) and f(x=x) are two separate cache entries unless the call is normalised to keyword form first.

Tests

43 new tests in tests/test_misc.py (TestFileSignature, TestCacheOnFiles, TestCacheOnFilesAuto, TestCacheOnFilesChained): invalidation on change, lists of files, positional vs keyword calls, two decorated functions not colliding in one cache dir, .uncached / .cached.clear(), auto-detection of non-paths and unusable strings, and the chained form landing in the right Memory.

pytest tests/test_misc.py → 92 passed, 2 failed. The two failures are in TestLongDfToArraySubsetColumns and are pre-existing — they fail on a clean main checkout too.

Note

The branch also carries the in-progress hash_file(use_cache=) and list_files changes that were sitting in the working tree; they live in the same regions of the same files, so they could not be separated into their own commit.

🤖 Generated with Claude Code

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.