Skip to content

Cache template features per query in TemplateHitFeaturizer - #22

Open
ssiddhantsharma wants to merge 2 commits into
aurekaresearch:mainfrom
ssiddhantsharma:cache-template-features
Open

ssiddhantsharma wants to merge 2 commits into
aurekaresearch:mainfrom
ssiddhantsharma:cache-template-features

Conversation

@ssiddhantsharma

@ssiddhantsharma ssiddhantsharma commented Aug 13, 2026 •

Copy link
Copy Markdown

TemplateHitFeaturizer.get_templates runs the full template pipeline, prefilter, dedup, remote mmCIF fetch (PDBe), and kalign — on every call. In iterative / optimization-in-the-loop inference (e.g. hallucination), the same fixed target chain is featurized on every optimization step, so this identical work (including the network fetch) repeats hundreds of times per trajectory.

With fetch_remote=True a single step can block for tens of minutes when PDBe is slow (up to ~20 candidate requests.get(..., timeout=30) + kalign, re-run each step). Observed in a run as a 34-minute gap between two consecutive featurization steps of the same trajectory:

[11:47:45] opendde.data.inference.infer_dataloader: Featurizing design...
[12:21:27] opendde.data.inference.infer_dataloader: Featurizing design...

Fix

Memoize get_templates on (sequence_uid, query_sequence) — computed once per unique query, reused thereafter. Single-shot inference is unaffected (each sequence is featurized once, so the cache never hits).

After the change, featurization steps run in seconds:

[19:22:55] infer_dataloader: Featurizing design...
[19:23:09] infer_dataloader: Featurizing design...
[19:23:19] infer_dataloader: Featurizing design...

Notes

  • The cache lives on the featurizer instance and is bounded by the number of unique query sequences it sees.
  • make_template_feature calls get_templates with max_template_date=None; a caller that varies max_template_date per call for the same sequence should add it to the key.
  • Orthogonal to Template search silently no-ops in several cases (empty templates, no error raised) #20 (silent no-op on empty templates): this only avoids recomputation and does not change the empty-template path.

get_templates re-runs the full template search (remote mmCIF fetch + kalign)
on every call. In iterative inference (e.g. hallucination) the same fixed
target chain is featurized every optimization step, repeating identical
remote fetches and stalling the loop. Memoize by (sequence_uid, query_sequence)
so templates are computed once per unique query.
@ssiddhantsharma

ssiddhantsharma commented Sep 3, 2026 •

Copy link
Copy Markdown
Author

Memoizes template features per (sequence_uid, query_sequence) in TemplateHitFeaturizer.get_templates, which removes a repeated remote-template-fetch stall in iterative / optimization-in-the-loop inference (e.g. hallucination), where the same fixed target chain is re-featurized on every step. I saw a 34-minute gap between two consecutive featurization steps of one trajectory when PDBe was slow. It's small (+8/-1), now rebased on v1.1.1 and mergeable.

ssiddhantsharma pushed a commit to ssiddhantsharma/jopendde that referenced this pull request Sep 3, 2026
… OpenDDE#22)

Memoizes TemplateHitFeaturizer.get_templates by (sequence_uid, query_sequence),
removing a repeated remote-template-fetch stall in iterative/hallucination
inference where the same fixed target chain is re-featurized every step (a
34-minute gap between consecutive featurization steps was observed when PDBe
was slow). This is the same 8-line fix as aurekaresearch/OpenDDE#22; carried in
the vendored copy here until that lands upstream and flows in via a re-vendor.
@ssiddhantsharma

ssiddhantsharma commented Sep 26, 2026 •

Copy link
Copy Markdown
Author

Gentle bump, is there anything needed from my side (rebase, tests, or scoping) to move it forward?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant