Repository navigation
Capture printed-on-card and multi-image stimulus metadata (Identifying Pictures, Sounds, Story Recall v2) - #330
Merged
Merged
Conversation
…istry
Several image-based tasks encode their target on the shown image, where it was
being dropped from the sidecars (the metadata carried only the image URL):
- Identifying Pictures (peds): each of the 37 flashcards prints the target WORD.
Several are sight words ("MY", "GO", "ONE") that cannot be named from the
picture, so this is a read (WER-scorable) target. Transcribed once into an
index-keyed bank; speech_type -> read, stimulus_text = the word, kept the
pinned image asset.
- Noisy / Silly / Long Sounds (peds): each card prints the target sound token
(e.g. "aa aa aa", "puh puh puh.....", "Ahhhhhh... (take a breath)..."). These
stay non-lexical (a sound repeated to the time limit, not a WER target); the
verbatim token is captured as stimulus_text, image asset kept.
- Story Recall v2 (adult): the participant views a wordless 10-panel picture
story before retelling. Recorded as ONE scalar template URL (StoryRecall_{n}.jpg)
plus a companion stimulus_asset_n_images count -- never a list, so the sidecar
stays flat/parquet-friendly. Wordless => language-independent (same panels for
every language; only the reference narrative differs). The Spanish narrative
already resolves from the es-419 static bank.
Field-format contract documented in resources/metadata.json: an inapplicable
optional field is an absent key in a JSON sidecar and a null/None value in a
parquet/TSV column -- the two are equivalent. The metadata bundler casts
stimulus_asset_n_images to nullable Int64 so the count renders as an integer
(not 10.0) with missing rows as empty cells; proven by a round-trip test.
Tests: identifying-pictures read targets, sound-card tokens, story-recall
sequence + n_images, single-image tasks omit the count, and a bundling
TSV round-trip proving absent-key == null-cell. Task-prompt suite 39/39.
A general declared-dtype schema for all packaged artifacts (this Int64 cast is a
stopgap) is tracked in #329.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…banks
- Extract _raw_github_url so _asset_url and _asset_sequence_url build the
commit-pinned GitHub URL in one place (no drift in host/encoding/pinning).
- _asset_url now returns None for any '{n}'-templated path, so a multi-image
sequence can never be percent-encoded into a broken single URL regardless of
task ptype; _asset_sequence_url owns those. Regression test added.
- @lru_cache _bank so a bank's JSON is parsed once, not per recording (hot loop
in large pediatric builds).
Deliberately kept: the stimulus_asset_n_images Int64 cast in commands.py (a
stopgap folded into #329's general dtype schema), and _bank raising on a missing
vendored bank (fail-loud beats silently shipping empty stimulus_text).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
@Evan8456 Mostly standard, does a weird thing with story_recall where we use *_n for the image of the story so we don't have to use lists of URLs since I removed the list of prompts from before though we could return to that if you think this solution is a bit wack |
Contributor
|
seems reasonable. I think urls are more troublesome long term, might be the less of 2 evils. |
Evan8456
approved these changes
Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Capture stimulus targets that live on the shown image for several tasks — previously the sidecars carried only the image URL (or nothing), dropping the actual target.
speech_type: read,stimulus_text= the word (WER-scorable; several are sight words like "MY"/"GO"/"ONE" that can't be named from the picture), image asset kept"aa aa aa","puh puh puh.....","Ahhhhhh... (take a breath)...") → staysnon-lexical, verbatim token asstimulus_text, image asset keptStoryRecall_{n}.jpg+ companionstimulus_asset_n_imagescountDesign points
stimulus_asset_n_imagesis present only for sequences; its absence is the documented signal thatstimulus_assetis a single, directly-resolvable image.metadata.json: an inapplicable optional field is an absent key in a JSON sidecar and a null/None value in a parquet/TSV column — the two are equivalent.stimulus_textdiffers (Spanish narrative already resolves from the es-419 static bank).Int64so it renders as10(not10.0) with missing rows as empty cells.How the words/tokens were obtained
Transcribed directly from the pinned redcap images (no OCR pipeline) and hand-verified.
Tests
tests/test_task_prompts.py— identifying-pictures read targets, sound-card tokens, Story Recall sequence +n_images, single-image tasks omit the count, and a bundling TSV round-trip proving absent-key == null-cell and the integer render. 39/39 pass.Follow-up
The
Int64cast is a stopgap; a general declared-dtype schema for all packaged artifacts (TSV + parquet) is tracked in #329.