[atao2004]

Closes #91

Summary

Migrates the Haystack + Valkey cookbook from the archive/main-pre-reorg branch, fully updated to conform with CONTRIBUTING.md guidelines (PR #40).

Depends on #45 (semantic-caching) — this branch is based on cookbook-semantic-caching and extends its shared cookbook-python.yml CI workflow. Merge #45 first.

Changes from the Original (archive/main-pre-reorg)

• Replaced sentence-transformers embedder with OllamaDocumentEmbedder / OllamaTextEmbedder (vendor neutral, no API key)
• Replaced OpenAIChatGenerator (paid) with OllamaChatGenerator as default (OpenAI in collapsible optional section)
• Pinned Valkey to valkey/valkey-bundle:9.1.0 (not :latest)
• Added security callout, audience line, config reference, teardown sections
• Added sample/ directory with runnable main.py and CI-friendly test_haystack.py
• Tests use Haystack's built-in MockDocumentEmbedder / MockTextEmbedder — no Ollama needed in CI
• Extended cookbook-python.yml matrix with framework-integrations/haystack
• Requires Python 3.10+ (all three packages require it)

Cookbook Structure

cookbooks/framework-integrations/haystack/
├── README.md                    # Index with 2-cookbook table
├── 01-getting-started.md        # Beginner · ~15 min
├── 02-rag-pipeline.md           # Intermediate · ~20 min
└── sample/
    ├── README.md                # How to run + expected output
    ├── docker-compose.yml       # Valkey Bundle 9.1.0 with healthcheck
    ├── requirements.txt         # haystack-ai>=3.0.0, valkey-haystack==0.2.2, ollama-haystack>=6.0.0
    ├── main.py                  # Full RAG demo (requires Ollama)
    ├── test_haystack.py         # CI tests with mock embedders (no Ollama)
    └── .gitignore

Technical Details

Component	Choice	Rationale
Embedding model	nomic-embed-text (768-dim, 274MB)	Matches ValkeyDocumentStore default
Chat model	llama3.2:1b (1.3GB)	Smallest instruction-tuned model for RAG
Embedder components	OllamaDocumentEmbedder + OllamaTextEmbedder	Free, local, no API key
Generator	OllamaChatGenerator	Free, local, vendor neutral
Valkey image	valkey/valkey-bundle:9.1.0	Required for Search + JSON modules
CI test strategy	MockDocumentEmbedder / MockTextEmbedder	Deterministic, no model downloads in CI
Client library	valkey-glide (via valkey-haystack)	Pre-built manylinux wheels for cp311/x86_64

Validation

• Markdown lint: 0 issues across all 4 .md files
• Python syntax: Both .py files compile cleanly
• QA review: 1 critical finding caught and fixed (MockDocumentEmbedder param name), all others clean

Vendor Neutrality

• No paid API dependencies in the default path
• Ollama runs entirely locally (nomic-embed-text for embeddings, llama3.2:1b for chat)
• OpenAI shown as optional alternative in a collapsible <details> section
• valkey-haystack is the official Haystack integration (deepset-maintained)

Known Risks

• haystack-ai>=3.0.0 requirement (MockDocumentEmbedder only exists in 3.0+). If valkey-haystack==0.2.2 has runtime incompatibilities with 3.0, fallback: use pre-computed embeddings directly on Document objects instead of mock embedders.
• ValkeyDocumentStore uses JSON storage (not Hash) — requires both search and json Valkey modules (hence bundle image).

Relationship to Other PRs

• docs+ci: add semantic-caching cookbook with Python CI (MERGE FIRST FOR PYTHON CI) #45 (semantic-caching) — base branch, provides shared cookbook-python.yml
• docs: strengthen CONTRIBUTING.md with acceptance criteria and review process #40 (CONTRIBUTING.md) — defines the structure this cookbook follows

[MatthiasHowellYopp]

COMMENT — Large, well-structured PR with two cookbooks (Haystack + Semantic Caching). Security posture is strong: FT.SEARCH uses parameterized values, TAG filters sanitized, TEXT search escaped. 9 substantive findings below (all warnings, none blocking).

[MatthiasHowellYopp]

Re-review: All findings addressed ✅

Commit 53fe53ff resolves all 9 findings from the initial review:

1. ✅ Key injection — store_conversation_cache now sanitizes user_id before using it in the cache key
2. ✅ Port binding — both docker-compose.yml files bind to 127.0.0.1
3. ✅ Unpinned dep — ollama-haystack pinned to ==6.5.0
4. ✅ Client never closed — client.close() called at end of main()
5. ✅ Test fixture client — client.close() in fixture teardown
6. ✅ Missing version requirement — README now states Valkey Bundle 9.0+
7. ✅ Config reference — request_timeout added to table
8. ✅ Cross-test ordering — self-contained tests
9. ✅ Workflow self-trigger — .github/workflows/cookbook-python.yml in paths filter

Additional positives: env var overrides in main.py, try/finally cleanup pattern, failure-path test (cleanup_closes_store_when_deletion_fails), CI tests against both 8.1.4 and 9.1.1.

Verdict: Approve — will formally approve once undrafted.

[jbrinkman]

Cookbook Review — Haystack + Semantic Caching

[AI Review] Reviewed against CONTRIBUTING.md standards:

• ❌ Directory structure & naming
• ✅ Issue context & requirements verification
• ✅ Upstream Valkey support verification
• ✅ Markdown structure & mandatory elements
• ✅ Acceptance criteria & vendor neutrality
• ✅ Sample code quality
• ✅ CI validation

Summary

• Total findings: 3
• Blocking: 2
• Suggestions: 1

Key Issues

• ❌ Missing python/ language directory for framework-integrations/haystack/
• ❌ Missing python/ language directory for use-cases/semantic-caching/
• ⚠️ Inline docker run commands bind to all interfaces instead of localhost

Please address the blocking issues above before merging.

[jbrinkman]

The use-cases/python/semantic-caching cookbook should be in it's own PR and not included in the Haystack cookbook. Please split this into a separate PR and move to the Bit-quill repo as a draft.

[MatthiasHowellYopp]

One small naming nit on the sample — non-blocking. Thanks for the embedder rework and the cookbook isolation; both look great.

[MatthiasHowellYopp]

Re-reviewed at f766fbf against the current cookbook guidelines. This is in strong shape — real SentenceTransformers embedders (no more deterministic stub), all deps pinned, valkey_store naming, try/finally + close(), structured filter dicts (no injection), 127.0.0.1 binding, 9.1.2, and a vendor-neutral OpenRouter alternative in a <details>. I've resolved my earlier threads that this commit addressed (ollama-haystack pin, request_timeout in the config table, the store→valkey_store rename) and resolved the semantic-caching threads since that cookbook was split into its own draft.

Two minor, non-blocking polish items below (same single-source-the-version / standalone-runnable pattern we're applying across the cookbooks), plus one scope observation:

Scope note (judgment call, no change requested): Under our "scope each recipe to Valkey's role" guideline, generation is normally handed off with a pointer since Valkey's role ends at retrieval. 02 keeps the full ChatPromptBuilder/OllamaChatGenerator generation — and I think that's defensible here in a way it wasn't for langchain4j: there the generation was dead commented-out code; here it's live, local, and the composable Pipeline wiring (retriever.documents → prompt_builder.documents → generator) is the Haystack integration story. Flagging only so the choice is deliberate — your call whether to keep the full generator or trim to the retriever hand-off + a pointer to Haystack's generator docs.

[Jonathan-Improving]

APPROVE — Well-structured Haystack + Valkey cookbook.

Reviewed: All 10 files (4 markdown docs, sample code, tests, docker-compose, requirements, .gitignore, markdownlint config).

Highlights:

• Correct haystack-ai 2.x import paths and component API usage
• Proper GLIDE lifecycle management via valkey-haystack (try/finally + close())
• Security callout present; docker binds to 127.0.0.1 only
• Internal markdown cross-links verified (01↔02, README→both)
• Tests use real SentenceTransformers embeddings (384-dim) — deterministic without external services
• Clean separation: sample uses sentence-transformers (no Ollama needed), cookbook docs teach Ollama workflow
• All dependencies pinned to exact versions
• Filter dict format ({"field": ..., "operator": ..., "value": ...}) correct for Haystack 2.x

No blocking findings. Previous reviewer concerns (directory structure, localhost binding, store naming) all addressed in this revision.

[jbrinkman]

Closing this PR as the cookbook work is being tracked internally. The branch and all commits have been preserved. Thank you for the contribution!