Skip to content

PRD: Migrate C++ baseline extraction to Rust-owned indexing #678

Description

@jununfly

Problem Statement

ZCodeGraph is migrating supported source languages from TypeScript-owned fallback extraction to Rust-owned per-file indexing. C has already moved to the Rust-owned path, but C++ is still TypeScript-owned under rust-hybrid.

From a maintainer and user perspective, this leaves a mixed ownership gap: C++ source files still depend on the TypeScript fallback extractor even though the adjacent C migration has established the Rust-owned pattern, header classification boundary, metadata expectations, fixture style, and corpus validation workflow.

The target in the Rust-owned migration roadmap is:

C++: functions, classes, structs, enums, typedefs/aliases, includes, calls, namespace/member boundary.

This PRD turns that roadmap checklist item into an implementation-ready issue. The work should migrate C++ baseline extraction to Rust-owned indexing, validate it on a suitably sized GitHub project with real C++ coverage, and remove the migrated TypeScript-owned C++ extraction path after success.

Solution

Add Rust-owned C++ baseline extraction to rust-hybrid indexing.

The Rust indexer should recognize C++ source and header extensions, parse C++ with a tree-sitter C++ parser, emit schema-compatible graph facts for the baseline C++ constructs, and expose C++ as Rust-owned in rust-hybrid metadata.

After fixture and real-corpus validation prove the Rust-owned path is sufficient for this migration slice, remove the corresponding TypeScript-owned C++ extractor implementation and registry entry. The TypeScript shell should remain responsible for cross-language orchestration, finalization, diagnostics, import resolution, and any framework/runtime semantics not explicitly migrated in this PRD.

The implementation should preserve the current C/C++/Objective-C header classification boundary. C++-looking .h files should be classified as C++ and handled by Rust-owned C++ indexing; C-looking .h files should continue through Rust-owned C indexing; Objective-C-looking .h files should remain Objective-C.

User Stories

  1. As a C++ library user, I want .cpp files indexed by the Rust-owned path, so that C++ projects do not depend on TypeScript fallback extraction.
  2. As a C++ application user, I want .cc files indexed by the Rust-owned path, so that common Unix-style C++ repositories are covered.
  3. As a C++ application user, I want .cxx files indexed by the Rust-owned path, so that alternate C++ source suffixes are covered.
  4. As a C++ header user, I want .hpp files indexed by the Rust-owned path, so that header-declared types and functions are visible in the graph.
  5. As a C++ header user, I want .hxx files indexed by the Rust-owned path, so that alternate header suffixes are covered.
  6. As a mixed C/C++ user, I want C++-looking .h files to be classified as C++, so that C++ declarations in headers are not treated as C.
  7. As a mixed C/C++ user, I want C-looking .h files to remain classified as C, so that the C migration does not regress.
  8. As an Objective-C user, I want Objective-C-looking .h files to remain Objective-C, so that C++ migration does not steal Objective-C headers.
  9. As a C++ developer, I want free functions indexed, so that search, callers, callees, and explore can find procedural entry points.
  10. As a C++ developer, I want class declarations indexed, so that object-oriented APIs appear as first-class graph nodes.
  11. As a C++ developer, I want struct declarations indexed, so that data types and POD-style APIs are visible.
  12. As a C++ developer, I want enum declarations indexed, so that domain constants and state machines are visible.
  13. As a C++ developer, I want enum members indexed, so that individual constants can be searched and connected.
  14. As a C++ developer, I want typedef declarations indexed, so that legacy type aliases are visible.
  15. As a C++ developer, I want using alias declarations indexed, so that modern type aliases are visible.
  16. As a C++ developer, I want local includes indexed as import nodes and unresolved import references, so that header dependencies can resolve later.
  17. As a C++ developer, I want system includes indexed as import nodes and unresolved import references, so that dependency shape remains visible even when external headers are not in the repo.
  18. As a C++ developer, I want direct function calls indexed as unresolved call references, so that reference resolution can connect calls to functions.
  19. As a C++ developer, I want method/member calls indexed with a useful target name, so that common call relationships can resolve when the target symbol is present.
  20. As a C++ developer, I want namespace declarations represented or preserved in qualified names, so that duplicate names in different namespaces can be disambiguated.
  21. As a C++ developer, I want class member functions indexed as methods when possible, so that class APIs are navigable.
  22. As a C++ developer, I want out-of-class method definitions recognized with their owning receiver where possible, so that Foo::bar does not collapse into an unqualified free function.
  23. As a C++ developer, I want constructors and destructors represented as methods, so that class lifecycle APIs are visible.
  24. As a C++ developer, I want nested namespaces and nested types to preserve enough qualified context, so that search results are useful in real codebases.
  25. As a C++ developer, I want the extractor to avoid obvious tree-sitter macro misparse false positives, so that macro-heavy files do not create junk symbols.
  26. As a C++ developer, I want the extractor to continue extracting useful symbols from parse trees that contain recoverable C++ parse errors, so that macro-heavy real projects are not dropped wholesale.
  27. As a maintainer, I want rust-hybrid metadata to report cpp as Rust-owned, so that status and doctor explain the new ownership model.
  28. As a maintainer, I want C++ fallback count to drop to zero for migrated C++ files, so that the roadmap acceptance criteria can be verified.
  29. As a maintainer, I want C++ parse or extraction gaps to surface as Rust-owned diagnostics, so that maintainers can distinguish Rust-owned gaps from language-level TypeScript fallback.
  30. As a maintainer, I want a fixture test covering core C++ syntax, so that future changes cannot silently regress baseline C++ extraction.
  31. As a maintainer, I want a header classification test covering C, C++, and Objective-C .h files, so that the migration boundary remains explicit.
  32. As a maintainer, I want a real GitHub C++ corpus validation artifact, so that the migration is backed by evidence beyond synthetic fixtures.
  33. As a maintainer, I want the selected corpus to be suitably sized, so that it is meaningful but does not turn the migration issue into a benchmark project.
  34. As a maintainer, I want failed first corpus attempts to be recorded as boundary evidence, so that stress-corpus discoveries are not lost.
  35. As a maintainer, I want the TypeScript-owned C++ extractor removed after successful validation, so that ownership is not duplicated.
  36. As a maintainer, I want C extraction to remain Rust-owned after C++ migration, so that the previous C migration does not regress.
  37. As a maintainer, I want Objective-C extraction to remain TypeScript-owned unless explicitly migrated, so that C++ migration does not broaden scope.
  38. As an agent using ZCodeGraph, I want C++ symbols and call relationships available through the same search and explore APIs, so that I do not need to read files manually for basic C++ structure.
  39. As an agent using ZCodeGraph, I want no change to MCP tool semantics, so that the ownership migration is transparent at query time.
  40. As a release maintainer, I want a changelog entry under Unreleased, so that users can discover the C++ Rust-owned indexing improvement.

Implementation Decisions

  • Add Rust-owned C++ source language support in the Rust core.
  • Add the tree-sitter C++ parser dependency and parser setup to the Rust-owned indexing path.
  • Register C++ source extensions for Rust-owned indexing: .cpp, .cc, .cxx, .hpp, .hxx, and content-classified C++ .h.
  • Keep C/C++/Objective-C .h classification explicit and content-aware.
  • Emit file nodes with language cpp.
  • Emit baseline C++ symbols for functions, classes, structs, enums, enum members, typedefs, and aliases.
  • Emit import nodes and unresolved imports references for C++ include directives.
  • Emit unresolved calls references for direct calls and member/function call expressions where the target name can be extracted safely.
  • Preserve or derive useful qualified names for namespace and member contexts.
  • Treat namespace/member boundaries as baseline extraction scope, not framework/runtime semantics.
  • Continue extracting useful facts from recoverable C++ parse trees where practical; do not mark an entire C++ file as failed merely because macro-heavy syntax creates recoverable parse nodes if useful symbols can still be extracted.
  • Preserve Rust-owned gap diagnostics for true parse/extraction failures.
  • Add cpp to rust-hybrid Rust-owned language metadata only after Rust-owned extraction is wired and covered.
  • Remove the TypeScript-owned C++ extractor registration and migrated implementation after fixture and corpus validation pass.
  • Keep TypeScript shell ownership for reference resolver orchestration, import resolution, finalization, status/doctor output, MCP APIs, and any dynamic-dispatch or framework semantics not explicitly included here.
  • Keep C extraction Rust-owned and do not reintroduce a TypeScript-owned C path.
  • Keep Objective-C extraction unchanged.
  • Update the Rust-owned migration roadmap checklist for the C++ item after successful implementation.
  • Add an Unreleased changelog note written for users, not implementation internals.

Testing Decisions

  • Test at the highest practical seam: run the built CLI through rust-hybrid and inspect the resulting graph through the public SDK APIs and status metadata.
  • Add or extend a rust-hybrid language smoke test that creates a small C++ fixture project and verifies:
    • C++ files are indexed.
    • C++ symbols are present with language cpp.
    • Includes are represented.
    • Calls resolve through the normal graph flow when local targets exist.
    • rustOwnedLanguages includes cpp.
    • engineByLanguage.cpp is rust.
    • fallbackByLanguage does not include C++ for the migrated fixture.
  • Add or extend a rust-hybrid assignment planner test for ambiguous headers:
    • Plain C .h remains c and Rust-owned through the C path.
    • C++-looking .h becomes cpp and Rust-owned through the C++ path.
    • Objective-C-looking .h remains Objective-C and does not become C++.
  • Keep focused extraction regression tests for C++ behavior that still matters while the TypeScript fallback exists during migration, then remove or adjust TypeScript-owned C++ extractor tests after the migrated path is removed.
  • Run build-level validation:
    • Rust core build.
    • TypeScript build.
    • Focused rust-hybrid language and fallback test suites.
    • Focused extraction guard tests for C/C++ imports, Objective-C headers, and language detection.
    • Whitespace check.
  • Validate on a suitably sized GitHub project with real C++ files. A good candidate should:
    • Be public and easy to clone shallowly.
    • Include enough .cpp or .cc files and headers to exercise the baseline.
    • Avoid being so macro-heavy that the first migration slice becomes a stress-test project.
    • Produce evidence that C++ files are Rust-owned and do not produce C++ language-level TypeScript fallback.
  • Record corpus evidence in a durable benchmark or validation document, including repository, commit, file counts, indexed file count, node/edge counts, language ownership metadata, fallback summary, and decision.
  • If the first selected corpus fails due to macro-heavy or unsupported syntax, record it as future stress-corpus evidence and choose a better baseline corpus rather than weakening the migration gate.

Out of Scope

  • C++ dynamic-dispatch semantics, including virtual override synthesis, beyond whatever baseline static call/reference extraction can already support.
  • Full C++ template semantic resolution.
  • Full overload resolution.
  • Full type inference.
  • Build-system-aware include path discovery beyond the existing TypeScript shell resolver behavior.
  • Framework-specific C++ semantics.
  • Objective-C migration.
  • C migration changes, except for preserving the header classification boundary.
  • MCP tool behavior changes.
  • Status/doctor health vocabulary changes.
  • Performance benchmarking beyond a basic real-corpus validation gate.
  • Migration of shared TypeScript shell layers such as ReferenceResolver, name matching, cleanup protocol execution, and MCP/read APIs.

Further Notes

  • This PRD follows the same ownership migration pattern as the completed C baseline migration.
  • The validation project should be a real GitHub C++ project or a mixed C/C++ project with meaningful C++ files. If the wording “C project” is interpreted literally, the implementation should still ensure the corpus exercises the C++ migration target rather than only C files.
  • The roadmap item should be checked off only after the TypeScript-owned C++ extractor path has been removed and the real-corpus validation evidence is durable.
  • The issue should be ready for an implementation agent without additional product clarification.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified and ready for an AFK agent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions