Skip to content

Add Zarr, muon, and SpatialData integration capabilities #46

Description

@d33bs

Recommended roadmap

  1. Add a collection API above OMEArrow.

    • Introduce OMEArrowCollection (or similar) for a table of related assets per sample/FOV (image, profiles, labels, etc.).
    • Keep current OMEArrow unchanged as the single-record primitive.
  2. Make dtype and annotation support first-class.

    • Replace pa.list_(pa.uint16()) planes/chunks with a dtype-flexible representation.
    • Implement real masks/labels schema (instead of null) and typed spatial geometries/regions.
    • This is critical for profiles/segmentation and spatial ecosystem compatibility.
  3. Build adapters instead of forcing external ecosystems into core.

    • Add ome_arrow.adapters.muon: from_mudata, to_mudata.
    • Add ome_arrow.adapters.spatial: from_spatialdata, to_spatialdata.
    • Keep these as optional dependencies (muon, anndata, spatialdata) so base install stays light.
  4. Focus performance work on Arrow-native execution paths.

    • Minimize as_py() conversions.
    • Add chunk-aware ROI slicing that works without requiring dense planes.
    • Support direct ROI from Zarr chunks for lazy reads instead of per-plane decode loops.
  5. Strengthen benchmarking around real domain cases.

    • Keep the current lazy canary benchmark.
    • Add benchmarks for multi-array Zarr group ingest, ROI over chunked stores, and adapter round-trips.
    • Use these to gate regressions before wider adoption.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions