Repository navigation
Conversation
added 3 commits
October 10, 2026 12:48
Route timeseries create/update/datapoints through the Charon CDF-writer service instead of directly to CDF, behind an opt-in config switch. All new types live under Cognite.Extractor.Utils.Unstable.Charon, since Charon is a POC with a changing contract. The wire format is isolated behind ICharonClient. - CharonClient: typed /cdfwriter/setup and /cdfwriter/payload client, reusing the existing bearer-token HTTP flow; logs only status/counts, never tokens or item bodies. - CharonWriter: façade to declare the task set, register it at startup, and create upload queues. - CharonUploadQueue / CharonDatapointsUploadQueue: IUploadQueue<T> writers that chunk to CDF limits and map results to QueueUploadResult<T>. - OrderedStringMap: preserves space_routing mapping order (first match wins). - Config: opt-in cdf-writer section (disabled by default -> direct-to-CDF path, unchanged); wired through DestinationUtilsUnstable.AddCharonWriter and gated in the runtime. Schema updated, version bumped to 1.46.0.
- Wire-contract: camelCase field names, insertion-ordered mapping serialization, error-body and payload-response parsing. - Client: setup 200/422, payload 200/207/422, 401 and 5xx. - Queues: payload body shape, chunking to CDF limits, datapoint one-item-per-reading expansion, result mapping, and opt-in DI registration.
Document the opt-in cdf-writer config, task registration, write path, and the fallback/limits/POC caveats. Register it in the tutorials TOC.
nithinb
marked this pull request as draft
October 13, 2026 04:13
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why?
Charon is a new CDF-writer service (POC, changing contract). Instead of extractors writing timeseries straight to CDF, they can register their task config once and stream data through Charon, which decorates/routes/shapes items (Kuiper expressions, regex space routing) and forwards to CDF with the caller's own bearer token. We want extractors built on this library to opt into Charon without disturbing the existing direct-to-CDF path, and without committing the POC's unstable wire contract to the stable public API.
What?
Opt-in support, entirely under
Cognite.Extractor.Utils.Unstable.Charon. No stable type changes behavior or signature. Disabled by default (cdf-writer.enabled: false) → direct-to-CDF path is byte-for-byte unchanged.ICharonClient/CharonClient— typed client for/cdfwriter/setupand/cdfwriter/payload; the single isolation seam for the wire format. Reuses the existing bearer-token HTTP flow; logs only status/counts, never tokens or item bodies.CharonWriter— façade to declare the task set (CharonTask.SpaceRouting/.Custom), register it at startup (SetupAsync), and create upload queues.CharonUploadQueue/CharonDatapointsUploadQueue—IUploadQueue<T>writers that chunk each task's items to CDF limits and map results toQueueUploadResult<T>.OrderedStringMap— preservesspace_routingmapping order (first regex match wins).cdf-writersection onConnectionConfig,DestinationUtilsUnstable.AddCharonWriter, gated in the runtime. Schema + docs updated;version→ 1.46.0.Key decisions:
enabled: falseto use the direct path.JsonElementescape hatch for task registration and item bodies.Carry-forward: regex anchoring for space routing is not yet frozen Charon-side (full-match vs substring) — client sends patterns verbatim, noted in builder docs.
How?
Extractor authors:
Config (
connection.yml):Testing
19 new tests: wire-contract (camelCase + mapping-order), client (200/207/422/401/5xx), queues (chunking, datapoint expansion, DI). Full Unstable suite 154/154 green;
dotnet buildof the solution clean.