fix(cpp): never seal empty chunks and fix dangling ref in parallel tablet write - #909
fix(cpp): never seal empty chunks and fix dangling ref in parallel tablet write#909kkzi wants to merge 2 commits into
Conversation
…blet write Two fixes in TsFileWriter: 1. flush_chunk_group / flush_chunk_group_encoded: skip registered-but-empty measurement columns. A measurement that received no data in a window used to be sealed as an EMPTY chunk (count=0, dataSize=0). Java readers (TsFileSequenceReader self-check) treat such a file as crashed and refuse to load it. Mirror the aligned branch's existing hasData() check so empty columns never produce a chunk. 2. write_table (aligned parallel path): the submitted tasks run asynchronously on the thread pool, but the lambdas captured the loop variables (ctx, vt) by reference. Once the loop advances, every queued task reads the same / already-destroyed loop variable. Capture the per-iteration addresses by value instead.
There was a problem hiding this comment.
Pull request overview
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Fixes two correctness issues in the C++ TsFile writer: avoiding creation of invalid empty chunks in the non-aligned flush path, and preventing dangling reference captures in the aligned parallel tablet write path.
Changes:
- Skip sealing registered-but-empty chunk writers in
flush_chunk_group()andflush_chunk_group_encoded()viaChunkWriter::hasData(). - Capture per-iteration
DeviceWriteCtx/ValueTaskpointers by value when submitting thread-pool tasks inwrite_table().
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
Pushed CI runs for the new commit are waiting for approval ( |
Fixes #908
Two fixes in
cpp/src/writer/tsfile_writer.cc1. Never seal empty chunks (non-aligned flush path)
flush_chunk_group()/flush_chunk_group_encoded()now skip registered-but-empty measurement columns via the existingChunkWriter::hasData()check — mirroring what the aligned branch already does.Before: a measurement that received no data in a window was sealed as an EMPTY chunk (
count=0, dataSize=0). Java readers (TsFileSequenceReaderself-check /TsFileSketchTool) treat the whole file as crashed and refuse to load it, even though the other chunks are valid.After: empty columns produce no chunk at all.
2. Fix dangling reference capture in parallel aligned tablet write
In
write_table()'s aligned parallel path, the tasks submitted to the thread pool previously captured the loop variables (ctx,vt) by reference. Since the pool executes tasks asynchronously (after the loop advances/exits), the references dangle and tasks can read wrong or out-of-scope state. The fix captures the per-iteration addresses by value (ctx_ptr/vt_ptr), so each task reads its ownDeviceWriteCtx/ValueTask.Verification
tsfile_writer.ccchanged, no build config touched).count=0chunk), andTsFileSketchTool/print-tsfile.batread it completely (END of TsFile,TsFile Sketch End, no errors).