Skip to content

Return byte arrays from typed BLOB reads - #384

Merged
Giorgi merged 2 commits into
Giorgi:developfrom
alaricd:fix/blob-getfieldvalue-byte-array
Oct 26, 2026
Merged

Giorgi merged 2 commits into
Giorgi:developfrom
alaricd:fix/blob-getfieldvalue-byte-array

Conversation

@alaricd

@alaricd alaricd commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Problem

DbDataReader.GetFieldValue<byte[]>() on DuckDB BLOB columns returned the provider's Stream value, causing an invalid cast for consumers that request materialized bytes.

Fix

  • Materialize BLOB data as byte[] for GetFieldValue<byte[]>().
  • Preserve existing GetValue() and GetStream() behavior as streams.
  • Add coverage for non-empty and empty BLOBs.

Validation

  • DuckDB blob tests: 6 passed.
  • Focused coverage: new GetBytes method 100% line, branch, and method coverage.
  • ADO.NET conformance test added separately and passes on .NET 8 and .NET 10.

@coveralls

coveralls commented Oct 26, 2026 •

Copy link
Copy Markdown

Coverage Report for CI Build 40294592054

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Coverage increased (+0.1%) to 92.914%

Details

  • Coverage increased (+0.1%) from the base build.
  • Patch coverage: 6 of 6 lines across 1 file are fully covered (100%).
  • 13 coverage regressions across 4 files.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

13 previously-covered lines in 4 files lost coverage.

File Lines Losing Coverage Coverage
DuckDB.NET.Data/DataChunk/Reader/VectorDataReaderBase.cs 4 92.88%
DuckDB.NET.Data/DuckDBDataReader.cs 3 97.18%
DuckDB.NET.Data/DuckDBTransaction.cs 3 87.8%
DuckDB.NET.Data/PreparedStatement/PreparedStatement.cs 3 93.18%

Coverage Stats

Coverage Status
Relevant Lines: 4287
Covered Lines: 4046
Line Coverage: 94.38%
Relevant Branches: 2572
Covered Branches: 2327
Branch Coverage: 90.47%
Branches in Coverage %: Yes
Coverage Strength: 967896.98 hits per line

💛 - Coveralls

@Giorgi Giorgi left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! Reading a BLOB as byte[] is a gap people hit often, and the change is small and fits the reader's conventions (typeof(T) == typeof(byte[]), nothing else in the type test).

One thing should be fixed before merging, and two smaller ones would be nice to have.

1. The object path still returns a Stream for a byte[] target

The PR changes the typed path (GetValidValue<T>). The override GetValue(ulong offset, Type targetType) still has DuckDBType.Blob => GetStream(offset) for every target type.

The composite readers read their children through that object path, so a BLOB inside a struct, map or list still fails:

  • Struct: a byte[] property is read with reader.GetValue(offset, property.PropertyType), gets a Stream, and the setter's cast throws InvalidCastException.
  • Map: for Dictionary<string, byte[]>, the values are read with valueReader.GetValue(childOffset, valueTargetType), and adding a Stream through IDictionary.Add throws.
  • List: List<byte[]> goes through BuildOtherList, whose items come from the same GetValue(…, typeof(byte[])).

So reader.GetFieldValue<byte[]>(0) works, but GetFieldValue<List<byte[]>>(0) and a struct with a byte[] property don't. (I worked this out from the code; I haven't run the branch.)

The same branch in the object path fixes it. GetValue(offset) asks for the column's own type, Stream, so GetValue() still returns a stream:

DuckDBType.Blob => targetType == typeof(byte[]) ? GetBytes(offset) : GetStream(offset),

Could you add tests for a BLOB struct field read into a byte[] property, a MAP(VARCHAR, BLOB) read as Dictionary<string, byte[]>, and a BLOB[] read as List<byte[]>?

2. GetBytes can copy straight from the string data

There's no need for a stream just to read it into an array:

private unsafe byte[] GetBytes(ulong offset)
{
    var data = (DuckDBString*)DataPointer + offset;
    return new ReadOnlySpan<byte>((void*)data->Data, checked((int)data->Length)).ToArray();
}

3. Tests

The two new tests (non-empty and empty BLOB) look good. One more assertion, reader.GetValue(0).Should().BeOfType<UnmanagedMemoryStream>() next to the byte[] read, would pin that GetValue() is unchanged, as the description says.

Docs: once this is in, I'll add byte[] for BLOB to docs/type-mapping.md in the docs repository.

@Giorgi
Giorgi merged commit f86315d into Giorgi:develop Oct 26, 2026
12 of 13 checks passed

This branch was successfully deployed

1 active deployment
external — ed1afff8 Deployed Oct 26, 2026 by alaricd via Authorize #717
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants