Repository navigation
Return byte arrays from typed BLOB reads - #384
Conversation
Coverage Report for CI Build 40294592054Warning Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes. Coverage increased (+0.1%) to 92.914%Details
Uncovered ChangesNo uncovered changes found. Coverage Regressions13 previously-covered lines in 4 files lost coverage.
Coverage Stats💛 - Coveralls |
Giorgi
left a comment
There was a problem hiding this comment.
Thanks for this! Reading a BLOB as byte[] is a gap people hit often, and the change is small and fits the reader's conventions (typeof(T) == typeof(byte[]), nothing else in the type test).
One thing should be fixed before merging, and two smaller ones would be nice to have.
1. The object path still returns a Stream for a byte[] target
The PR changes the typed path (GetValidValue<T>). The override GetValue(ulong offset, Type targetType) still has DuckDBType.Blob => GetStream(offset) for every target type.
The composite readers read their children through that object path, so a BLOB inside a struct, map or list still fails:
- Struct: a
byte[]property is read withreader.GetValue(offset, property.PropertyType), gets aStream, and the setter's cast throwsInvalidCastException. - Map: for
Dictionary<string, byte[]>, the values are read withvalueReader.GetValue(childOffset, valueTargetType), and adding aStreamthroughIDictionary.Addthrows. - List:
List<byte[]>goes throughBuildOtherList, whose items come from the sameGetValue(…, typeof(byte[])).
So reader.GetFieldValue<byte[]>(0) works, but GetFieldValue<List<byte[]>>(0) and a struct with a byte[] property don't. (I worked this out from the code; I haven't run the branch.)
The same branch in the object path fixes it. GetValue(offset) asks for the column's own type, Stream, so GetValue() still returns a stream:
DuckDBType.Blob => targetType == typeof(byte[]) ? GetBytes(offset) : GetStream(offset),Could you add tests for a BLOB struct field read into a byte[] property, a MAP(VARCHAR, BLOB) read as Dictionary<string, byte[]>, and a BLOB[] read as List<byte[]>?
2. GetBytes can copy straight from the string data
There's no need for a stream just to read it into an array:
private unsafe byte[] GetBytes(ulong offset)
{
var data = (DuckDBString*)DataPointer + offset;
return new ReadOnlySpan<byte>((void*)data->Data, checked((int)data->Length)).ToArray();
}3. Tests
The two new tests (non-empty and empty BLOB) look good. One more assertion, reader.GetValue(0).Should().BeOfType<UnmanagedMemoryStream>() next to the byte[] read, would pin that GetValue() is unchanged, as the description says.
Docs: once this is in, I'll add byte[] for BLOB to docs/type-mapping.md in the docs repository.
Problem
DbDataReader.GetFieldValue<byte[]>()on DuckDB BLOB columns returned the provider'sStreamvalue, causing an invalid cast for consumers that request materialized bytes.Fix
byte[]forGetFieldValue<byte[]>().GetValue()andGetStream()behavior as streams.Validation
GetBytesmethod 100% line, branch, and method coverage.