Fix metadata xs:dateTime serialization and GZip compression handshake for C# subscriber interop#10
Merged
Conversation
The Python publisher serialized timezone-aware UTC datetimes in metadata XML
with both a UTC offset and a 'Z' designator (e.g. 2024-01-01T00:00:00.000+00:Z),
an invalid xs:dateTime lexical value that the STTP C# subscriber rejects when
parsing metadata via .NET DataSet.ReadXml. The malformed string came from
isoformat() emitting a +00:00 offset, rstrip('0') then corrupting the offset
digits, and an unconditional 'Z' suffix.
Rewrite xsdformat() as a single canonical formatter that emits a bare
xs:dateTime (yyyy-MM-ddTHH:mm:ss[.ffffff], no Z, no offset) matching the .NET
DataSet reference format, normalizing timezone-aware values to UTC wall-clock and
trimming fractional zeros without leaving a dangling dot. _value_to_xml_text()
now delegates to it, unifying the two previously divergent datetime serializers
and fixing xsdformat's own latent tz-aware bug.
Add a pytest regression test (test/test_metadata_datetime_xml.py) and a
cross-language interop harness (test/interop/) that runs a real Python publisher
against the STTP C# subscriber to verify metadata date/time exchange end-to-end.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
The publisher GZip-compressed metadata (and the signal index cache) whenever the CompressMetadata / CompressSignalIndexCache operational mode was set, ignoring the negotiated GZip compression mode. The .NET DataSubscriber default advertises CompressMetadata WITHOUT GZip and therefore does not inflate the payload, so it failed to parse the metadata (0x1F "invalid character" on the gzip magic byte). Add should_gzip_compress() and gate both the metadata and signal-index-cache compression on the content flag AND CompressionModes.GZIP, matching the STTP C# reference (DataPublisher.SerializeMetadata / SerializeSignalIndexCache). This is a publisher-only change; the Python subscriber already always advertises GZip. Extend the interop harness to exercise both the uncompressed (.NET-default) and GZip compression negotiations, and add compression-decision unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
| from .signalindexcache import SignalIndexCache | ||
| from .constants import OperationalModes, OperationalEncoding, ServerCommand, ServerResponse | ||
| from .constants import DataPacketFlags, BufferBlockFlags, Defaults | ||
| from .constants import DataPacketFlags, BufferBlockFlags, Defaults, CompressionModes |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two metadata interop bugs between the STTP Python publisher and the STTP C# subscriber (
gsfapireference). Python's lenient implementations accepted both on a Python↔Python round-trip; the C# subscriber did not.1.
xs:dateTimeserializationThe Python publisher serialized timezone-aware UTC datetimes in metadata XML with both a UTC offset and a
Zdesignator (e.g.2024-01-01T00:00:00.000+00:Z) — an invalidxs:dateTimelexical value that the C# subscriber rejects when parsing metadata via .NETDataSet.ReadXml.Root cause (
DataSet._value_to_xml_text()insrc/sttp/data/dataset.py):isoformat()emitted a+00:00offset,rstrip('0')then corrupted the offset, and aZwas appended unconditionally.Fix: rewrote
xsdformat()as a single canonical formatter emitting a barexs:dateTime(yyyy-MM-ddTHH:mm:ss[.ffffff], noZ/offset), matching the .NETDataSetreference and round-tripping without a timezone shift._value_to_xml_text()delegates to it, unifying the two previously divergent datetime serializers.2. Metadata / signal-index-cache GZip compression handshake
The Python publisher GZip-compressed metadata (and the signal index cache) whenever
CompressMetadata/CompressSignalIndexCachewas set, ignoring the negotiated GZip compression mode. The.NET DataSubscriberdefault advertisesCompressMetadatawithout GZip, so it received gzip bytes it does not inflate and failed on the0x1Fmagic byte.Fix (
src/sttp/transport/subscriberconnection.py): addedshould_gzip_compress()and gated both compression sites on the content flag ANDCompressionModes.GZIP, matching the C# reference (DataPublisher.SerializeMetadata/SerializeSignalIndexCache). Publisher-only — the Python subscriber already always advertises GZip.Tests
test/test_metadata_datetime_xml.py(bare-datetime output + round-trip) andtest/test_metadata_compression.py(compress only when flag and GZip). Full suite: 36 passed.test/interop/): runs a real Python publisher against the real C# subscriber (gsfapiInteropTest --metadata), by default exercising both compression negotiations with a tz-aware UTC sentinel. Verified end-to-end:2033-03-03T03:33:33.123in both uncompressed and GZip modes;'…+00:Z' is not a valid AllXsd value;--compression plain) → C# failshexadecimal value 0x1F, is an invalid character.Reviewer notes
InteropTest --metadata [--gzip]harness mode this uses (separate repo).test_pubsub, which negotiates GZip).🤖 Generated with Claude Code