Skip to content

Fix metadata xs:dateTime serialization and GZip compression handshake for C# subscriber interop#10

Merged
ritchiecarroll merged 5 commits into
mainfrom
claude/sttp-python-date-format-e77bec
Jul 15, 2026
Merged

Fix metadata xs:dateTime serialization and GZip compression handshake for C# subscriber interop#10
ritchiecarroll merged 5 commits into
mainfrom
claude/sttp-python-date-format-e77bec

Conversation

@ritchiecarroll

@ritchiecarroll ritchiecarroll commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary

Fixes two metadata interop bugs between the STTP Python publisher and the STTP C# subscriber (gsfapi reference). Python's lenient implementations accepted both on a Python↔Python round-trip; the C# subscriber did not.

1. xs:dateTime serialization

The Python publisher serialized timezone-aware UTC datetimes in metadata XML with both a UTC offset and a Z designator (e.g. 2024-01-01T00:00:00.000+00:Z) — an invalid xs:dateTime lexical value that the C# subscriber rejects when parsing metadata via .NET DataSet.ReadXml.

Root cause (DataSet._value_to_xml_text() in src/sttp/data/dataset.py): isoformat() emitted a +00:00 offset, rstrip('0') then corrupted the offset, and a Z was appended unconditionally.

Fix: rewrote xsdformat() as a single canonical formatter emitting a bare xs:dateTime (yyyy-MM-ddTHH:mm:ss[.ffffff], no Z/offset), matching the .NET DataSet reference and round-tripping without a timezone shift. _value_to_xml_text() delegates to it, unifying the two previously divergent datetime serializers.

2. Metadata / signal-index-cache GZip compression handshake

The Python publisher GZip-compressed metadata (and the signal index cache) whenever CompressMetadata / CompressSignalIndexCache was set, ignoring the negotiated GZip compression mode. The .NET DataSubscriber default advertises CompressMetadata without GZip, so it received gzip bytes it does not inflate and failed on the 0x1F magic byte.

Fix (src/sttp/transport/subscriberconnection.py): added should_gzip_compress() and gated both compression sites on the content flag AND CompressionModes.GZIP, matching the C# reference (DataPublisher.SerializeMetadata / SerializeSignalIndexCache). Publisher-only — the Python subscriber already always advertises GZip.

Tests

  • Unit / regression: test/test_metadata_datetime_xml.py (bare-datetime output + round-trip) and test/test_metadata_compression.py (compress only when flag and GZip). Full suite: 36 passed.
  • Cross-language harness (test/interop/): runs a real Python publisher against the real C# subscriber (gsfapi InteropTest --metadata), by default exercising both compression negotiations with a tz-aware UTC sentinel. Verified end-to-end:
    • fixed → C# parses 2033-03-03T03:33:33.123 in both uncompressed and GZip modes;
    • revert date fix → C# fails '…+00:Z' is not a valid AllXsd value;
    • revert compression fix (--compression plain) → C# fails hexadecimal value 0x1F, is an invalid character.

Reviewer notes

🤖 Generated with Claude Code

The Python publisher serialized timezone-aware UTC datetimes in metadata XML
with both a UTC offset and a 'Z' designator (e.g. 2024-01-01T00:00:00.000+00:Z),
an invalid xs:dateTime lexical value that the STTP C# subscriber rejects when
parsing metadata via .NET DataSet.ReadXml. The malformed string came from
isoformat() emitting a +00:00 offset, rstrip('0') then corrupting the offset
digits, and an unconditional 'Z' suffix.

Rewrite xsdformat() as a single canonical formatter that emits a bare
xs:dateTime (yyyy-MM-ddTHH:mm:ss[.ffffff], no Z, no offset) matching the .NET
DataSet reference format, normalizing timezone-aware values to UTC wall-clock and
trimming fractional zeros without leaving a dangling dot. _value_to_xml_text()
now delegates to it, unifying the two previously divergent datetime serializers
and fixing xsdformat's own latent tz-aware bug.

Add a pytest regression test (test/test_metadata_datetime_xml.py) and a
cross-language interop harness (test/interop/) that runs a real Python publisher
against the STTP C# subscriber to verify metadata date/time exchange end-to-end.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread test/interop/run_metadata_interop.py Fixed
Comment thread test/interop/run_metadata_interop.py Fixed
Comment thread test/interop/run_metadata_interop.py Fixed
ritchiecarroll and others added 4 commits July 15, 2026 16:32
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
The publisher GZip-compressed metadata (and the signal index cache) whenever the
CompressMetadata / CompressSignalIndexCache operational mode was set, ignoring the
negotiated GZip compression mode. The .NET DataSubscriber default advertises
CompressMetadata WITHOUT GZip and therefore does not inflate the payload, so it
failed to parse the metadata (0x1F "invalid character" on the gzip magic byte).

Add should_gzip_compress() and gate both the metadata and signal-index-cache
compression on the content flag AND CompressionModes.GZIP, matching the STTP C#
reference (DataPublisher.SerializeMetadata / SerializeSignalIndexCache). This is a
publisher-only change; the Python subscriber already always advertises GZip.

Extend the interop harness to exercise both the uncompressed (.NET-default) and
GZip compression negotiations, and add compression-decision unit tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
from .signalindexcache import SignalIndexCache
from .constants import OperationalModes, OperationalEncoding, ServerCommand, ServerResponse
from .constants import DataPacketFlags, BufferBlockFlags, Defaults
from .constants import DataPacketFlags, BufferBlockFlags, Defaults, CompressionModes
@ritchiecarroll ritchiecarroll changed the title Fix xs:dateTime metadata serialization for C# subscriber interop Fix metadata xs:dateTime serialization and GZip compression handshake for C# subscriber interop Jul 15, 2026
@ritchiecarroll ritchiecarroll merged commit 731f746 into main Jul 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant