Repository navigation
Conversation
go-mysql decodes INT24 as a sign-extended int32, so unsigned values of 2^23 and above reach the binlog writer as 4286782170-style uint32s. The binlog INSERT IGNORE clamps them to 16777215 on the target.
grodowski
approved these changes
Oct 15, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #299. Supersedes #301.
Problem
go-mysql decodes
MYSQL_TYPE_INT24as a sign-extendedint32(mysql.ParseBinaryInt24). Our unsigned fix-up inNewBinlogDMLEventsconverts it withuint32(v), so an unsignedMEDIUMINTof 2^23 or above turns into a value outside the column's range. For example, 8592090 arrives asint32(-8185126)and becomes4286782170, the value reported in #299.What happens on the target depends on the statement (target runs
STRICT_ALL_TABLES):INSERT IGNORE): the out-of-range value is clamped to 16777215 with a warning. The data is silently corrupted and Ghostferry exits 0. Only the inline verifier notices (cutover verification failed ... PKs: 3).ERROR 1264 (22003): Out of range valueand the run aborts.The data iterator isn't affected; only values read from the binlog are.
Fix
For unsigned
TYPE_MEDIUM_INTcolumns, keep only the low 24 bits:uint32(v) & 0xFFFFFF. This matches go-mysql's owncanalfix-up (canal/rows.go). We usereplication.BinlogSyncerdirectly, so that code never runs for us.Why not upgrade go-mysql?
v1.15.0 (go-mysql#1117) decodes unsigned integers in
replication, but only when the TABLE_MAP event carries signedness metadata (binlog_row_metadata, MySQL 8.0+). MySQL 5.7, our default test target, never sends it, so the fix-up is still needed. v1.15 also changes theMysqlGTIDSet/UUIDSettypes thatbinlog_streamer.gobuilds directly. If go-mysql is upgraded later, values arrive asuint32, the type switch doesn't match, and this code does nothing.Repro
The first commit adds the tests on their own so they can be seen failing:
TestBinlogUpdateEventWithUnsignedMediumInt(Go unit test, no MySQL). Before the fix:SET unsigned_col=4294967295 ... WHERE unsigned_col=4286782170.test_unsigned_mediumint_with_inline_verifier(Ruby integration test). It inserts and updates rows with values ≥ 2^23 during binlog streaming. Before the fix the table checksums differ: the target holds16777215where the source has8592090.Verification (local, MySQL 8.0.32)
test/integration/types_test.rb: 11 tests, 0 failures.go test ./test/go -run 'TestDMLEventsTestSuite|TestBinlogWriter|Types': passes.