Skip to content

sort queued items before dispatching in light mode for non-zero confirmation - #186

Merged
peterbroadhurst merged 1 commit into
mainfrom
light-mode-dispatch-order-fix
Sep 11, 2026
Merged

peterbroadhurst merged 1 commit into
mainfrom
light-mode-dispatch-order-fix

Conversation

@Chengxuan

@Chengxuan Chengxuan commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

In light chain tracking mode with confirmationsRequired > 0, events dispatched from the confirmation manager weren't guaranteed to be in block order. This could occasionally lead to a later-arriving event being treated by the event stream as a re-detection of an earlier one, so it wasn't forwarded. This path isn't used when confirmationsRequired = 0 or during catch-up.

Root cause

checkAndDispatchConfirmationsUsingBlockHeight builds its list of newly-confirmable items from bcm.pending, a Go map. Map iteration order isn't guaranteed, so when a head block update confirmed several pending events at once, they were dispatched in map order rather than block order.

Full chain tracking mode already handles this correctly in processBlock via sort.Sort(notifications) before dispatch. Light mode was missing the equivalent sort.

Fix

Sort pending items by block number, transaction index, and log index (using the existing pendingItems sort type) before dispatching in checkAndDispatchConfirmationsUsingBlockHeight, bringing it in line with full mode.

Testing

  • Added TestBlockConfirmationManagerHeadBlockNumberDispatchesInBlockOrder, which confirms a batch of light-mode events together via one head block update and asserts dispatch order. It reproduced the ordering issue consistently before the fix and passes consistently after.
  • go build ./..., go vet ./..., and the internal/confirmations suite all pass.
  • The full go test ./... run has two unrelated pre-existing failures in persistence/dbmigration and persistence/postgres (a golang-migrate panic in this environment), present on main before this change too.

🤖 Generated with Claude Code

Signed-off-by: Chengxuan Xing <chengxuan.xing@kaleido.io>
@Chengxuan
Chengxuan requested a review from a team as a code owner September 11, 2026 11:27
@Chengxuan Chengxuan changed the title sort queued items before dispatching in light mode sort queued items before dispatching in light mode for non-zero confirmation Sep 11, 2026
// processBlock's notifications in full chain tracking mode - we must sort by block order
// before dispatching, or events can be delivered out of order (and then dropped downstream
// as apparent re-detections once the checkpoint moves past them).
sort.Sort(items)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool the ordering is here:

type pendingItems []*pendingItem
func (pi pendingItems) Len() int { return len(pi) }
func (pi pendingItems) Swap(i, j int) { pi[i], pi[j] = pi[j], pi[i] }
func (pi pendingItems) Less(i, j int) bool {
// At the point we emit the confirmations, we ensure to sort them by:
// - Block number
// - Transaction index within the block
// - Log index within the transaction (only for events)
return pi[i].blockNumber < pi[j].blockNumber ||
(pi[i].blockNumber == pi[j].blockNumber && (pi[i].transactionIndex < pi[j].transactionIndex ||
(pi[i].transactionIndex == pi[j].transactionIndex && pi[i].logIndex < pi[j].logIndex)))
}

@peterbroadhurst
peterbroadhurst merged commit 35e3ade into main Sep 11, 2026
3 checks passed
@peterbroadhurst
peterbroadhurst deleted the light-mode-dispatch-order-fix branch September 11, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants