CF-4076 : Add stable sort to paginated CMF list commands - #3413
Open
Paras Negi (paras-negi-flink) wants to merge 1 commit into
Open
CF-4076 : Add stable sort to paginated CMF list commands#3413Paras Negi (paras-negi-flink) wants to merge 1 commit into
Paras Negi (paras-negi-flink) wants to merge 1 commit into
Conversation
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
Copilot started reviewing on behalf of
Paras Negi (paras-negi-flink)
July 24, 2026 13:56
View session
There was a problem hiding this comment.
Pull request overview
This PR addresses CF-4076 by making CMF (on-prem) Flink list commands deterministic under offset pagination, preventing silent omissions/duplications when result sets exceed the 100-item page size.
Changes:
- Add stable, unique sort keys (
sortquery param) to all paginated CMF list API calls to ensure deterministic paging. - Preserve intended ordering where applicable (e.g., newest-first application events) while adding a unique tiebreaker for stability.
- Add unit tests that (1) regress the >100-items paging behavior and (2) lock in the exact per-resource sort keys sent by each paginated list call.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| pkg/flink/cmf_rest_client.go | Adds stable Sort(...) parameters to all paginated CMF list requests to make offset pagination deterministic. |
| pkg/flink/cmf_rest_client_test.go | Adds regression + table-driven tests asserting all items are returned across pages and that each list call sends the correct unique sort key. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+245
to
+246
| // createdTime DESC keeps events newest-first; "name" (unique id) breaks ties for stable paging (CF-4076). | ||
| eventsPage, httpResponse, err := cmfClient.FlinkApplicationsApi.GetApplicationEvents(ctx, environment, application).Sort([]string{"creationTimestamp,desc", "name"}).Page(currentPageNumber).Size(pageSize).Execute() |
airlock-confluentinc
Bot
force-pushed
the
CF-4076-cmf-list-stable-sort
branch
2 times, most recently
from
July 24, 2026 14:22
210f314 to
370ce69
Compare
The on-prem (CMF) Flink list commands page through results with offset
pagination but requested no sort order. With no ORDER BY, the server can
return rows in a different order across separate page requests, so rows near
a page boundary can be silently skipped or duplicated once a result set
exceeds the page size of 100.
Request a unique, stable sort on every paginated list call so offset paging
is deterministic:
- most resources sort by "name" (maps server-side to each unique id column)
- secret mappings sort by "name","uid" ("name" has no unique constraint)
- application events sort by "creationTimestamp,desc","name" (newest-first
plus a unique tiebreaker)
Add regression tests asserting a >100-item list returns every item exactly
once and that each list call sends the expected sort key.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Paras Negi (paras-negi-flink)
marked this pull request as ready for review
July 24, 2026 16:43
airlock-confluentinc
Bot
force-pushed
the
CF-4076-cmf-list-stable-sort
branch
from
July 24, 2026 16:44
370ce69 to
a91ce7a
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release Notes
Breaking Changes
New Features
Bug Fixes
confluent flink ... listcommands silently omitting or duplicating items when a resource has more than 100 items.Checklist
Whatsection below whether this PR applies to Confluent Cloud, Confluent Platform, or both.Test & Reviewsection below.Blast Radiussection below.What
This PR fixes CF-4076 for Confluent Platform / CP Flink (CMF on-prem).
The on-prem (CMF)
confluent flink ... listcommands page through results with offset pagination (.Page(n).Size(100)) but requested no sort order. CMF applies no defaultORDER BY, so each page is a separateLIMIT/OFFSETquery with no ordering. PostgreSQL does not guarantee a stable row order across separate queries, so once a (filtered) result set exceeds the page size of 100, rows near a page boundary can be silently skipped or duplicated between page requests — with no error. The most-frequently-updated row (e.g. an activelyRUNNINGstatement) is the most likely to shift and disappear.Fix: request a unique, stable sort on every paginated CMF list call in
pkg/flink/cmf_rest_client.go, so offset paging is deterministic."name"maps server-side (via each resource'sSortPropertyMapper) to that resource's unique id column.Per-resource sort keys:
["name"]secret-mapping list→["name", "uid"]—namehas no unique constraint;uid(the PK) is the tiebreakerapplication event list→["creationTimestamp,desc", "name"]— CMF applies no ordering to events today, so this both introduces newest-first ordering (the useful order for an event log) and makes paging stable;creationTimestampis not unique on its own, sonameis the tiebreakerThis works against existing CMF servers (incl. 2.3.0); no server change is required.
Blast Radius
confluent flink ... listcommands; the change only adds asortquery parameter to the existing paginated list requests. No writes, no persistence changes, no config or migration changes.output.NewListalready sorts client-side); the only user-visible change is that-o json/-o yamllist output is now deterministically ordered — and forapplication event listspecifically, that order is now newest-first rather than arbitrary.confluent flinkand allconfluent kafkacommands are unaffected.sortparam, which is verified incp-flink-cmf(SortPropertyMapper.transformSort+ per-resource mappers mapnameto each resource's unique id).References
cp-flink-cmf.Test & Review
Automated tests (
pkg/flink/cmf_rest_client_test.go):TestListStatements_ReturnsAllItemsAcrossPages— paginates a 250-item list (spanning the 100/200 page boundaries) and asserts every item is returned exactly once; the mock server rejects any request that omitssort.TestListCommands_SendUniqueSortKey— table-driven across all 13 paginated call sites, asserting the exact sort key each sends (name;name,uidfor secret-mappings;creationTimestamp,desc+namefor events).Results
pkg/flinkandinternal/flinkunit suites pass (including-race).List*(golden) tests pass; output unchanged (the mock ignoressort).gofmtandgo vetclean.Deterministic repro of the bug — driving the real
ListStatementsagainst a server that mimics unorderedOFFSETpaging: without a sort, 149/150 statements were returned (one silently dropped); with the fix's sort, 150/150 were returned.