feat(python): make rdma get/set available in python glide-sync by anitarua · Pull Request #2 · anitarua/valkey-glide

anitarua · 2026-10-26T18:07:21Z

Summary

This is a continuation of the work to add client support for a new Valkey module for remote direct memory access (RDMA). This PR makes the glide-rdma features available in the Python glide-sync client, which will be used in KV caching adapters (e.g. LMCache and vLLM).

Issue link

This Pull Request is linked to issue: ValkeyLargeObj Large-Object Module
Closes valkey-io#4743

Features / Behaviour Changes

Introduces RdmaConfiguration, register_rdma_region, rdma_set, rdma_get, region.close, and other functions for memory management.

Implementation

The new configuration options are in python/glide-shared/glide_shared/config.py.
The bulk of the implementation is in python/glide-sync/glide_sync/glide_client.py.
Updated .github/workflows/pypi-cd.yml to build the python release with the RDMA feature turned on.

Limitations

RDMA not supported in pooled clients or async clients.

Testing

New tests in the python/tests directory and I've tested the full client integration against the module on real efa hardware.

Checklist

Before submitting the PR make sure the following are checked:

This Pull Request is related to one issue.
Commit message has a detailed description of what changed and why.
Tests are added or updated.
CHANGELOG.md and documentation files are updated.
Linters have been run (make *-lint targets) and Prettier has been run (make prettier-fix).
Destination branch is correct - main or release
Create merge commit if merging release branch into main, squash otherwise.
Make sure to update the documentation in the valkey-glide-docs repository if necessary

jeremyprime · 2026-10-28T16:18:51Z

LGTM, I think this is ready to opened against upstream once valkey-io#7412 (and #1) is merged. Then CI will run and we'll see if any comments from the review bots.

jamesx-improving

Thanks for the quick rework; the cancel/close path and both of jeremyprime's leaks look right now. One packaging blocker left, inline.

…-io#7454) * fix(python): stop async client hang after pool borrow/release Signed-off-by: Jeremy Parr-Pearson <jeremy.parr-pearson@improving.com> * fix(ffi): gate glide_allocate_client_id behind pool-support Signed-off-by: Jeremy Parr-Pearson <jeremy.parr-pearson@improving.com> * fix(python): make regression test id-assertion xdist-safe Signed-off-by: Jeremy Parr-Pearson <jeremy.parr-pearson@improving.com> * fix(python): make regression test xdist-safe and consistency-clean Signed-off-by: Jeremy Parr-Pearson <jeremy.parr-pearson@improving.com> * fix(ffi,python): ungate glide_allocate_client_id; trim comments Signed-off-by: Jeremy Parr-Pearson <jeremy.parr-pearson@improving.com> --------- Signed-off-by: Jeremy Parr-Pearson <jeremy.parr-pearson@improving.com>

jamesx-improving

LGTM, thanks for the fixes. It now conflicts with the rebased feat/rdma-3-ffi-integration (valkey-io#7458); replaying it onto e0fe021, the first conflict is CHANGELOG.md.

…7457) glide_pool_metrics wrote three counts through out-pointers and returned an int32 code, -1 for both an unknown pool and a contended lock, writing nothing. The counts and the verdict on whether they were read travelled separately, so a binding could take zero-filled buffers as the pool's state (valkey-io#7129). The function now returns a repr(C) PoolMetrics { status, idle, active, total } by value, so no caller holds counts without the status. When a read is declined is unchanged (try_lock); valkey-io#7130 changes that. Go and Python read the struct and report the same numbers as before, zeros on a non-Read status. Java and Node keep their own copies. Refs valkey-io#7129 Signed-off-by: Neil Derraugh <neil.derraugh@improving.com>

…rseding refresh (valkey-io#7435) When routing a key-based command to a specific node, if the connection is missing because a refresh task is currently in progress, SpecificNode routing previously waited on a single reconnect notifier once. If another refresh was triggered or superseded the original refresh task, the route could stop waiting prematurely while the refresh was still ongoing. This change: 1. Loops while a refresh task remains registered for the target route. 2. Acquires the owned notification handle under the same read lock as the connection lookup, preventing missed wakeups between lock drop and await. 3. Adds unit tests verifying that SpecificNode routing continues waiting across in-progress and superseded refresh tasks. Fixes valkey-io#7434 Signed-off-by: Kevin Friedland <347626243+KevinFriedland88@users.noreply.github.com> Signed-off-by: James Xin <james.xin@improving.com> Co-authored-by: Kevin Friedland <347626243+KevinFriedland88@users.noreply.github.com>

feat(ffi): expose RDMA get/set through the FFI (valkey-io#7458) Make glide-rdma available to the FFI layer behind an opt-in `rdma` Cargo feature, so the Python glide-sync client can transfer large values with the valkey-large-object module over RDMA. This serves KV-cache adapters such as LMCache and vLLM. - New C API in ffi/src/rdma_ffi.rs: register_rdma_region, rdma_get, rdma_set, rdma_close_regions and free_rdma_region, plus memory-hold helpers that tell released memory apart from parked or leaked memory. - glide-core and glide-rdma gain register-by-address (register_rdma_region_raw / register_raw). This lets memory a C caller owns be registered without forming a Rust reference to it. - close_client revokes the client's RDMA regions. Pools reject an RDMA configuration. - CI runs the rdma feature tests, and glide-sync vendors glide-rdma. Limitations: no RDMA in client pools yet, and sync client only. Related: valkey-io/valkey#4743 --------- Signed-off-by: Anita Ruangrotsakun <anita@momentohq.com> Co-authored-by: valkey-review-bot[bot] <282604435+valkey-review-bot[bot]@users.noreply.github.com>

Signed-off-by: anitarua <anita@momentohq.com>

…s in use Signed-off-by: anitarua <anita@momentohq.com>

Signed-off-by: anitarua <anita@momentohq.com>

…close_client Signed-off-by: anitarua <anita@momentohq.com>

Signed-off-by: anitarua <anita@momentohq.com>

anitarua · 2026-10-29T23:16:52Z

closing in favor of valkey-io#7464

anitarua mentioned this pull request Oct 26, 2026

GLIDE Client Support Implementation valkey-io/valkey-large-object#63

Closed

anitarua changed the title ~~Feat/rdma 4 python sync~~ feat(python): make rdma get/set available in python glide-sync Oct 26, 2026

anitarua force-pushed the feat/rdma-4-python-sync branch from c86753c to 6fbb2bf Compare October 26, 2026 20:56

jeremyprime reviewed Oct 27, 2026

View reviewed changes

Comment thread python/glide-sync/glide_sync/glide_client.py Resolved

jeremyprime reviewed Oct 27, 2026

View reviewed changes

Comment thread python/glide-sync/glide_sync/glide_client.py Resolved

anitarua force-pushed the feat/rdma-3-ffi-integration branch from 3d44ae6 to 7b5b711 Compare October 27, 2026 21:04

anitarua force-pushed the feat/rdma-4-python-sync branch from 6fbb2bf to a911957 Compare October 27, 2026 23:08

anitarua requested a review from jeremyprime October 27, 2026 23:13

nderraugh reviewed Oct 28, 2026

View reviewed changes

Comment thread python/glide-sync/glide_sync/glide_client.py Resolved

jamesx-improving mentioned this pull request Oct 28, 2026

feat(core): integrate glide-rdma into glide-core valkey-io/valkey-glide#7412

Merged

8 tasks done

jamesx-improving reviewed Oct 28, 2026

View reviewed changes

Comment thread python/glide-sync/setup.py Resolved

anitarua requested review from jamesx-improving and nderraugh October 28, 2026 19:00

anitarua force-pushed the feat/rdma-3-ffi-integration branch from 7b5b711 to fc9bafc Compare October 28, 2026 21:50

jamesx-improving approved these changes Oct 29, 2026

View reviewed changes

anitarua mentioned this pull request Oct 29, 2026

feat(ffi): integrate rdma into ffi valkey-io/valkey-glide#7458

Merged

8 tasks done

KevinFriedland88 and others added 11 commits October 29, 2026 14:26

feat(python): add rdma get/set to sync client

abe8133

Signed-off-by: anitarua <anita@momentohq.com>

chore(ci): smoke test glide-sync with rdma

972ac3a

Signed-off-by: anitarua <anita@momentohq.com>

fix(python): BLOB.RDMA_* protocol updates

ad4135e

Signed-off-by: anitarua <anita@momentohq.com>

fix(python): better error handling for regions closed by rdma transfers

4e2a3d0

Signed-off-by: anitarua <anita@momentohq.com>

chore(python): clean up docs/comments

679ac4a

Signed-off-by: anitarua <anita@momentohq.com>

fix(python): raise instead of corrupting memory when an rdma region i…

bb45546

…s in use Signed-off-by: anitarua <anita@momentohq.com>

fix(python): free unclosed rdma regions on garbage collection

2375232

Signed-off-by: anitarua <anita@momentohq.com>

chore(python): rebase and fix conflicts

6f3630b

Signed-off-by: anitarua <anita@momentohq.com>

fix(python): avoid segfault on python 3.9 and add regression test

294ab9c

Signed-off-by: anitarua <anita@momentohq.com>

anitarua added 7 commits October 29, 2026 16:07

chore(python): update changelog and other docs

0cd7a3f

Signed-off-by: anitarua <anita@momentohq.com>

fix(python): take rdma region capacity from the registered memory

495fcd8

Signed-off-by: anitarua <anita@momentohq.com>

fix(python): cancel rdma transfers with rdma_close_regions and defer …

59731e0

…close_client Signed-off-by: anitarua <anita@momentohq.com>

fix(python): free an rdma region whose RdmaRegion fails to build

aad52cd

Signed-off-by: anitarua <anita@momentohq.com>

fix(python): free before warning to avoid potential leak

c107958

Signed-off-by: anitarua <anita@momentohq.com>

fix(python): additional python 3.9 segfault fixes

bf3cbb1

Signed-off-by: anitarua <anita@momentohq.com>

chore(python): fix test mismatch after rebase

02bb313

Signed-off-by: anitarua <anita@momentohq.com>

anitarua force-pushed the feat/rdma-4-python-sync branch from 135286f to 02bb313 Compare October 29, 2026 23:14

anitarua closed this Oct 29, 2026

Navigation Menu

Repository navigation

feat(python): make rdma get/set available in python glide-sync - #2

feat(python): make rdma get/set available in python glide-sync#2
anitarua wants to merge 20 commits into
feat/rdma-3-ffi-integrationfrom
feat/rdma-4-python-sync

anitarua commented Oct 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

jeremyprime commented Oct 28, 2026

Uh oh!

Uh oh!

jamesx-improving left a comment

Uh oh!

Uh oh!

jamesx-improving left a comment

Uh oh!

anitarua commented Oct 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

anitarua commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Issue link

Features / Behaviour Changes

Implementation

Limitations

Testing

Checklist

Uh oh!

Uh oh!

Uh oh!

jeremyprime commented Oct 28, 2026

Uh oh!

Uh oh!

jamesx-improving left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jamesx-improving left a comment

Choose a reason for hiding this comment

Uh oh!

anitarua commented Oct 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

anitarua commented Oct 26, 2026 •

edited

Loading