Skip to content

build: make outputs independent of the checkout path - #226

Open
xrviv wants to merge 3 commits into
Blockstream:masterfrom
xrviv:reproducible-build-paths
Open

build: make outputs independent of the checkout path#226
xrviv wants to merge 3 commits into
Blockstream:masterfrom
xrviv:reproducible-build-paths

Conversation

@xrviv

@xrviv xrviv commented Jul 30, 2026

Copy link
Copy Markdown

Summary

Make GDK’s C++ and Rust outputs independent of the absolute checkout path.

This fixes #225 and allows third parties to reproduce GDK without recreating Blockstream’s CI directory structure.

Problem

GDK embedded its absolute source directory into compiled output.

Using release_0.77.6, the same source, builder image, dependencies, and build command produced different Android libraries at different checkout paths:

Checkout path libgreen_gdk_java.so SHA-256
/root/gdk b60ce800…
/root/gdk/gdk 440ca6ed…
/builds/blockstream/gdk fbcd5f11…
/builds/blockstream/green/gdk 13d4ba62…

Only the final path reproduced the published library.

Changes

  • Add -ffile-prefix-map=${CMAKE_SOURCE_DIR}=. to supported C and C++ builds.
  • Pass the top-level source root into the Rust build script.
  • Add Rust’s --remap-path-prefix without discarding inherited flags.
  • Preserve Android’s required -L…/libgcc flag.
  • Respect CARGO_ENCODED_RUSTFLAGS when Cargo uses it instead of RUSTFLAGS.
  • Pass --locked to Cargo, as requested in Android release_0.77.6 tarball does not reproduce from documented GitLab CI recipe #225.

The C++ and Rust changes are separate commits. I kept --locked in a third commit because dependency locking has a separate rationale, but I am happy to fold it into the Rust commit if preferred.

Verification

Tested commit 4cf8443ad17aaae66a13a0de2b5f927b925c3d9c with the pinned builder:

docker.io/blockstream/gdk-android-builder@sha256:e08f5a77cc5472c9539cd0a83ee376efffab02eff4e768eae2c16c243f587f1e

I made two clean arm64 Android Release builds with identical inputs but different checkout paths:

  • Build A: /builds/blockstream/green/gdk
  • Build B: /root/gdk

All four outputs were byte-for-byte identical:

Output SHA-256 for both builds
src/libgreen_gdk.a 48ce7e4313decc8220b0479032632a77b7b79e73292619ee147475ecb88d1ab0
gdk-rust/libgdk_rust.a a2157fcadb08e9990e277e43a64e8c7819450665faeab6e6439d71c9d7471f45
libgreen_gdk_java.so 1aa34e6fd3f565f759350909d16ee57ee58aa4c4d709f5b211b7b49a4515644e
libgreen_gdk_java.syms 48b8dcb14034b1c0b7eb52c86e4f838d5958348923e01271add8201875c99ac5

Additional checks:

  • Neither absolute checkout path remained in the four outputs.
  • Neither path remained in the tested C/C++ object files.
  • Verbose rustc output contained the correct --remap-path-prefix for each build.
  • Android’s -L…/libgcc flag remained present.
  • Both builds used --locked.
  • The .syms file still resolved a machine address to a real function and source line.

The patched hashes are intentionally different from the old release hash because path remapping changes the recorded strings. The relevant result is that the two patched builds match each other.

Scope

The acceptance test covers the arm64 Android Release build. No wallet behavior, protocol handling, or key-management code is changed.

xrviv added 3 commits July 30, 2026 15:46
Add -ffile-prefix-map so the absolute build directory is not embedded in compiled objects. The same commit built at different paths currently produces different binaries; see Blockstream#225.
Pass the top-level source root to buildgdk_rust.sh and add --remap-path-prefix so file!() expansions and debug info do not embed the absolute build directory. RUSTFLAGS was assigned rather than appended for Android targets, so caller flags were silently dropped; flags are now collected and applied to whichever source Cargo consults. See Blockstream#225.
Fail rather than silently update Cargo.lock, pinning dependency versions for both security and reproducibility. Requested in Blockstream#225.
@xrviv xrviv changed the title Reproducible build paths - build: make outputs independent of the checkout path build: make outputs independent of the checkout path Jul 30, 2026
@xrviv

xrviv commented Aug 11, 2026

Copy link
Copy Markdown
Author

Hi @jgriffiths — friendly nudge on this one.

The three commits are split as you asked: C++ path remap, Rust path remap, and --locked as its own commit.

Since opening the PR I also ran a two-root acceptance check on the patched tree. The same commit built at /builds/blockstream/green/gdk and at /root/gdk produced byte-identical arm64 Android output:

Output SHA-256
build-android-arm64-v8a/src/libgreen_gdk.a 48ce7e4313decc8220b0479032632a77b7b79e73292619ee147475ecb88d1ab0
build-android-arm64-v8a/gdk-rust/libgdk_rust.a a2157fcadb08e9990e277e43a64e8c7819450665faeab6e6439d71c9d7471f45
gdk-android-jni/lib/arm64-v8a/libgreen_gdk_java.so 1aa34e6fd3f565f759350909d16ee57ee58aa4c4d709f5b211b7b49a4515644e
gdk-android-jni/lib/arm64-v8a/libgreen_gdk_java.syms 48b8dcb14034b1c0b7eb52c86e4f838d5958348923e01271add8201875c99ac5

Neither checkout path appears in the finished artifacts or in the C/C++ object files. Two checks worth calling out specifically: the Rust remap does not discard Android's required libgcc search path (both builds still pass -L<root>/subprojects/gdk_rust/libgcc), and the .syms file retains usable line information — the matching NDK llvm-addr2line resolved an address in it to a real function and source line.

These patched hashes are not expected to match the old 0.77.6 release hash, since remapping changes the output by design. The meaningful comparison is patched build A against patched build B.

Scope is the arm64 Android build only; I have not tested other Android architectures, iOS, macOS, Windows or Linux.

One other thing worth flagging: no CI has run on this PR at all, so it may be waiting on maintainer approval for workflows from a first-time contributor's fork.

Happy to rebase, split things differently, or extend coverage to other targets — just say the word. Build logs available if useful.

@jgriffiths

Copy link
Copy Markdown
Contributor

Hi @xrviv

We have been inundated with questions/scans/reports (many automated/AI) for all products/libraries as a result of the Coldcard issue last week, and have to do due diligence by investigating them all for validity/duplicates etc and mitigation if required. This backlog will take some time to complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Android release_0.77.6 tarball does not reproduce from documented GitLab CI recipe

2 participants