Skip to content

test(e2e): run four peered validators per managed Besu chain - #1588

Merged
gjermundgaraba merged 4 commits into
mainfrom
fou-1903-harness
Oct 23, 2026
Merged

gjermundgaraba merged 4 commits into
mainfrom
fou-1903-harness

Conversation

@gjermundgaraba

@gjermundgaraba gjermundgaraba commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Description

Stacked on #1583. Every managed Besu chain now runs four validators, the smallest BFT set that tolerates a faulty one, so tests can show quorum and forge evidence a light client accepts as validly sealed. A single validator sealed every header alone.

  • Validators (e2e/internal/harness/chain/evm/besu/besu.go): the four start concurrently on a Docker network of their own and are peered through admin_addPeer once all listen. Static nodes dial at startup, and a peer still booting costs a 30s+ redial backoff while the BFT round timeouts escalate. JVM options cap each node's heap and nearly halve its startup CPU, since a two-chain environment boots eight Besu JVMs at once.
  • Followers and voting (changes test(e2e): relay across a Besu validator turnover #1583's harness): ManagedBesu.Nodes becomes ManagedBesu.Followers, nodes beyond the four validators that join the same network and are peered the same way, replacing the static-nodes follower path. Chain.Validators() now covers every managed Besu chain, and its Keys() returns the genesis validator keys. The turnover tests vote three followers in and three genesis validators out: with four trusted validators, a one-for-one swap leaves the client enough trusted signers to skip the change. The matrix labels such a chain "Besu QBFT (+3 followers)".
  • Timeout refunds (e2e/transfer_test.go): the packet gets ten destination block intervals, enough for the approval and the send to land through a BFT round change, and the test waits for the packet's own timeout to pass. Five seconds raced the two source transactions on four-validator chains.
  • Router access (cli/routeraccess): one public package owns the router methods opened to PUBLIC_ROLE, so ibc deploy and the harness deployer share the list. The harness now also opens multicall and submitMisbehaviour, and checks every method with CanCall.
  • Docker: harness networks need only the e2e labels, so Besu creates its network through network.New; make clean also removes labeled networks. Both e2e AGENTS.md files allow importing any public CLI package.

Part of FOU-1903 (Linear).

Verification

  • On top of the full stack (test(e2e): generate the Besu light-client fixtures from the e2e harness #1590): every deploy and Besu e2e test passes in complete mode against Besu 26.5.0, including both turnover tests (QBFT 204s, IBFT 2.0 241s) and the timeout refund. No containers or networks were left behind.
  • The timeout-refund change passed twice in a row for both the Anvil and Besu tests, with four tests running at once.
  • make -C e2e lint, make -C cli lint, check-matrix, the harness unit and integration tests and go test ./... in cli pass; refgen.py all --check is clean.

  • Linked to GitHub issue with discussion and accepted design, OR link to spec that describes this work. (Linear FOU-1903)
  • Include changelog entry when appropriate (e.g. chores should be omitted from changelog). (n/a: CHANGELOG.md only tracks the spec)
  • Wrote unit and e2e tests if relevant.
  • Updated documentation (docs/) if anything is changed. (e2e/README.md, AGENTS.md files)
  • Added godoc comments if relevant.
  • Self-reviewed Files changed in the GitHub PR explorer.
  • Provide a conventional commit message to follow the repository standards.

🤖 Generated with Claude Code

@gjermundgaraba
gjermundgaraba added this pull request to stack #1577 October 21, 2026 08:30
@linear-code

linear-code Bot commented Oct 21, 2026

Copy link
Copy Markdown

FOU-1903

@gjermundgaraba gjermundgaraba changed the title fou 1903 harness test(e2e): run four peered validators per managed Besu chain Oct 21, 2026
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Vulnerable dependencies

2 advisories across 2 libraries need attention, 2 accepted.

🚨 Needs attention

github.com/cosmos/evm

In use: v0.7.3 — reached from e2e

Advisory CVE Fixed in Summary
GO-2025-3684 — no fix yet Cosmos EVM Allows Partial Precompile State Writes in github.com/cosmos/evm

Go standard library

In use: go1.26.9 — reached from cli, e2e, gen/go/solidity-abi

Advisory CVE Fixed in Summary
GO-2026-6809 CVE-2026-111943 go1.26.10 Distrust compromised intermediate certificates in crypto/x509

Run make vulncheck for the call paths. Either upgrade the library, or add the
advisory to .govulncheck-ignore with a reason if no fix exists yet.

Accepted

Listed in .govulncheck-ignore with a reason.

github.com/pion/dtls/v2

In use: v2.2.12 — reached from cli

Advisory CVE Fixed in Summary
GO-2026-4479 CVE-2026-26014 no fix yet Usage of random nonce generation with AES GCM ciphers risks leaking the authentication key in github.com/pion/dtls

golang.org/x/crypto

In use: v0.57.0 — reached from cli, e2e

Advisory CVE Fixed in Summary
GO-2026-5932 — no fix yet The golang.org/x/crypto/openpgp package is unmaintained, unsafe by design, and has known security issues

From make vulncheck-json on this run: logs.

@gjermundgaraba
gjermundgaraba marked this pull request as ready for review October 21, 2026 10:09
@gjermundgaraba
gjermundgaraba requested a review from a team as a code owner October 21, 2026 10:09
@greptile-apps

greptile-apps Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] This PR appears safe to merge; no actionable defects were found.

Summary

Managed Besu chains now run four validators on a dedicated Docker network.

  • Managed Besu chains start with four validators and can test validator turnover.
  • Timeout-refund tests wait for each packet’s actual deadline.
  • CLI and e2e deployments share the router calls opened to everyone.

No actionable issues were found. Tests were not run during this review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Start managed Besu chain] --> B[Generate four validator keys and genesis]
  B --> C[Create labeled Docker network]
  C --> D[Start four validators and optional followers together]
  D --> E[Wait for every node to listen]
  E --> F[Connect node pairs through admin_addPeer]
  F --> G[Wait for first validator to produce blocks]
  G --> H[Expose RPC, voting, and genesis keys]
  H --> I[Run test]
  I --> J[Remove containers]
  J --> K[Remove network]
Loading

Reviews (2) · Last reviewed commit: "refactor(e2e): stage Besu node dirs in o..." · Reviewed by Greptile

@gjermundgaraba

Copy link
Copy Markdown
Contributor Author

@greptile review

@gjermundgaraba
gjermundgaraba force-pushed the fou-1903-harness branch 3 times, most recently from a66795e to d43f74c Compare October 22, 2026 15:40
@gjermundgaraba
gjermundgaraba force-pushed the fou-1903-harness branch 2 times, most recently from 1e59706 to 84888ba Compare October 22, 2026 19:49
dhfang
dhfang previously approved these changes Oct 22, 2026
Base automatically changed from fou-1612-e2e to main October 22, 2026 20:40
A single validator seals every header alone, so tests cannot show quorum
or forge evidence a light client accepts as validly sealed. Every managed
Besu chain now runs four validators, the smallest BFT set that tolerates a
faulty one, and exposes their keys.

- Besu boots four validators concurrently on a Docker network of its own,
  then peers each with the validators before it through admin_addPeer once
  all listen. Static nodes dial at startup, and a peer still booting costs
  a 30s+ redial backoff while the BFT round timeouts escalate.
- JVM options cap each validator's heap and nearly halve its startup CPU,
  since a two-chain environment boots eight Besu JVMs at once.
- Follower nodes, which the validator turnover tests vote in, now join
  the four validators on the chain's network and are peered the same way.
  Chain.Validators covers every managed Besu chain; Keys returns the
  genesis validator keys. The turnover tests vote three followers in and
  three genesis validators out, since a one-for-one swap leaves the client
  enough trusted signers to skip the change.
- The timeout-refund tests give the packet ten destination block intervals,
  enough for the approval and the send to land through a BFT round change,
  and wait for the packet's own timeout to pass.
- cli/routeraccess owns the router methods opened to PUBLIC_ROLE, so
  ibc deploy and the harness deployer share one list; the harness now
  also opens multicall and submitMisbehaviour, and checks every one.
- Harness networks need only the e2e labels, not the name prefix, so
  Besu creates its network through network.New. make clean also removes
  labeled e2e Docker networks.

Refs: FOU-1903
Signed-off-by: Gjermund Garaba <gjermund@garaba.net>
Parallel tests booting their four-node chains together ran thirty-odd
Besu JVMs on CI's four cores, starving nodes past the 90s wait for their
RPC to listen. Booting is the CPU-bound part, so queue chains at the
boot; running ones cost little.

Signed-off-by: Gjermund Garaba <gjermund@garaba.net>
Signed-off-by: Gjermund Garaba <gjermund@garaba.net>
A QBFT turnover run in CI ended with two voted-out genesis validators
back in the set; ten local runs did not reproduce it. Nothing in the
test casts the add vote Besu would need to re-add them, so the vote
wait most likely passed early and the deferred discard then cancelled
the removal's votes. The wait read only node 0, the first validator
voted out, and counted a null answer, which Besu gives when it has no
header at its head, as a removal.

Every node's head must now show the change, a node reporting no
validators is an error, and the turnover tests vote out genesis
validators 1-3, keeping node 0, which serves the chain's RPC.

Signed-off-by: Gjermund Garaba <gjermund@garaba.net>
@gjermundgaraba
gjermundgaraba dismissed dhfang’s stale review October 22, 2026 20:40

The merge-base changed after approval.

@gjermundgaraba
gjermundgaraba added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit 1e957fe Oct 23, 2026
20 of 21 checks passed
@gjermundgaraba
gjermundgaraba deleted the fou-1903-harness branch October 23, 2026 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants