Skip to content

core/vm: compute stack operations in place - #35156

Merged
jrhea merged 2 commits into
ethereum:masterfrom
jrhea:evm-stack-ops-ptr
Jun 16, 2026
Merged

core/vm: compute stack operations in place#35156
jrhea merged 2 commits into
ethereum:masterfrom
jrhea:evm-stack-ops-ptr

Conversation

@jrhea

@jrhea jrhea commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

The stack primitives pop by value: pop() returns the 32-byte value itself, so every popped operand is copied out of the stack arena before it is used. The result side was already in place, peek returns a pointer and binary ops write into the new stack top. This PR fixes the operand side: pointer-returning primitives (popPtr, popPtrPeek, etc), with the handlers rewritten to read operands directly from their arena slots. Every popped operand paid the copy, whatever the op went on to do with it, so this optimization covers the arithmetic and comparison ops as much as JUMP, MSTORE, SSTORE and RETURN.

The copy is visible in the assembly. On arm64, master's opLt spends four instructions moving the popped value through the frame, and the comparison then reads it back from there:

LDP  (R5), (R6, R7)              ; load words 0 and 1 of the popped value from the arena
LDP  16(R5), (R5, R8)            ; load words 2 and 3
STP  (R6, R7), vm.~r0-64(SP)     ; store words 0 and 1 into a frame slot
STP  (R5, R8), vm.~r0-48(SP)     ; store words 2 and 3

With popPtrPeek those four instructions are gone, the frame shrinks from locals=0x58 to locals=0x18, and the function from 336 to 288 bytes. The compiler cannot remove the copy itself: uint256.Int is a four-element array, and Go's SSA does not promote arrays longer than one element to registers, so a by-value pop pays this round trip no matter how far inlining gets, for LT exactly as for ADD.

The CALL and CREATE families are deliberately not converted: a child frame reuses the same stack arena, so parent pointers into popped slots die when the child pushes. The rule is recorded on the primitives: pointers stay valid until the next push or any sub call. Converting the call family safely means materializing scalars before the child call, left for later work with a call-heavy benchmark to justify it.

Benchmarks

Measured with the benchmark suite from #35144 (the evm-bench contract workloads and the block import benchmark), which is not part of this PR's diff. Apple M4 Max, fixed iteration counts, n=10, all p=0.000. B/op and allocs/op are statistically identical on every benchmark:

benchmark master PR vs master
Snailtracer 60.0 ms 54.1 ms -9.8%
TenThousandHashes 13.2 ms 12.2 ms -7.8%
ERC20Transfer 11.7 ms 11.0 ms -5.5%
ERC20Mint 7.49 ms 7.02 ms -6.2%
ERC20ApprovalTransfer 8.92 ms 8.44 ms -5.4%

This PR is independent of #35144 but plays nicely with it: the generated dispatch there splices these handler bodies, so the in-place forms land in its fast path too, where they measure larger.

Testing

The rewritten handlers run on the interpreter's only execution path, so correctness rests on references outside the change:

  • Consensus fixtures. The full tests package passes: state tests, the execution-spec families, blockchain tests.
  • Opcode testcases. The JSON testcases compare individual opcode results against committed expected values.
  • Tracer fixtures. The tracetest reference files pin exact log and return data shapes, covering the rewritten LOG and RETURN paths.
  • Cross-build differential. A goevmlab campaign running this branch's evm against master's evm over generated state tests across four forks (Prague, Cancun, London, Osaka) with full trace comparison: 160,566 tests, zero divergences.

@jrhea
jrhea requested a review from rjl493456442 as a code owner June 11, 2026 22:52
Comment thread core/vm/eips.go
)
// These values are checked for overflow during memory expansion calculation
// (the memorySize function on the opcode).
scope.Memory.Copy(dst.Uint64(), src.Uint64(), length.Uint64())

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this helps a handler that immediately collapses its operands to integers:

dst, src, length := scope.Stack.popPtr3()
scope.Memory.Copy(dst.Uint64(), src.Uint64(), length.Uint64())

Uint64() is return z[0], one 8-byte load. The high three words are never read! The old pop() materialized all four words anyway: 4 loads plus 4 stores per operand, then one load back from the frame local. 9 memory operations per operand, 8 of them wasted, versus 1 through the pointer. Across three operands that is 27 memory operations reduced down to 3.

Comment thread core/vm/eips.go Outdated
@s1na

s1na commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

For completeness sake a block import benchmark:

Baseline: master @ 906727089
Workload: 1,000 mainnet blocks (24,950,066 → 24,951,065), 3 runs each
Hardware: Intel Ultra 7 255H, governor=performance, ZFS clone per run

Summary

Metric master pr-35156 Δ
Throughput 189.7 MGas/s (±0.2%) 192.4 MGas/s (±0.5%) +1.4%
Mean newPayload 160 ms (±0.2%) 158 ms (±0.5%) −1.4%
p50 150 ms 147 ms −1.7%
p95 263 ms (±0.6%) 262 ms (±2.7%) −0.4%
p99 379 ms (±3.1%) 383 ms (±3.4%) +1.2% (within noise)

Execution breakdown (avg/block)

Stage master pr-35156 Δ
EVM execution 91 ms 88 ms −3.4%
State read 13 ms 13 ms +1.0%
State hash 9 ms 9 ms +1.4%
Commit 22 ms 22 ms +1.4%

Per-run throughput (MGas/s)

run 1 run 2 run 3
master 189.2 189.9 189.8
pr-35156 193.0 193.0 191.1

@MariusVanDerWijden MariusVanDerWijden left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jrhea
jrhea merged commit 6e62cc5 into ethereum:master Jun 16, 2026
20 of 23 checks passed
@jrhea
jrhea deleted the evm-stack-ops-ptr branch June 16, 2026 12:47
@jrhea jrhea added this to the 1.17.4 milestone Jun 16, 2026
AdityaSripal pushed a commit to AdityaSripal/go-ethereum that referenced this pull request Jun 21, 2026
The stack primitives pop by value: pop() returns the 32-byte value
itself, so every popped operand is copied out of the stack arena before
it is used. The result side was already in place, peek returns a pointer
and binary ops write into the new stack top. This PR fixes the operand
side: pointer-returning primitives (popPtr, popPtrPeek, etc), with the
handlers rewritten to read operands directly from their arena slots.
Every popped operand paid the copy, whatever the op went on to do with
it, so this optimization covers the arithmetic and comparison ops as
much as JUMP, MSTORE, SSTORE and RETURN.

The copy is visible in the assembly. On arm64, master's opLt spends four
instructions moving the popped value through the frame, and the
comparison then reads it back from there:

LDP (R5), (R6, R7) ; load words 0 and 1 of the popped value from the
arena
    LDP  16(R5), (R5, R8)            ; load words 2 and 3
STP (R6, R7), vm.~r0-64(SP) ; store words 0 and 1 into a frame slot
    STP  (R5, R8), vm.~r0-48(SP)     ; store words 2 and 3

With popPtrPeek those four instructions are gone, the frame shrinks from
locals=0x58 to locals=0x18, and the function from 336 to 288 bytes. The
compiler cannot remove the copy itself: uint256.Int is a four-element
array, and Go's SSA does not promote arrays longer than one element to
registers, so a by-value pop pays this round trip no matter how far
inlining gets, for LT exactly as for ADD.

The CALL and CREATE families are deliberately not converted: a child
frame reuses the same stack arena, so parent pointers into popped slots
die when the child pushes. The rule is recorded on the primitives:
pointers stay valid until the next push or any sub call. Converting the
call family safely means materializing scalars before the child call,
left for later work with a call-heavy benchmark to justify it.

### Benchmarks

Measured with the benchmark suite from ethereum#35144 (the evm-bench contract
workloads and the block import benchmark), which is not part of this
PR's diff. Apple M4 Max, fixed iteration counts, n=10, all p=0.000. B/op
and allocs/op are statistically identical on every benchmark:

| benchmark | master | PR | vs master |
|---|---|---|---|
| Snailtracer | 60.0 ms | 54.1 ms | -9.8% |
| TenThousandHashes | 13.2 ms | 12.2 ms | -7.8% |
| ERC20Transfer | 11.7 ms | 11.0 ms | -5.5% |
| ERC20Mint | 7.49 ms | 7.02 ms | -6.2% |
| ERC20ApprovalTransfer | 8.92 ms | 8.44 ms | -5.4% |

This PR is independent of ethereum#35144 but plays nicely with it: the generated
dispatch there splices these handler bodies, so the in-place forms land
in its fast path too, where they measure larger.

### Testing

The rewritten handlers run on the interpreter's only execution path, so
correctness rests on references outside the change:

- **Consensus fixtures.** The full tests package passes: state tests,
the execution-spec families, blockchain tests.
- **Opcode testcases.** The JSON testcases compare individual opcode
results against committed expected values.
- **Tracer fixtures.** The tracetest reference files pin exact log and
return data shapes, covering the rewritten LOG and RETURN paths.
- **Cross-build differential.** A goevmlab campaign running this
branch's evm against master's evm over generated state tests across four
forks (Prague, Cancun, London, Osaka) with full trace comparison:
160,566 tests, zero divergences.

---------

Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants