Increase memory limit

[sauliusgrigaitis]

Component

sp1-zkvm

Describe the feature you would like

SP1 has memory limit of 1-2GB (depending on the allocator). Grandine is snarkifying Beacon Chain and this limit is not enough for mainnet size STF.

Additional context

No response

[nhtyy]

One option is to use the bump allocator for now.

[mr-ma]

It seems like the bump allocator is the one with the memory limits — see here.
Note that the panic!("Memory limit exceeded (0x78000000)") only occurs when the embedded feature is not enabled.

From the docs, it looks like the embedded allocator is intended to overcome those limits:
https://docs.succinct.xyz/docs/sp1/generating-proofs/advanced#embedded-allocator

[quietBlockchain]

Hi everyone, thanks for highlighting this issue.
I’d like to add that enabling the embedded-allocator by default or at least warning users when it's not enabled and memory is about to exceed limits might improve developer experience significantly.

It might also help to:

Detect the allocator type at runtime or build time and provide a clear error message when the embedded allocator is missing.

Offer a CLI flag or config option that explicitly toggles embedded-allocator, with documentation pointing to its purpose and limitations.

Let me know if this is something worth exploring in a PR I’d be happy to help contribute.

[ArtiomTr]

For Grandine STF, even embedded allocator is not enough to prove mainnet state.
Enabling it via "embedded" feature like this:

sp1-zkvm = { version = '5.2.1', features = ['embedded'] }

At some point, error occurs:

stderr: memory allocation of 239075332 bytes failed
Error: got unimplemented as opcode