Add TinyGo JSON/Regex serialization benchmarks #302
Conversation
benchmarks/tinygo/build.sh
Outdated
| cd "$dir" | ||
|
|
||
| # Build with TinyGo | ||
| "$TINYGO" build -o "$OUTPUT_DIR/tinygo-$benchmark.wasm" -target=wasi . |
There was a problem hiding this comment.
Here I think, at a minimum, we want -opt=2 and possibly -gc=leaking to match how this is used in serverless (ty @dgryski). I'm going to enable both of those and we can decide later if we want a bench that covers that tinygo gc.
- Create Dockerfile.tinygo template for TinyGo benchmarks - Add tinygo-json benchmark with encoding/json - Add tinygo-regex benchmark using standard library regexp
There's still a large divide with the rust regex bench but this brings things a bit closer from 32x slower to 19x slower in a quick benchmark.
posborne
force-pushed
the
tinygo-benchmarks
branch
from
2392a49 to
adbebd4
Compare
|
I didn't do it with this pass, but it should be possible to build the identical benchmarks with "big" go. That's probably not super useful in terms of coverage, but I may confirm that it works in order to allow for the top-level directory be renamed to just "go" vs. "tinygo". |
|
Big-go is a no-go here at present due to an interaction between:
I tried a few workaround local to the test; apart from fully embedding the input file we probably need to change the environment which would require changes to wasmtime (which is fine but less than ideal in terms of being able to benchmark older engines). |
| cd "$dir" | ||
|
|
||
| # Build with TinyGo | ||
| "$TINYGO" build -o "$OUTPUT_DIR/tinygo-$benchmark.wasm" -target=wasi -opt=2 -gc=leaking . |
There was a problem hiding this comment.
Can you add a comment here documenting why -gc=leaking is used here?
2 participants
Sample size of one, but in some quick comparison the regex with tinygo is fairly slow taking ~32x which is somewhat surprising. CC @dgryski to review to ensure the workloads here are useful in representing some proxy of real-world workloads.
Given how much slower tinygo appears to be here it may make sense to truncate the input file to reduce the per-iteration time of the regex benchmark.
As with the previous round of benchmarks, portions of this PR were generated using an LLM though reworked a fair bit. I opted to use a single tinygo/ benchmarks subdirectory as it was easier to achieve reuse in that way that make it simpler to add other tinygo benchmarks as we see fit.