Skip to content

Commit 6ea06fa

Browse files
author
Hendrik van Antwerpen
committed
Add benchmark instructions
1 parent f05019b commit 6ea06fa

File tree

1 file changed

+17
-0
lines changed

1 file changed

+17
-0
lines changed

crates/bpe/README.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -216,3 +216,20 @@ Results for encoding o200k tokens for random 1000 bytes. The backtracking encode
216216
Results for incrementally encoding o200k tokens by appending 10000 random bytes. The appending encoder is slower by a constant factor but overall has similar performance curve as the backtracking encoder encoding all data at once.
217217

218218
![appending runtime comparison](./benches/result/appending-o200k.svg)
219+
220+
### Running
221+
222+
Run the benchmark as follows (required [cargo-criterion](https://crates.io/crates/cargo-criterion) installed):
223+
224+
```sh
225+
cargo criterion
226+
```
227+
228+
(Using `cargo bench` ignores the settings in `criterion.toml`!)
229+
Open the full report which should be located in `target/criterion/reports/index.html`.
230+
231+
Update the figures in this repo as follows (requires `rsvg-convert` from `librsvg` installed):
232+
233+
```sh
234+
script/copy-benchmark-results
235+
```

0 commit comments

Comments
 (0)