Codspeed [now* supports the pytest-benchmark's pedantic API](https://github.com/CodSpeedHQ/pytest-codspeed/issues/77), making it possibly to do setup/teardown. We should update the benchmarking tests to use this where appropriate. *May not be released yet.