Add support for Rekor v2

[haydentherapper]

Description

As outlined on our blog, we are working on a new revision of Rekor v2. Following the developments from Certificate Transparency deployments, Rekor v2 will be backed by a tile-based log, which will simplify maintenance, be cheaper to run, and be more scalable and cacheable.

Sigstore clients will need to make a number of changes to support Rekor v2, as its API is not compatible with Rekor v1. We have also made a significant change to sharding the log, where we create new instances periodically, which will require implementing support for an updated SigningConfig. We have also removed timestamping (SignedEntryTimestamps) from the log, so clients must support fetching timestamps from timestamping authorities.

Note that we will eventually turn down the write path for Rekor v1, but we will announce that at least a year before turndown.

We have provided documentation for the necessary changes. We have followed this guide to implement support in sigstore-go, sigstore-java and sigstore-python, and any of the maintainers of these libraries can answer implementation questions. The conformance test suite has been updated with Rekor v2 and timestamp authority tests as well. If you would like to experiment with Rekor v2, you can spin up a local instance following the documentation, or use Cosign (at HEAD) to test against our staging deployment.

Roughly, the changes include:

• Supporting two new entry types, hashedrekord v0.0.2 and dsse v0.0.2. No other types are implemented in Rekor v2.
• Gracefully handling longer request times. All requests are now batched with checkpoints published every ~2 seconds, and we will later include cosignatures from witnesses which may add another few seconds.
• Handle v2 transparency log bundle fields differently than Rekor v1
• Log ID is now a checkpoint key ID
• Support SigningConfig v0.2, which adds validity windows, API versions and operators to services so that clients can select the newest Rekor deployment. This will also enable us to seamlessly deploy new log instances without any client configuration.
• Support RFC3161 timestamping on signing and verification, as Rekor v2 drops support for timestamps
• No online lookups
• Verify spec-compliant log checkpoints

If you have any questions, feel free to reach out!