Describe the bug

If you try to run ~1,000+ concurrent requests, the mutex on the credentials cache becomes a bottleneck

To Reproduce

Run several thousand concurrent requests, and you'll see performance drop (for our tests, p50 GET latency was 800ms on S3). Tracing revealed that almost all the time is spent acquiring the credentials mutex.

Expected behavior

Since the read:write ratio is so high, we should be able to avoid taking a lock in this case. Tests show that if you do, performance returns to the expected (8-20ms) p50 for HEAD/GET requests on S3.