Weird runtime variations - are there any caching effects?

[flackbash]

Dear Tom,

First of all thank you for publishing this awesome and easy to use entity linker.

I've been running experiments with ReFinED for a while but only started using it on GPU a few days ago. I noticed some weird variations in the runtime on GPU (maybe they were there on CPU as well, and I didn't pay close attention to the runtime before, but I think I would have noticed):
If I run ReFinED over a benchmark for the first time (or for the first time after linking over several other benchmarks), it takes quite a while (in fact at least as long as on my CPU-only machine: 76s for the Wiki-Fair benchmark). If I run it again immediately on the same benchmark it is lightning fast and links the whole thing in 4s.

Is there any caching used that might explain this behavior? If so, can I disable it to get comparable runtime measurements?

The loading of the model does not count towards my time measurement. The model is loaded before the measurement is started:

self.refined = Refined.from_pretrained(model_name=model_name, entity_set=entity_set)

I'm using ReFinED from inside the ELEVANT entity linking evaluation tool with the AIDA model and the 33M entity set.

Thanks in advance,
Natalie