Seeding notebook tests

[natibek]

Hello,

I started working on a project to create notebook tests with seeding support and multithreaded testing. I noticed that many of the core ideas overlap with nbval, and I’d love to avoid duplicating effort if there's a path for collaboration.

My implementation injects seeding into the notebook (maybe into the first code cell is better to avoid ):

func_id = self._generate_id()
seed_code = f"from random import seed as seed_{func_id};seed_{func_id}({seed});"
if seed_numpy:
    numpy_seed_code = f"from numpy.random import seed as n_seed_{func_id}; n_seed_{func_id}({seed});"
    seed_code += numpy_seed_code

executor(seed_code)

Also, I hope to use multithreading to speed up the testing of multiple notebooks.

I am reaching out to ask if this seems like a feature that can be included in nbval or if I should continue working on it independently. The issue this might cause is with diffing notebooks with nbdime. Also, it may cause more differences in outputs with the original outputs since they were likely run with a different seed (can avoid this by using a flag to disable seeding?).

Would this feature be of interest for inclusion in nbval (perhaps as an optional mode)? If so, would you be open to a PR? Maybe these changes are too breaking to the philosophy of notebook testing. I would love to hear what you have to say on this.