Set specific cache directories per test function call
@lhoestq, I think this reaches some memory limit on Linux instances... (?) It looks like the `comet` metric test fails because it tries to load a model in memory.
In the tests I think we have `patch_comet` that mocks the model download + inference. Not sure why it didn't work though.
I can take a look tomorrow (this afternoon is the pytorch ecosystem day) @lhoestq thanks for the hint: I'm going to have a look at that mock... ;) @lhoestq finally I did not find out why the mock is not used... If you can give me some other hint tomorrow...
Implement specific cache directories (datasets, metrics and modules) per test function call.

Currently, the cache directories are set within the temporary test directory, but they are shared across all test function calls.

This PR implements specific cache directories for each test function call, so that tests are atomic and there are no side effects.

