i do not like this you re partially parametrizing the test along a different dimension variable  in a nested for loop i see two options  write out every single case as a separate   dumb   test  or parametrize properly i suggested parametrization due to the shape of the problem  you are just testing cases in the cartesian product of various sets of dimensions web  logdir  build x  short  long  x  true  false web  logdir  build x  short  long  x  servo  firmware this is the worst of both worlds  you re copy pasting the test across  web  logdir  build and parametrizing across  short  long  x  server  firmware you are paying the complexity price for the parametrization while still keeping some duplication that the parametrization is supposed to remove 