original_docstring: Null distributions for an alternative hypothesis that the:
      a) mean coefficient value over cross validations is not random
      b) mean contribution value over cross validations is not random
  
      Arguments:
          pipeline: a multi-block pipeline
          method: randomization method (permute, bootstrap or custom callback)
          block: x or y
          permutations: number of permutations
          **kwargs: passed to cross_validate_and_test method
      
docstring: Null distributions for an alternative hypothesis that the:
  a) mean coefficient value over cross validations is not random
  b) mean contribution value over cross validations is not random
docstring_tokens: Null distributions for an alternative hypothesis that the : a ) mean coefficient value over cross validations is not random b ) mean contribution value over cross validations is not random
code: def null_distributions_over_cv(
      pipeline: MultiBlockPipeline, train_data, response, block, method='permute', permutations=100,
      groups=('test', 'cross_validation'),
      kinds=('coefficients', 'contributions'),
      **kwargs
  ):
      """Null distributions for an alternative hypothesis that the:
      a) mean coefficient value over cross validations is not random
      b) mean contribution value over cross validations is not random
  
      Arguments:
          pipeline: a multi-block pipeline
          method: randomization method (permute, bootstrap or custom callback)
          block: x or y
          permutations: number of permutations
          **kwargs: passed to cross_validate_and_test method
      """
  
      nulls = {
          group: {
              kind: []
              for kind in kinds
          }
          for group in groups
      }
  
      if not isinstance(pipeline, LeakyPipeline):
          warn('Using early normalization without leaky pipeline - is this intended?')
  
      for i in tqdm(range(permutations)):
          result = cross_validate_and_test(
              pipeline, train_data, response=response,
              randomize=method, progress_bar=False,
              only_coefficients=True, stripped=True,
              warn_early=False,
              **kwargs
          )
  
          for group in groups:
              parent = result if group == 'test' else result.cross_validation
              for kind in kinds:
                  store: AttributesStore = getattr(parent, kind)
                  nulls[group][kind].append(
                      getattr(store, block).data
                  )
  
      return SimpleNamespace(**{
          group: SimpleNamespace(**{
              f'{kind}': pd.concat(values, axis=1)
              for kind, values in kinds.items()
          })
          for group, kinds in nulls.items()
      })
code_tokens: def null_distributions_over_cv ( pipeline : MultiBlockPipeline , train_data , response , block , method = 'permute' , permutations = 100 , groups = ( 'test' , 'cross_validation' ) , kinds = ( 'coefficients' , 'contributions' ) , ** kwargs ) : nulls = { group : { kind : [ ] for kind in kinds } for group in groups } if not isinstance ( pipeline , LeakyPipeline ) : warn ( 'Using early normalization without leaky pipeline - is this intended?' ) for i in tqdm ( range ( permutations ) ) : result = cross_validate_and_test ( pipeline , train_data , response = response , randomize = method , progress_bar = False , only_coefficients = True , stripped = True , warn_early = False , ** kwargs ) for group in groups : parent = result if group == 'test' else result . cross_validation for kind in kinds : store : AttributesStore = getattr ( parent , kind ) nulls [ group ] [ kind ] . append ( getattr ( store , block ) . data ) return SimpleNamespace ( ** { group : SimpleNamespace ( ** { f'{kind}' : pd . concat ( values , axis = 1 ) for kind , values in kinds . items ( ) } ) for group , kinds in nulls . items ( ) } )
short_docstring: Null distributions for an alternative hypothesis that the:
  a) mean coefficient value over cross validations is not random
  b) mean contribution value over cross validations is not random
short_docstring_tokens: Null distributions for an alternative hypothesis that the : a ) mean coefficient value over cross validations is not random b ) mean contribution value over cross validations is not random
comment: """Null distributions for an alternative hypothesis that the:
      a) mean coefficient value over cross validations is not random
      b) mean contribution value over cross validations is not random
  
      Arguments:
          pipeline: a multi-block pipeline
          method: randomization method (permute, bootstrap or custom callback)
          block: x or y
          permutations: number of permutations
          **kwargs: passed to cross_validate_and_test method
      """