original_docstring: Returns the attention inputs associated to the inputs of the model.
docstring: Returns the attention inputs associated to the inputs of the model.
docstring_tokens: Returns the attention inputs associated to the inputs of the model .
code: def init_attention_inputs(self, inputs_embeds, attention_mask=None, token_type_ids=None):
          """Returns the attention inputs associated to the inputs of the model."""
          # inputs_embeds has shape batch_size x seq_len x d_model
          # attention_mask and token_type_ids have shape batch_size x seq_len
          self.pooling_mult = 1
          self.seq_len = seq_len = inputs_embeds.size(1)
          position_embeds = self.get_position_embeds(seq_len, inputs_embeds.dtype, inputs_embeds.device)
          token_type_mat = self.token_type_ids_to_mat(token_type_ids) if token_type_ids is not None else None
          cls_mask = (
              nn.functional.pad(inputs_embeds.new_ones([seq_len - 1, seq_len - 1]), (1, 0, 1, 0))
              if self.config.separate_cls
              else None
          )
          return (position_embeds, token_type_mat, attention_mask, cls_mask)
code_tokens: def init_attention_inputs ( self , inputs_embeds , attention_mask = None , token_type_ids = None ) : self . pooling_mult = 1 self . seq_len = seq_len = inputs_embeds . size ( 1 ) position_embeds = self . get_position_embeds ( seq_len , inputs_embeds . dtype , inputs_embeds . device ) token_type_mat = self . token_type_ids_to_mat ( token_type_ids ) if token_type_ids is not None else None cls_mask = ( nn . functional . pad ( inputs_embeds . new_ones ( [ seq_len - 1 , seq_len - 1 ] ) , ( 1 , 0 , 1 , 0 ) ) if self . config . separate_cls else None ) return ( position_embeds , token_type_mat , attention_mask , cls_mask )
short_docstring: Returns the attention inputs associated to the inputs of the model.
short_docstring_tokens: Returns the attention inputs associated to the inputs of the model .
comment: """Returns the attention inputs associated to the inputs of the model."""$SEP$# inputs_embeds has shape batch_size x seq_len x d_model$SEP$# attention_mask and token_type_ids have shape batch_size x seq_len