title: How to get tensorflow BERT transformer to predict?
body: <p>Might be a novice question, but I'm using a fine-tuned base cased BERT model trained on the yelp dataset and cannot get predict() to work. My code is below.</p>
  <p>Code:</p>
  <pre class="lang-py prettyprint-override"><code>from datasets import load_dataset
  from transformers import AutoTokenizer
  from transformers import DefaultDataCollator
  import tensorflow as tf
  
  # going to train and test on yelp review dataset
  dataset = load_dataset(&quot;yelp_review_full&quot;)
  print(dataset[&quot;train&quot;][10])
  
  tokenizer = AutoTokenizer.from_pretrained(&quot;bert-base-cased&quot;)
  
  def tokenize_function(examples):
      return tokenizer(examples[&quot;text&quot;], padding=&quot;max_length&quot;, truncation=True)
  
  tokenized_datasets = dataset.map(tokenize_function, batched=True)
  # print(tokenized_datasets[&quot;train&quot;][100])
  
  small_train_dataset = tokenized_datasets[&quot;train&quot;].shuffle(seed=42).select(range(1000))
  small_eval_dataset = tokenized_datasets[&quot;test&quot;].shuffle(seed=42).select(range(1000))
  
  data_collator = DefaultDataCollator(return_tensors=&quot;tf&quot;)
  
  tf_train_dataset = small_train_dataset.to_tf_dataset(
      columns=[&quot;attention_mask&quot;, &quot;input_ids&quot;, &quot;token_type_ids&quot;],
      label_cols=[&quot;labels&quot;],
      shuffle=True,
      collate_fn=data_collator,
      batch_size=8,
  )
  
  tf_validation_dataset = small_eval_dataset.to_tf_dataset(
      columns=[&quot;attention_mask&quot;, &quot;input_ids&quot;, &quot;token_type_ids&quot;],
      label_cols=[&quot;labels&quot;],
      shuffle=True,
      collate_fn=data_collator,
      batch_size=8,
  )
  
  small_train_dataset = tokenized_datasets[&quot;train&quot;].shuffle(seed=42).select(range(1000))
  small_eval_dataset = tokenized_datasets[&quot;test&quot;].shuffle(seed=42).select(range(1000))
  
  data_collator = DefaultDataCollator(return_tensors=&quot;tf&quot;)
  
  tf_train_dataset = small_train_dataset.to_tf_dataset(
      columns=[&quot;attention_mask&quot;, &quot;input_ids&quot;, &quot;token_type_ids&quot;],
      label_cols=[&quot;labels&quot;],
      shuffle=True,
      collate_fn=data_collator,
      batch_size=8,
  )
  
  tf_validation_dataset = small_eval_dataset.to_tf_dataset(
      columns=[&quot;attention_mask&quot;, &quot;input_ids&quot;, &quot;token_type_ids&quot;],
      label_cols=[&quot;labels&quot;],
      shuffle=True,
      collate_fn=data_collator,
      batch_size=8,
  )
  </code></pre>
  <p>I have it trained now, but am trying to use model.predict() with a string or list of strings as input with no success. How would I do this? Appreciate any insight!</p>