title: Initializing BERT embeddings in a classification model
body: <p>I'm quite new to TensorFlow and trying to do a multitask classification with BERT (I have done this with GloVe in another part of the project). My problem is with the concept of placeholder in TensorFlow. I know that it is just a placeholder of some variables and will be filled. See this is the part of my classification model that I have problem with. I'll explain the exact problem down here.</p>
  
  <pre><code>def bert_emb_lookup(input_ids):
      # TODO to be implemented;
      """
      X is the input IDs, but a placeholder
      """
      pass
  
  class BertClassificationModel(object):
      def __init__(self, num_class, args):
          self.embedding_size = args.embedding_size
          self.num_layers = args.num_layers
          self.num_hidden = args.num_hidden
  
          self.input_ids = tf.placeholder(tf.int32, [None, args.max_document_len])
          self.Y1 = tf.placeholder(tf.int32, [None])
          self.Y2 = tf.placeholder(tf.int32, [None])
          self.dropout = tf.placeholder(tf.float64, [])
  
          self.input_len = tf.reduce_sum(tf.sign(self.input_ids), 1)
  
          with tf.name_scope("embedding"):
              self.input_emb = bert_emb_lookup(self.)
  ...
  </code></pre>
  
  <p>It was easy to get the word embeddings from GloVe; I first loaded the glove vectors and then simply used <code>tf.nn.embedding_lookup(embeddings, self.input_ids)</code> to fetch the embeddings.</p>
  
  <p>So in BERT classification model, I'm trying to do something similar by defining a function whose argument is <code>input_ids</code>, where I want to match input ids with their associated vocab (string). Thereafter, I'll use an API (BERT as a service) that gives BERT embeddings of any given list of strings at string-level/token-level. The problem is that since <code>self.input_ids</code> is just a placeholder, it shows it a NULL object. Is there any workaround that helps me with this? </p>
  
  <p>Thanks!</p>