title: Tensorflow - TextSum model: How to create my own training data
body: <p>I am trying to create my own training data for TextSum model. As my understanding, I need to put my articles and abstracts to a binary file (in TFRecords). However, I can not create my own training data from raw text files. I don't understand format very clearly, so I am trying to create a very simple binary file using the following code:</p>
  
  <pre><code>files = os.listdir(path)
  writer = tf.python_io.TFRecordWriter("test_data")
  for i, file in enumerate(files):
      content = open(os.path.join(path, file), "r").read()
      example = tf.train.Example(
          features = tf.train.Features(
              feature = {
                  'content': tf.train.Feature(bytes_list=tf.train.BytesList(value=[content]))
              }
          )
      )
  
      serialized = example.SerializeToString()
      writer.write(serialized)
  </code></pre>
  
  <p>And I try to use the following code to read out the value of this test_data file</p>
  
  <pre><code>reader = open("test_data", 'rb')
  len_bytes = reader.read(8)
  str_len = struct.unpack('q', len_bytes)[0]
  example_str = struct.unpack('%ds' % str_len, reader.read(str_len))[0]
  example_pb2.Example.FromString(example_str)
  </code></pre>
  
  <p>But I always get the following error:</p>
  
  <pre><code>  File "dailymail_corpus_to_tfrecords.py", line 34, in check_file
      example_pb2.Example.FromString(example_str)
    File "/home/s1510032/anaconda2/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 770, in FromString
      message.MergeFromString(s)
    File "/home/s1510032/anaconda2/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 1091, in MergeFromString
      if self._InternalParse(serialized, 0, length) != length:
    File "/home/s1510032/anaconda2/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 1117, in InternalParse
      new_pos = local_SkipField(buffer, new_pos, end, tag_bytes)
    File "/home/s1510032/anaconda2/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 850, in SkipField
      return WIRETYPE_TO_SKIPPER[wire_type](buffer, pos, end)
    File "/home/s1510032/anaconda2/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 791, in _SkipLengthDelimited
      raise _DecodeError('Truncated message.')
  google.protobuf.message.DecodeError: Truncated message.
  </code></pre>
  
  <p>I have no idea what is wrong. Please let me know if you have any suggestions to solve this issue.</p>