title: Corrupted TFrecord
body: <p>I've converted MNIST to TFRecords. I validated that the records are of valid format using this code:</p>
  <pre><code>example.ParseFromString(raw_record.numpy())
  print(example)
  </code></pre>
  <p>Output:</p>
  <pre><code>features {
    feature {
      key: &quot;depth&quot;
      value {
        int64_list {
          value: 1
        }
      }
    }
  ...
  </code></pre>
  <p>I then use the TFRecordDataset to read and parse the files using map function:</p>
  <pre><code>def parse(record):
          feature = {
              'height': tf.io.FixedLenSequenceFeature([], tf.int64, allow_missing=True),
              'width': tf.io.FixedLenSequenceFeature([], tf.int64, allow_missing=True),
              'depth': tf.io.FixedLenSequenceFeature([], tf.int64, allow_missing=True),
              'label': tf.io.FixedLenSequenceFeature([], tf.int64, allow_missing=True),
              'image_raw': tf.io.FixedLenFeature([], tf.string)
          }
      features = tf.io.parse_single_example(record, feature)
  
      image = tf.io.decode_raw(features['image_raw'], tf.uint8)
      image.set_shape([28 * 28])
      image = tf.cast(image, tf.float32) * (1. / 255) - 0.5
      image = tf.reshape(image, [28, 28, 1])
  
      label = tf.cast(features['label'], tf.int32)
  
      return image, label
  
  BATCH_SIZE = 64 * 10
  NUM_PARALLEL_BATCHES = 2
  LOCAL_PATTERN = './data/*'
  ds = tf.data.Dataset.list_files(LOCAL_PATTERN)
  ds = tf.data.TFRecordDataset(filenames=ds) \
      .map(parse) \
      .batch(batch_size=NUM_PARALLEL_BATCHES)
  </code></pre>
  <p>This results in an error:</p>
  <blockquote>
  <p>File
  &quot;/Users/Projects/mobileye/lib/python3.7/site-packages/tensorflow/python/eager/execute.py&quot;,
  line 60, in quick_execute
  inputs, attrs, num_outputs) tensorflow.python.framework.errors_impl.DataLossError:  corrupted
  record at 0    [[node IteratorGetNext (defined at
  cnn_mnist_pipe_gcp.py:94) ]] [Op:__inference_train_function_885]</p>
  </blockquote>
  <p>EDIT: The code for writing the tfrecords:</p>
  <pre><code>  with tf.python_io.TFRecordWriter(filename) as writer:
        image_raw = images[index].tostring()
        example = tf.train.Example(
            features=tf.train.Features(
                feature={
                    'height': _int64_feature(rows),
                    'width': _int64_feature(cols),
                    'depth': _int64_feature(depth),
                    'label': _int64_feature(int(labels[index])),
                    'image_raw': _bytes_feature(image_raw)
                }))
        writer.write(example.SerializeToString())
  </code></pre>