title: Allocator (GPU_0_bfc) ran out of memory trying to allocate
body: <p>I know this is recurrent problem and question, and I'm aware of some solutions, specially if this problem occurs during training (reducing batch size, setting gpu option <code>allow_grouth=True</code>), but I'm facing this problem when using the <em>already trained</em> model to predict an outcome. 
  So, I was able to train the model (a Faster RCNN from the <a href="https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md" rel="nofollow noreferrer">TensorFlow Object Detection Model Zoo</a> with a batch size of 1 -- otherwise I would receive OOM error during training). </p>
  
  <p>To apply the trained model, I load it with the following code:</p>
  
  <pre><code>class Model:
  
      def __init__(self, conf):
          self.threshold = conf["threshold"]
          self.gpu = conf['gpu']
          self.scope = conf['scope']
          self.frozen_inference_graph = conf['frozen_inference_graph']
          self.detection_graph = self.load_model(self.gpu, self.scope, self.frozen_inference_graph)
  
          gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=conf['gpu_usage'], allocator_type='BFC')
          self.session = tf.Session(graph=self.detection_graph,
                                    config=tf.ConfigProto(gpu_options=gpu_options,
                                                          log_device_placement=True,
                                                          allow_soft_placement=True)
  
      def load_model(gpu, scope, frozen_inference_graph):
          detection_graph = tf.Graph()
          with detection_graph.device(gpu):
              with detection_graph.as_default():
                  od_graph_def = tf.GraphDef()
  
                  with tf.gfile.GFile(frozen_inference_graph, 'rb') as fid:
                      serialized_graph = fid.read()
                      od_graph_def.ParseFromString(serialized_graph)
                      tf.import_graph_def(od_graph_def, name=scope)
                      return detection_graph
  
  </code></pre>
  
  <p>Where my conf file looks like:</p>
  
  <pre><code>{
      "frozen_inference_graph": "/config/plate_detector/faster_rcnn/frozen_inference_graph.pb",
      "gpu": "/gpu:0",
      "gpu_usage": 0.05,
      "threshold": 0.5,
      "scope": "testing"
    }
  </code></pre>
  
  <p>To apply the model, the class <code>Model</code> has the following method:</p>
  
  <pre><code>    def detect(self, image):
          image_tensor = self.detection_graph.get_tensor_by_name(self.scope + "/image_tensor:0")
          detection_boxes = self.detection_graph.get_tensor_by_name(self.scope + "/detection_boxes:0")
          detection_scores = self.detection_graph.get_tensor_by_name(self.scope + "/detection_scores:0")
          detection_classes = self.detection_graph.get_tensor_by_name(self.scope + "/detection_classes:0")
          num_detections = self.detection_graph.get_tensor_by_name(self.scope + "/num_detections:0")
  
          height, width = image.shape[:2]
          image_np = image.copy()
          # Expands dimensions since the model expects images to have shape: [1, None, None, 3]
          image_np_expanded = np.expand_dims(image_np, axis=0)
          # Actual detection
          boxes, scores, classes, num = self.session.run(
              [detection_boxes, detection_scores, detection_classes, num_detections],
              feed_dict={image_tensor: image_np_expanded}
          )
  
          output_dict = dict()
          output_dict["num_dets"] = int(num[0])
          output_dict["classes"] = classes[0].astype(np.uint8)
          output_dict["bboxes"] = boxes[0]
          output_dict["confidences"] = scores[0]
  
          detections = []
          for i, b in enumerate(output_dict["bboxes"]):
              if output_dict["confidences"][i] &gt;= self.threshold:
                  x1 = int(output_dict["bboxes"][i][1] * width)
                  y1 = int(output_dict["bboxes"][i][0] * height)
                  x2 = int(output_dict["bboxes"][i][3] * width)
                  y2 = int(output_dict["bboxes"][i][2] * height)
                  label = self.class_mapping.get(output_dict["classes"][i].astype(str))
                  confidence = output_dict["confidences"][i]
                  detections.append(detection.Detection([x1, y1, x2, y2, confidence, label]))
  
          # Sort based on x1 to return the right order
          detections = sorted(detections, key=lambda x: x.x1)
  
          return detections
  </code></pre>
  
  <p>In my <code>main.py</code> file, I simply do (pseudo code):</p>
  
  <pre><code>model = Model(*path to config*)
  images = fetch_images()
  for image in images:
      detections = model.detect(image)
      // processing over the detections
  </code></pre>
  
  <p>However, sometimes, in the middle of the process, I get a message such as:</p>
  
  <blockquote>
    <p>2019-10-16 21:42:32.394227: W tensorflow/core/common_runtime/bfc_allocator.cc:314] Allocator (GPU_0_bfc) ran out of memory trying to allocate 76.56MiB (rounded to 80281600).  Current allocation summary follows.</p>
  </blockquote>
  
  <p><strong>This doesn't happens all the time, it seems to happen at random, which I don't understand.</strong></p>
  
  <p>I've tried using <code>allow_growth=True</code>, which causes the error to happens when I load the model, and also <code>allocator_type='BFC'</code>, which didn't help (OOM still happens randomly).</p>
  
  <p><code>watch -n 1 nvidia-smi</code> looks like this:</p>
  
  <pre><code>+-----------------------------------------------------------------------------+
  | NVIDIA-SMI 435.21       Driver Version: 435.21       CUDA Version: 10.1     |
  |-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |===============================+======================+======================|
  |   0  Quadro RTX 4000     Off  | 00000000:01:00.0  On |                  N/A |
  | 30%   40C    P0    96W / 125W |   1589MiB /  7979MiB |     83%      Default |
  +-------------------------------+----------------------+----------------------+
  </code></pre>