<p>This is additional answer for whom seeking for SSD architecture, the Tensorflow Object Detection API 1 has include <code>vgg</code> architecture from <code>slim</code> folder, we can import it <code>from nets import vgg</code> directly. I have just only tried with SSD architecture. I follow the ssd-mobilenet config to create respective models from 2 layers 'fc7' and 'conv4_3' as stated in the <a href="https://arxiv.org/pdf/1512.02325.pdf" rel="nofollow noreferrer">paper</a>. Then, save your new <code>SSD-VGG16_feature_extractor.py</code> inside <code>models</code> folder.</p>
<p><strong>Notice</strong> For a correct matching with the <code>vgg</code> in the <a href="https://arxiv.org/pdf/1512.02325.pdf" rel="nofollow noreferrer">paper</a>, you should change 4096 to 1024 and kernel size <code>[7,7]</code> to <code>[3,3]</code> for correct depth features <a href="https://github.com/tensorflow/models/blob/master/research/slim/nets/vgg.py#L206-L209" rel="nofollow noreferrer">https://github.com/tensorflow/models/blob/master/research/slim/nets/vgg.py#L206-L209</a></p>
<pre><code>import tensorflow.compat.v1 as tf
import tf_slim as slim

from object_detection.meta_architectures import ssd_meta_arch
from object_detection.models import feature_map_generators
from object_detection.utils import ops
from object_detection.utils import shape_utils
from nets import vgg


class SSDVgg16FeatureExtractor(ssd_meta_arch.SSDFeatureExtractor):
  &quot;&quot;&quot;SSD Feature Extractor using Vgg16 features.&quot;&quot;&quot;

  def __init__(self,
               is_training,
               depth_multiplier,
               min_depth,
               pad_to_multiple,
               conv_hyperparams_fn,
               reuse_weights=None,
               use_explicit_padding=False,
               use_depthwise=False,
               override_base_feature_extractor_hyperparams=False):
    &quot;&quot;&quot;Vgg16 Feature Extractor for SSD Models.
    Args:
      is_training: whether the network is in training mode.
      depth_multiplier: float depth multiplier for feature extractor.
      min_depth: minimum feature extractor depth.
      pad_to_multiple: the nearest multiple to zero pad the input height and
        width dimensions to.
      conv_hyperparams_fn: A function to construct tf slim arg_scope for conv2d
        and separable_conv2d ops in the layers that are added on top of the
        base feature extractor.
      reuse_weights: Whether to reuse variables. Default is None.
      use_explicit_padding: Whether to use explicit padding when extracting
        features. Default is False.
      use_depthwise: Whether to use depthwise convolutions. Default is False.
      num_layers: Number of SSD layers.
      override_base_feature_extractor_hyperparams: Whether to override
        hyperparameters of the base feature extractor with the one from
        `conv_hyperparams_fn`.
    Raises:
      ValueError: If `override_base_feature_extractor_hyperparams` is False.
    &quot;&quot;&quot;
    super(SSDVgg16FeatureExtractor, self).__init__(
        is_training=is_training,
        depth_multiplier=depth_multiplier,
        min_depth=min_depth,
        pad_to_multiple=pad_to_multiple,
        conv_hyperparams_fn=conv_hyperparams_fn,
        reuse_weights=reuse_weights,
        use_explicit_padding=use_explicit_padding,
        use_depthwise=use_depthwise,
        override_base_feature_extractor_hyperparams=
        override_base_feature_extractor_hyperparams)
    if not self._override_base_feature_extractor_hyperparams:
      raise ValueError('SSD Vgg16 feature extractor always uses'
                       'scope returned by `conv_hyperparams_fn` for both the '
                       'base feature extractor and the additional layers '
                       'added since there is no arg_scope defined for the base '
                       'feature extractor.')

  def preprocess(self, resized_inputs):
    &quot;&quot;&quot;SSD preprocessing.
    Maps pixel values to the range [-1, 1].
    Args:
      resized_inputs: a [batch, height, width, channels] float tensor
        representing a batch of images.
    Returns:
      preprocessed_inputs: a [batch, height, width, channels] float tensor
        representing a batch of images.
    &quot;&quot;&quot;
    return (2.0 / 255.0) * resized_inputs - 1.0

  def extract_features(self, preprocessed_inputs):
    &quot;&quot;&quot;Extract features from preprocessed inputs.
    Args:
      preprocessed_inputs: a [batch, height, width, channels] float tensor
        representing a batch of images.
    Returns:
      feature_maps: a list of tensors where the ith tensor has shape
        [batch, height_i, width_i, depth_i]
    &quot;&quot;&quot;
    preprocessed_inputs = shape_utils.check_min_image_dim(
        33, preprocessed_inputs)

    feature_map_layout = {
        'from_layer': ['FeatureExtractor/vgg_16/conv4/conv4_3', 'FeatureExtractor/vgg_16/fc7', '', '', '', ''],
        'layer_depth': [-1, -1, 512, 256, 256, 128],
        'use_explicit_padding': self._use_explicit_padding,
        'use_depthwise': self._use_depthwise,
    }
    with slim.arg_scope(self._conv_hyperparams_fn()):
      with slim.arg_scope(vgg.vgg_arg_scope()):
        _, image_features = vgg.vgg_16(
            ops.pad_to_multiple(preprocessed_inputs, self._pad_to_multiple),
            num_classes=0)
    print(&quot;Available output head: &quot;)
    print([k for k,v in image_features.items()])
    with slim.arg_scope(self._conv_hyperparams_fn()):
      feature_maps = feature_map_generators.multi_resolution_feature_maps(
        feature_map_layout=feature_map_layout,
        depth_multiplier=self._depth_multiplier,
        min_depth=self._min_depth,
        insert_1x1_conv=True,
        image_features=image_features)

    return list(feature_maps.values())
</code></pre>
<p>Then, you just need to update <code>'ssd_vgg16': SSDVgg16FeatureExtractor</code> in <code>SSD_FEATURE_EXTRACTOR_CLASS_MAP</code> dict in <code>builder/model_builder.py</code> to complete the model.</p>
<p>I have tested and it works like charm</p>
<pre><code>INFO:tensorflow:global_step/sec: 0.195851
I1223 18:19:21.963316 139974845604416 basic_session_run_hooks.py:692] global_step/sec: 0.195851
INFO:tensorflow:loss = 3.674446, step = 700 (510.592 sec)
I1223 18:19:21.964789 139974845604416 basic_session_run_hooks.py:260] loss = 3.674446, step = 700 (510.592 sec)
</code></pre>
Tensorflow object detection API: Custom VGG 16 model
<p>I am in the process of creating a Custom VGG model as a feature extractor of Faster RCNN model in Tensorflow object detection API. As mentioned on in the document <a href="https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/defining_your_own_model.md" rel="nofollow noreferrer">https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/defining_your_own_model.md</a> the feature extractor code consists of <code>extract_proposal_features</code> and  <code>extract_classifier_features</code>. I am using TF slim code of creating the convolution layers (since they Tensorflow team uses it). As a reference please find the model structure of VGG 16 returned using by TF slim</p>

<pre><code>      ([('vgg_16/conv1/conv1_1',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv1/conv1_1/Relu:0' shape=(?, 224, 224, 64) dtype=float32&gt;),
        ('vgg_16/conv1/conv1_2',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv1/conv1_2/Relu:0' shape=(?, 224, 224, 64) dtype=float32&gt;),
        ('vgg_16/vgg_16/pool1',
         &lt;tf.Tensor 'vgg_16/vgg_16/pool1/MaxPool:0' shape=(?, 112, 112, 64) dtype=float32&gt;),
        ('vgg_16/conv2/conv2_1',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv2/conv2_1/Relu:0' shape=(?, 112, 112, 128) dtype=float32&gt;),
        ('vgg_16/conv2/conv2_2',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv2/conv2_2/Relu:0' shape=(?, 112, 112, 128) dtype=float32&gt;),
        ('vgg_16/vgg_16/pool2',
         &lt;tf.Tensor 'vgg_16/vgg_16/pool2/MaxPool:0' shape=(?, 56, 56, 128) dtype=float32&gt;),
        ('vgg_16/conv3/conv3_1',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv3/conv3_1/Relu:0' shape=(?, 56, 56, 256) dtype=float32&gt;),
        ('vgg_16/conv3/conv3_2',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv3/conv3_2/Relu:0' shape=(?, 56, 56, 256) dtype=float32&gt;),
        ('vgg_16/conv3/conv3_3',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv3/conv3_3/Relu:0' shape=(?, 56, 56, 256) dtype=float32&gt;),
        ('vgg_16/vgg_16/pool3',
         &lt;tf.Tensor 'vgg_16/vgg_16/pool3/MaxPool:0' shape=(?, 28, 28, 256) dtype=float32&gt;),
        ('vgg_16/conv4/conv4_1',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv4/conv4_1/Relu:0' shape=(?, 28, 28, 512) dtype=float32&gt;),
        ('vgg_16/conv4/conv4_2',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv4/conv4_2/Relu:0' shape=(?, 28, 28, 512) dtype=float32&gt;),
        ('vgg_16/conv4/conv4_3',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv4/conv4_3/Relu:0' shape=(?, 28, 28, 512) dtype=float32&gt;),
        ('vgg_16/vgg_16/pool4',
         &lt;tf.Tensor 'vgg_16/vgg_16/pool4/MaxPool:0' shape=(?, 14, 14, 512) dtype=float32&gt;),
        ('vgg_16/conv5/conv5_1',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv5/conv5_1/Relu:0' shape=(?, 14, 14, 512) dtype=float32&gt;),
        ('vgg_16/conv5/conv5_2',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv5/conv5_2/Relu:0' shape=(?, 14, 14, 512) dtype=float32&gt;),
        ('vgg_16/conv5/conv5_3',
         &lt;tf.Tensor 'vgg_16/vgg_16/conv5/conv5_3/Relu:0' shape=(?, 14, 14, 512) dtype=float32&gt;),
        ('vgg_16/vgg_16/pool5',
         &lt;tf.Tensor 'vgg_16/vgg_16/pool5/MaxPool:0' shape=(?, 7, 7, 512) dtype=float32&gt;),
        ('vgg_16/fc6',
         &lt;tf.Tensor 'vgg_16/vgg_16/fc6/Relu:0' shape=(?, 1, 1, 4096) dtype=float32&gt;),
        ('vgg_16/fc7',
         &lt;tf.Tensor 'vgg_16/vgg_16/fc7/Relu:0' shape=(?, 1, 1, 4096) dtype=float32&gt;)])
</code></pre>

<p>My question is that, which convolution layer needs to be included and returned in <code>extract_proposal_features</code> method and which convolution layers needs to be included and returned in <code>extract_classifier_features</code>. Please let me know.</p>