{"action":"created","issue":{"url":"https://api.github.com/repos/CharlesShang/TFFRCNN/issues/7","repository_url":"https://api.github.com/repos/CharlesShang/TFFRCNN","labels_url":"https://api.github.com/repos/CharlesShang/TFFRCNN/issues/7/labels{/name}","comments_url":"https://api.github.com/repos/CharlesShang/TFFRCNN/issues/7/comments","events_url":"https://api.github.com/repos/CharlesShang/TFFRCNN/issues/7/events","html_url":"https://github.com/CharlesShang/TFFRCNN/issues/7","id":192755595,"number":7,"title":"Using Resnet50 for FRCN","user":{"login":"miraclebiu","id":16793970,"avatar_url":"https://avatars.githubusercontent.com/u/16793970?v=3","gravatar_id":"","url":"https://api.github.com/users/miraclebiu","html_url":"https://github.com/miraclebiu","followers_url":"https://api.github.com/users/miraclebiu/followers","following_url":"https://api.github.com/users/miraclebiu/following{/other_user}","gists_url":"https://api.github.com/users/miraclebiu/gists{/gist_id}","starred_url":"https://api.github.com/users/miraclebiu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/miraclebiu/subscriptions","organizations_url":"https://api.github.com/users/miraclebiu/orgs","repos_url":"https://api.github.com/users/miraclebiu/repos","events_url":"https://api.github.com/users/miraclebiu/events{/privacy}","received_events_url":"https://api.github.com/users/miraclebiu/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-12-01T05:05:49Z","updated_at":"2016-12-01T14:05:38Z","closed_at":"2016-12-01T14:05:38Z","body":"## Sorry to bother you again, I have tried to repalce the VGG to Resnet50 ,so I add two layers: add_layer and batch_normalization layer:\r\n    @layer \r\n    def add(self,input,name):\r\n        return tf.add(input[0],input[1])\r\n\r\n    @layer\r\n    def batch_normalization(self,input,name,relu=True,is_training=False):\r\n        if relu:\r\n            temp_layer=tf.contrib.layers.batch_norm(input,scale=True,center=True,is_training=is_training,scope=name)\r\n            return tf.nn.relu(temp_layer)\r\n        else:\r\n            return tf.contrib.layers.batch_norm(input,scale=True,center=True,is_training=is_training,scope=name)\r\n\r\n## the model file is convert from the caffe-tensorflow project on the  github(you can search this on the github),  and I convert the bn_layer's data in the caffe to the moving_average and so on , but the result is really really bad ,map is about 55% , and I don't the reason ,maybe the model is not convert right?\r\n\r\n## Have you tried to replace VGG to resnet50\r\n\r\n## below is the resnet50_train.py: \r\n\r\n\r\nclass Resnet50_train(Network):\r\n    def __init__(self, trainable=True):\r\n        self.inputs = []\r\n        self.data = tf.placeholder(tf.float32, shape=[None, None, None, 3], name='data')\r\n        self.im_info = tf.placeholder(tf.float32, shape=[None, 3], name='im_info')\r\n        self.gt_boxes = tf.placeholder(tf.float32, shape=[None, 5], name='gt_boxes')\r\n        self.gt_ishard = tf.placeholder(tf.int32, shape=[None], name='gt_ishard')\r\n        self.dontcare_areas = tf.placeholder(tf.float32, shape=[None, 4], name='dontcare_areas')\r\n        self.keep_prob = tf.placeholder(tf.float32)\r\n        self.layers = dict({'data':self.data, 'im_info':self.im_info, 'gt_boxes':self.gt_boxes,\\\r\n                            'gt_ishard': self.gt_ishard, 'dontcare_areas': self.dontcare_areas})\r\n        self.trainable = trainable\r\n        self.setup()\r\n\r\n    def setup(self):\r\n\r\n        n_classes = cfg.NCLASSES\r\n        # anchor_scales = [8, 16, 32]\r\n        anchor_scales = cfg.ANCHOR_SCALES\r\n        _feat_stride = [16, ]\r\n        (self.feed('data')\r\n             .conv(7, 7, 64, 2, 2, relu=False, name='conv1')\r\n             .batch_normalization(relu=True, name='bn_conv1', is_training=False)\r\n             .max_pool(3, 3, 2, 2, padding='VALID',name='pool1')\r\n             .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res2a_branch1')\r\n             .batch_normalization(name='bn2a_branch1',is_training=False,relu=False))\r\n\r\n        (self.feed('pool1')\r\n             .conv(1, 1, 64, 1, 1, biased=False, relu=False, name='res2a_branch2a')\r\n             .batch_normalization(relu=True, name='bn2a_branch2a',is_training=False)\r\n             .conv(3, 3, 64, 1, 1, biased=False, relu=False, name='res2a_branch2b')\r\n             .batch_normalization(relu=True, name='bn2a_branch2b',is_training=False)\r\n             .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res2a_branch2c')\r\n             .batch_normalization(name='bn2a_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('bn2a_branch1', \r\n                   'bn2a_branch2c')\r\n             .add(name='res2a')\r\n             .relu(name='res2a_relu')\r\n             .conv(1, 1, 64, 1, 1, biased=False, relu=False, name='res2b_branch2a')\r\n             .batch_normalization(relu=True, name='bn2b_branch2a',is_training=False)\r\n             .conv(3, 3, 64, 1, 1, biased=False, relu=False, name='res2b_branch2b')\r\n             .batch_normalization(relu=True, name='bn2b_branch2b',is_training=False)\r\n             .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res2b_branch2c')\r\n             .batch_normalization(name='bn2b_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res2a_relu', \r\n                   'bn2b_branch2c')\r\n             .add(name='res2b')\r\n             .relu(name='res2b_relu')\r\n             .conv(1, 1, 64, 1, 1, biased=False, relu=False, name='res2c_branch2a')\r\n             .batch_normalization(relu=True, name='bn2c_branch2a',is_training=False)\r\n             .conv(3, 3, 64, 1, 1, biased=False, relu=False, name='res2c_branch2b')\r\n             .batch_normalization(relu=True, name='bn2c_branch2b',is_training=False)\r\n             .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res2c_branch2c')\r\n             .batch_normalization(name='bn2c_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res2b_relu', \r\n                   'bn2c_branch2c')\r\n             .add(name='res2c')\r\n             .relu(name='res2c_relu')\r\n             .conv(1, 1, 512, 2, 2, biased=False, relu=False, name='res3a_branch1', padding='VALID')\r\n             .batch_normalization(name='bn3a_branch1',is_training=False,relu=False))\r\n\r\n        (self.feed('res2c_relu')\r\n             .conv(1, 1, 128, 2, 2, biased=False, relu=False, name='res3a_branch2a', padding='VALID')\r\n             .batch_normalization(relu=True, name='bn3a_branch2a',is_training=False)\r\n             .conv(3, 3, 128, 1, 1, biased=False, relu=False, name='res3a_branch2b')\r\n             .batch_normalization(relu=True, name='bn3a_branch2b',is_training=False)\r\n             .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res3a_branch2c')\r\n             .batch_normalization(name='bn3a_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('bn3a_branch1', \r\n                   'bn3a_branch2c')\r\n             .add(name='res3a')\r\n             .relu(name='res3a_relu')\r\n             .conv(1, 1, 128, 1, 1, biased=False, relu=False, name='res3b_branch2a')\r\n             .batch_normalization(relu=True, name='bn3b_branch2a',is_training=False)\r\n             .conv(3, 3, 128, 1, 1, biased=False, relu=False, name='res3b_branch2b')\r\n             .batch_normalization(relu=True, name='bn3b_branch2b',is_training=False)\r\n             .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res3b_branch2c')\r\n             .batch_normalization(name='bn3b_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res3a_relu', \r\n                   'bn3b_branch2c')\r\n             .add(name='res3b')\r\n             .relu(name='res3b_relu')\r\n             .conv(1, 1, 128, 1, 1, biased=False, relu=False, name='res3c_branch2a')\r\n             .batch_normalization(relu=True, name='bn3c_branch2a',is_training=False)\r\n             .conv(3, 3, 128, 1, 1, biased=False, relu=False, name='res3c_branch2b')\r\n             .batch_normalization(relu=True, name='bn3c_branch2b',is_training=False)\r\n             .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res3c_branch2c')\r\n             .batch_normalization(name='bn3c_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res3b_relu', \r\n                   'bn3c_branch2c')\r\n             .add(name='res3c')\r\n             .relu(name='res3c_relu')\r\n             .conv(1, 1, 128, 1, 1, biased=False, relu=False, name='res3d_branch2a')\r\n             .batch_normalization(relu=True, name='bn3d_branch2a',is_training=False)\r\n             .conv(3, 3, 128, 1, 1, biased=False, relu=False, name='res3d_branch2b')\r\n             .batch_normalization(relu=True, name='bn3d_branch2b',is_training=False)\r\n             .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res3d_branch2c')\r\n             .batch_normalization(name='bn3d_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res3c_relu', \r\n                   'bn3d_branch2c')\r\n             .add(name='res3d')\r\n             .relu(name='res3d_relu')\r\n             .conv(1, 1, 1024, 2, 2, biased=False, relu=False, name='res4a_branch1', padding='VALID')\r\n             .batch_normalization(name='bn4a_branch1',is_training=False,relu=False))\r\n\r\n        (self.feed('res3d_relu')\r\n             .conv(1, 1, 256, 2, 2, biased=False, relu=False, name='res4a_branch2a', padding='VALID')\r\n             .batch_normalization(relu=True, name='bn4a_branch2a',is_training=False)\r\n             .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4a_branch2b')\r\n             .batch_normalization(relu=True, name='bn4a_branch2b',is_training=False)\r\n             .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4a_branch2c')\r\n             .batch_normalization(name='bn4a_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('bn4a_branch1', \r\n                   'bn4a_branch2c')\r\n             .add(name='res4a')\r\n             .relu(name='res4a_relu')\r\n             .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4b_branch2a')\r\n             .batch_normalization(relu=True, name='bn4b_branch2a',is_training=False)\r\n             .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4b_branch2b')\r\n             .batch_normalization(relu=True, name='bn4b_branch2b',is_training=False)\r\n             .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4b_branch2c')\r\n             .batch_normalization(name='bn4b_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res4a_relu', \r\n                   'bn4b_branch2c')\r\n             .add(name='res4b')\r\n             .relu(name='res4b_relu')\r\n             .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4c_branch2a')\r\n             .batch_normalization(relu=True, name='bn4c_branch2a',is_training=False)\r\n             .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4c_branch2b')\r\n             .batch_normalization(relu=True, name='bn4c_branch2b',is_training=False)\r\n             .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4c_branch2c')\r\n             .batch_normalization(name='bn4c_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res4b_relu', \r\n                   'bn4c_branch2c')\r\n             .add(name='res4c')\r\n             .relu(name='res4c_relu')\r\n             .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4d_branch2a')\r\n             .batch_normalization(relu=True, name='bn4d_branch2a',is_training=False)\r\n             .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4d_branch2b')\r\n             .batch_normalization(relu=True, name='bn4d_branch2b',is_training=False)\r\n             .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4d_branch2c')\r\n             .batch_normalization(name='bn4d_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res4c_relu', \r\n                   'bn4d_branch2c')\r\n             .add(name='res4d')\r\n             .relu(name='res4d_relu')\r\n             .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4e_branch2a')\r\n             .batch_normalization(relu=True, name='bn4e_branch2a',is_training=False)\r\n             .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4e_branch2b')\r\n             .batch_normalization(relu=True, name='bn4e_branch2b',is_training=False)\r\n             .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4e_branch2c')\r\n             .batch_normalization(name='bn4e_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res4d_relu', \r\n                   'bn4e_branch2c')\r\n             .add(name='res4e')\r\n             .relu(name='res4e_relu')\r\n             .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4f_branch2a')\r\n             .batch_normalization(relu=True, name='bn4f_branch2a',is_training=False)\r\n             .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4f_branch2b')\r\n             .batch_normalization(relu=True, name='bn4f_branch2b',is_training=False)\r\n             .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4f_branch2c')\r\n             .batch_normalization(name='bn4f_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res4e_relu', \r\n                   'bn4f_branch2c')\r\n             .add(name='res4f')\r\n             .relu(name='res4f_relu'))\r\n\r\n        \r\n\r\n        #========= RPN ============\r\n        (self.feed('res4f_relu')\r\n             .conv(3,3,512,1,1,name='rpn_conv/3x3')\r\n             .conv(1,1,len(anchor_scales)*3*2 ,1 , 1, padding='VALID', relu = False, name='rpn_cls_score'))\r\n  \r\n        (self.feed('rpn_cls_score', 'gt_boxes', 'gt_ishard', 'dontcare_areas', 'im_info')\r\n             .anchor_target_layer(_feat_stride, anchor_scales, name = 'rpn-data' ))\r\n        # Loss of rpn_cls & rpn_boxes\r\n\r\n        (self.feed('rpn_conv/3x3')\r\n             .conv(1,1,len(anchor_scales)*3*4, 1, 1, padding='VALID', relu = False, name='rpn_bbox_pred'))\r\n\r\n        #========= RoI Proposal ============\r\n        (self.feed('rpn_cls_score')\r\n             .spatial_reshape_layer(2, name = 'rpn_cls_score_reshape')\r\n             .spatial_softmax(name='rpn_cls_prob'))\r\n\r\n        (self.feed('rpn_cls_prob')\r\n             .spatial_reshape_layer(len(anchor_scales)*3*2, name = 'rpn_cls_prob_reshape'))\r\n\r\n        (self.feed('rpn_cls_prob_reshape','rpn_bbox_pred','im_info')\r\n             .proposal_layer(_feat_stride, anchor_scales, 'TRAIN',name = 'rpn_rois'))\r\n\r\n        (self.feed('rpn_rois','gt_boxes', 'gt_ishard', 'dontcare_areas')\r\n             .proposal_target_layer(n_classes,name = 'roi-data'))\r\n\r\n        #========= RCNN ============        \r\n        (self.feed('res4f_relu','roi-data')\r\n             .roi_pool(7,7,1.0/16,name='res5a_branch2a_roipooling')\r\n             .conv(1, 1, 512, 2, 2, biased=False, relu=False, name='res5a_branch2a', padding='VALID')\r\n             .batch_normalization(relu=True, name='bn5a_branch2a',is_training=False)\r\n             .conv(3, 3, 512, 1, 1, biased=False, relu=False, name='res5a_branch2b')\r\n             .batch_normalization(relu=True, name='bn5a_branch2b',is_training=False)\r\n             .conv(1, 1, 2048, 1, 1, biased=False, relu=False, name='res5a_branch2c')\r\n             .batch_normalization(name='bn5a_branch2c',is_training=False,relu=False))\r\n\r\n        (self.feed('res5a_branch2a_roipooling')\r\n             .conv(1,1,2048,2,2,biased=False, relu=False, name='res5a_branch1', padding='VALID')\r\n             .batch_normalization(name='bn5a_branch1',is_training=False,relu=False))\r\n\r\n        (self.feed('bn5a_branch2c','bn5a_branch1')\r\n             .add(name='res5a')\r\n             .relu(name='res5a_relu')\r\n             .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res5b_branch2a')\r\n             .batch_normalization(relu=True, name='bn5b_branch2a',is_training=False)\r\n             .conv(3, 3, 512, 1, 1, biased=False, relu=False, name='res5b_branch2b')\r\n             .batch_normalization(relu=True, name='bn5b_branch2b',is_training=False)\r\n             .conv(1, 1, 2048, 1, 1, biased=False, relu=False, name='res5b_branch2c')\r\n             .batch_normalization(name='bn5b_branch2c',is_training=False,relu=False))\r\n        #pdb.set_trace()\r\n        (self.feed('res5a_relu', \r\n                   'bn5b_branch2c')\r\n             .add(name='res5b')\r\n             .relu(name='res5b_relu')\r\n             .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res5c_branch2a')\r\n             .batch_normalization(relu=True, name='bn5c_branch2a',is_training=False)\r\n             .conv(3, 3, 512, 1, 1, biased=False, relu=False, name='res5c_branch2b')\r\n             .batch_normalization(relu=True, name='bn5c_branch2b',is_training=False)\r\n             .conv(1, 1, 2048, 1, 1, biased=False, relu=False, name='res5c_branch2c')\r\n             .batch_normalization(name='bn5c_branch2c',is_training=False,relu=False))\r\n        #pdb.set_trace()\r\n        (self.feed('res5b_relu',\r\n        \t       'bn5c_branch2c')\r\n             .add(name='res5c')\r\n             .relu(name='res5c_relu')\r\n             .fc(n_classes, relu=False, name='cls_score')\r\n             .softmax(name='cls_prob'))\r\n\r\n        (self.feed('res5c_relu')\r\n             .fc(n_classes*4, relu=False, name='bbox_pred'))\r\n\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/CharlesShang/TFFRCNN/issues/comments/264181014","html_url":"https://github.com/CharlesShang/TFFRCNN/issues/7#issuecomment-264181014","issue_url":"https://api.github.com/repos/CharlesShang/TFFRCNN/issues/7","id":264181014,"user":{"login":"CharlesShang","id":2253266,"avatar_url":"https://avatars.githubusercontent.com/u/2253266?v=3","gravatar_id":"","url":"https://api.github.com/users/CharlesShang","html_url":"https://github.com/CharlesShang","followers_url":"https://api.github.com/users/CharlesShang/followers","following_url":"https://api.github.com/users/CharlesShang/following{/other_user}","gists_url":"https://api.github.com/users/CharlesShang/gists{/gist_id}","starred_url":"https://api.github.com/users/CharlesShang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/CharlesShang/subscriptions","organizations_url":"https://api.github.com/users/CharlesShang/orgs","repos_url":"https://api.github.com/users/CharlesShang/repos","events_url":"https://api.github.com/users/CharlesShang/events{/privacy}","received_events_url":"https://api.github.com/users/CharlesShang/received_events","type":"User","site_admin":false},"created_at":"2016-12-01T14:05:38Z","updated_at":"2016-12-01T14:05:38Z","body":"@miraclebiu Congratulations! Very glad you made it. You can try pull request to send you awesome code to this repo. check this link https://help.github.com/articles/about-pull-requests/ \r\n\r\nAnd there are something very confusing in the original code, like the inconsistency of axis. some where the rpn_score is ordered by (1, H, W, A, 2) and some where (1, H, W, 2, A)."}}
{"actor":{"display_login":"CharlesShang"}}