{"action":"opened","issue":{"url":"https://api.github.com/repos/matterport/Mask_RCNN/issues/829","repository_url":"https://api.github.com/repos/matterport/Mask_RCNN","labels_url":"https://api.github.com/repos/matterport/Mask_RCNN/issues/829/labels{/name}","comments_url":"https://api.github.com/repos/matterport/Mask_RCNN/issues/829/comments","events_url":"https://api.github.com/repos/matterport/Mask_RCNN/issues/829/events","html_url":"https://github.com/matterport/Mask_RCNN/issues/829","id":346924512,"node_id":"MDU6SXNzdWUzNDY5MjQ1MTI=","number":829,"title":"Unexpected Error while using my own dataset.","user":{"login":"MikeXlYang","id":30222595,"node_id":"MDQ6VXNlcjMwMjIyNTk1","avatar_url":"https://avatars1.githubusercontent.com/u/30222595?v=4","gravatar_id":"","url":"https://api.github.com/users/MikeXlYang","html_url":"https://github.com/MikeXlYang","followers_url":"https://api.github.com/users/MikeXlYang/followers","following_url":"https://api.github.com/users/MikeXlYang/following{/other_user}","gists_url":"https://api.github.com/users/MikeXlYang/gists{/gist_id}","starred_url":"https://api.github.com/users/MikeXlYang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MikeXlYang/subscriptions","organizations_url":"https://api.github.com/users/MikeXlYang/orgs","repos_url":"https://api.github.com/users/MikeXlYang/repos","events_url":"https://api.github.com/users/MikeXlYang/events{/privacy}","received_events_url":"https://api.github.com/users/MikeXlYang/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-02T09:08:47Z","updated_at":"2018-08-02T09:08:47Z","closed_at":null,"author_association":"NONE","body":"I'm using MICCAI 2017 EndoVis to do instance segmentation on surgical tools. The datasets includes 1800 images(25 of which have none instances and were excluded). I write my own tool.py as shown below. But when I started training the follow error occurs and the training process doesn't stop until sever iterations.\r\nThe same error happened while another user using 8-channel images.\r\nBut I'm only using simple 3-channel ones.\r\nI could really use some help as I'm pretty bad in tf and python. \r\nReally appreciate your help.\r\n`ERROR:root:Error processing image {'id': '7frame032', 'path': '/home/user/MaskRCNN/Mask_RCNN-master/ReArranged_origin/Left_frames/7frame032.png', 'source': 'tools'}\r\nTraceback (most recent call last):\r\n  File \"/home/user/.local/lib/python3.5/site-packages/mask_rcnn-2.1-py3.5.egg/mrcnn/model.py\", line 1704, in data_generator\r\n    use_mini_mask=config.USE_MINI_MASK)\r\n  File \"/home/user/.local/lib/python3.5/site-packages/mask_rcnn-2.1-py3.5.egg/mrcnn/model.py\", line 1227, in load_image_gt\r\n    mask = utils.resize_mask(mask, scale, padding, crop)\r\n  File \"/home/user/.local/lib/python3.5/site-packages/mask_rcnn-2.1-py3.5.egg/mrcnn/utils.py\", line 517, in resize_mask\r\n    mask = scipy.ndimage.zoom(mask, zoom=[scale, scale, 1], order=0)\r\n  File \"/home/user/.local/lib/python3.5/site-packages/scipy/ndimage/interpolation.py\", line 573, in zoom\r\n    zoom = _ni_support._normalize_sequence(zoom, input.ndim)\r\n  File \"/home/user/.local/lib/python3.5/site-packages/scipy/ndimage/_ni_support.py\", line 65, in _normalize_sequence\r\n`\r\n\r\nAnd the following are my tools.py\r\n`import os\r\nimport sys\r\nimport json\r\nimport datetime\r\nimport numpy as np\r\nimport skimage.io\r\nfrom imgaug import augmenters as iaa\r\n\r\n# Root directory of the project\r\nROOT_DIR = os.path.abspath(\"../../\")\r\n\r\n# Import Mask RCNN\r\nsys.path.append(ROOT_DIR)  # To find local version of the library\r\nfrom mrcnn.config import Config\r\nfrom mrcnn import utils\r\nfrom mrcnn import model as modellib\r\nfrom mrcnn import visualize\r\n\r\n# Path to trained weights file\r\nCOCO_WEIGHTS_PATH = os.path.join(ROOT_DIR, \"mask_rcnn_coco.h5\")\r\n\r\n# Directory to save logs and model checkpoints, if not provided\r\n# through the command line argument --logs\r\nDEFAULT_LOGS_DIR = os.path.join(ROOT_DIR, \"logs\")\r\n\r\n# Results directory\r\n# Save submission files here\r\n# RESULTS_DIR = os.path.join(ROOT_DIR, \"results/tools/\") \r\n# Under Linux\r\nRESULTS_DIR = os.path.join(ROOT_DIR, r\"results\\tools\")\r\n# To acquire val sets, I choose to segment the sequences with \r\n# constant gaps. \r\ntemplenum = 0\r\ntemplename = ''\r\ntemplename2 = ''\r\nvalname = ''\r\nfullidname = ''\r\nVAL_IMAGE_IDS = []\r\nfor idcount in range(1,226):\r\n    if idcount % 9 == 1:\r\n        templenum = str(idcount)\r\n        while len(templenum) < 3:\r\n            templenum = '0' + templenum\r\n        VAL_IMAGE_IDS.append('frame' + str(templenum))\r\n\r\nWRONG_IMAGE_IDS = ['5frame092','5frame093','7frame077',\r\n                   '7frame078','7frame079','7frame081',\r\n                   '7frame086','7frame092','7frame093',\r\n                   '7frame094','7frame095','7frame137',\r\n                   '7frame138','7frame139','7frame140',\r\n                   '7frame141','7frame144','7frame157',\r\n                   '7frame158','7frame159']\r\n############################################################\r\n#  Configurations\r\n############################################################\r\n\r\nclass ToolsConfig(Config):\r\n    \"\"\"Configuration for training on the nucleus segmentation dataset.\"\"\"\r\n    # Give the configuration a recognizable name\r\n    NAME = \"tools\"\r\n\r\n    # Adjust depending on your GPU memory\r\n    IMAGES_PER_GPU = 1\r\n\r\n    # Number of classes (including background)\r\n    NUM_CLASSES = 1 + 6  # Background + nucleus\r\n\r\n    USE_MINI_MASK = False\r\n    # Backbone network architecture\r\n    # Supported values are: resnet50, resnet101\r\n    BACKBONE = \"resnet101\"\r\n\r\n    # Input image resizing\r\n    # Random crops of size 512x512\r\n\r\n\r\n\r\nclass ToolsDataset(utils.Dataset):\r\n\r\n    def load_tools(self, dataset_dir, subset):\r\n        \"\"\"Load a subset of the nuclei dataset.\r\n        dataset_dir: Root directory of the dataset\r\n        subset: Subset to load. Either train or val\r\n        \"\"\"\r\n        # Add classes. We have one class.\r\n        # Naming the dataset nucleus, and the class nucleus\r\n        self.add_class(\"tools\", 1, \"Bipolar_Forceps\")\r\n        self.add_class(\"tools\", 2, \"Curved_Scissors\")\r\n        self.add_class(\"tools\", 3, \"Grasping_Retractor\")\r\n        self.add_class(\"tools\", 4, \"Needle_Driver\")\r\n        self.add_class(\"tools\", 5, \"Prograsp_Forceps\")\r\n        self.add_class(\"tools\", 6, \"Vessel_Sealer\")\r\n        \r\n        # Which subset?\r\n        # \"val\": use hard-coded list above\r\n        # \"train\": use data from stage1_train minus the hard-coded list above\r\n        # else: use the data from the specified sub-directory\r\n        assert subset in [\"train\", \"val\"]\r\n        trimage_ids = []\r\n        # validation set usage\r\n        if subset == \"val\":            \r\n            for imgcount in range(1,9):\r\n                for valname in VAL_IMAGE_IDS:\r\n                    if valname not in WRONG_IMAGE_IDS:\r\n                        templename = str(imgcount) + valname\r\n                        trimage_ids.append(templename)\r\n            for trimage_id in trimage_ids:\r\n                self.add_image(\r\n                        \"tools\",\r\n                        image_id = trimage_id,\r\n                        path = os.path.join(dataset_dir,\"Left_frames\", trimage_id + \".png\"))\r\n        # training set usage\r\n        elif subset == 'train':\r\n            # Get image ids from directory names\r\n            for imgcount in range(1,9):\r\n                for templenum in range(225):\r\n                    templename2 = str(templenum)\r\n                    while len(templename2) < 3:\r\n                        templename2 = '0' + templename2\r\n                    fullidname = 'frame' + templename2\r\n                    if fullidname not in WRONG_IMAGE_IDS:\r\n                        fullidname = str(imgcount) + fullidname\r\n                        if fullidname not in WRONG_IMAGE_IDS:\r\n                            trimage_ids.append(fullidname)\r\n            for trimage_id in trimage_ids:\r\n                self.add_image(\r\n                        \"tools\",\r\n                        image_id = trimage_id,\r\n                        path = os.path.join(dataset_dir,\"Left_frames\", trimage_id + \".png\"))\r\n\r\n    def load_mask(self, image_id):\r\n        \"\"\"Generate instance masks for an image.\r\n       Returns:\r\n        masks: A bool array of shape [height, width, instance count] with\r\n            one mask per instance.\r\n        class_ids: a 1D array of class IDs of the instance masks.\r\n        \"\"\"\r\n        # os.listdir(dataset_dir)\r\n        instance_names = ['Prograsp_Forceps_Right', 'Grasping_Retractor_Right',\r\n                          'Bipolar_Forceps', 'Prograsp_Forceps_Left',\r\n                          'Needle_Driver_Right', 'Vessel_Sealer_Right',\r\n                          'Bipolar_Forceps_Left', 'Needle_Driver_Left', \r\n                          'Prograsp_Forceps', \r\n                          'Grasping_Retractor_Left', 'Curved_Scissors', \r\n                          'Grasping_Retractor', 'Vessel_Sealer']\r\n        instancedic = {'Prograsp_Forceps_Right':5 , 'Grasping_Retractor_Right':3,\r\n                          'Bipolar_Forceps':1, 'Prograsp_Forceps_Left':5,\r\n                          'Needle_Driver_Right':4, 'Vessel_Sealer_Right':6,\r\n                          'Bipolar_Forceps_Left':1, 'Needle_Driver_Left':4, \r\n                          'Prograsp_Forceps':5, \r\n                          'Grasping_Retractor_Left':3, 'Curved_Scissors':2, \r\n                          'Grasping_Retractor':3, 'Vessel_Sealer':6}\r\n        \r\n        mask_dirs = []\r\n        mask = []\r\n        classids = []\r\n        info = self.image_info[image_id]\r\n        for instance_name in instance_names:\r\n            mask_dirs = os.path.join(ROOT_DIR,\"ReArranged_origin\",\r\n                                         instance_name, info['id'] + \".png\" )\r\n            m = skimage.io.imread(mask_dirs).astype(np.bool)\r\n            if np.sum(m) > 0:\r\n                mask.append(m)\r\n                classids.append(instancedic[instance_name])                \r\n        if np.sum(mask) > 0:            \r\n            mask = np.stack(mask, axis=-1)\r\n            classids = np.stack(classids)\r\n        elif np.sum(mask) == 0:\r\n            print(info['id'])\r\n        # Return mask, and array of class IDs of each instance. Since we have\r\n        # one class ID, we return an array of ones\r\n        return mask, classids\r\n\r\n    def image_reference(self, image_id):\r\n        \"\"\"Return the path of the image.\"\"\"\r\n        info = self.image_info[image_id]\r\n        if info[\"source\"] == \"tools\":\r\n            return info[\"id\"]\r\n        else:\r\n            super(self.__class__, self).image_reference(image_id)\r\n\r\n\r\n############################################################\r\n#  Training\r\n############################################################\r\n\r\ndef train(model, dataset_dir, subset):\r\n    \"\"\"Train the model.\"\"\"\r\n    # Training dataset.\r\n    dataset_train = ToolsDataset()\r\n    dataset_train.load_tools(dataset_dir, \"train\")\r\n    dataset_train.prepare()\r\n\r\n    # Validation dataset\r\n    dataset_val = ToolsDataset()\r\n    dataset_val.load_tools(dataset_dir, \"val\")\r\n    dataset_val.prepare()\r\n\r\n    # If starting from imagenet, train heads only for a bit\r\n    # since they have random weights\r\n    print(\"Train network heads\")\r\n    model.train(dataset_train, dataset_val,\r\n                learning_rate=config.LEARNING_RATE,\r\n                epochs=20,\r\n                layers='heads')\r\n\r\n    print(\"Train all layers\")\r\n    model.train(dataset_train, dataset_val,\r\n                learning_rate=config.LEARNING_RATE,\r\n                epochs=40,\r\n                layers='all')\r\n\r\n\r\n\r\nconfig = ToolsConfig()\r\nconfig.display()\r\nmodel = modellib.MaskRCNN(mode=\"training\", config=config,\r\n                                  model_dir=DEFAULT_LOGS_DIR)\r\nweights_path = COCO_WEIGHTS_PATH\r\nmodel.load_weights(weights_path, by_name=True, exclude=[\r\n            \"mrcnn_class_logits\", \"mrcnn_bbox_fc\",\r\n            \"mrcnn_bbox\", \"mrcnn_mask\"])\r\ndatasetsdir = os.path.join(ROOT_DIR,'ReArranged_origin')\r\ntrain(model, datasetsdir, 'train')`"}}
{"actor":{"display_login":"MikeXlYang"}}