{"action":"closed","issue":{"url":"https://api.github.com/repos/pierluigiferrari/ssd_keras/issues/119","repository_url":"https://api.github.com/repos/pierluigiferrari/ssd_keras","labels_url":"https://api.github.com/repos/pierluigiferrari/ssd_keras/issues/119/labels{/name}","comments_url":"https://api.github.com/repos/pierluigiferrari/ssd_keras/issues/119/comments","events_url":"https://api.github.com/repos/pierluigiferrari/ssd_keras/issues/119/events","html_url":"https://github.com/pierluigiferrari/ssd_keras/issues/119","id":323092372,"number":119,"title":"Custom DataGenerator to load diffrent format of image lables","user":{"login":"mwindowshz","id":17617515,"avatar_url":"https://avatars3.githubusercontent.com/u/17617515?v=4","gravatar_id":"","url":"https://api.github.com/users/mwindowshz","html_url":"https://github.com/mwindowshz","followers_url":"https://api.github.com/users/mwindowshz/followers","following_url":"https://api.github.com/users/mwindowshz/following{/other_user}","gists_url":"https://api.github.com/users/mwindowshz/gists{/gist_id}","starred_url":"https://api.github.com/users/mwindowshz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwindowshz/subscriptions","organizations_url":"https://api.github.com/users/mwindowshz/orgs","repos_url":"https://api.github.com/users/mwindowshz/repos","events_url":"https://api.github.com/users/mwindowshz/events{/privacy}","received_events_url":"https://api.github.com/users/mwindowshz/received_events","type":"User","site_admin":false},"labels":[{"id":753501642,"url":"https://api.github.com/repos/pierluigiferrari/ssd_keras/labels/stale","name":"stale","color":"ffffff","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-05-15T07:10:03Z","updated_at":"2018-05-29T20:39:42Z","closed_at":"2018-05-29T20:39:42Z","author_association":"NONE","body":"Hi\r\n\r\nI am using data that is saved in diffrent format then xml of VOC\r\nI wrote parse function to read my data\r\nI fill the list of \r\n\r\n\r\nI checked that the format of information is saved in the same way as it comes from parse_xml() function\r\nI started running training on PASCAL VOC using ssd300training notebook\r\nwhen using the original xml training worked as you described\r\nwhen using my input it did not work meaning, training is very slow and after 120 epoch loss is still high 13.xx\r\nwhile using data from xml I got to loss of 4.xx \r\n\r\nTraining loss reduction rate is very slow, I still did not try running more than 120 epoch maybe it would get lower.\r\n\r\nI am trying to understand what data is missing from my parser that causes this difference, since I am running basically the same training on VOC. (later I want to run on my own images, but this is a test scenario to see that the parse function works , for now it does not.)\r\nMy function fills \r\n>   self.filenames\r\n>   self.labels\r\n>   self.classes \r\n>  self.image_ids\r\n>  self.eval_neutral -here I dont have the values of difficult image or not, so all is set to **False**\r\n\r\nI run parse_xml, and my parse function and checked the lists, of labels, filenames etc, and cannot find a difference, it seems to be in the same format.\r\nhere is an example of self.filenames, and self.labels after loading 2 images from PASCAL using my method:\r\n\r\n**labels:**\r\n![image](https://user-images.githubusercontent.com/17617515/40040283-51a880d0-5822-11e8-9d4c-ce1a64e64bb2.png)\r\n**filenames:**\r\n![image](https://user-images.githubusercontent.com/17617515/40040301-679f567a-5822-11e8-8efd-17b6c6380558.png)\r\n\r\nHere are is example of using the parse_xml() function\r\n\r\n\r\n**lables:**\r\n![image](https://user-images.githubusercontent.com/17617515/40040613-a1470a34-5823-11e8-81c7-d9560c0c411a.png)\r\n\r\n**filenames:**\r\n![image](https://user-images.githubusercontent.com/17617515/40040591-8c376da0-5823-11e8-9fa5-fb6b36c8fd49.png)\r\n\r\n**As you can see both functions produce the same data in the same format.**\r\n\r\n\r\n**Can you please point me to the error.**\r\n\r\n\r\nmy function is : \r\n ```\r\n   def parse_yolo_labels(self,\r\n                          images_list_file_name=None, VocLabelsDirReplace=True,\r\n                          ret=False,classes=['background',\r\n                           'aeroplane', 'bicycle', 'bird', 'boat',\r\n                           'bottle', 'bus', 'car', 'cat',\r\n                           'chair', 'cow', 'diningtable', 'dog',\r\n                           'horse', 'motorbike', 'person', 'pottedplant',\r\n                           'sheep', 'sofa', 'train', 'tvmonitor']):\r\n        '''\r\n        Arguments:\r\n            images_list_file_name (str): images list file containing full path for each file files are the jpg images\r\n            next to each jpg file there should be a txt file containing the yolo_marks\r\n            Yolo marks are as follows, x,y center of rect, width height of rect\r\n            all the values are relative to image size, this is why we need to read the image to get its dimensions\r\n\r\n            VOC labels are not in the same folder as the jpeg but in labels folder, so we can read the corresponding\r\n            label from the correct path - this is the use of next argumen:\r\n\r\n           VocLabelsDirReplace (bool):  are VOC files jpg and labels in different folders. if True, then replace the\r\n           'JPEGImages','labels', and load the txt file from labels folder\r\n            ret (bool, optional): Whether or not the image filenames and labels are to be returned.\r\n                Defaults to `False`.\r\n\r\n        Returns:\r\n            None by default, optionally the image filenames and labels.\r\n        '''\r\n\r\n        # Erase data that might have been parsed before\r\n        self.filenames = []\r\n        self.labels = []\r\n        self.image_ids = []\r\n        self.eval_neutral = []\r\n        self.classes = classes\r\n        with open(images_list_file_name) as f:\r\n            # read one image from list, and add its information\r\n            for line in f:\r\n                # open the txt file, so change the ending of file name to be .txt\r\n\r\n                imageFile = str(line.rstrip())\r\n                print(imageFile)\r\n                img = cv2.imread(imageFile)\r\n                height, width, c = img.shape\r\n                # self.filenames.append(imageFile)  only add a file if there are labesl\r\n                current_labels = []\r\n                eval_neutr = []  # We'll store whether a box is annotated as \"difficult\" here.\r\n                labelFile = imageFile.replace('jpg', 'txt')\r\n                if VocLabelsDirReplace:\r\n                    labelFile = labelFile.replace('JPEGImages', 'labels')\r\n\r\n                with open(labelFile) as csvfile:\r\n                    readCSV = csv.reader(csvfile, delimiter=' ')\r\n                    boxes = []\r\n                    for row in readCSV:\r\n                        classID = int(row[0]) + 1  # we add 1 because in ssd 0 is background\r\n                        rectHeight = int(float(row[4]) * height)\r\n                        rectWidth = int(float(row[3]) * width)\r\n                        xmin = int(float(row[1]) * width) - int(rectWidth / 2)\r\n                        ymin = int(float(row[2]) * height) - int(rectHeight / 2)\r\n                        xmax = xmin + rectWidth\r\n                        ymax = ymin + rectHeight\r\n                        imageName = os.path.split(imageFile)\r\n                        \r\n                        box = []\r\n                        \r\n                        item_dict = {'image_name': imageFile,\r\n                                     'class_id': classID,                                     \r\n                                     'xmin': xmin,\r\n                                     'ymin': ymin,\r\n                                     'xmax': xmax,\r\n                                     'ymax': ymax}\r\n                        box = []\r\n                        for item in self.labels_output_format:\r\n                            box.append(item_dict[item])\r\n                        boxes.append(box)\r\n\r\n                       \r\n                        current_labels.append(box)\r\n                        eval_neutr.append(False)\r\n                       \r\n                if current_labels.__len__() > 0:  # only add file and labels if there are labels\r\n                    \r\n                    self.labels.append(boxes)\r\n                    self.filenames.append(imageFile)\r\n                    head,tail = os.path.split(imageFile)\r\n                    imgID = os.path.splitext(tail)[0]\r\n                    self.image_ids.append(imgID)\r\n                    self.eval_neutral.append(eval_neutr)\r\n\r\n        if ret:  # In case we want to return these\r\n            return self.filenames, self.labels\r\n```\r\n\r\nMy data for example is\r\n[ssd.zip](https://github.com/pierluigiferrari/ssd_keras/files/2003755/ssd.zip)\r\nfull code:\r\n[object_detection_2d_data_generator.zip](https://github.com/pierluigiferrari/ssd_keras/files/2003702/object_detection_2d_data_generator.zip)\r\n(should I send other information? full notebook?)\r\n\r\nI am writing to you after trying and training and retraining trying to find the issue.\r\nTried to find answer in other issues posted but could not find. \r\nI know that this is a function I wrote, but I hope you can give me a pointer of what is missing.\r\nHope this is ok.\r\nThanks in advance,\r\nM.\r\n\r\n"}}
{"actor":{"display_login":"stale"}}