{"action":"created","issue":{"url":"https://api.github.com/repos/minosworld/minos/issues/135","repository_url":"https://api.github.com/repos/minosworld/minos","labels_url":"https://api.github.com/repos/minosworld/minos/issues/135/labels{/name}","comments_url":"https://api.github.com/repos/minosworld/minos/issues/135/comments","events_url":"https://api.github.com/repos/minosworld/minos/issues/135/events","html_url":"https://github.com/minosworld/minos/issues/135","id":391464482,"node_id":"MDU6SXNzdWUzOTE0NjQ0ODI=","number":135,"title":"Evaluation does not follow the episode schedule in point_goal task / goal position not passed.","user":{"login":"mcimpoi","id":1849908,"node_id":"MDQ6VXNlcjE4NDk5MDg=","avatar_url":"https://avatars2.githubusercontent.com/u/1849908?v=4","gravatar_id":"","url":"https://api.github.com/users/mcimpoi","html_url":"https://github.com/mcimpoi","followers_url":"https://api.github.com/users/mcimpoi/followers","following_url":"https://api.github.com/users/mcimpoi/following{/other_user}","gists_url":"https://api.github.com/users/mcimpoi/gists{/gist_id}","starred_url":"https://api.github.com/users/mcimpoi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mcimpoi/subscriptions","organizations_url":"https://api.github.com/users/mcimpoi/orgs","repos_url":"https://api.github.com/users/mcimpoi/repos","events_url":"https://api.github.com/users/mcimpoi/events{/privacy}","received_events_url":"https://api.github.com/users/mcimpoi/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-12-16T12:52:54Z","updated_at":"2018-12-24T00:05:09Z","closed_at":"2018-12-24T00:05:09Z","author_association":"NONE","body":"Hi, \r\n\r\nI tried several runs of the same agent (unreal), in the attempt to reproduce the results in Table2 in the paper, namely PoingGoal for Matterport3D small and medium.\r\n\r\nHowever, looking at the logs, filtered by \"EPISODE\", for the first episode, at test time, the goal is at different coordinates (* * was added manually ):\r\n```\r\n{\r\n  'sceneId': 'mp3d.2t7WUuJeko7',\r\n  'task': 'point_goal',\r\n  'start': {\r\n    'angle': 1.386,\r\n    'position': [0.672, 0.551, 2.339],\r\n    'cell': {\r\n      'j': 39,\r\n      'isValid': True,\r\n      'i': 80,\r\n      'id': 3707\r\n    }\r\n  },\r\n  'goal': {\r\n    'angle': 1.8403473282415732,\r\n\r\n    **'position': [-4.813260249729765, 0.5735506993532181, 1.2931640217578368],**\r\n\r\n    'room': 'R5',\r\n    'cell': {\r\n      'j': 29,\r\n      'isValid': True,\r\n      'i': 25,\r\n      'id': 2722\r\n    },\r\n    'radius': 0.25\r\n  },\r\n  'shortestPath': {\r\n    'distance': 7.601219330881972,\r\n    'isValid': True,\r\n    'doors': [],\r\n    'rooms': [1, 5, 6]\r\n  }\r\n}\r\n\r\n\r\n{\r\n  'start': {\r\n    'position': [0.672, 0.551, 2.339],\r\n    'cell': {\r\n      'id': 3707,\r\n      'j': 39,\r\n      'i': 80,\r\n      'isValid': True\r\n    },\r\n    'angle': 1.386\r\n  },\r\n  'task': 'point_goal',\r\n  'sceneId': 'mp3d.2t7WUuJeko7',\r\n  'goal': {\r\n    'radius': 0.25,\r\n\r\n    **'position': [-1.235677195070037, 0.5514386199228466, -0.9918128308026164],**\r\n\r\n    'cell': {\r\n      'id': 619,\r\n      'j': 6,\r\n      'i': 61,\r\n      'isValid': True\r\n    },\r\n    'room': 'R0',\r\n    'angle': 0.08224559970336502\r\n  },\r\n  'shortestPath': {\r\n    'distance': 4.262741699796953,\r\n    'doors': [],\r\n    'isValid': True,\r\n    'rooms': [1]\r\n  }\r\n}\r\n```\r\n\r\nThe goal coordinates are different for different runs, and also different from what the episode scheduler for test is providing: \r\n\r\n```\r\n{\r\n  \"pathNumDoors\": 0,\r\n  \"pathNumRooms\": 3,\r\n  \"episode_id\": 10714,\r\n  \"level\": 0,\r\n  \"pathDoorIds\": \"\",\r\n  \"dist\": 5.209,\r\n  \"pathDist\": 6.36,\r\n  \"goal\": {\r\n    \"position\": [-4.473, 0.029, 1.72],\r\n    \"id\": \"\"\r\n  },\r\n  \"scene_id\": \"2t7WUuJeko7\",\r\n  \"room_id\": \"R5\",\r\n  \"start\": {\r\n    \"position\": [0.672, 0.551, 2.339],\r\n    \"angle\": 1.386\r\n  }\r\n}\r\n```\r\nLooking in RoomSimulator.py, seems the problem is in L96-103\r\n```\r\n        if 'goal' in ep_settings:\r\n            if self.params['task'] == 'room_goal':\r\n                room_id = ep_settings['room_id']\r\n                config['goal'] = {'roomIds': [room_id]}\r\n            else:\r\n                object_id = ep_settings['goal']['id']\r\n                if object_id:\r\n                    config['goal'] = {'objectIds': [object_id]}\r\n```\r\n\r\nWhich doesn't handle the point_goal case.\r\nIf I try to set config['goal'] I get the simulator crash with the following stack trace:\r\n\r\n```\r\n 20:36:41,008 INFO b'set active level 0'\r\n 20:36:41,015 ERROR b\"Error fetching model-mp3d.2t7WUuJeko7-matterport-glb TypeError: Cannot read property 'position' of undefined\"\r\n 20:36:41,016 ERROR b'    at NavScene.getCellId (/research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:101809:42)'\r\n 20:36:41,016 ERROR b'    at /research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:101988:21'\r\n 20:36:41,016 ERROR b'    at arrayMap (/research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:4651:24)'\r\n 20:36:41,016 ERROR b'    at Function.map (/research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:13568:15)'\r\n 20:36:41,016 ERROR b'    at NavScene.computeShortestPath (/research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:101987:24)'\r\n 20:36:41,016 ERROR b'    at NavScene.reset (/research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:101679:31)'\r\n 20:36:41,016 ERROR b'    at SimState.reset (/research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:111700:24)'\r\n 20:36:41,016 ERROR b'    at Simulator.__setSceneState (/research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:110007:15)'\r\n 20:36:41,016 ERROR b'    at /research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:110243:12'\r\n 20:36:41,017 ERROR b'    at wrappedCallback (/research/minos_world_forks/minos/minos/server/node_modules/sstk/client/build/STK-core.bundle.js:30066:10)'\r\n 20:36:41,017 ERROR b\"Cannot load: TypeError: Cannot read property 'position' of undefined\"\r\n```\r\n\r\nAny hints in how to fix this? \r\n\r\nThank you!\r\n"},"comment":{"url":"https://api.github.com/repos/minosworld/minos/issues/comments/449671118","html_url":"https://github.com/minosworld/minos/issues/135#issuecomment-449671118","issue_url":"https://api.github.com/repos/minosworld/minos/issues/135","id":449671118,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0OTY3MTExOA==","user":{"login":"angelxuanchang","id":2142116,"node_id":"MDQ6VXNlcjIxNDIxMTY=","avatar_url":"https://avatars0.githubusercontent.com/u/2142116?v=4","gravatar_id":"","url":"https://api.github.com/users/angelxuanchang","html_url":"https://github.com/angelxuanchang","followers_url":"https://api.github.com/users/angelxuanchang/followers","following_url":"https://api.github.com/users/angelxuanchang/following{/other_user}","gists_url":"https://api.github.com/users/angelxuanchang/gists{/gist_id}","starred_url":"https://api.github.com/users/angelxuanchang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/angelxuanchang/subscriptions","organizations_url":"https://api.github.com/users/angelxuanchang/orgs","repos_url":"https://api.github.com/users/angelxuanchang/repos","events_url":"https://api.github.com/users/angelxuanchang/events{/privacy}","received_events_url":"https://api.github.com/users/angelxuanchang/received_events","type":"User","site_admin":false},"created_at":"2018-12-24T00:05:09Z","updated_at":"2018-12-24T00:05:09Z","author_association":"CONTRIBUTOR","body":"Thanks for letting us know about this - we have fixed `override_args` to default to `{}` in `sim_config.py`\r\nI'm closing this issue now - please let us know if you encounter other issues. "}}
{"actor":{"display_login":"angelxuanchang"}}