{"action":"closed","issue":{"url":"https://api.github.com/repos/tensorflow/models/issues/2890","repository_url":"https://api.github.com/repos/tensorflow/models","labels_url":"https://api.github.com/repos/tensorflow/models/issues/2890/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/models/issues/2890/comments","events_url":"https://api.github.com/repos/tensorflow/models/issues/2890/events","html_url":"https://github.com/tensorflow/models/issues/2890","id":276717982,"node_id":"MDU6SXNzdWUyNzY3MTc5ODI=","number":2890,"title":"Can't train detector model using Google Cloud","user":{"login":"lucasharada","id":18669473,"node_id":"MDQ6VXNlcjE4NjY5NDcz","avatar_url":"https://avatars3.githubusercontent.com/u/18669473?v=4","gravatar_id":"","url":"https://api.github.com/users/lucasharada","html_url":"https://github.com/lucasharada","followers_url":"https://api.github.com/users/lucasharada/followers","following_url":"https://api.github.com/users/lucasharada/following{/other_user}","gists_url":"https://api.github.com/users/lucasharada/gists{/gist_id}","starred_url":"https://api.github.com/users/lucasharada/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lucasharada/subscriptions","organizations_url":"https://api.github.com/users/lucasharada/orgs","repos_url":"https://api.github.com/users/lucasharada/repos","events_url":"https://api.github.com/users/lucasharada/events{/privacy}","received_events_url":"https://api.github.com/users/lucasharada/received_events","type":"User","site_admin":false},"labels":[{"id":387356495,"node_id":"MDU6TGFiZWwzODczNTY0OTU=","url":"https://api.github.com/repos/tensorflow/models/labels/stat:awaiting%20response","name":"stat:awaiting response","color":"5319e7","default":false},{"id":473246237,"node_id":"MDU6TGFiZWw0NzMyNDYyMzc=","url":"https://api.github.com/repos/tensorflow/models/labels/type:build/install","name":"type:build/install","color":"c5def5","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":16,"created_at":"2017-11-25T01:55:18Z","updated_at":"2018-12-12T22:19:01Z","closed_at":"2018-12-12T22:19:01Z","author_association":"NONE","body":"I'm trying to train my own Detector model based on Tensorflow sample and [this post](https://towardsdatascience.com/how-to-train-your-own-object-detector-with-tensorflows-object-detector-api-bec72ecfe1d9). And I did succeed on training locally on a Macbook Pro. The problem is that I don't have a GPU and doing it on the CPU is too slow (about 25-40s per iteration).\r\n\r\nThis way, I'm trying to run on Google Cloud ML Engine following the tutorial, but I can't make it run properly.\r\n\r\nMy folder structures is described below:\r\n```\r\n+ data\r\n - train.record\r\n - test.record\r\n+ models\r\n + train\r\n + eval\r\n+ training\r\n - ssd_mobilenet_v1_coco\r\n```\r\nMy steps to change from local training to Google Cloud training were:\r\n\r\n1. Create a bucket in Google Cloud storage and copy my local folder structure with files;\r\n2. Edit my pipeline.config file and change all paths from Users/dev/detector/ to gcc://bucketname/;\r\n3. Create a YAML file with the default configuration provided in the tutorial;\r\n4. Run\r\n```\r\ngcloud ml-engine jobs submit training object_detection_date +%s \\ \r\n--job-dir=gs://bucketname/models/train \\ \r\n--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz \\ \r\n--module-name object_detection.train \\ \r\n--region us-east1 \\ \r\n--config /Users/dev/detector/training/cloud.yml \\ \r\n-- \\ \r\n--train_dir=gs://bucketname/models/train \\ \r\n--pipeline_config_path=gs://bucketname/data/pipeline.config\r\n```\r\nDoing so, gives me the following error message from the MLUnits:\r\n\r\n> The replica ps 0 exited with a non-zero status of 1. Termination reason: Error. Traceback (most recent call last): File \"/usr/lib/python2.7/runpy.py\", line 162, in _run_module_as_main \"__main__\", fname, loader, pkg_name) File \"/usr/lib/python2.7/runpy.py\", line 72, in _run_code exec code in run_globals File \"/root/.local/lib/python2.7/site-packages/object_detection/train.py\", line 49, in from object_detection import trainer File \"/root/.local/lib/python2.7/site-packages/object_detection/trainer.py\", line 27, in from object_detection.builders import preprocessor_builder File \"/root/.local/lib/python2.7/site-packages/object_detection/builders/preprocessor_builder.py\", line 21, in from object_detection.protos import preprocessor_pb2 File \"/root/.local/lib/python2.7/site-packages/object_detection/protos/preprocessor_pb2.py\", line 71, in options=None, file=DESCRIPTOR), TypeError: __new__() got an unexpected keyword argument 'file'\r\n\r\nThanks in advance."}}
{"actor":{"display_login":"ymodak"}}