{"action":"created","issue":{"url":"https://api.github.com/repos/Xilinx/CHaiDNN/issues/95","repository_url":"https://api.github.com/repos/Xilinx/CHaiDNN","labels_url":"https://api.github.com/repos/Xilinx/CHaiDNN/issues/95/labels{/name}","comments_url":"https://api.github.com/repos/Xilinx/CHaiDNN/issues/95/comments","events_url":"https://api.github.com/repos/Xilinx/CHaiDNN/issues/95/events","html_url":"https://github.com/Xilinx/CHaiDNN/issues/95","id":375353012,"node_id":"MDU6SXNzdWUzNzUzNTMwMTI=","number":95,"title":"XportDNN tool having an error when trying to convert a model using Xilinx quantization method","user":{"login":"DanielsJusts","id":37595234,"node_id":"MDQ6VXNlcjM3NTk1MjM0","avatar_url":"https://avatars3.githubusercontent.com/u/37595234?v=4","gravatar_id":"","url":"https://api.github.com/users/DanielsJusts","html_url":"https://github.com/DanielsJusts","followers_url":"https://api.github.com/users/DanielsJusts/followers","following_url":"https://api.github.com/users/DanielsJusts/following{/other_user}","gists_url":"https://api.github.com/users/DanielsJusts/gists{/gist_id}","starred_url":"https://api.github.com/users/DanielsJusts/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DanielsJusts/subscriptions","organizations_url":"https://api.github.com/users/DanielsJusts/orgs","repos_url":"https://api.github.com/users/DanielsJusts/repos","events_url":"https://api.github.com/users/DanielsJusts/events{/privacy}","received_events_url":"https://api.github.com/users/DanielsJusts/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"anilmartha","id":24727351,"node_id":"MDQ6VXNlcjI0NzI3MzUx","avatar_url":"https://avatars1.githubusercontent.com/u/24727351?v=4","gravatar_id":"","url":"https://api.github.com/users/anilmartha","html_url":"https://github.com/anilmartha","followers_url":"https://api.github.com/users/anilmartha/followers","following_url":"https://api.github.com/users/anilmartha/following{/other_user}","gists_url":"https://api.github.com/users/anilmartha/gists{/gist_id}","starred_url":"https://api.github.com/users/anilmartha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anilmartha/subscriptions","organizations_url":"https://api.github.com/users/anilmartha/orgs","repos_url":"https://api.github.com/users/anilmartha/repos","events_url":"https://api.github.com/users/anilmartha/events{/privacy}","received_events_url":"https://api.github.com/users/anilmartha/received_events","type":"User","site_admin":false},"assignees":[{"login":"anilmartha","id":24727351,"node_id":"MDQ6VXNlcjI0NzI3MzUx","avatar_url":"https://avatars1.githubusercontent.com/u/24727351?v=4","gravatar_id":"","url":"https://api.github.com/users/anilmartha","html_url":"https://github.com/anilmartha","followers_url":"https://api.github.com/users/anilmartha/followers","following_url":"https://api.github.com/users/anilmartha/following{/other_user}","gists_url":"https://api.github.com/users/anilmartha/gists{/gist_id}","starred_url":"https://api.github.com/users/anilmartha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anilmartha/subscriptions","organizations_url":"https://api.github.com/users/anilmartha/orgs","repos_url":"https://api.github.com/users/anilmartha/repos","events_url":"https://api.github.com/users/anilmartha/events{/privacy}","received_events_url":"https://api.github.com/users/anilmartha/received_events","type":"User","site_admin":false},{"login":"smenon009","id":36900075,"node_id":"MDQ6VXNlcjM2OTAwMDc1","avatar_url":"https://avatars1.githubusercontent.com/u/36900075?v=4","gravatar_id":"","url":"https://api.github.com/users/smenon009","html_url":"https://github.com/smenon009","followers_url":"https://api.github.com/users/smenon009/followers","following_url":"https://api.github.com/users/smenon009/following{/other_user}","gists_url":"https://api.github.com/users/smenon009/gists{/gist_id}","starred_url":"https://api.github.com/users/smenon009/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/smenon009/subscriptions","organizations_url":"https://api.github.com/users/smenon009/orgs","repos_url":"https://api.github.com/users/smenon009/repos","events_url":"https://api.github.com/users/smenon009/events{/privacy}","received_events_url":"https://api.github.com/users/smenon009/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2018-10-30T07:24:49Z","updated_at":"2018-10-31T07:30:33Z","closed_at":null,"author_association":"NONE","body":"Hello!\r\n\r\nI'm having trouble using XportDNN tool when using \"Xilinx\" quantization method. I also posted on Xilinx Forums, but I had no response there, so I thought I might try to get help here.\r\n\r\n[Here](https://drive.google.com/open?id=1FtFBInakbVYGw3emFRZn1xHWdo3NJCLG) is a link to my Google Drive folder, where you can access the .prototxt and .caffemodel files of my model, if anyone needs them.\r\n\r\nThe Python script throws an error:\r\n```\r\nTraceback (most recent call last):\r\n  File \"./tools_binaries/quantize.py\", line 1307, in <module>\r\n    print sys.argv[0], \"--deploy_model\", args.deploy_model, \"--weights\", args.weights, \"--quantized_deploy_model\", args.quantized_deploy_model, \"--quantized_weights\", args.quantized_weights, \"--calibration_directory\", args.calibration_directory, \"--calibration_size\", args.calibration_size, \"--calibration_indices\", calibration_indices_str, \"--bitwidths\", args.bitwidths, \"--dims\", args.dims, \"--transpose\", args.transpose, \"--channel_swap\", args.channel_swap, \"--raw_scale\", args.raw_scale, \"--mean_value\", args.mean_value, \"--input_scale\", args.input_scale, \"--gpu \"\r\n  File \"./tools_binaries/quantize.py\", line 1299, in main\r\n    \r\n  File \"./tools_binaries/quantize.py\", line 1059, in execute_calibration\r\n    net, bw_layer_in, th_layer_in, bw_layer_out, th_layer_out = quantize_flatten_reshape(name, net, net_parameter, bw_layer_in, th_layer_in, bw_layer_out, th_layer_out)\r\n  File \"./tools_binaries/quantize.py\", line 249, in quantize_batchnorm\r\n    data = net.blobs[net.top_names[name][0]].data[...]\r\n  File \"./tools_binaries/quantize.py\", line 102, in ThresholdLayerOutputs_cpu\r\n    \r\n  File \"./tools_binaries/quantize.py\", line 40, in compute_threshold\r\n    mn = 0\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/numpy/lib/histograms.py\", line 710, in histogram\r\n    bin_edges, uniform_bins = _get_bin_edges(a, bins, range, weights)\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/numpy/lib/histograms.py\", line 333, in _get_bin_edges\r\n    first_edge, last_edge = _get_outer_edges(a, range)\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/numpy/lib/histograms.py\", line 253, in _get_outer_edges\r\n    \"supplied range of [{}, {}] is not finite\".format(first_edge, last_edge))\r\nValueError: supplied range of [0, inf] is not finite\r\n```\r\n\r\nI have no clue as to where to look for the issue. Only \"weird\" thing I noticed is that the threshold values for the layer tend to grow bigger from layer to layer, so that lead me to believe that the issue might come from there, but I don't know how to debug that. An example:\r\n```\r\n--------------------------------------------------------------------------------\r\n0 95\r\ndata, Input\r\n[], ['data']\r\nbw_layer_out:  8\r\nth_layer_out:  150.8812280483544\r\n--------------------------------------------------------------------------------\r\n1 95\r\nconv1, Convolution\r\n['data'], ['conv1']\r\nbw_layer_in:  8\r\nth_layer_in:  150.8812280483544\r\nbw_layer_out:  8\r\nth_layer_out:  265.59699538722634\r\n--------------------------------------------------------------------------------\r\n2 95\r\nbn1, BatchNorm\r\n['conv1'], ['bn1']\r\nbw_layer_in:  8\r\nth_layer_in:  265.59699538722634\r\nbw_layer_out:  8\r\nth_layer_out:  62792.91298723221\r\n--------------------------------------------------------------------------------\r\n3 95\r\nscale1, Scale\r\n['bn1'], ['scale1']\r\nbw_layer_in:  8\r\nth_layer_in:  62792.91298723221\r\nbw_layer_out:  8\r\nth_layer_out:  326706.5521259308\r\n--------------------------------------------------------------------------------\r\n4 95\r\nrelu1, ReLU\r\n['scale1'], ['scale1']\r\n--------------------------------------------------------------------------------\r\n5 95\r\npool1, Pooling\r\n['scale1'], ['pool1']\r\nbw_layer_in:  8\r\nth_layer_in:  326706.5521259308\r\nbw_layer_out:  8\r\nth_layer_out:  326706.5521259308\r\n--------------------------------------------------------------------------------\r\n6 95\r\nconv2, Convolution\r\n['pool1'], ['conv2']\r\nbw_layer_in:  8\r\nth_layer_in:  326706.5521259308\r\nbw_layer_out:  8\r\nth_layer_out:  645658.5820999146\r\n--------------------------------------------------------------------------------\r\n7 95\r\nbn2, BatchNorm\r\n['conv2'], ['bn2']\r\nbw_layer_in:  8\r\nth_layer_in:  645658.5820999146\r\nbw_layer_out:  8\r\nth_layer_out:  80895660.97851562\r\n--------------------------------------------------------------------------------\r\n```\r\n\r\nAnd right before the error:\r\n```\r\n64 95\r\nbn15, BatchNorm\r\n['conv15'], ['bn15']\r\nbw_layer_in:  8\r\nth_layer_in:  2.271687002092953e+35\r\nbw_layer_out:  8\r\nth_layer_out:  2.7895644236416093e+37\r\n--------------------------------------------------------------------------------\r\n65 95\r\nscale15, Scale\r\n['bn15'], ['scale15']\r\nbw_layer_in:  8\r\nth_layer_in:  2.7895644236416093e+37\r\nbw_layer_out:  8\r\nth_layer_out:  4.709880642132973e+37\r\n--------------------------------------------------------------------------------\r\n66 95\r\nrelu15, ReLU\r\n['scale15'], ['scale15']\r\n--------------------------------------------------------------------------------\r\n67 95\r\nconv16, Convolution\r\n['scale15'], ['conv16']\r\nbw_layer_in:  8\r\nth_layer_in:  4.709880642132973e+37\r\nbw_layer_out:  8\r\nth_layer_out:  3.603168268599782e+37\r\n--------------------------------------------------------------------------------\r\n```\r\n\r\nDoes anyone have any ideas as to what to do now and how to diagnose this issue?\r\nThanks in advance!"},"comment":{"url":"https://api.github.com/repos/Xilinx/CHaiDNN/issues/comments/434588087","html_url":"https://github.com/Xilinx/CHaiDNN/issues/95#issuecomment-434588087","issue_url":"https://api.github.com/repos/Xilinx/CHaiDNN/issues/95","id":434588087,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNDU4ODA4Nw==","user":{"login":"DanielsJusts","id":37595234,"node_id":"MDQ6VXNlcjM3NTk1MjM0","avatar_url":"https://avatars3.githubusercontent.com/u/37595234?v=4","gravatar_id":"","url":"https://api.github.com/users/DanielsJusts","html_url":"https://github.com/DanielsJusts","followers_url":"https://api.github.com/users/DanielsJusts/followers","following_url":"https://api.github.com/users/DanielsJusts/following{/other_user}","gists_url":"https://api.github.com/users/DanielsJusts/gists{/gist_id}","starred_url":"https://api.github.com/users/DanielsJusts/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DanielsJusts/subscriptions","organizations_url":"https://api.github.com/users/DanielsJusts/orgs","repos_url":"https://api.github.com/users/DanielsJusts/repos","events_url":"https://api.github.com/users/DanielsJusts/events{/privacy}","received_events_url":"https://api.github.com/users/DanielsJusts/received_events","type":"User","site_admin":false},"created_at":"2018-10-31T07:30:33Z","updated_at":"2018-10-31T07:30:33Z","author_association":"NONE","body":"Hello, @adu81020799!\r\nThank you for your reply.\r\n\r\nIf I understood you correctly, then you think that I should change from this syntax:\r\n```\r\nlayer {\r\n  name: \"conv1\"\r\n  type: \"Convolution\"\r\n  bottom: \"data\"\r\n  top: \"conv1\"\r\n  convolution_param {\r\n    num_output: 32\r\n    kernel_size: 3\r\n    pad: 1\r\n    bias_term: true\r\n  }\r\n}\r\n```\r\n\r\nto this:\r\n```\r\nlayer [\r\n  name: \"conv1\"\r\n  type: \"Convolution\"\r\n  bottom: \"data\"\r\n  top: \"conv1\"\r\n  convolution_param [\r\n    num_output: 32\r\n    kernel_size: 3\r\n    pad: 1\r\n    bias_term: true\r\n  ]\r\n]\r\n```\r\n\r\nDid I understand you correctly?\r\nThis doesn't work, XportDNN tool throws an error:\r\n```\r\nTraceback (most recent call last):\r\n  File \"XportDNN.py\", line 400, in <module>\r\n  File \"XportDNN.py\", line 355, in main\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/google/protobuf/text_format.py\", line 536, in Merge\r\n    descriptor_pool=descriptor_pool)\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/google/protobuf/text_format.py\", line 590, in MergeLines\r\n    return parser.MergeLines(lines, message)\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/google/protobuf/text_format.py\", line 623, in MergeLines\r\n    self._ParseOrMerge(lines, message)\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/google/protobuf/text_format.py\", line 638, in _ParseOrMerge\r\n    self._MergeField(tokenizer, message)\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/google/protobuf/text_format.py\", line 757, in _MergeField\r\n    merger(tokenizer, message, field)\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/google/protobuf/text_format.py\", line 805, in _MergeMessageField\r\n    tokenizer.Consume('{')\r\n  File \"/home/edi/Workspaces/SDx/CHaiDNN_repo/tools/chaidnn_tools_ENV/local/lib/python2.7/site-packages/google/protobuf/text_format.py\", line 1101, in Consume\r\n    raise self.ParseError('Expected \"%s\".' % token)\r\ngoogle.protobuf.text_format.ParseError: 3:2 : Expected \"{\".\r\n```\r\n\r\nAlso, I cannot import the model with this syntax in PyCaffe as well.\r\n\r\n> You can try using the solution which i used for it .#90 (comment)\r\nIt seems that you had to convert a Darknet model to Caffe, but I have a TensorFlow checkpoint file, which I converted to Caffe, so I cannot use the method you used (to my knowledge)."}}
{"actor":{"display_login":"DanielsJusts"}}