{"action":"created","comment":{"url":"https://api.github.com/repos/pytorch/pytorch/pulls/comments/212661412","pull_request_review_id":149344749,"id":212661412,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIxMjY2MTQxMg==","diff_hunk":"@@ -1047,67 +1140,409 @@ struct CPUFusionFunction : public CompiledFusionFunction {\n   void (*kernel)(uint32_t, void**) = nullptr;\n };\n \n-std::shared_ptr<CompiledFusionFunction> FusionCompiler::getOrCompile(AnnotatedGraph & agraph) {\n-  std::stringstream key;\n-  key << *agraph.graph << \"\\n\";\n-  key << \"device \" << agraph.device << \"\\n\";\n-  for(auto & i : agraph.input_desc)\n-    key << i << \"\\n\";\n-  for(auto & i : agraph.output_desc)\n-    key << i << \"\\n\";\n-  std::string key_ = key.str();\n+////////////////////////////////////////////////////////////////////////////////\n+// FusedKernelCache\n+\n+// Note [Run-time shape checking code]\n+// There are multiple assumptions that our codegen makes, which we can't check\n+// in the fusion pass, because we don't have the shape information. Most notably,\n+// that all values (post-input-chunk, and pre-output-concat) have the same shape\n+// (hereinafter referred to as map size). One way to check this would be to run\n+// shape propagation for every size configuration we get as an input, but that\n+// requires a full graph traversal, and might incur unnecessary overhead. The code\n+// below uses a few nice properties of broadcasting rules and their interactions with\n+// pointwise operations, and takes a smarter approach, to quickly verify validity of\n+// the kernel.\n+//\n+// Notation:\n+//   - a.s when a is a tensor is a shorthand for a.shape.\n+//   - B is a shorthand for the broadcasting/expanding function. It is used as a\n+//     vararg function.\n+//   - E is a shorthand for expand function.\n+//   - Every pointwise operation can be equivalently rewritten as\n+//     f(a, b) = f^(E(a, B(a.s, b.s)), E(b, B(a.s, b.s))),\n+//     where f^ is a non-broadcasting verison of f.\n+//   - A set of inputs that are used to produce a certain graph output is referred to\n+//     as the output's broadcasting group (see Lemma 2. for explanation why).\n+//\n+// Lemma 1. Set of lists of integers (shapes) + { _|_ (bottom/error marker) }, with the\n+//          operation of broadcasting (returning bottom upon shape mismatch) forms a monoid.\n+//          In simpler terms: broadcasting is associative, i.e. B(a, B(b, c)) == B(B(a, b), c).\n+//\n+// Proof.   Satisfies all monoid laws:\n+//            - Closed under broadcasting (trivial)\n+//            - Empty shape is the identity element: B(a, []) == B([], a) == a\n+//            - Associativity: A simple visual proof is that you can expand 3 tensors\n+//                at the same time by stacking their sizes (with alignment to the right),\n+//                just as you'd do in the case of 2 tensors, but with an intermediate\n+//                (the algorithm ends up being pretty much the same).\n+//\n+// Lemma 2. Shape of an output of an arbitrary DAG of pointwise ops depends only on the set\n+//          of inputs used in this DAG and is equal to B([i.shape for i in used_inputs]).\n+//\n+// Proof.   Let G be any DAG of pointwise ops and < be any valid topological\n+//          ordering on nodes of G. Proof by induction over <.\n+//          Base case (graph input):\n+//            Trivial (input is also an output).\n+//          Step (n = f(q, r)):\n+//            Let QS (RS) be the set of shapes of inputs that q (r) depends on.\n+//            Note that the set of inputs that n depends on is exactly QS + RS.\n+//            shape(n) == shape(f(q, r))\n+//                          (def of f)\n+//                     == shape(f^(E(q, B(q.s, r.s)), E(r, B(q.s, r.s))))\n+//                          (output shape of f^ is equal to either of argument shapes)\n+//                     == shape(E(q, B(q.s, r.s)))\n+//                          (property of expand)\n+//                     == B(q.s, r.s)\n+//                          (induction assumption)\n+//                     == B(B(QS...), B(RS...))\n+//                          (Lemma 1.)\n+//                     == B(QS..., RS...)\n+//                          (repeated shapes don't matter for broadcasting)\n+//                     == B((QS + RS)...)\n+//\n+// Lemma 3. Expands are distributive over pointwise ops, i.e. E(f(a, b), s) = f(E(a, s), E(b, s))\n+// Lemma 4. Expands can be collapsed, i.e. E(E(x, s1), s2) = E(x, B(s1, s2)).\n+// Proof.   A simple exercise for the reader :)\n+//\n+// Theorem. If all (pre-concat-)outputs have equal shapes, then we can push the expands to\n+//          (pre-chunk-)inputs, and have all intermediates of the same shape\n+//          (no broadcasting happening in the body).\n+//\n+// Proof.   Using the above lemmas we can easily show that a graph with a single output\n+//          can be easily rewritten by taking the shape given by B applied to all input\n+//          shapes, expanding inputs to it, and using only non-broadcasting operations.\n+//          Example:\n+//\n+//          let d = f(a, b) in\n+//          let e = h(b, c) in\n+//          g(d, e)\n+//\n+//          (By def. of broadcasting pointwise ops applied to g, f and h)\n+//          (Lemma 2. for a closed formula for the size of g = gs)\n+//\n+//          let gs = B(a.s, b.s, c.s) in\n+//          let d' = E(f^(E(a, B(a.s, b.s)), E(b, B(a.s, b.s))), gs) in\n+//          let e' = E(h^(E(b, B(b.s, c.s)), E(c, B(b.s, c.s))), gs) in\n+//          g^(d', e')\n+//\n+//          (Lemma 3.)\n+//\n+//          let gs = B(a.s, b.s, c.s) in\n+//          let d' = f^(E(E(a, B(a.s, b.s)), gs), E(E(b, B(a.s, b.s)), gs)) in\n+//          let e' = h^(E(E(b, B(b.s, c.s)), gs), E(E(c, B(b.s, c.s)), gs)) in\n+//          g^(d', e')\n+//\n+//          (Lemma 4. + Lemma 1. to simplify broadcasting function)\n+//\n+//          let gs = B(a.s, b.s, c.s) in\n+//          let d' = f^(E(a, gs), E(b, gs)) in\n+//          let e' = h^(E(b, gs), E(c, gs)) in\n+//          g^(d', e')\n+//\n+//          (Simple rewrite)\n+//\n+//          let gs = B(a.s, b.s, c.s) in\n+//          let a' = E(a, gs) in\n+//          let b' = E(b, gs) in\n+//          let c' = E(c, gs) in\n+//          let d' = f^(a', b') in\n+//          let e' = h^(b', c') in\n+//          g^(d', e')\n+//\n+//          This example can be easily formalized to arbitrary DAGs using induction\n+//          over topological ordering, similar to Lemma 2. Now, if broadcasting groups\n+//          for all outputs have the same shape, then performing an expand to this size\n+//          on all inputs will ensure that all intermediates on all paths to outputs\n+//          will have the same shape, proving that the body of the kernel is valid.\n+//\n+//          This shows the part until post-chunk-inputs. Extending it to pre-chunk-inputs\n+//          is straightforward (needs a simple lemma for moving expands through chunks).\n+\n+// Register implementations of fused operators, so that we can reuse the fused graph\n+// to generate fallback code.\n+RegisterOperators reg_fused_operators({\n+  Operator(\n+    prim::FusedChunk,\n+    [](Node* node) {\n+      int64_t dim = node->i(attr::dim);\n+      int64_t chunks = node->outputs().size();\n+      return [dim, chunks](Stack& stack) {\n+        auto result = at::chunk(std::move(peek(stack, 0, 1)).toTensor(), chunks, dim);\n+        drop(stack, 1);\n+        pack(stack, std::move(result));\n+        return 0;\n+      };\n+    }),\n+  Operator(\n+    prim::FusedConcat,\n+    [](Node* node) {\n+      int64_t dim = node->i(attr::dim);\n+      int64_t num_inputs = node->inputs().size();\n+      return [dim, num_inputs](Stack& stack) {\n+        auto result = at::cat(\n+          fmap(last(stack, num_inputs), [](const IValue& i) { return i.toTensor(); }),\n+          dim\n+        );\n+        drop(stack, num_inputs);\n+        pack(stack, std::move(result));\n+        return 0;\n+      };\n+    })\n+});\n+\n+FusedKernelCache::FusedKernelCache(FusionCompiler& compiler, std::shared_ptr<Graph> _graph, int device)\n+  : device(device)\n+  , fallback_code(_graph)\n+  , compiler(compiler)\n+  , graph(std::move(_graph))\n+  , input_broadcast_groups(getInputBroadcastGroups())\n+  , input_chunks(getInputChunkDescriptors())\n+  , kernels() {}\n+\n+std::atomic<size_t> FusedKernelCache::next_kernel_id {0};\n+\n+auto FusedKernelCache::getInputChunkDescriptors() -> std::vector<PartitionInfo> {\n+  std::vector<PartitionInfo> descs;\n+  descs.reserve(graph->inputs().size());\n+  for (Value * input : graph->inputs()) {\n+    if (Node * chunk = usedInFusedChunk(input)) {\n+      descs.emplace_back(chunk->i(attr::chunks), chunk->i(attr::dim));\n+    } else {\n+      descs.emplace_back(1, 0);\n+    }\n+  }\n+  return descs;\n+}\n \n-  auto it = cache.find(key_);\n-  if (it == cache.end()) {\n-    std::string name = \"kernel_\" + std::to_string(cache.size());\n-    CompiledFusionFunction * raw_func;\n-    if(agraph.device != kCPUDevice) {\n-#ifdef USE_CUDA\n-      raw_func = new CUDAFusionFunction(name, agraph);\n-#else\n-      throw std::runtime_error(\"cannot compile a CUDA fusion group, CUDA is not enabled.\");\n-#endif\n+// NB: this vector is really a set, but we want to keep it contiguous in memory for faster access\n+static std::vector<int64_t> getInputDependencies(Value* output) {\n+  // Run a DFS traversal to find all inputs that affect a given output value\n+  std::vector<Value*> queue { output };\n+  std::unordered_set<Value*> inputs;\n+  std::unordered_set<Value*> seen;\n+  while (!queue.empty()) {\n+    Value * val = queue.back(); queue.pop_back();\n+    Node * producer = val->node();\n+    if (producer->kind() == prim::Param) {\n+      inputs.insert(val);\n+      continue;\n+    }\n+    for (Value * input : producer->inputs()) {\n+      if (/*bool inserted = */seen.insert(input).second) {\n+        queue.push_back(input);\n+      }\n+    }\n+  }\n+\n+  // Convert Value* into offsets into the graph's input list\n+  std::vector<int64_t> offsets;\n+  offsets.reserve(inputs.size());\n+  for (Value * input : inputs) {\n+    offsets.push_back(input->offset());\n+  }\n+  std::sort(offsets.begin(), offsets.end());\n+  return offsets;\n+}\n+\n+std::vector<std::vector<int64_t>> FusedKernelCache::getInputBroadcastGroups() {\n+  std::unordered_set<std::vector<int64_t>, torch::hash<std::vector<int64_t>>> broadcast_groups;\n+  for (Value * output : graph->outputs()) {\n+    broadcast_groups.insert(getInputDependencies(output));\n+  }\n+  return std::vector<std::vector<int64_t>>{ broadcast_groups.begin(), broadcast_groups.end() };\n+}\n+\n+void FusedKernelCache::run(Stack& stack) {\n+  int64_t num_inputs = graph->inputs().size();\n+  auto args = fmap(last(stack, num_inputs), [](const IValue& i) {\n+                return i.toTensor();\n+              });\n+\n+  auto maybe_map_size = canRunKernel(args);\n+  if (!maybe_map_size) {\n+    return runFallback(stack);\n+  }\n+  expandArgs(args, *maybe_map_size);\n+\n+  FusedKernelArgSpec spec { args };\n+  auto it = kernels.find(spec);\n+  if (it == kernels.end()) {\n+    std::tie(it, std::ignore) = kernels.emplace(spec, compileSpec(spec, *maybe_map_size));\n+  }\n+  auto & fn = it->second;\n+\n+  std::vector<at::Tensor> outputs;\n+  fn->launch(args, outputs);\n+  drop(stack, num_inputs);\n+  pack(stack, std::move(outputs));\n+}\n+\n+at::optional<std::vector<int64_t>> FusedKernelCache::getMapSize(at::TensorList args, at::IntList arg_subset) {\n+  // NB: we leave this uninitialized, because an empty size is trivially\n+  // broadcastable to any other size.\n+  // TODO: this keeps reallocating map_size at every iteration, but we know\n+  // exactly how much storage do we need, so this could be fixed in-place at\n+  // every step. We're just missing a few functions for ATen, but the fix\n+  // should be straightforward.\n+  int64_t dim_after_broadcast = 0;\n+  for (int64_t arg_idx : arg_subset) {\n+    dim_after_broadcast = std::max(dim_after_broadcast, args[arg_idx].dim());\n+  }\n+  std::vector<int64_t> map_size;\n+  for (size_t i = 0; i < arg_subset.size(); ++i) {\n+    auto & arg = args.at(arg_subset[i]);\n+    auto & chunk_desc = input_chunks.at(arg_subset[i]);\n+    if (chunk_desc.nSubtensors == 1) {\n+      try {\n+        map_size = at::infer_size(map_size, arg.sizes());\n+      } catch (std::exception& e) {\n+        return at::nullopt;\n+      }\n     } else {\n-      JIT_ASSERT(canCompileOnCPU());\n-      raw_func = new CPUFusionFunction(name, agraph, config_);\n+      auto tensor_sizes = arg.sizes().vec();\n+      int64_t num_chunks = chunk_desc.nSubtensors;\n+      int64_t dim = chunk_desc.dim;\n+      if (dim < 0) {\n+        dim += arg.dim();\n+      }\n+      JIT_ASSERT(dim >= 0 && dim <= static_cast<int64_t>(tensor_sizes.size()));\n+      if (tensor_sizes[dim] % num_chunks != 0) {\n+        return at::nullopt;\n+      }\n+      tensor_sizes[dim] /= num_chunks;\n+      try {\n+        map_size = at::infer_size(map_size, tensor_sizes);\n+      } catch (std::exception& e) {\n+        return at::nullopt;\n+      }\n     }\n-    it = cache.emplace(key_, std::shared_ptr<CompiledFusionFunction>(raw_func)).first;\n   }\n-  return it->second;\n+\n+  return {map_size};\n }\n \n-std::shared_ptr<CompiledFusionFunction> FusionCompiler::getOrCompile(Node* fusion_group) {\n-  auto & graph = *fusion_group->g(attr::Subgraph);\n-  AnnotatedGraph agraph(graph, fusion_group->i(attr::device));\n-  for(auto & input : graph.inputs()) {\n-    auto t = input->type()->expect<TensorType>();\n-    agraph.input_desc.emplace_back(t);\n+// See Note [Run-time shape checking code] for more explanation on the algorithm.\n+at::optional<std::vector<int64_t>> FusedKernelCache::canRunKernel(at::TensorList args) {\n+  AT_CHECK(args.size() == input_chunks.size(),\n+           \"Expected \", input_chunks.size(), \" arguments, but got \", args.size());\n+\n+  at::optional<std::vector<int64_t>> map_size;\n+  for (const auto & broadcast_group : input_broadcast_groups) {\n+    if (!map_size) {\n+      map_size = getMapSize(args, broadcast_group);\n+      if (!map_size) {\n+        return at::nullopt;\n+      }\n+    } else {\n+      auto group_map_size = getMapSize(args, broadcast_group);\n+      // NB: this checks that group_map_size is defined AND equal to map_size\n+      if (map_size != group_map_size) {\n+        return at::nullopt;\n+      }\n+    }\n   }\n-  for(auto & output : graph.outputs()) {\n-    auto t = output->type()->expect<TensorType>();\n-    agraph.output_desc.emplace_back(t);\n+  return map_size;\n+}\n+\n+void FusedKernelCache::runFallback(Stack& stack) {\n+  InterpreterState(fallback_code).runOneStage(stack);\n+}\n+\n+void FusedKernelCache::expandArgs(std::vector<at::Tensor>& args, std::vector<int64_t>& map_size) {\n+  for (size_t i = 0; i < args.size(); ++i) {\n+    auto & arg = args[i];\n+    auto & pdesc = input_chunks[i];\n+    if (pdesc.nSubtensors == 1) {\n+      if (arg.sizes().equals(map_size)) continue;\n+      arg = arg.expand(map_size);\n+    } else {\n+      map_size.at(pdesc.dim) *= pdesc.nSubtensors;\n+      if (!arg.sizes().equals(map_size)) {\n+        arg = arg.expand(map_size);\n+      }\n+      map_size.at(pdesc.dim) /= pdesc.nSubtensors;","path":"torch/csrc/jit/fusion_compiler.cpp","position":686,"original_position":686,"commit_id":"5c300b6e70abb13121d37deac2f2bb9563d92a7b","original_commit_id":"5c300b6e70abb13121d37deac2f2bb9563d92a7b","user":{"login":"ezyang","id":13564,"node_id":"MDQ6VXNlcjEzNTY0","avatar_url":"https://avatars0.githubusercontent.com/u/13564?v=4","gravatar_id":"","url":"https://api.github.com/users/ezyang","html_url":"https://github.com/ezyang","followers_url":"https://api.github.com/users/ezyang/followers","following_url":"https://api.github.com/users/ezyang/following{/other_user}","gists_url":"https://api.github.com/users/ezyang/gists{/gist_id}","starred_url":"https://api.github.com/users/ezyang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ezyang/subscriptions","organizations_url":"https://api.github.com/users/ezyang/orgs","repos_url":"https://api.github.com/users/ezyang/repos","events_url":"https://api.github.com/users/ezyang/events{/privacy}","received_events_url":"https://api.github.com/users/ezyang/received_events","type":"User","site_admin":false},"body":"Yuck. Understandable, but yuck.","created_at":"2018-08-24T15:12:51Z","updated_at":"2018-08-24T15:12:51Z","html_url":"https://github.com/pytorch/pytorch/pull/10844#discussion_r212661412","pull_request_url":"https://api.github.com/repos/pytorch/pytorch/pulls/10844","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/pytorch/pytorch/pulls/comments/212661412"},"html":{"href":"https://github.com/pytorch/pytorch/pull/10844#discussion_r212661412"},"pull_request":{"href":"https://api.github.com/repos/pytorch/pytorch/pulls/10844"}}},"pull_request":{"url":"https://api.github.com/repos/pytorch/pytorch/pulls/10844","id":210618558,"node_id":"MDExOlB1bGxSZXF1ZXN0MjEwNjE4NTU4","html_url":"https://github.com/pytorch/pytorch/pull/10844","diff_url":"https://github.com/pytorch/pytorch/pull/10844.diff","patch_url":"https://github.com/pytorch/pytorch/pull/10844.patch","issue_url":"https://api.github.com/repos/pytorch/pytorch/issues/10844","number":10844,"state":"open","locked":false,"title":"Prevent JIT from overspecializing to every single size configuration","user":{"login":"apaszke","id":4583066,"node_id":"MDQ6VXNlcjQ1ODMwNjY=","avatar_url":"https://avatars3.githubusercontent.com/u/4583066?v=4","gravatar_id":"","url":"https://api.github.com/users/apaszke","html_url":"https://github.com/apaszke","followers_url":"https://api.github.com/users/apaszke/followers","following_url":"https://api.github.com/users/apaszke/following{/other_user}","gists_url":"https://api.github.com/users/apaszke/gists{/gist_id}","starred_url":"https://api.github.com/users/apaszke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apaszke/subscriptions","organizations_url":"https://api.github.com/users/apaszke/orgs","repos_url":"https://api.github.com/users/apaszke/repos","events_url":"https://api.github.com/users/apaszke/events{/privacy}","received_events_url":"https://api.github.com/users/apaszke/received_events","type":"User","site_admin":false},"body":"Please review the expects carefully to make sure there are no regressions. I tried to go over them one by one when they changed, but it's sometimes easy to miss finer details.\r\n\r\nSummary of changes:\r\n\r\n- Renamed `TensorType` to `CompleteTensorType`. Added a new `TensorType` which records only the scalar type, number of dimensions, and device of a value. The argument behind the rename is to encourage people to use `CompleteTensorType` less, as most passes will only have limited information available. To make transition easier `complete_type->cast<TensorType>()` works, and makes our passes work with both kinds of specialization if they don't need extra the extra detail.\r\n- Renamed `ArgumentSpec` to `CompleteArgumentSpec`. Added a new `ArgumentSpec`, which matches argument only at the level of the new `TensorType`.\r\n- Shape analysis can process graphs with both `CompleteTensorType` and `TensorType`.\r\n- Fuser was a part that heavily relied on full shape information being available. Now, we simply try to fuse the largest possible graphs, and have to do run-time checks to make sure they match the code we generate. If they don't, we fall back to regular interpretation. The shape checks are implementing using an optimized method exploiting algebraic properties of shapes with broadcasting, and the relations of broadcasting with pointwise ops. A full written proof of correctness of the shape checking algorithm is included in a comment in `graph_fuser.cpp`.\r\n\r\n@zdevito @ezyang @mruberry @ngimel @csarofeen","created_at":"2018-08-24T01:45:06Z","updated_at":"2018-08-24T15:12:51Z","closed_at":null,"merged_at":null,"merge_commit_sha":"424489e8dc8eedcb02af80665a16ae7a09655381","assignee":null,"assignees":[],"requested_reviewers":[{"login":"zdevito","id":370202,"node_id":"MDQ6VXNlcjM3MDIwMg==","avatar_url":"https://avatars0.githubusercontent.com/u/370202?v=4","gravatar_id":"","url":"https://api.github.com/users/zdevito","html_url":"https://github.com/zdevito","followers_url":"https://api.github.com/users/zdevito/followers","following_url":"https://api.github.com/users/zdevito/following{/other_user}","gists_url":"https://api.github.com/users/zdevito/gists{/gist_id}","starred_url":"https://api.github.com/users/zdevito/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zdevito/subscriptions","organizations_url":"https://api.github.com/users/zdevito/orgs","repos_url":"https://api.github.com/users/zdevito/repos","events_url":"https://api.github.com/users/zdevito/events{/privacy}","received_events_url":"https://api.github.com/users/zdevito/received_events","type":"User","site_admin":false},{"login":"Yangqing","id":551151,"node_id":"MDQ6VXNlcjU1MTE1MQ==","avatar_url":"https://avatars1.githubusercontent.com/u/551151?v=4","gravatar_id":"","url":"https://api.github.com/users/Yangqing","html_url":"https://github.com/Yangqing","followers_url":"https://api.github.com/users/Yangqing/followers","following_url":"https://api.github.com/users/Yangqing/following{/other_user}","gists_url":"https://api.github.com/users/Yangqing/gists{/gist_id}","starred_url":"https://api.github.com/users/Yangqing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Yangqing/subscriptions","organizations_url":"https://api.github.com/users/Yangqing/orgs","repos_url":"https://api.github.com/users/Yangqing/repos","events_url":"https://api.github.com/users/Yangqing/events{/privacy}","received_events_url":"https://api.github.com/users/Yangqing/received_events","type":"User","site_admin":false},{"login":"colesbury","id":655866,"node_id":"MDQ6VXNlcjY1NTg2Ng==","avatar_url":"https://avatars1.githubusercontent.com/u/655866?v=4","gravatar_id":"","url":"https://api.github.com/users/colesbury","html_url":"https://github.com/colesbury","followers_url":"https://api.github.com/users/colesbury/followers","following_url":"https://api.github.com/users/colesbury/following{/other_user}","gists_url":"https://api.github.com/users/colesbury/gists{/gist_id}","starred_url":"https://api.github.com/users/colesbury/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/colesbury/subscriptions","organizations_url":"https://api.github.com/users/colesbury/orgs","repos_url":"https://api.github.com/users/colesbury/repos","events_url":"https://api.github.com/users/colesbury/events{/privacy}","received_events_url":"https://api.github.com/users/colesbury/received_events","type":"User","site_admin":false},{"login":"soumith","id":1310570,"node_id":"MDQ6VXNlcjEzMTA1NzA=","avatar_url":"https://avatars0.githubusercontent.com/u/1310570?v=4","gravatar_id":"","url":"https://api.github.com/users/soumith","html_url":"https://github.com/soumith","followers_url":"https://api.github.com/users/soumith/followers","following_url":"https://api.github.com/users/soumith/following{/other_user}","gists_url":"https://api.github.com/users/soumith/gists{/gist_id}","starred_url":"https://api.github.com/users/soumith/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/soumith/subscriptions","organizations_url":"https://api.github.com/users/soumith/orgs","repos_url":"https://api.github.com/users/soumith/repos","events_url":"https://api.github.com/users/soumith/events{/privacy}","received_events_url":"https://api.github.com/users/soumith/received_events","type":"User","site_admin":false},{"login":"anderspapitto","id":1388690,"node_id":"MDQ6VXNlcjEzODg2OTA=","avatar_url":"https://avatars3.githubusercontent.com/u/1388690?v=4","gravatar_id":"","url":"https://api.github.com/users/anderspapitto","html_url":"https://github.com/anderspapitto","followers_url":"https://api.github.com/users/anderspapitto/followers","following_url":"https://api.github.com/users/anderspapitto/following{/other_user}","gists_url":"https://api.github.com/users/anderspapitto/gists{/gist_id}","starred_url":"https://api.github.com/users/anderspapitto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anderspapitto/subscriptions","organizations_url":"https://api.github.com/users/anderspapitto/orgs","repos_url":"https://api.github.com/users/anderspapitto/repos","events_url":"https://api.github.com/users/anderspapitto/events{/privacy}","received_events_url":"https://api.github.com/users/anderspapitto/received_events","type":"User","site_admin":false},{"login":"smessmer","id":2373925,"node_id":"MDQ6VXNlcjIzNzM5MjU=","avatar_url":"https://avatars3.githubusercontent.com/u/2373925?v=4","gravatar_id":"","url":"https://api.github.com/users/smessmer","html_url":"https://github.com/smessmer","followers_url":"https://api.github.com/users/smessmer/followers","following_url":"https://api.github.com/users/smessmer/following{/other_user}","gists_url":"https://api.github.com/users/smessmer/gists{/gist_id}","starred_url":"https://api.github.com/users/smessmer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/smessmer/subscriptions","organizations_url":"https://api.github.com/users/smessmer/orgs","repos_url":"https://api.github.com/users/smessmer/repos","events_url":"https://api.github.com/users/smessmer/events{/privacy}","received_events_url":"https://api.github.com/users/smessmer/received_events","type":"User","site_admin":false},{"login":"gchanan","id":3768583,"node_id":"MDQ6VXNlcjM3Njg1ODM=","avatar_url":"https://avatars2.githubusercontent.com/u/3768583?v=4","gravatar_id":"","url":"https://api.github.com/users/gchanan","html_url":"https://github.com/gchanan","followers_url":"https://api.github.com/users/gchanan/followers","following_url":"https://api.github.com/users/gchanan/following{/other_user}","gists_url":"https://api.github.com/users/gchanan/gists{/gist_id}","starred_url":"https://api.github.com/users/gchanan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gchanan/subscriptions","organizations_url":"https://api.github.com/users/gchanan/orgs","repos_url":"https://api.github.com/users/gchanan/repos","events_url":"https://api.github.com/users/gchanan/events{/privacy}","received_events_url":"https://api.github.com/users/gchanan/received_events","type":"User","site_admin":false},{"login":"jamesr66a","id":4685384,"node_id":"MDQ6VXNlcjQ2ODUzODQ=","avatar_url":"https://avatars2.githubusercontent.com/u/4685384?v=4","gravatar_id":"","url":"https://api.github.com/users/jamesr66a","html_url":"https://github.com/jamesr66a","followers_url":"https://api.github.com/users/jamesr66a/followers","following_url":"https://api.github.com/users/jamesr66a/following{/other_user}","gists_url":"https://api.github.com/users/jamesr66a/gists{/gist_id}","starred_url":"https://api.github.com/users/jamesr66a/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jamesr66a/subscriptions","organizations_url":"https://api.github.com/users/jamesr66a/orgs","repos_url":"https://api.github.com/users/jamesr66a/repos","events_url":"https://api.github.com/users/jamesr66a/events{/privacy}","received_events_url":"https://api.github.com/users/jamesr66a/received_events","type":"User","site_admin":false},{"login":"bddppq","id":9300575,"node_id":"MDQ6VXNlcjkzMDA1NzU=","avatar_url":"https://avatars2.githubusercontent.com/u/9300575?v=4","gravatar_id":"","url":"https://api.github.com/users/bddppq","html_url":"https://github.com/bddppq","followers_url":"https://api.github.com/users/bddppq/followers","following_url":"https://api.github.com/users/bddppq/following{/other_user}","gists_url":"https://api.github.com/users/bddppq/gists{/gist_id}","starred_url":"https://api.github.com/users/bddppq/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bddppq/subscriptions","organizations_url":"https://api.github.com/users/bddppq/orgs","repos_url":"https://api.github.com/users/bddppq/repos","events_url":"https://api.github.com/users/bddppq/events{/privacy}","received_events_url":"https://api.github.com/users/bddppq/received_events","type":"User","site_admin":false},{"login":"dzhulgakov","id":17890620,"node_id":"MDQ6VXNlcjE3ODkwNjIw","avatar_url":"https://avatars2.githubusercontent.com/u/17890620?v=4","gravatar_id":"","url":"https://api.github.com/users/dzhulgakov","html_url":"https://github.com/dzhulgakov","followers_url":"https://api.github.com/users/dzhulgakov/followers","following_url":"https://api.github.com/users/dzhulgakov/following{/other_user}","gists_url":"https://api.github.com/users/dzhulgakov/gists{/gist_id}","starred_url":"https://api.github.com/users/dzhulgakov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dzhulgakov/subscriptions","organizations_url":"https://api.github.com/users/dzhulgakov/orgs","repos_url":"https://api.github.com/users/dzhulgakov/repos","events_url":"https://api.github.com/users/dzhulgakov/events{/privacy}","received_events_url":"https://api.github.com/users/dzhulgakov/received_events","type":"User","site_admin":false},{"login":"houseroad","id":30275821,"node_id":"MDQ6VXNlcjMwMjc1ODIx","avatar_url":"https://avatars0.githubusercontent.com/u/30275821?v=4","gravatar_id":"","url":"https://api.github.com/users/houseroad","html_url":"https://github.com/houseroad","followers_url":"https://api.github.com/users/houseroad/followers","following_url":"https://api.github.com/users/houseroad/following{/other_user}","gists_url":"https://api.github.com/users/houseroad/gists{/gist_id}","starred_url":"https://api.github.com/users/houseroad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/houseroad/subscriptions","organizations_url":"https://api.github.com/users/houseroad/orgs","repos_url":"https://api.github.com/users/houseroad/repos","events_url":"https://api.github.com/users/houseroad/events{/privacy}","received_events_url":"https://api.github.com/users/houseroad/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/pytorch/pytorch/pulls/10844/commits","review_comments_url":"https://api.github.com/repos/pytorch/pytorch/pulls/10844/comments","review_comment_url":"https://api.github.com/repos/pytorch/pytorch/pulls/comments{/number}","comments_url":"https://api.github.com/repos/pytorch/pytorch/issues/10844/comments","statuses_url":"https://api.github.com/repos/pytorch/pytorch/statuses/5c300b6e70abb13121d37deac2f2bb9563d92a7b","head":{"label":"apaszke:jit_type_unspecialize","ref":"jit_type_unspecialize","sha":"5c300b6e70abb13121d37deac2f2bb9563d92a7b","user":{"login":"apaszke","id":4583066,"node_id":"MDQ6VXNlcjQ1ODMwNjY=","avatar_url":"https://avatars3.githubusercontent.com/u/4583066?v=4","gravatar_id":"","url":"https://api.github.com/users/apaszke","html_url":"https://github.com/apaszke","followers_url":"https://api.github.com/users/apaszke/followers","following_url":"https://api.github.com/users/apaszke/following{/other_user}","gists_url":"https://api.github.com/users/apaszke/gists{/gist_id}","starred_url":"https://api.github.com/users/apaszke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apaszke/subscriptions","organizations_url":"https://api.github.com/users/apaszke/orgs","repos_url":"https://api.github.com/users/apaszke/repos","events_url":"https://api.github.com/users/apaszke/events{/privacy}","received_events_url":"https://api.github.com/users/apaszke/received_events","type":"User","site_admin":false},"repo":{"id":91262910,"node_id":"MDEwOlJlcG9zaXRvcnk5MTI2MjkxMA==","name":"pytorch-1","full_name":"apaszke/pytorch-1","owner":{"login":"apaszke","id":4583066,"node_id":"MDQ6VXNlcjQ1ODMwNjY=","avatar_url":"https://avatars3.githubusercontent.com/u/4583066?v=4","gravatar_id":"","url":"https://api.github.com/users/apaszke","html_url":"https://github.com/apaszke","followers_url":"https://api.github.com/users/apaszke/followers","following_url":"https://api.github.com/users/apaszke/following{/other_user}","gists_url":"https://api.github.com/users/apaszke/gists{/gist_id}","starred_url":"https://api.github.com/users/apaszke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apaszke/subscriptions","organizations_url":"https://api.github.com/users/apaszke/orgs","repos_url":"https://api.github.com/users/apaszke/repos","events_url":"https://api.github.com/users/apaszke/events{/privacy}","received_events_url":"https://api.github.com/users/apaszke/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/apaszke/pytorch-1","description":"Tensors and Dynamic neural networks in Python  with strong GPU acceleration","fork":true,"url":"https://api.github.com/repos/apaszke/pytorch-1","forks_url":"https://api.github.com/repos/apaszke/pytorch-1/forks","keys_url":"https://api.github.com/repos/apaszke/pytorch-1/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apaszke/pytorch-1/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apaszke/pytorch-1/teams","hooks_url":"https://api.github.com/repos/apaszke/pytorch-1/hooks","issue_events_url":"https://api.github.com/repos/apaszke/pytorch-1/issues/events{/number}","events_url":"https://api.github.com/repos/apaszke/pytorch-1/events","assignees_url":"https://api.github.com/repos/apaszke/pytorch-1/assignees{/user}","branches_url":"https://api.github.com/repos/apaszke/pytorch-1/branches{/branch}","tags_url":"https://api.github.com/repos/apaszke/pytorch-1/tags","blobs_url":"https://api.github.com/repos/apaszke/pytorch-1/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apaszke/pytorch-1/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apaszke/pytorch-1/git/refs{/sha}","trees_url":"https://api.github.com/repos/apaszke/pytorch-1/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apaszke/pytorch-1/statuses/{sha}","languages_url":"https://api.github.com/repos/apaszke/pytorch-1/languages","stargazers_url":"https://api.github.com/repos/apaszke/pytorch-1/stargazers","contributors_url":"https://api.github.com/repos/apaszke/pytorch-1/contributors","subscribers_url":"https://api.github.com/repos/apaszke/pytorch-1/subscribers","subscription_url":"https://api.github.com/repos/apaszke/pytorch-1/subscription","commits_url":"https://api.github.com/repos/apaszke/pytorch-1/commits{/sha}","git_commits_url":"https://api.github.com/repos/apaszke/pytorch-1/git/commits{/sha}","comments_url":"https://api.github.com/repos/apaszke/pytorch-1/comments{/number}","issue_comment_url":"https://api.github.com/repos/apaszke/pytorch-1/issues/comments{/number}","contents_url":"https://api.github.com/repos/apaszke/pytorch-1/contents/{+path}","compare_url":"https://api.github.com/repos/apaszke/pytorch-1/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apaszke/pytorch-1/merges","archive_url":"https://api.github.com/repos/apaszke/pytorch-1/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apaszke/pytorch-1/downloads","issues_url":"https://api.github.com/repos/apaszke/pytorch-1/issues{/number}","pulls_url":"https://api.github.com/repos/apaszke/pytorch-1/pulls{/number}","milestones_url":"https://api.github.com/repos/apaszke/pytorch-1/milestones{/number}","notifications_url":"https://api.github.com/repos/apaszke/pytorch-1/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apaszke/pytorch-1/labels{/name}","releases_url":"https://api.github.com/repos/apaszke/pytorch-1/releases{/id}","deployments_url":"https://api.github.com/repos/apaszke/pytorch-1/deployments","created_at":"2017-05-14T18:21:28Z","updated_at":"2017-05-14T18:21:33Z","pushed_at":"2018-08-24T04:22:37Z","git_url":"git://github.com/apaszke/pytorch-1.git","ssh_url":"git@github.com:apaszke/pytorch-1.git","clone_url":"https://github.com/apaszke/pytorch-1.git","svn_url":"https://github.com/apaszke/pytorch-1","homepage":"http://pytorch.org","size":67916,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"pytorch:master","ref":"master","sha":"df2d48b42ccbc55ce42ce8e5c03da4c44cc58877","user":{"login":"pytorch","id":21003710,"node_id":"MDEyOk9yZ2FuaXphdGlvbjIxMDAzNzEw","avatar_url":"https://avatars3.githubusercontent.com/u/21003710?v=4","gravatar_id":"","url":"https://api.github.com/users/pytorch","html_url":"https://github.com/pytorch","followers_url":"https://api.github.com/users/pytorch/followers","following_url":"https://api.github.com/users/pytorch/following{/other_user}","gists_url":"https://api.github.com/users/pytorch/gists{/gist_id}","starred_url":"https://api.github.com/users/pytorch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pytorch/subscriptions","organizations_url":"https://api.github.com/users/pytorch/orgs","repos_url":"https://api.github.com/users/pytorch/repos","events_url":"https://api.github.com/users/pytorch/events{/privacy}","received_events_url":"https://api.github.com/users/pytorch/received_events","type":"Organization","site_admin":false},"repo":{"id":65600975,"node_id":"MDEwOlJlcG9zaXRvcnk2NTYwMDk3NQ==","name":"pytorch","full_name":"pytorch/pytorch","owner":{"login":"pytorch","id":21003710,"node_id":"MDEyOk9yZ2FuaXphdGlvbjIxMDAzNzEw","avatar_url":"https://avatars3.githubusercontent.com/u/21003710?v=4","gravatar_id":"","url":"https://api.github.com/users/pytorch","html_url":"https://github.com/pytorch","followers_url":"https://api.github.com/users/pytorch/followers","following_url":"https://api.github.com/users/pytorch/following{/other_user}","gists_url":"https://api.github.com/users/pytorch/gists{/gist_id}","starred_url":"https://api.github.com/users/pytorch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pytorch/subscriptions","organizations_url":"https://api.github.com/users/pytorch/orgs","repos_url":"https://api.github.com/users/pytorch/repos","events_url":"https://api.github.com/users/pytorch/events{/privacy}","received_events_url":"https://api.github.com/users/pytorch/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/pytorch/pytorch","description":"Tensors and Dynamic neural networks in Python  with strong GPU acceleration","fork":false,"url":"https://api.github.com/repos/pytorch/pytorch","forks_url":"https://api.github.com/repos/pytorch/pytorch/forks","keys_url":"https://api.github.com/repos/pytorch/pytorch/keys{/key_id}","collaborators_url":"https://api.github.com/repos/pytorch/pytorch/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/pytorch/pytorch/teams","hooks_url":"https://api.github.com/repos/pytorch/pytorch/hooks","issue_events_url":"https://api.github.com/repos/pytorch/pytorch/issues/events{/number}","events_url":"https://api.github.com/repos/pytorch/pytorch/events","assignees_url":"https://api.github.com/repos/pytorch/pytorch/assignees{/user}","branches_url":"https://api.github.com/repos/pytorch/pytorch/branches{/branch}","tags_url":"https://api.github.com/repos/pytorch/pytorch/tags","blobs_url":"https://api.github.com/repos/pytorch/pytorch/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/pytorch/pytorch/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/pytorch/pytorch/git/refs{/sha}","trees_url":"https://api.github.com/repos/pytorch/pytorch/git/trees{/sha}","statuses_url":"https://api.github.com/repos/pytorch/pytorch/statuses/{sha}","languages_url":"https://api.github.com/repos/pytorch/pytorch/languages","stargazers_url":"https://api.github.com/repos/pytorch/pytorch/stargazers","contributors_url":"https://api.github.com/repos/pytorch/pytorch/contributors","subscribers_url":"https://api.github.com/repos/pytorch/pytorch/subscribers","subscription_url":"https://api.github.com/repos/pytorch/pytorch/subscription","commits_url":"https://api.github.com/repos/pytorch/pytorch/commits{/sha}","git_commits_url":"https://api.github.com/repos/pytorch/pytorch/git/commits{/sha}","comments_url":"https://api.github.com/repos/pytorch/pytorch/comments{/number}","issue_comment_url":"https://api.github.com/repos/pytorch/pytorch/issues/comments{/number}","contents_url":"https://api.github.com/repos/pytorch/pytorch/contents/{+path}","compare_url":"https://api.github.com/repos/pytorch/pytorch/compare/{base}...{head}","merges_url":"https://api.github.com/repos/pytorch/pytorch/merges","archive_url":"https://api.github.com/repos/pytorch/pytorch/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/pytorch/pytorch/downloads","issues_url":"https://api.github.com/repos/pytorch/pytorch/issues{/number}","pulls_url":"https://api.github.com/repos/pytorch/pytorch/pulls{/number}","milestones_url":"https://api.github.com/repos/pytorch/pytorch/milestones{/number}","notifications_url":"https://api.github.com/repos/pytorch/pytorch/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/pytorch/pytorch/labels{/name}","releases_url":"https://api.github.com/repos/pytorch/pytorch/releases{/id}","deployments_url":"https://api.github.com/repos/pytorch/pytorch/deployments","created_at":"2016-08-13T05:26:41Z","updated_at":"2018-08-24T15:11:50Z","pushed_at":"2018-08-24T15:10:48Z","git_url":"git://github.com/pytorch/pytorch.git","ssh_url":"git@github.com:pytorch/pytorch.git","clone_url":"https://github.com/pytorch/pytorch.git","svn_url":"https://github.com/pytorch/pytorch","homepage":"http://pytorch.org","size":68727,"stargazers_count":18114,"watchers_count":18114,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":4299,"mirror_url":null,"archived":false,"open_issues_count":1649,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":4299,"open_issues":1649,"watchers":18114,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/pytorch/pytorch/pulls/10844"},"html":{"href":"https://github.com/pytorch/pytorch/pull/10844"},"issue":{"href":"https://api.github.com/repos/pytorch/pytorch/issues/10844"},"comments":{"href":"https://api.github.com/repos/pytorch/pytorch/issues/10844/comments"},"review_comments":{"href":"https://api.github.com/repos/pytorch/pytorch/pulls/10844/comments"},"review_comment":{"href":"https://api.github.com/repos/pytorch/pytorch/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/pytorch/pytorch/pulls/10844/commits"},"statuses":{"href":"https://api.github.com/repos/pytorch/pytorch/statuses/5c300b6e70abb13121d37deac2f2bb9563d92a7b"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"ezyang"}}