{"push_id":4050577204,"size":20,"distinct_size":0,"ref":"refs/heads/gh/ezyang/347/head","head":"f3a7ace0a0466d1c1fb7d557a9ffce94e94ceabc","before":"9da34e950b9419fb313e0d1a4721108c7759a644","commits":[{"sha":"72aeafd3d05fc71fba475737fea7a3e29a530767","author":{"name":"Lu Fang","email":"6e6bc3b7f607f5d674fb8a406040e26b26ffec27@fb.com"},"message":"Fix no tab check (#26399)\n\nSummary:\nignore the folder and its children ios/TestApp\nPull Request resolved: https://github.com/pytorch/pytorch/pull/26399\n\nDifferential Revision: D17451239\n\nPulled By: houseroad\n\nfbshipit-source-id: d6ba666bf955454eca4a10c00784ee5947a70f59","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/72aeafd3d05fc71fba475737fea7a3e29a530767"},{"sha":"98ccae09afc8e1d67ff62b3e6fd0ecec21328993","author":{"name":"Pavel Belevich","email":"ab480f254afd36010cb195cde42eb0e19ef1f5bc@fb.com"},"message":"C++ API parity: at::Tensor::grad\n\nSummary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/26150\n\nTest Plan: Imported from OSS\n\nDifferential Revision: D17427579\n\nPulled By: pbelevich\n\nfbshipit-source-id: 68d012076aa86dee9f23fad71a2d265d75f56d22","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/98ccae09afc8e1d67ff62b3e6fd0ecec21328993"},{"sha":"ed09704899253be4c3810f96202655d4ed3bfb04","author":{"name":"Yanli Zhao","email":"017d10c40d56ad915e5ffc6d458f5225efe5bef3@fb.com"},"message":"use allgatherv for sparse all reduce (#23917)\n\nSummary:\nper https://github.com/pytorch/pytorch/issues/22226, The current sparse allreduce in ProcessGroupGloo pads the indices and values tensors to the maximum length across all processes and then performs a regular allgather (because they'll have equal size across processes). Instead, we can use allgatherv. This is mostly a win for memory usage if there is severe size imbalance between processes.\n\nclose https://github.com/pytorch/pytorch/issues/22226\nPull Request resolved: https://github.com/pytorch/pytorch/pull/23917\n\nTest Plan:\nbuck run mode/dev-nosan caffe2/test:c10d -- test_c10d.ProcessGroupGlooTest.test_sparse_allreduce_basics\n\nbuck run mode/dev-nosan caffe2/test:c10d -- test_c10d.ProcessGroupGlooTest.test_sparse_allreduce_basics_cuda\n\nbuck run mode/dev-nosan caffe2/test:c10d -- test_c10d.ProcessGroupGlooTest.test_sparse_allreduce_checks\n\nDifferential Revision: D16664985\n\nPulled By: zhaojuanmao\n\nfbshipit-source-id: e7d3c0770cbc09f9175b3027b527e95053724843","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/ed09704899253be4c3810f96202655d4ed3bfb04"},{"sha":"388cfdf2ac58c79fe5427fe545e181fa601cd671","author":{"name":"Mike Ruberry","email":"3b6c81a5bb1dcc1d5f19bc15b1d7e79960d88113@users.noreply.github.com"},"message":"Removes torchtest, expands generic device testing (#26374)\n\nSummary:\n- Removes torchtest\n- <s>Moves test_torch tests skipped on ROCm to generic device test class</s>\n- Creates test_nn generic device test class\n\nNext: adding dtypes to generic device testing framework.\nPull Request resolved: https://github.com/pytorch/pytorch/pull/26374\n\nTest Plan: Change is to tests themselves.\n\nDifferential Revision: D17442218\n\nPulled By: mruberry\n\nfbshipit-source-id: d7e4451d09fc9049478b35a7efb8bb580071e8c8","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/388cfdf2ac58c79fe5427fe545e181fa601cd671"},{"sha":"b0b0f2c65f4e029e36e17dd8e0f4ebb1c24cb80f","author":{"name":"Shihao Xu","email":"a39c2ff8ffdf6475090a395ebc9d68130a6ff73e@fb.com"},"message":"Make ProcessGroupAgent take num_send_recv_threads as constructor argument (#26313)\n\nSummary:\n# Problem\n\nIf there is not enough number of thread in the RPC Agent thread pool. Some circular dependent works could cause deadlock.\n\nThe current to way to get around this deadlock is to provide abundant number of threads.\n\n# Solution\n\nas titled\nPull Request resolved: https://github.com/pytorch/pytorch/pull/26313\n\nDifferential Revision: D17405491\n\nPulled By: xush6528\n\nfbshipit-source-id: a1d9b6a84db0371cd4b63328fa00f651c0808485","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/b0b0f2c65f4e029e36e17dd8e0f4ebb1c24cb80f"},{"sha":"2801df5ba16b393c3486989a0b0c18f2078906ba","author":{"name":"Hong Xu","email":"0b3307686866342c46008b0805da3549c5788939@topbug.net"},"message":"Add a float version of calc_erfinv (by templating) on CPU (#26070)\n\nSummary:\nCurrently calc_erfinv's float version on CPU is missing. This commit adds the float version (by templating).\n\nI also used this opportunity to clean up calc_erfinv a bit.\nPull Request resolved: https://github.com/pytorch/pytorch/pull/26070\n\nReviewed By: ezyang\n\nDifferential Revision: D17368024\n\nPulled By: VitalyFedyunin\n\nfbshipit-source-id: 00cc3097f340022b3788143e6c12b01c35d72f13","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/2801df5ba16b393c3486989a0b0c18f2078906ba"},{"sha":"ecb82ed5a210da3206c7a1af1132c3d28c9857c2","author":{"name":"Tao Xu","email":"ce4f0bffc749a47f8d7f141d8d386a6e12403111@fb.com"},"message":"clean up the PR job script for iOS build (#26353)\n\nSummary:\nPull Request resolved: https://github.com/pytorch/pytorch/pull/26353\n\n### Summary\n\nAs the new iOS building script has been landed, this PR will clean up some redundant code for the PR jobs.\n\n### Test Plan\n\n- Don't break any existing CI jobs\n- Don't break the old iOS CI jobs\n\nTest Plan: Imported from OSS\n\nDifferential Revision: D17457253\n\nPulled By: xta0\n\nfbshipit-source-id: 0d85117533a62d0b9b7b859b0044fd4388c3c9d4","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/ecb82ed5a210da3206c7a1af1132c3d28c9857c2"},{"sha":"76fb909beb0b01ed2816adbeafd1775f1e500913","author":{"name":"Richard Zou","email":"32cb0331e7af743fc44515966c14887fb6c0bc3c@gmail.com"},"message":"Change \"named_guard\" in native_functions to \"supports_named_tensor\" (#26352)\n\nSummary:\nPull Request resolved: https://github.com/pytorch/pytorch/pull/26352\n\n\"named_guard: P\" is the same as \"supports_named_tensor: !P\".\nAlso changed the error message to be more understandable to users.\n\nTest Plan:\n- `TEST_NAMEDTENSOR=1 pytest test/test_namedtensor.py -v`\n- [namedtensor ci]\n\nDifferential Revision: D17426234\n\nPulled By: zou3519\n\nfbshipit-source-id: 4cab780e6e29e184e79cdd3690f41df9ebb2ecb5","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/76fb909beb0b01ed2816adbeafd1775f1e500913"},{"sha":"18eb92e2afbbdabe73044453419580d5d2b276ba","author":{"name":"Nikolay Korovaiko","email":"528632496459efd07ba3c391cb8ec30c03b8f97f@gmail.com"},"message":"Add support for lists for prim::min and prim::max\n\nSummary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/26155\n\nDifferential Revision: D17455540\n\nPulled By: Krovatkin\n\nfbshipit-source-id: e3aee465d108b59691d6c68f85fbf212a5d6a125","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/18eb92e2afbbdabe73044453419580d5d2b276ba"},{"sha":"13b544602e4bbe7fe5420c072b7a6d3814f208e3","author":{"name":"Hong Xu","email":"0b3307686866342c46008b0805da3549c5788939@topbug.net"},"message":"Fix many type mismatches in the CUDA version of calc_digamma and calc_trigamma (#25791)\n\nSummary:\n- There are some missing casts.\n- Functions like ::log, ::sin will potentially always invoke the double version on host. For\n  example, compiling the following code:\n\n  ```c++\n  #include <cmath>\n\n  float log_float(float f) {\n      return ::logf(f);\n  }\n\n  double log_double(double f) {\n      return ::log(f);\n  }\n\n  float log_float2(float f) {\n      return ::log(f);\n  }\n\n  float log_float3(float f) {\n      return std::log(f);\n  }\n  ```\n\n  using `g++ -c -O3` leads to:\n\n      log_float(float):\n              jmp     logf\n      log_double(double):\n              jmp     log\n      log_float2(float):\n              subq    $8, %rsp\n              cvtss2sd        %xmm0, %xmm0\n              call    log\n              addq    $8, %rsp\n              cvtsd2ss        %xmm0, %xmm0\n              ret\n      log_float3(float):\n              jmp     logf\n\n  Note that log_float2 delegates the call to the double version of log\n  (surrounded by cast), while log_float3 delegates the call correctly to\n  logf. See https://godbolt.org/z/KsRWwW\nPull Request resolved: https://github.com/pytorch/pytorch/pull/25791\n\nDifferential Revision: D17452312\n\nPulled By: izdeby\n\nfbshipit-source-id: 6276a011a373cd7cb144f9ecd84116aa206e7d1b","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/13b544602e4bbe7fe5420c072b7a6d3814f208e3"},{"sha":"a561660241defbfa5448222ce6e1816bcacf1178","author":{"name":"Mike Ruberry","email":"42fe9a903ac79e22ebdbdd651d8a00cb91c679a8@devfair044.maas"},"message":"Puts ROCm tests on default stream (#26394)\n\nSummary:\nThis PR has been updated. Since ORIGINAL PR comment below.\n\nROCm CI builds have been hanging as we've been refactoring tests, even when these refactors seem entirely innocuous. This PR started by commenting out test_stft, for example, a Python test never run on ROCm, and that was sufficient to reliably hang the ROCm build in CI.\n\nPutting ROCm tests back on the default stream appears to remove this hang. So this PR now does that. This is likely to unblock development.\n\nORIGINAL: Some test changes appear to be causing ROCm builds to hang in CI. This PR is an attempt to diagnose the source of the hang.\nPull Request resolved: https://github.com/pytorch/pytorch/pull/26394\n\nTest Plan: Change is to test themselves.\n\nDifferential Revision: D17456678\n\nPulled By: mruberry\n\nfbshipit-source-id: 38d00d01c64b5055c1dfed01687ce3e1c9372887","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/a561660241defbfa5448222ce6e1816bcacf1178"},{"sha":"52d999e17345594403a38af4909703f1944e28ef","author":{"name":"Supriya Rao","email":"a7ec7e3a604005fb51034121a99592309b633095@fb.com"},"message":"Disable QNNPACK tests if pytorch is not built with it. (#26427)\n\nSummary:\nPull Request resolved: https://github.com/pytorch/pytorch/pull/26427\n\nUse the new macro USE_PYTORCH_QNNPACK to enable testing with qnnpack\n\nTest Plan:\ntest caffe2/test:quantized -- TestQNNPackOps\nSummary (total time 4.96s):\n  PASS: 0\n  FAIL: 0\n  SKIP: 4\n    caffe2/test:quantized - test_qlinear_qnnpack (test_quantized.TestQNNPackOps)\n    caffe2/test:quantized - test_qnnpack_add (test_quantized.TestQNNPackOps)\n    caffe2/test:quantized - test_qnnpack_relu (test_quantized.TestQNNPackOps)\n    caffe2/test:quantized - test_qnnpack_maxpool2d (test_quantized.TestQNNPackOps)\n  FATAL: 0\n  TIMEOUT: 0\n  OMIT: 0\n\nReviewed By: ljk53\n\nDifferential Revision: D17459791\n\nfbshipit-source-id: 3798fc270d22123b8807c9c63f12b9940981b115","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/52d999e17345594403a38af4909703f1944e28ef"},{"sha":"248d5857ae8eb441482e04dcbb75235774b68069","author":{"name":"Mike Ruberry","email":"42fe9a903ac79e22ebdbdd651d8a00cb91c679a8@devfair044.maas"},"message":"Adds dtypes decorators to and allows helper methods in device generic test classes (#26375)\n\nSummary:\n- Adds dtypes, dtypesIfCPU, and dtypesIfCUDA decorators.\n- Eliminates the need for nontest members to be defined in an inherited base.\n- Updates one test to use the decorators and updates TestTorchDeviceType with helpers.\n\nThis PR appears to be hanging the ROCm build, which is not entirely surprising. See https://github.com/pytorch/pytorch/issues/26394, which demonstrates that the ROCm build can be hung by commenting out a Python test that was never run on ROCm.\n\ngchanan - what type list, if any, do you want to expose? I imagine most test suites will define their own lists like today. SCALAR_TYPES, QUANTIZED_TYPES, and ALL_TYPES seem reasonable to me. DOCUMENTED_TENSOR_TYPES will be removed, of course.\nPull Request resolved: https://github.com/pytorch/pytorch/pull/26375\n\nTest Plan: Edit is to tests themselves.\n\nDifferential Revision: D17462294\n\nPulled By: mruberry\n\nfbshipit-source-id: f8259ec66709749b1bf8077efc737676af901436","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/248d5857ae8eb441482e04dcbb75235774b68069"},{"sha":"d02369dac268a51f87bb17a193182d7645cf4a85","author":{"name":"BowenBao","email":"9bfbab3ad59df0277a8852270ab6e45f822f60ac@microsoft.com"},"message":"add pass for onnx scalar type conversion (#24378)\n\nSummary:\nThis pass tries to resolve scalar type mismatch issues between input tensors introduced by the implicit type conversions on scalars.\n\ne.g. https://github.com/pytorch/pytorch/issues/23724\nPull Request resolved: https://github.com/pytorch/pytorch/pull/24378\n\nReviewed By: hl475\n\nDifferential Revision: D17088682\n\nPulled By: houseroad\n\nfbshipit-source-id: 3de710f70c3b70b9f76fd36a7c4c76e168dbc756","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/d02369dac268a51f87bb17a193182d7645cf4a85"},{"sha":"ad0af1127b09f9ee9aef2ab88e4831a50ca8a6aa","author":{"name":"Zachary DeVito","email":"ffbacb804f60e1eea39c96029c00fe9a377f84d7@fb.com"},"message":"Add ivalue::type(), part 1 (#25439)\n\nSummary:\nPull Request resolved: https://github.com/pytorch/pytorch/pull/25439\n\nThis introduces a type() method on IValue that returns the tagged type\nof the IValue. The intention is that this value is always present/accurate,\nmaking it possible for clients to recover the Type from an IValue.\nCurrently our APIs here are incomplete: they can sometimes recover a type but not always.\n\nThis PR adds the function, and cleans up remaining cases where Lists/Dicts are not\ntagged. However, this information does not survive serialization unchanged.\n\nA second PR will use the type information in the ClassType being serialized\nto fixup the serialized ivalues to have the correct types again.\nAfter this patch it will be save to remove our incomplete APIs for recovering types.\n\nTest Plan: Imported from OSS\n\nDifferential Revision: D17125595\n\nPulled By: zdevito\n\nfbshipit-source-id: 71c8c1a0e44762647e8f15f45d8ed73af8e6cb92","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/ad0af1127b09f9ee9aef2ab88e4831a50ca8a6aa"},{"sha":"12762cd586f2197b30a7c3ba670ff86b9726194b","author":{"name":"Zachary DeVito","email":"ffbacb804f60e1eea39c96029c00fe9a377f84d7@fb.com"},"message":"Use static type information to restore type tags (#25447)\n\nSummary:\nPull Request resolved: https://github.com/pytorch/pytorch/pull/25447\n\nWhen we unpickle IValues, we lose type information for List[T]\nand Dict[K, V]. We can restore this information using the static\ntype information contained in the top-level Module/Class type.\n\nThis ensures that even after serialization we can always get the\ndynamic type of an ivalue using its type() method.\n\nTest Plan: Imported from OSS\n\nDifferential Revision: D17127872\n\nPulled By: zdevito\n\nfbshipit-source-id: 1ffb5e37a7c35c71ac9d3fb7b2edbc7ce3fbec72","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/12762cd586f2197b30a7c3ba670ff86b9726194b"},{"sha":"99b9e92e3a3d7766d503f6c6ca45ed8c546a4538","author":{"name":"Edward Z. Yang","email":"dbd597f5635f432486c5d365e9bb585b3eaa1853@fb.com"},"message":"Update base for Update on \"Implement multiple dispatch\"\n\n\r\nInstead of considering only the TensorTypeSet of the first argument, we collect all Tensor and TensorList arguments and union them together before computing the dispatch type id.\r\n\r\nXLA companion patch at https://github.com/pytorch/xla/pull/1031\r\n\r\nBilling of changes:\r\n* ATenDispatch fallback code (i.e., what gets run if there is no entry for a function in the table) now lives out-of-line in a function `getFallbackOp`. This gave me an opportunity to write a more detailed error message, providing information about what registrations were available. There is a TODO in the fallback code, suggesting that we could automatically redispatch in the event that there is no handler for the key. But this is a bit of a design question, because it's not clear if automatic redispatch would cover up errors in the dispatch table (i.e., there *should* have been something registered at some key, but there wasn't.)\r\n* Collection of Tensor/TensorList arguments is done using the trusty old IterArgs helper class. A minor bit of refactoring I had to do to get here was move the IterArgs functionality in torch/csrc/utils/variadic.h into ATen/core.  There's some refactoring due on that file too (it has copies of some C++ helper pieces which already live in c10--you can't actually move the whole thing because it is literally incompatible with other code in the codebase). So instead of calling `type_set()` to get the type set of the dispatch argument, now we just call `at::detail::multi_dispatch_tensor_type_set` on all of the tensor/tensor list arguments.\r\n* The code generator is adjusted to codegen collection of arguments as needed. There is a little bit of a hack in the code generator to turn 'self' arguments into '*this'.  I think this may be duplicated with some logic somewhere else but I have to double check.\r\n\r\nThe new generated code looks like this:\r\n\r\n```\r\ninline Tensor & Tensor::copy_(const Tensor & src, bool non_blocking) const {\r\n    static auto table = globalATenDispatch().getOpTable(\"aten::copy_(Tensor(a!) self, Tensor src, bool non_blocking=False) -> Tensor(a!)\");\r\n    return table->getOp<Tensor & (Tensor &, const Tensor &, bool)>(at::detail::multi_dispatch_tensor_type_set(*this, src))(const_cast<Tensor&>(*this), src, non_blocking);\r\n}          \r\n```\r\n\r\nThe key difference is that previously we wrote `type_set()` as argument to getOp; now it is a call to `multi_dispatch_tensor_type_set` which collects the type ids together.\r\n\r\nAfter turning on multi-dispatch, I had to refactor existing code which previously dispatched one place, but now dispatches somewhere else. The primary component affected by this is sparse.\r\n\r\n* Binary operations (add/sub/mul/div/addmm) now dispatch to sparse kernels even if you did add(dense, sparse). So I delete all the sparse handling code from dense kernels, and bulk up the sparse error handling to handle when the first argument is dense. In the case of addmm, I can eliminate the bridge code entirely (well, not quite: more on this below). I also updated the dispatch on sparse to actually point at sparse kernels. Pay special attention to the handling of `div_` by scalar: previously this logic lived in the \"dense\" `div_` implementation, but there is actually not any sparse kernel we dispatch to. I solved this particular problem by making a redispatch, but another valid approach would have been to add specific dispatches for sparse div on scalar. This codepath is poorly tested because it is only exercised from C++.\r\n* One minor annoyance is that because I now want separate dispatch for dense and sparse, I also need to replicate the `add`, `add_`, `add_out` trifecta on the sparse side. I opted for a compromise here: I wrote new a new `add_sparse` trifecta, but reused the implementation between CPU and CUDA. This means that I hav to do another dispatch once I get to `add_out`. The alternative would have been to do twice as many copies for CPU and CUDA (thereby eliminating the extra dispatch) but that seemed distinctly not worth it.\r\n* A lot of kernels in sparse assumed that the dispatch argument must be sparse. This is no longer true with dispatch, so I converted the asserts into plain error checking. This also means that we've perturbed the error message in the case of TestSparseOneOff.test_cuda_sparse_cpu_dense_add (I just updated the saved error message)\r\n* `addmm` is a little bit even more special: the bridge code also handled broadcasting. I replicated the broadcasting logic between CPU and CUDA implementations to avoid an extra dispatch.\r\n* `_sparse_addmm` gave me a bit of trouble, because I had forgotten why we had `torch.sparse.addmm` in the first place. But in the end, its changes followed along with the structural changes I made in addmm. I opted for an extra dispatch here for simplicity.\r\n* c10d has some Variable-Tensor confusion in its sparse code. I've worked around it by judiciously inserting \"no variable type\" guards, but a more correct fix would be to just solve the confusion entirely.\r\n\r\nBenchmark:\r\n\r\nApply the following patch to the base commit and this commit:\r\n\r\n```\r\ndiff --git a/aten/src/ATen/native/Const.cpp b/aten/src/ATen/native/Const.cpp\r\nnew file mode 100644\r\nindex 0000000000..b66f4d3ece\r\n--- /dev/null\r\n+++ b/aten/src/ATen/native/Const.cpp\r\n@@ -0,0 +1,10 @@\r\n+#include <ATen/ATen.h>\r\n+\r\n+namespace at {\r\n+namespace native {\r\n+\r\n+Tensor _const5(const Tensor& self, const Tensor& second, const Tensor& third, const Tensor& fourth, const Tensor& fifth) {\r\n+  return self;\r\n+}\r\n+\r\n+}} // namespace at::native\r\ndiff --git a/aten/src/ATen/native/native_functions.yaml b/aten/src/ATen/native/native_functions.yaml\r\nindex b494ed7950..fddae638bb 100644\r\n--- a/aten/src/ATen/native/native_functions.yaml\r\n+++ b/aten/src/ATen/native/native_functions.yaml\r\n@@ -5878,3 +5878,9 @@\r\n   dispatch:\r\n     CPU: im2col_backward_cpu\r\n     CUDA: im2col_backward_cuda\r\n+\r\n+# For benchmarking\r\n+- func: _const5(Tensor self, Tensor second, Tensor third, Tensor fourth, Tensor fifth) -> Tensor\r\n+  variants: function\r\n+  dispatch:\r\n+    CPU: _const5\r\n```\r\n\r\nComparisons with timeit:\r\n\r\nOne-argument, representative case:\r\n\r\nBefore:\r\n\r\n```\r\nIn [6]: %timeit x.reshape(1, 1)\r\n1.46 µs ± 1.38 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [7]: %timeit x.reshape(1, 1)\r\n1.48 µs ± 29.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [8]: %timeit x.reshape(1, 1)\r\n1.52 µs ± 61.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n```\r\n\r\nAfter:\r\n\r\n```\r\nIn [3]: %timeit x.reshape(1, 1)\r\n1.42 µs ± 1.34 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [4]: %timeit x.reshape(1, 1)\r\n1.43 µs ± 1.01 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [5]: %timeit x.reshape(1, 1)\r\n1.42 µs ± 0.982 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n```\r\n\r\nFive-argument, synthetic case (we expect, with enough Tensor arguments, for there to be a slowdown, as we scale `O(n)` with number of arguments, compared to old dispatcher which is `O(1)` with number of arguments):\r\n\r\nBefore:\r\n\r\n```\r\nIn [1]: import torch\r\n\r\nIn [2]: x = torch.zeros(1)\r\n\r\nIn [3]: %timeit torch._const5(x, x, x, x, x)\r\n949 ns ± 1.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [4]: %timeit torch._const5(x, x, x, x, x)\r\n954 ns ± 1.96 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [5]: %timeit torch._const5(x, x, x, x, x)\r\n947 ns ± 0.601 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n```\r\n\r\nAfter:\r\n\r\n```\r\nIn [3]: %timeit torch._const5(x, x, x, x, x)                               \r\n985 ns ± 9.11 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [4]: %timeit torch._const5(x, x, x, x, x)                            \r\n984 ns ± 1.17 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n                                                    \r\nIn [5]: %timeit torch._const5(x, x, x, x, x)\r\n988 ns ± 0.555 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n```\r\n\r\nSigned-off-by: Edward Z. Yang <ezyang@fb.com>\r\n\r\nDifferential Revision: [D17265918](https://our.internmc.facebook.com/intern/diff/D17265918)\n\n[ghstack-poisoned]","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/99b9e92e3a3d7766d503f6c6ca45ed8c546a4538"},{"sha":"3082b6377c61d71f6cdb43a783791ecfac1f6881","author":{"name":"Edward Z. Yang","email":"dbd597f5635f432486c5d365e9bb585b3eaa1853@fb.com"},"message":"Update on \"Implement multiple dispatch\"\n\n\r\nInstead of considering only the TensorTypeSet of the first argument, we collect all Tensor and TensorList arguments and union them together before computing the dispatch type id.\r\n\r\nXLA companion patch at https://github.com/pytorch/xla/pull/1031\r\n\r\nBilling of changes:\r\n* ATenDispatch fallback code (i.e., what gets run if there is no entry for a function in the table) now lives out-of-line in a function `getFallbackOp`. This gave me an opportunity to write a more detailed error message, providing information about what registrations were available. There is a TODO in the fallback code, suggesting that we could automatically redispatch in the event that there is no handler for the key. But this is a bit of a design question, because it's not clear if automatic redispatch would cover up errors in the dispatch table (i.e., there *should* have been something registered at some key, but there wasn't.)\r\n* Collection of Tensor/TensorList arguments is done using the trusty old IterArgs helper class. A minor bit of refactoring I had to do to get here was move the IterArgs functionality in torch/csrc/utils/variadic.h into ATen/core.  There's some refactoring due on that file too (it has copies of some C++ helper pieces which already live in c10--you can't actually move the whole thing because it is literally incompatible with other code in the codebase). So instead of calling `type_set()` to get the type set of the dispatch argument, now we just call `at::detail::multi_dispatch_tensor_type_set` on all of the tensor/tensor list arguments.\r\n* The code generator is adjusted to codegen collection of arguments as needed. There is a little bit of a hack in the code generator to turn 'self' arguments into '*this'.  I think this may be duplicated with some logic somewhere else but I have to double check.\r\n\r\nThe new generated code looks like this:\r\n\r\n```\r\ninline Tensor & Tensor::copy_(const Tensor & src, bool non_blocking) const {\r\n    static auto table = globalATenDispatch().getOpTable(\"aten::copy_(Tensor(a!) self, Tensor src, bool non_blocking=False) -> Tensor(a!)\");\r\n    return table->getOp<Tensor & (Tensor &, const Tensor &, bool)>(at::detail::multi_dispatch_tensor_type_set(*this, src))(const_cast<Tensor&>(*this), src, non_blocking);\r\n}          \r\n```\r\n\r\nThe key difference is that previously we wrote `type_set()` as argument to getOp; now it is a call to `multi_dispatch_tensor_type_set` which collects the type ids together.\r\n\r\nAfter turning on multi-dispatch, I had to refactor existing code which previously dispatched one place, but now dispatches somewhere else. The primary component affected by this is sparse.\r\n\r\n* Binary operations (add/sub/mul/div/addmm) now dispatch to sparse kernels even if you did add(dense, sparse). So I delete all the sparse handling code from dense kernels, and bulk up the sparse error handling to handle when the first argument is dense. In the case of addmm, I can eliminate the bridge code entirely (well, not quite: more on this below). I also updated the dispatch on sparse to actually point at sparse kernels. Pay special attention to the handling of `div_` by scalar: previously this logic lived in the \"dense\" `div_` implementation, but there is actually not any sparse kernel we dispatch to. I solved this particular problem by making a redispatch, but another valid approach would have been to add specific dispatches for sparse div on scalar. This codepath is poorly tested because it is only exercised from C++.\r\n* One minor annoyance is that because I now want separate dispatch for dense and sparse, I also need to replicate the `add`, `add_`, `add_out` trifecta on the sparse side. I opted for a compromise here: I wrote new a new `add_sparse` trifecta, but reused the implementation between CPU and CUDA. This means that I hav to do another dispatch once I get to `add_out`. The alternative would have been to do twice as many copies for CPU and CUDA (thereby eliminating the extra dispatch) but that seemed distinctly not worth it.\r\n* A lot of kernels in sparse assumed that the dispatch argument must be sparse. This is no longer true with dispatch, so I converted the asserts into plain error checking. This also means that we've perturbed the error message in the case of TestSparseOneOff.test_cuda_sparse_cpu_dense_add (I just updated the saved error message)\r\n* `addmm` is a little bit even more special: the bridge code also handled broadcasting. I replicated the broadcasting logic between CPU and CUDA implementations to avoid an extra dispatch.\r\n* `_sparse_addmm` gave me a bit of trouble, because I had forgotten why we had `torch.sparse.addmm` in the first place. But in the end, its changes followed along with the structural changes I made in addmm. I opted for an extra dispatch here for simplicity.\r\n* c10d has some Variable-Tensor confusion in its sparse code. I've worked around it by judiciously inserting \"no variable type\" guards, but a more correct fix would be to just solve the confusion entirely.\r\n\r\nBenchmark:\r\n\r\nApply the following patch to the base commit and this commit:\r\n\r\n```\r\ndiff --git a/aten/src/ATen/native/Const.cpp b/aten/src/ATen/native/Const.cpp\r\nnew file mode 100644\r\nindex 0000000000..b66f4d3ece\r\n--- /dev/null\r\n+++ b/aten/src/ATen/native/Const.cpp\r\n@@ -0,0 +1,10 @@\r\n+#include <ATen/ATen.h>\r\n+\r\n+namespace at {\r\n+namespace native {\r\n+\r\n+Tensor _const5(const Tensor& self, const Tensor& second, const Tensor& third, const Tensor& fourth, const Tensor& fifth) {\r\n+  return self;\r\n+}\r\n+\r\n+}} // namespace at::native\r\ndiff --git a/aten/src/ATen/native/native_functions.yaml b/aten/src/ATen/native/native_functions.yaml\r\nindex b494ed7950..fddae638bb 100644\r\n--- a/aten/src/ATen/native/native_functions.yaml\r\n+++ b/aten/src/ATen/native/native_functions.yaml\r\n@@ -5878,3 +5878,9 @@\r\n   dispatch:\r\n     CPU: im2col_backward_cpu\r\n     CUDA: im2col_backward_cuda\r\n+\r\n+# For benchmarking\r\n+- func: _const5(Tensor self, Tensor second, Tensor third, Tensor fourth, Tensor fifth) -> Tensor\r\n+  variants: function\r\n+  dispatch:\r\n+    CPU: _const5\r\n```\r\n\r\nComparisons with timeit:\r\n\r\nOne-argument, representative case:\r\n\r\nBefore:\r\n\r\n```\r\nIn [6]: %timeit x.reshape(1, 1)\r\n1.46 µs ± 1.38 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [7]: %timeit x.reshape(1, 1)\r\n1.48 µs ± 29.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [8]: %timeit x.reshape(1, 1)\r\n1.52 µs ± 61.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n```\r\n\r\nAfter:\r\n\r\n```\r\nIn [3]: %timeit x.reshape(1, 1)\r\n1.42 µs ± 1.34 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [4]: %timeit x.reshape(1, 1)\r\n1.43 µs ± 1.01 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [5]: %timeit x.reshape(1, 1)\r\n1.42 µs ± 0.982 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n```\r\n\r\nFive-argument, synthetic case (we expect, with enough Tensor arguments, for there to be a slowdown, as we scale `O(n)` with number of arguments, compared to old dispatcher which is `O(1)` with number of arguments):\r\n\r\nBefore:\r\n\r\n```\r\nIn [1]: import torch\r\n\r\nIn [2]: x = torch.zeros(1)\r\n\r\nIn [3]: %timeit torch._const5(x, x, x, x, x)\r\n949 ns ± 1.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [4]: %timeit torch._const5(x, x, x, x, x)\r\n954 ns ± 1.96 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [5]: %timeit torch._const5(x, x, x, x, x)\r\n947 ns ± 0.601 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n```\r\n\r\nAfter:\r\n\r\n```\r\nIn [3]: %timeit torch._const5(x, x, x, x, x)                               \r\n985 ns ± 9.11 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n\r\nIn [4]: %timeit torch._const5(x, x, x, x, x)                            \r\n984 ns ± 1.17 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n                                                    \r\nIn [5]: %timeit torch._const5(x, x, x, x, x)\r\n988 ns ± 0.555 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\r\n```\r\n\r\nSigned-off-by: Edward Z. Yang <ezyang@fb.com>\r\n\r\nDifferential Revision: [D17265918](https://our.internmc.facebook.com/intern/diff/D17265918)\n\n[ghstack-poisoned]","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/3082b6377c61d71f6cdb43a783791ecfac1f6881"},{"sha":"df88b83d602b5021b47dc8d57b3b7ca5542f9f5c","author":{"name":"Edward Z. Yang","email":"dbd597f5635f432486c5d365e9bb585b3eaa1853@fb.com"},"message":"Update on \"Delete backwards compatibility Backend overload for registerOp\"\n\n\nSigned-off-by: Edward Z. Yang <ezyang@fb.com>\n\nDifferential Revision: [D17284083](https://our.internmc.facebook.com/intern/diff/D17284083)\n\n[ghstack-poisoned]","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/df88b83d602b5021b47dc8d57b3b7ca5542f9f5c"},{"sha":"f3a7ace0a0466d1c1fb7d557a9ffce94e94ceabc","author":{"name":"Edward Z. Yang","email":"dbd597f5635f432486c5d365e9bb585b3eaa1853@fb.com"},"message":"Update on \"Implement multiple dispatch in boxed c10 dispatcher\"\n\n\nSigned-off-by: Edward Z. Yang <ezyang@fb.com>\n\nDifferential Revision: [D17404367](https://our.internmc.facebook.com/intern/diff/D17404367)\n\n[ghstack-poisoned]","distinct":false,"url":"https://api.github.com/repos/pytorch/pytorch/commits/f3a7ace0a0466d1c1fb7d557a9ffce94e94ceabc"}]}
{"actor":{"display_login":"ezyang"}}