{"push_id":2718405094,"size":2,"distinct_size":2,"ref":"refs/heads/export-D8820351","head":"a3a0f328f3058f8e76f7e8a232e3b36d44f1707f","before":"7be69e6721440077190a658c6db2774d3e163b4a","commits":[{"sha":"cc9dcdff16b26fee9b336388e42d7827f83fd7f4","author":{"name":"Michael Carilli","email":"5598654a278f0d7fc5fab046d4b818491d625f6a@nvidia.com"},"message":"Improving THCReduce.cuh's performance on latency-bound non-contiguous reductions (#9214)\n\nSummary:\nThis PR improves perfomance of (formerly) latency-bound non-contig-dim reduction kernels by up to 20X, while maintaining determinism.\n\nCurrently, reducing across a non-contiguous dimension uses the parallelism exposed across the number of output elements.  This means that performance suffers if the number of output elements is small.  Example:\n```\na = torch.cuda.FloatTensor(32768, 32)\na.sum(dim=0)\n```\nBefore this PR, `a.sum`'s kernel (kernelReduceNoncontigDim_shared) took 138 microseconds on my machine.  The speed-of-light estimate (based on a bandwidth of 700 GB/s) should be around 6 microseconds.  After this PR's changes, `a.sum(dim=0)`'s kernel takes 6.9 microseconds on my machine.\n\nChristian implemented some nice logic to squeeze out better performance for cases like `a.sum` using intra-block and instruction-level parallelism across the dimension being reduced, but his kernel still only launched one block for every 32 output elements.  This was insufficient to saturate the device in many cases, like `a.sum` here (where only one block is launched).\n\nMy PR adds block cooperation across the dimension being reduced.  Many blocks, instead of one block, help to reduce into each 32 output elements.  Internally, each block leverages all of Christian's nice logic to compute a partial reduction into a per-block staging buffer, then the last block to finish combines the results to compute the final output.\n\nBlock cooperation does require THCudaMalloc-ing staging and semaphore buffers, so it's not always worthwhile.  I included a set of rough heuristics to decide when the kernel should choose to use block cooperation.  These heuristics are based on Python-side timings of calling sum() many times in a loop, and comparing to the old implementation.\n\n I tested a wide range of sizes (to determine heuristics) and as long as the number of output elements is greater than 16ish, I don't think there are any remaining pathological sizes where users will encounter unexpectedly poor performance.\nPull Request resolved: https://github.com/pytorch/pytorch/pull/9214\n\nReviewed By: gchanan\n\nDifferential Revision: D8808127\n\nPulled By: colesbury\n\nfbshipit-source-id: 139f310fc6ea6d187a7c983128f8eb8e1c9b4be3","distinct":true,"url":"https://api.github.com/repos/nateanl/pytorch/commits/cc9dcdff16b26fee9b336388e42d7827f83fd7f4"},{"sha":"a3a0f328f3058f8e76f7e8a232e3b36d44f1707f","author":{"name":"Zhaoheng Ni","email":"25d508652c8799722bd3413ffcb42f11fee3f49f@fb.com"},"message":"Add BatchBucketizeOp in caffe2 (#9385)\n\nSummary:\nPull Request resolved: https://github.com/pytorch/pytorch/pull/9385\n\nThe operator transform dense features to sparse features by bucketizing. Only the feature in indices tensor will be transformed and output.\n\nDifferential Revision: D8820351\n\nfbshipit-source-id: 08264929289d1a976b53623ff87f5c5ba3556f5b","distinct":true,"url":"https://api.github.com/repos/nateanl/pytorch/commits/a3a0f328f3058f8e76f7e8a232e3b36d44f1707f"}]}
{"actor":{"display_login":"nateanl"}}