{"push_id":2929353673,"size":3,"distinct_size":3,"ref":"refs/heads/_v2,3_3__powerpc_powernv_npu_Remove_atsd_threshold_debugfs_setting_mbox_next","head":"c110c64c734d7a9e4d82c2cf5dac530490c3a7bd","before":"90d69849ad92346ba23b2183f70ee2cd730a81ee","commits":[{"sha":"86c727a8d9c96abc385e30dbc1ffabe1904e7b04","author":{"name":"Mark Hairgrove","email":"54959fc8b60e3604fe2c52d8937efb756b452305@nvidia.com"},"message":"powerpc/powernv/npu: Reduce eieio usage when issuing ATSD invalidates\n\nThere are two types of ATSDs issued to the NPU: invalidates targeting a\nspecific virtual address and invalidates targeting the whole address\nspace. In both cases prior to this change, the sequence was:\n\n    for each NPU\n        - Write the target address to the XTS_ATSD_AVA register\n        - EIEIO\n        - Write the launch value to issue the ATSD\n\nFirst, a target address is not required when invalidating the whole\naddress space, so that write and the EIEIO have been removed. The AP\n(size) field in the launch is not needed either.\n\nSecond, for per-address invalidates the above sequence is inefficient in\nthe common case of multiple NPUs because an EIEIO is issued per NPU. This\nunnecessarily forces the launches of later ATSDs to be ordered with the\nlaunches of earlier ones. The new sequence only issues a single EIEIO:\n\n    for each NPU\n        - Write the target address to the XTS_ATSD_AVA register\n    EIEIO\n    for each NPU\n        - Write the launch value to issue the ATSD\n\nPerformance results were gathered using a microbenchmark which creates a\n1G allocation then uses mprotect with PROT_NONE to trigger invalidates in\nstrides across the allocation.\n\nWith only a single NPU active (one GPU) the difference is in the noise for\nboth types of invalidates (+/-1%).\n\nWith two NPUs active (on a 6-GPU system) the effect is more noticeable:\n\n         mprotect rate (GB/s)\nStride   Before      After      Speedup\n64K         5.9        6.5          10%\n1M         31.2       33.4           7%\n2M         36.3       38.7           7%\n4M        322.6      356.7          11%\n\nSigned-off-by: Mark Hairgrove <mhairgrove@nvidia.com>","distinct":true,"url":"https://api.github.com/repos/ruscur/linux/commits/86c727a8d9c96abc385e30dbc1ffabe1904e7b04"},{"sha":"93181157c44fb81e6f188c9795a387fd071ae95a","author":{"name":"Mark Hairgrove","email":"54959fc8b60e3604fe2c52d8937efb756b452305@nvidia.com"},"message":"powerpc/powernv/npu: Use size-based ATSD invalidates\n\nPrior to this change only two types of ATSDs were issued to the NPU:\ninvalidates targeting a single page and invalidates targeting the whole\naddress space. The crossover point happened at the configurable\natsd_threshold which defaulted to 2M. Invalidates that size or smaller\nwould issue per-page invalidates for the whole range.\n\nThe NPU supports more invalidation sizes however: 64K, 2M, 1G, and all.\nThese invalidates target addresses aligned to their size. 2M is a common\ninvalidation size for GPU-enabled applications because that is a GPU\npage size, so reducing the number of invalidates by 32x in that case is a\nclear improvement.\n\nATSD latency is high in general so now we always issue a single invalidate\nrather than multiple. This will over-invalidate in some cases, but for any\ninvalidation size over 2M it matches or improves the prior behavior.\nThere's also an improvement for single-page invalidates since the prior\nversion issued two invalidates for that case instead of one.\n\nWith this change all issued ATSDs now perform a flush, so the flush\nparameter has been removed from all the helpers.\n\nTo show the benefit here are some performance numbers from a\nmicrobenchmark which creates a 1G allocation then uses mprotect with\nPROT_NONE to trigger invalidates in strides across the allocation.\n\nOne NPU (1 GPU):\n\n         mprotect rate (GB/s)\nStride   Before      After      Speedup\n64K         5.3        5.6           5%\n1M         39.3       57.4          46%\n2M         49.7       82.6          66%\n4M        286.6      285.7           0%\n\nTwo NPUs (6 GPUs):\n\n         mprotect rate (GB/s)\nStride   Before      After      Speedup\n64K         6.5        7.4          13%\n1M         33.4       67.9         103%\n2M         38.7       93.1         141%\n4M        356.7      354.6          -1%\n\nAnything over 2M is roughly the same as before since both cases issue a\nsingle ATSD.\n\nSigned-off-by: Mark Hairgrove <mhairgrove@nvidia.com>","distinct":true,"url":"https://api.github.com/repos/ruscur/linux/commits/93181157c44fb81e6f188c9795a387fd071ae95a"},{"sha":"c110c64c734d7a9e4d82c2cf5dac530490c3a7bd","author":{"name":"Mark Hairgrove","email":"54959fc8b60e3604fe2c52d8937efb756b452305@nvidia.com"},"message":"powerpc/powernv/npu: Remove atsd_threshold debugfs setting\n\nThis threshold is no longer used now that all invalidates issue a single\nATSD to each active NPU.\n\nSigned-off-by: Mark Hairgrove <mhairgrove@nvidia.com>","distinct":true,"url":"https://api.github.com/repos/ruscur/linux/commits/c110c64c734d7a9e4d82c2cf5dac530490c3a7bd"}]}
{"actor":{"display_login":"ruscur"}}