{"push_id":3965074097,"size":1,"distinct_size":1,"ref":"refs/heads/gh/jamesr66a/80/head","head":"daddea5ddaf24327be2f868e390a556de74c4ecc","before":"0cf721aad2cf447d03dcbdb0821bbfd6b6309e62","commits":[{"sha":"daddea5ddaf24327be2f868e390a556de74c4ecc","author":{"name":"James Reed","email":"f6111144b37277d4771706e14416c5fc33d0fdf4@fb.com"},"message":"Update on \"[WIP] Quantized vec256 + vectorized quantized::add\"\n\n\r\nDifferential Revision: [D17061047](https://our.internmc.facebook.com/intern/diff/D17061047)\r\n\r\nThis PR adds:\r\n\r\n1) Vec256<{qint8,quint8,qint32}>, with facilities for loading, storing, quantizing and dequantizing vectors\r\n   of quantized data. Note this is generic infrastructure we can also apply to other ops\r\n2) Integration of Vec256 into `quantized::add`\r\n\r\nBenchmark script:\r\n\r\n```\r\nimport torch\r\nimport time\r\n\r\nx = torch.rand(1, 256, 56, 56)\r\ny = torch.rand(1, 256, 56, 56)\r\n\r\nprint('dtype', 'ms/iter (float)', 'ms/iter (quant)', 'quant / float', sep='\\t')\r\n\r\nfor dtype in [torch.quint8, torch.qint8, torch.qint32]:\r\n    qX = torch.quantize_linear(x, 0.1, 128, torch.quint8)\r\n    qY = torch.quantize_linear(y, 0.1, 128, torch.quint8)\r\n\r\n    NITER = 1000\r\n\r\n    # Test float\r\n    s = time.time()\r\n    for i in range(NITER):\r\n        x + y\r\n    elapsed_float = time.time() - s\r\n    ms_per_iter_float = elapsed_float / NITER * 1000    \r\n\r\n    # Test quantized\r\n    s = time.time()\r\n    for i in range(NITER):\r\n        torch.ops.quantized.add(qX, qY, 0.1, 128)\r\n    elapsed = time.time() - s\r\n    ms_per_iter = elapsed / NITER * 1000\r\n\r\n    print(str(dtype), ms_per_iter_float, ms_per_iter, ms_per_iter / ms_per_iter_float, sep='\\t')\r\n```\r\n\r\nResults on my machine with OMP disabled (results are similar with OMP enabled but it makes\r\nmore sense to report these numbers since we don't use OMP in prod):\r\n\r\n`OMP_NUM_THREADS=1 python benchmark.py`\r\n\r\n```\r\nWITHOUT OMP\r\n\r\nBefore vectorization\r\n\r\ndtype               ms/iter (float)                  ms/iter (quant)                 quant / float\r\ntorch.quint8     0.47158026695251465\t5.631141901016235\t11.94100409969711\r\ntorch.qint8\t0.45938873291015625\t5.78427267074585\t12.591237564977662\r\ntorch.qint32\t0.46388673782348633\t5.619501113891602\t12.113950789491808\r\n\r\nAfter vectorization\r\n\r\ndtype\t        ms/iter (float)\t                ms/iter (quant)\t        quant / float\r\ntorch.quint8\t0.4659900665283203\t1.7951319217681885\t3.8522965417313038\r\ntorch.qint8\t0.4428229331970215\t1.7832145690917969\t4.026924613451323\r\ntorch.qint32\t0.4432840347290039\t1.7856943607330322\t4.028329966416891\r\n```\r\n\r\n*NOTE*: These results required me to manually pass `-march=native` to the compiler in my pytorch\r\nbuild. This is because the files in `native/quantized/cpu` do not follow the same compilation\r\nrules as `native/cpu`. We likely need to replicate this for the quantized op implementations, which\r\nI plan to do in a subsequent PR","distinct":true,"url":"https://api.github.com/repos/pytorch/pytorch/commits/daddea5ddaf24327be2f868e390a556de74c4ecc"}]}
{"actor":{"display_login":"jamesr66a"}}