{"action":"closed","issue":{"url":"https://api.github.com/repos/dmlc/tvm/issues/1485","repository_url":"https://api.github.com/repos/dmlc/tvm","labels_url":"https://api.github.com/repos/dmlc/tvm/issues/1485/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/tvm/issues/1485/comments","events_url":"https://api.github.com/repos/dmlc/tvm/issues/1485/events","html_url":"https://github.com/dmlc/tvm/issues/1485","id":344278618,"node_id":"MDU6SXNzdWUzNDQyNzg2MTg=","number":1485,"title":"[RFC] Generalize compute to tensor region","user":{"login":"ZihengJiang","id":17693755,"node_id":"MDQ6VXNlcjE3NjkzNzU1","avatar_url":"https://avatars2.githubusercontent.com/u/17693755?v=4","gravatar_id":"","url":"https://api.github.com/users/ZihengJiang","html_url":"https://github.com/ZihengJiang","followers_url":"https://api.github.com/users/ZihengJiang/followers","following_url":"https://api.github.com/users/ZihengJiang/following{/other_user}","gists_url":"https://api.github.com/users/ZihengJiang/gists{/gist_id}","starred_url":"https://api.github.com/users/ZihengJiang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ZihengJiang/subscriptions","organizations_url":"https://api.github.com/users/ZihengJiang/orgs","repos_url":"https://api.github.com/users/ZihengJiang/repos","events_url":"https://api.github.com/users/ZihengJiang/events{/privacy}","received_events_url":"https://api.github.com/users/ZihengJiang/received_events","type":"User","site_admin":false},"labels":[{"id":540459405,"node_id":"MDU6TGFiZWw1NDA0NTk0MDU=","url":"https://api.github.com/repos/dmlc/tvm/labels/priority:%20high","name":"priority: high","color":"d93f0b","default":false},{"id":540459689,"node_id":"MDU6TGFiZWw1NDA0NTk2ODk=","url":"https://api.github.com/repos/dmlc/tvm/labels/status:%20RFC","name":"status: RFC","color":"15d3d0","default":false}],"state":"closed","locked":false,"assignee":{"login":"tqchen","id":2577440,"node_id":"MDQ6VXNlcjI1Nzc0NDA=","avatar_url":"https://avatars3.githubusercontent.com/u/2577440?v=4","gravatar_id":"","url":"https://api.github.com/users/tqchen","html_url":"https://github.com/tqchen","followers_url":"https://api.github.com/users/tqchen/followers","following_url":"https://api.github.com/users/tqchen/following{/other_user}","gists_url":"https://api.github.com/users/tqchen/gists{/gist_id}","starred_url":"https://api.github.com/users/tqchen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tqchen/subscriptions","organizations_url":"https://api.github.com/users/tqchen/orgs","repos_url":"https://api.github.com/users/tqchen/repos","events_url":"https://api.github.com/users/tqchen/events{/privacy}","received_events_url":"https://api.github.com/users/tqchen/received_events","type":"User","site_admin":false},"assignees":[{"login":"tqchen","id":2577440,"node_id":"MDQ6VXNlcjI1Nzc0NDA=","avatar_url":"https://avatars3.githubusercontent.com/u/2577440?v=4","gravatar_id":"","url":"https://api.github.com/users/tqchen","html_url":"https://github.com/tqchen","followers_url":"https://api.github.com/users/tqchen/followers","following_url":"https://api.github.com/users/tqchen/following{/other_user}","gists_url":"https://api.github.com/users/tqchen/gists{/gist_id}","starred_url":"https://api.github.com/users/tqchen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tqchen/subscriptions","organizations_url":"https://api.github.com/users/tqchen/orgs","repos_url":"https://api.github.com/users/tqchen/repos","events_url":"https://api.github.com/users/tqchen/events{/privacy}","received_events_url":"https://api.github.com/users/tqchen/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/dmlc/tvm/milestones/2","html_url":"https://github.com/dmlc/tvm/milestone/2","labels_url":"https://api.github.com/repos/dmlc/tvm/milestones/2/labels","id":3566224,"node_id":"MDk6TWlsZXN0b25lMzU2NjIyNA==","number":2,"title":"v0.5","description":null,"creator":{"login":"tqchen","id":2577440,"node_id":"MDQ6VXNlcjI1Nzc0NDA=","avatar_url":"https://avatars3.githubusercontent.com/u/2577440?v=4","gravatar_id":"","url":"https://api.github.com/users/tqchen","html_url":"https://github.com/tqchen","followers_url":"https://api.github.com/users/tqchen/followers","following_url":"https://api.github.com/users/tqchen/following{/other_user}","gists_url":"https://api.github.com/users/tqchen/gists{/gist_id}","starred_url":"https://api.github.com/users/tqchen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tqchen/subscriptions","organizations_url":"https://api.github.com/users/tqchen/orgs","repos_url":"https://api.github.com/users/tqchen/repos","events_url":"https://api.github.com/users/tqchen/events{/privacy}","received_events_url":"https://api.github.com/users/tqchen/received_events","type":"User","site_admin":false},"open_issues":3,"closed_issues":4,"state":"open","created_at":"2018-08-13T16:45:09Z","updated_at":"2018-10-18T21:56:06Z","due_on":null,"closed_at":null},"comments":19,"created_at":"2018-07-25T03:25:47Z","updated_at":"2018-10-18T21:56:06Z","closed_at":"2018-10-18T21:56:06Z","author_association":"MEMBER","body":"## Target:\r\n\r\n- Avoid verification in the original Tensorize, which we found it is not easy while detecting some patterns, like specifying abosolute/relative indexing.\r\n- Achieve better encapsulation. Vendors can build external package for their hardware, and user do not need to know the detail while writing their TVM declaration.\r\n- Propose a way to represent a block in tensor, which can be scheduled as an unit.\r\n\r\n## Motivation\r\n\r\nThe current `tensorize` method use `TensorIntrin` as a contract. While lowering, it will compare the body of the intrinsic‘s declaration and the body of the original declaration. After verifying the both are absolutely identical, the body of the original declaration will be replaced by intrinsic.\r\n\r\nHowever, we found that it is not easy to guarantee/express the identity in some situation, especially involving the value of an index. This problem is related to the implementation of `tensorize`: normally, `tensorize` will replace the index in intrinsic with the index in the original declaration, so that the lowered code can be perfectly matched. For example, if we want to tensorize an addition intrinsic:\r\n\r\n```\r\n# intrinsic declaration\r\nfor k in 0...16\r\n    A[k] = B[k] + C[k]\r\n\r\n# original declaration\r\nfor i in 0...32\r\n    for j in 0...16\r\n        A[i*16+j] = B[i*16+j] + C[i*16+j]\r\n```\r\nWe will replace `k` in intrinsic declaration with `i*16+j`, then to compare the ir whether is identical. However, if we require the value of index, things will change. The verification will fail unless we express absolute indexing and relative indexing clearly.\r\n\r\n\r\n## Design\r\n\r\nInstead of proposing a way to express absolute indexing, we would like to come up with generalizing `tvm.compute` to the region of tensor. That means we can view a tensor intrinsic as an operation between regions, just like `+` as an operation between elements. \r\n\r\n- **region**: following numpy, we use slice to express a region in tensor: `A[i, 0:16]`\r\n- **tensor_op**: different with `extern` operation, we hope that we can keep the ability to schedule the part out of tensorized region. It means there are two kinds of axes: 1. axes in tensorized region, which cannot be scheduled; 2. axes out of the tensorized region, which can be scheduled through `split`, `reorder`, etc.\r\n\r\nWith this generalization, we can use tensor intrinsic in declaration directly, so verification is no longer needed. Also, this way provides better encapsulation, user do not need to know the detail of tensor intrinsic.\r\n\r\n## Interface\r\n\r\n```python\r\ndef tensor_op(out_dims,\r\n              in_dims,\r\n              finputs,\r\n              tensor_intrin,\r\n              reduce_axis=[],\r\n              name='tensor_op',\r\n              tag=\"\"):\r\n    \"\"\"Construct new tensors with intrinsic.\r\n\r\n    Parameters\r\n    ----------\r\n    out_dims: tuple\r\n        The dimensions out of the tensorized region, which can be scheduled through `reorder`, `split`.\r\n\r\n    in_dims: tuple\r\n        The dimensions inside of the tensorized region, which cannot be manipulated.\r\n\r\n    finputs: lambda function of out_dims -> list of TensorSlice\r\n        Specifies involved regions of input tensors.\r\n\r\n    tensor_intrin : TensorIntrin\r\n        The tensor intrinsic used for computation.\r\n\r\n    reduce_axis : IterVar\r\n        An iteration variable representing the value.\r\n\r\n    name: str, optional\r\n        The name hint of the tensor\r\n\r\n    tag: str, optional\r\n        Additonal tag information about the compute.\r\n```\r\n\r\n## Demo:\r\n\r\n```python\r\n# tensor intrinsic\r\ndef intrin_vadd(n):\r\n    x = tvm.placeholder((n,), dtype=dtype, name='vx')\r\n    y = tvm.placeholder((n,), dtype=dtype, name='vy')\r\n    z = tvm.compute(x.shape, lambda i: x[i] + y[i], name='z')\r\n\r\n    def intrin_func(ins, outs):\r\n        ib = tvm.ir_builder.create()\r\n        ib.emit(tvm.call_extern(outs[0].dtype, 'vadd', ins[0].access_ptr(\"r\"), ins[1].access_ptr('r'), outs[0].access_ptr('wr'), 8, 1, 1, 1, 8, 8, 8))\r\n        return ib.get()\r\n\r\n    return tvm.decl_tensor_intrin(z.op, intrin_func)\r\n\r\n\r\n# tensorize way\r\ndef tensorize_vadd():\r\n    A = tvm.placeholder((m/factor, factor), name='A')\r\n    B = tvm.placeholder((m/factor, factor), name='B')\r\n    C = tvm.compute(A.shape, lambda *i: A(*i) + B(*i), name='C')\r\n\r\n    s = tvm.create_schedule(C.op)\r\n    xo, xi = C.op.axis\r\n    vadd = intrin_vadd(factor)\r\n    s[C].tensorize(xi, vadd)\r\n    print(tvm.lower(s, [A, B, C], simple_mode=True))\r\n\r\n\r\n# tensor_op way\r\ndef tensor_op_vadd():\r\n    A = tvm.placeholder((m/factor, factor), name=\"A\", dtype=dtype)\r\n    B = tvm.placeholder((m/factor, factor), name=\"B\", dtype=dtype)\r\n\r\n    intrin = intrin_vadd(16)\r\n    C = tvm.tensor_op([m/factor,], [factor,],\r\n                      lambda i: [A[i, 0:factor], B[i, 0:factor]], \r\n                      intrin, name='C')\r\n\r\n    s = tvm.create_schedule(C.op)\r\n    print(tvm.lower(s, [A, B, C], simple_mode=True))\r\n\r\n```\r\n\r\nlink to the PR: #1476 \r\n\r\n@tqchen @xqdan "}}
{"actor":{"display_login":"tqchen"}}