{"action":"created","comment":{"url":"https://api.github.com/repos/Theano/Theano/pulls/comments/67961830","id":67961830,"diff_hunk":"@@ -0,0 +1,236 @@\n+\n+.. _extending_theano_gpu:\n+\n+==============================\n+Extending Theano with a GPU Op\n+==============================\n+\n+.. note::\n+\n+    This covers the :ref:`gpuarray <gpuarray>` back-end for the GPU.\n+\n+This tutorial covers how to extend Theano with an op that offers a GPU\n+implementation.  It assumes you are familiar with how to write new\n+Theano ops.  If that is not the case you should probably follow the\n+:ref:`extending_theano` and :ref:`extending_theano_c` section before\n+continuing on.\n+\n+Writing a new GPU op can be done in python for some simple tasks, but\n+will usually done in C to access the complete API and avoid paying the\n+overhead of a python function call.\n+\n+Dealing With the Context\n+========================\n+\n+One of the major differences with GPU ops is that they require a\n+context (a.k.a. device) to execute.  Most of the time you can infer\n+the context to run on from your inputs.  There is a way for the user\n+to transfer things between contexts and to tag certain varaibles for\n+transfer.  It might also be the case that your inputs are not all from\n+the same context and you would have to choose which one to run on.\n+\n+In order to support all of those options and have a consistent\n+interface, :func:`theano.gpuarray.basic_ops.infer_context_name` was\n+written.  An example usage is below::\n+\n+    def make_node(self, a, b, c):\n+        ctx = infer_context_name(a, b, c)\n+        a = as_gpuarray_variable(a, ctx)\n+        b = as_gpuarray_variable(b, ctx)\n+        c = as_gpuarray_variable(c, ctx)\n+        return Apply(self, [a, b, c], [a.type()])\n+\n+In this example the Op takes three inputs, all on the GPU.  In case\n+one or more of your inputs is not supposed to be on the GPU, you\n+should not pass it to :func:`infer_context_name`.\n+\n+Also note that :func:`theano.gpuarray.basic_ops.as_gpuarray_variable`\n+takes ``context_name`` as a mandatory parameter.  This is because it's\n+not enough to know you want the value to be on the GPU, you also want\n+to know which GPU to put it on.  In almost all cases, you can pass in\n+the return value of :func:`infer_context_name` there.\n+\n+If you also need the context during runtime (for example to allocate\n+the output).  You can use the context of one of your inputs to know\n+which one to use.  Here is another example::\n+\n+    def perform(self, node, inputs, output_storage):\n+        A, B = inputs\n+        C, = output_storage\n+        C[0] = pygpu.empty([A.shape[0], B.shape[1]], dtype=A.dtype, A.context)\n+        pygpu.blas.gemm(1, A, B, 0, C, overwrite_c=True)\n+\n+Finally if you require the context before perform, such as during\n+make_thunk() to initialize kernels and such, you can access the\n+context of your inputs through the type if the variables::\n+\n+    def make_thunk(self, node, storage_map, compute_map, no_recycling):\n+        ctx = node.inputs[0].type.context\n+\n+Note that GpuArrayType objects also have a `context_name` attribute\n+which is the symbolic equivalent of `context`.  It can't be used for\n+calls to pygpu or libgpuarray, but it should be used for theano\n+operations and variables.\n+\n+The last place where you might need the context is in the C\n+initialization code.  For that you will have to use the :ref:`params\n+<extending_op_params>`.  The params type should be\n+:class:`theano.gpuarray.type.gpu_context_type` and the params object\n+should be a context object from one of your input variables::\n+\n+    def get_params(self, node):\n+        return node.inputs[0].type.context\n+\n+If you don't have any input variables on the GPU you can follow the\n+the example of :class:`theano.gpuarray.basic_ops.GpuFromHost` or\n+:class:`theano.gpuarray.basic_ops.GpuEye`.  This is not a case that\n+you should encounter often, so it will not be covered further.\n+\n+Defining New Kernels\n+====================\n+\n+If your op needs to do some transformation on the data, chances are\n+that you will need to write a new kernel.  The best way to do this is\n+to leverage GpuKernelBase (or CGpuKernelBase if you want to use the\n+COp functionality).\n+\n+For plain GpuKernelBase, you have to define a method called\n+gpu_kernels which returns a list of :class:`Kernel\n+<theano.gpuarray.basic_ops.Kernel>` objects.  You can define as many\n+kernels as you want for a single op.  An example would look like this::\n+\n+    def gpu_kernels(self, node, name):\n+        code = \"\"\"\n+    KERNEL void k(GLOBAL_MEM ga_double *a, ga_size n, ga_size m) {\n+        ga_size nb = n < m ? n : m;\n+        for (ga_size i = LID_0; i < nb; i += LDIM_0) {\n+            a[i*m + i] = 1;\n+        }\n+    }\"\"\"\n+        return [Kernel(\n+                code=code, name=\"k\",\n+                params=[gpuarray.GpuArray, gpuarray.SIZE, gpuarray.SIZE],\n+                flags=Kernel.get_flags('float64'))]\n+\n+If you want to use COp, then you should use ``CGpuKernelBase``\n+instead.  It adds a new section to the parsed files whose tag is\n+``kernels``.  Inside that section you can define some kernels with\n+``#kernel name:params:flags``.\n+\n+Here ``name`` is the name of the kernel function in the following\n+code, ``params`` is a comma-separeted list of numpy typecode names.\n+There are three exceptions for ``size_t`` which should be noted as\n+``size``, ``ssize_t`` which should be noted as ``ssize`` and a pointer\n+which should be noted as ``*``.\n+\n+``flags`` is a ``|``-separeted list of C kernel flag values (can be\n+empty).  The same kernel definition as above would look like this with\n+``CGpuKernelBase``::\n+\n+    #section kernels\n+\n+    #kernel k : *, size, size : GA_USE_DOUBLE\n+\n+    KERNEL void k(GLOBAL_MEM ga_double *a, ga_size n, ga_size m) {\n+        ga_size nb = n < m ? n : m;\n+        for (ga_size i = LID_0; i < nb; i += LDIM_0) {\n+        a[i*m + i] = 1;\n+        }\n+    }\n+\n+The second method is to handle the kernel compilation and cache on\n+your own.  This is not recommended because there are lots of details\n+to pay attention to that can cripple your performance if not done\n+right, which GpuKernelBase handles for you.  But if you really want to\n+go this way, then you can look up the C API for kernels in\n+libgpuarray.\n+\n+In any case you will need to call your compiled kernel with some data.","path":"doc/extending/extending_theano_gpu.txt","position":148,"original_position":148,"commit_id":"74138d60bbc8c6360d51d9cd5e64fceb9be9e4a4","original_commit_id":"74138d60bbc8c6360d51d9cd5e64fceb9be9e4a4","user":{"login":"lamblin","id":178249,"avatar_url":"https://avatars.githubusercontent.com/u/178249?v=3","gravatar_id":"","url":"https://api.github.com/users/lamblin","html_url":"https://github.com/lamblin","followers_url":"https://api.github.com/users/lamblin/followers","following_url":"https://api.github.com/users/lamblin/following{/other_user}","gists_url":"https://api.github.com/users/lamblin/gists{/gist_id}","starred_url":"https://api.github.com/users/lamblin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lamblin/subscriptions","organizations_url":"https://api.github.com/users/lamblin/orgs","repos_url":"https://api.github.com/users/lamblin/repos","events_url":"https://api.github.com/users/lamblin/events{/privacy}","received_events_url":"https://api.github.com/users/lamblin/received_events","type":"User","site_admin":false},"body":"Maybe mention that we usually want to do that in the `c_code` method of the Op.","created_at":"2016-06-21T22:11:02Z","updated_at":"2016-06-21T22:11:02Z","html_url":"https://github.com/Theano/Theano/pull/4584#discussion_r67961830","pull_request_url":"https://api.github.com/repos/Theano/Theano/pulls/4584","_links":{"self":{"href":"https://api.github.com/repos/Theano/Theano/pulls/comments/67961830"},"html":{"href":"https://github.com/Theano/Theano/pull/4584#discussion_r67961830"},"pull_request":{"href":"https://api.github.com/repos/Theano/Theano/pulls/4584"}}},"pull_request":{"url":"https://api.github.com/repos/Theano/Theano/pulls/4584","id":72560452,"html_url":"https://github.com/Theano/Theano/pull/4584","diff_url":"https://github.com/Theano/Theano/pull/4584.diff","patch_url":"https://github.com/Theano/Theano/pull/4584.patch","issue_url":"https://api.github.com/repos/Theano/Theano/issues/4584","number":4584,"state":"open","locked":false,"title":"Add some documentation on how to write gpu ops.","user":{"login":"abergeron","id":1002504,"avatar_url":"https://avatars.githubusercontent.com/u/1002504?v=3","gravatar_id":"","url":"https://api.github.com/users/abergeron","html_url":"https://github.com/abergeron","followers_url":"https://api.github.com/users/abergeron/followers","following_url":"https://api.github.com/users/abergeron/following{/other_user}","gists_url":"https://api.github.com/users/abergeron/gists{/gist_id}","starred_url":"https://api.github.com/users/abergeron/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/abergeron/subscriptions","organizations_url":"https://api.github.com/users/abergeron/orgs","repos_url":"https://api.github.com/users/abergeron/repos","events_url":"https://api.github.com/users/abergeron/events{/privacy}","received_events_url":"https://api.github.com/users/abergeron/received_events","type":"User","site_admin":false},"body":"This explains some of the specifics about making gpu ops and also introduces a class that merges COp and GpuKernelBase if you want to use both.\r\n\r\nI've explained all that I could think could be a problem when building new GPU ops, but if you think of something else, add a comment and I'll see what I can do about it.","created_at":"2016-06-03T19:49:32Z","updated_at":"2016-06-21T22:11:02Z","closed_at":null,"merged_at":null,"merge_commit_sha":"4921ae45b1fcb1da58e4c48a2878de47e0e25555","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/Theano/Theano/pulls/4584/commits","review_comments_url":"https://api.github.com/repos/Theano/Theano/pulls/4584/comments","review_comment_url":"https://api.github.com/repos/Theano/Theano/pulls/comments{/number}","comments_url":"https://api.github.com/repos/Theano/Theano/issues/4584/comments","statuses_url":"https://api.github.com/repos/Theano/Theano/statuses/74138d60bbc8c6360d51d9cd5e64fceb9be9e4a4","head":{"label":"abergeron:gpua_doc","ref":"gpua_doc","sha":"74138d60bbc8c6360d51d9cd5e64fceb9be9e4a4","user":{"login":"abergeron","id":1002504,"avatar_url":"https://avatars.githubusercontent.com/u/1002504?v=3","gravatar_id":"","url":"https://api.github.com/users/abergeron","html_url":"https://github.com/abergeron","followers_url":"https://api.github.com/users/abergeron/followers","following_url":"https://api.github.com/users/abergeron/following{/other_user}","gists_url":"https://api.github.com/users/abergeron/gists{/gist_id}","starred_url":"https://api.github.com/users/abergeron/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/abergeron/subscriptions","organizations_url":"https://api.github.com/users/abergeron/orgs","repos_url":"https://api.github.com/users/abergeron/repos","events_url":"https://api.github.com/users/abergeron/events{/privacy}","received_events_url":"https://api.github.com/users/abergeron/received_events","type":"User","site_admin":false},"repo":{"id":2264121,"name":"Theano","full_name":"abergeron/Theano","owner":{"login":"abergeron","id":1002504,"avatar_url":"https://avatars.githubusercontent.com/u/1002504?v=3","gravatar_id":"","url":"https://api.github.com/users/abergeron","html_url":"https://github.com/abergeron","followers_url":"https://api.github.com/users/abergeron/followers","following_url":"https://api.github.com/users/abergeron/following{/other_user}","gists_url":"https://api.github.com/users/abergeron/gists{/gist_id}","starred_url":"https://api.github.com/users/abergeron/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/abergeron/subscriptions","organizations_url":"https://api.github.com/users/abergeron/orgs","repos_url":"https://api.github.com/users/abergeron/repos","events_url":"https://api.github.com/users/abergeron/events{/privacy}","received_events_url":"https://api.github.com/users/abergeron/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/abergeron/Theano","description":"Optimizing GPU-meta-programming code generating array oriented optimizing math compiler in Python","fork":true,"url":"https://api.github.com/repos/abergeron/Theano","forks_url":"https://api.github.com/repos/abergeron/Theano/forks","keys_url":"https://api.github.com/repos/abergeron/Theano/keys{/key_id}","collaborators_url":"https://api.github.com/repos/abergeron/Theano/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/abergeron/Theano/teams","hooks_url":"https://api.github.com/repos/abergeron/Theano/hooks","issue_events_url":"https://api.github.com/repos/abergeron/Theano/issues/events{/number}","events_url":"https://api.github.com/repos/abergeron/Theano/events","assignees_url":"https://api.github.com/repos/abergeron/Theano/assignees{/user}","branches_url":"https://api.github.com/repos/abergeron/Theano/branches{/branch}","tags_url":"https://api.github.com/repos/abergeron/Theano/tags","blobs_url":"https://api.github.com/repos/abergeron/Theano/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/abergeron/Theano/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/abergeron/Theano/git/refs{/sha}","trees_url":"https://api.github.com/repos/abergeron/Theano/git/trees{/sha}","statuses_url":"https://api.github.com/repos/abergeron/Theano/statuses/{sha}","languages_url":"https://api.github.com/repos/abergeron/Theano/languages","stargazers_url":"https://api.github.com/repos/abergeron/Theano/stargazers","contributors_url":"https://api.github.com/repos/abergeron/Theano/contributors","subscribers_url":"https://api.github.com/repos/abergeron/Theano/subscribers","subscription_url":"https://api.github.com/repos/abergeron/Theano/subscription","commits_url":"https://api.github.com/repos/abergeron/Theano/commits{/sha}","git_commits_url":"https://api.github.com/repos/abergeron/Theano/git/commits{/sha}","comments_url":"https://api.github.com/repos/abergeron/Theano/comments{/number}","issue_comment_url":"https://api.github.com/repos/abergeron/Theano/issues/comments{/number}","contents_url":"https://api.github.com/repos/abergeron/Theano/contents/{+path}","compare_url":"https://api.github.com/repos/abergeron/Theano/compare/{base}...{head}","merges_url":"https://api.github.com/repos/abergeron/Theano/merges","archive_url":"https://api.github.com/repos/abergeron/Theano/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/abergeron/Theano/downloads","issues_url":"https://api.github.com/repos/abergeron/Theano/issues{/number}","pulls_url":"https://api.github.com/repos/abergeron/Theano/pulls{/number}","milestones_url":"https://api.github.com/repos/abergeron/Theano/milestones{/number}","notifications_url":"https://api.github.com/repos/abergeron/Theano/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/abergeron/Theano/labels{/name}","releases_url":"https://api.github.com/repos/abergeron/Theano/releases{/id}","deployments_url":"https://api.github.com/repos/abergeron/Theano/deployments","created_at":"2011-08-24T20:28:59Z","updated_at":"2015-05-08T20:34:01Z","pushed_at":"2016-06-21T20:13:57Z","git_url":"git://github.com/abergeron/Theano.git","ssh_url":"git@github.com:abergeron/Theano.git","clone_url":"https://github.com/abergeron/Theano.git","svn_url":"https://github.com/abergeron/Theano","homepage":"http://www.deeplearning.net/software/theano","size":54190,"stargazers_count":1,"watchers_count":1,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":0,"forks":1,"open_issues":0,"watchers":1,"default_branch":"master"}},"base":{"label":"Theano:master","ref":"master","sha":"e23c1310e0a649735d0faeadf8d586b2e3590239","user":{"login":"Theano","id":970517,"avatar_url":"https://avatars.githubusercontent.com/u/970517?v=3","gravatar_id":"","url":"https://api.github.com/users/Theano","html_url":"https://github.com/Theano","followers_url":"https://api.github.com/users/Theano/followers","following_url":"https://api.github.com/users/Theano/following{/other_user}","gists_url":"https://api.github.com/users/Theano/gists{/gist_id}","starred_url":"https://api.github.com/users/Theano/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Theano/subscriptions","organizations_url":"https://api.github.com/users/Theano/orgs","repos_url":"https://api.github.com/users/Theano/repos","events_url":"https://api.github.com/users/Theano/events{/privacy}","received_events_url":"https://api.github.com/users/Theano/received_events","type":"Organization","site_admin":false},"repo":{"id":2183193,"name":"Theano","full_name":"Theano/Theano","owner":{"login":"Theano","id":970517,"avatar_url":"https://avatars.githubusercontent.com/u/970517?v=3","gravatar_id":"","url":"https://api.github.com/users/Theano","html_url":"https://github.com/Theano","followers_url":"https://api.github.com/users/Theano/followers","following_url":"https://api.github.com/users/Theano/following{/other_user}","gists_url":"https://api.github.com/users/Theano/gists{/gist_id}","starred_url":"https://api.github.com/users/Theano/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Theano/subscriptions","organizations_url":"https://api.github.com/users/Theano/orgs","repos_url":"https://api.github.com/users/Theano/repos","events_url":"https://api.github.com/users/Theano/events{/privacy}","received_events_url":"https://api.github.com/users/Theano/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/Theano/Theano","description":"Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation.","fork":false,"url":"https://api.github.com/repos/Theano/Theano","forks_url":"https://api.github.com/repos/Theano/Theano/forks","keys_url":"https://api.github.com/repos/Theano/Theano/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Theano/Theano/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Theano/Theano/teams","hooks_url":"https://api.github.com/repos/Theano/Theano/hooks","issue_events_url":"https://api.github.com/repos/Theano/Theano/issues/events{/number}","events_url":"https://api.github.com/repos/Theano/Theano/events","assignees_url":"https://api.github.com/repos/Theano/Theano/assignees{/user}","branches_url":"https://api.github.com/repos/Theano/Theano/branches{/branch}","tags_url":"https://api.github.com/repos/Theano/Theano/tags","blobs_url":"https://api.github.com/repos/Theano/Theano/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Theano/Theano/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Theano/Theano/git/refs{/sha}","trees_url":"https://api.github.com/repos/Theano/Theano/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Theano/Theano/statuses/{sha}","languages_url":"https://api.github.com/repos/Theano/Theano/languages","stargazers_url":"https://api.github.com/repos/Theano/Theano/stargazers","contributors_url":"https://api.github.com/repos/Theano/Theano/contributors","subscribers_url":"https://api.github.com/repos/Theano/Theano/subscribers","subscription_url":"https://api.github.com/repos/Theano/Theano/subscription","commits_url":"https://api.github.com/repos/Theano/Theano/commits{/sha}","git_commits_url":"https://api.github.com/repos/Theano/Theano/git/commits{/sha}","comments_url":"https://api.github.com/repos/Theano/Theano/comments{/number}","issue_comment_url":"https://api.github.com/repos/Theano/Theano/issues/comments{/number}","contents_url":"https://api.github.com/repos/Theano/Theano/contents/{+path}","compare_url":"https://api.github.com/repos/Theano/Theano/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Theano/Theano/merges","archive_url":"https://api.github.com/repos/Theano/Theano/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Theano/Theano/downloads","issues_url":"https://api.github.com/repos/Theano/Theano/issues{/number}","pulls_url":"https://api.github.com/repos/Theano/Theano/pulls{/number}","milestones_url":"https://api.github.com/repos/Theano/Theano/milestones{/number}","notifications_url":"https://api.github.com/repos/Theano/Theano/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Theano/Theano/labels{/name}","releases_url":"https://api.github.com/repos/Theano/Theano/releases{/id}","deployments_url":"https://api.github.com/repos/Theano/Theano/deployments","created_at":"2011-08-10T03:48:06Z","updated_at":"2016-06-21T17:56:10Z","pushed_at":"2016-06-21T21:59:44Z","git_url":"git://github.com/Theano/Theano.git","ssh_url":"git@github.com:Theano/Theano.git","clone_url":"https://github.com/Theano/Theano.git","svn_url":"https://github.com/Theano/Theano","homepage":"http://www.deeplearning.net/software/theano","size":74704,"stargazers_count":3978,"watchers_count":3978,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1437,"mirror_url":null,"open_issues_count":686,"forks":1437,"open_issues":686,"watchers":3978,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/Theano/Theano/pulls/4584"},"html":{"href":"https://github.com/Theano/Theano/pull/4584"},"issue":{"href":"https://api.github.com/repos/Theano/Theano/issues/4584"},"comments":{"href":"https://api.github.com/repos/Theano/Theano/issues/4584/comments"},"review_comments":{"href":"https://api.github.com/repos/Theano/Theano/pulls/4584/comments"},"review_comment":{"href":"https://api.github.com/repos/Theano/Theano/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/Theano/Theano/pulls/4584/commits"},"statuses":{"href":"https://api.github.com/repos/Theano/Theano/statuses/74138d60bbc8c6360d51d9cd5e64fceb9be9e4a4"}}}}
{"actor":{"display_login":"lamblin"}}