{"action":"created","comment":{"url":"https://api.github.com/repos/gpudirect/libgdsync/pulls/comments/233745591","pull_request_review_id":175227055,"id":233745591,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIzMzc0NTU5MQ==","diff_hunk":"@@ -1871,7 +1875,32 @@ gds_create_cq(struct ibv_context *context, int cqe,\n \n //-----------------------------------------------------------------------------\n \n-struct gds_qp *gds_create_qp(struct ibv_pd *pd, struct ibv_context *context,\n+\n+int gds_add_dv_qp_ctx(struct gds_qp* gqp){\n+        struct mlx5dv_obj dv_obj = {};\n+        struct mlx5dv_qp* dv_qp = (struct mlx5dv_qp *)malloc(sizeof(struct mlx5dv_qp));\n+        memset((void *)&dv_obj, 0, sizeof(struct mlx5dv_obj));\n+\n+        dv_obj.qp.in = gqp->qp;\n+        dv_obj.qp.out = dv_qp;\n+        int ret = mlx5dv_init_obj(&dv_obj, MLX5DV_OBJ_QP);\n+\n+        if (ret)\n+          return ret;","path":"src/gdsync.cpp","position":27,"original_position":27,"commit_id":"4f47911742f20abd7395c0e0ee5d13a7af6899d6","original_commit_id":"58707396292ec51a8a188338326e188994c98d23","user":{"login":"haggaie","id":35070,"node_id":"MDQ6VXNlcjM1MDcw","avatar_url":"https://avatars3.githubusercontent.com/u/35070?v=4","gravatar_id":"","url":"https://api.github.com/users/haggaie","html_url":"https://github.com/haggaie","followers_url":"https://api.github.com/users/haggaie/followers","following_url":"https://api.github.com/users/haggaie/following{/other_user}","gists_url":"https://api.github.com/users/haggaie/gists{/gist_id}","starred_url":"https://api.github.com/users/haggaie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/haggaie/subscriptions","organizations_url":"https://api.github.com/users/haggaie/orgs","repos_url":"https://api.github.com/users/haggaie/repos","events_url":"https://api.github.com/users/haggaie/events{/privacy}","received_events_url":"https://api.github.com/users/haggaie/received_events","type":"User","site_admin":false},"body":"this leaks the dv_qp memory allocation.","created_at":"2018-11-15T08:18:13Z","updated_at":"2018-11-15T08:19:10Z","html_url":"https://github.com/gpudirect/libgdsync/pull/75#discussion_r233745591","pull_request_url":"https://api.github.com/repos/gpudirect/libgdsync/pulls/75","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/gpudirect/libgdsync/pulls/comments/233745591"},"html":{"href":"https://github.com/gpudirect/libgdsync/pull/75#discussion_r233745591"},"pull_request":{"href":"https://api.github.com/repos/gpudirect/libgdsync/pulls/75"}}},"pull_request":{"url":"https://api.github.com/repos/gpudirect/libgdsync/pulls/75","id":230488167,"node_id":"MDExOlB1bGxSZXF1ZXN0MjMwNDg4MTY3","html_url":"https://github.com/gpudirect/libgdsync/pull/75","diff_url":"https://github.com/gpudirect/libgdsync/pull/75.diff","patch_url":"https://github.com/gpudirect/libgdsync/pull/75.patch","issue_url":"https://api.github.com/repos/gpudirect/libgdsync/issues/75","number":75,"state":"open","locked":false,"title":"RFC - Expose send params driver bypass","user":{"login":"yanivbl6","id":24679884,"node_id":"MDQ6VXNlcjI0Njc5ODg0","avatar_url":"https://avatars2.githubusercontent.com/u/24679884?v=4","gravatar_id":"","url":"https://api.github.com/users/yanivbl6","html_url":"https://github.com/yanivbl6","followers_url":"https://api.github.com/users/yanivbl6/followers","following_url":"https://api.github.com/users/yanivbl6/following{/other_user}","gists_url":"https://api.github.com/users/yanivbl6/gists{/gist_id}","starred_url":"https://api.github.com/users/yanivbl6/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yanivbl6/subscriptions","organizations_url":"https://api.github.com/users/yanivbl6/orgs","repos_url":"https://api.github.com/users/yanivbl6/repos","events_url":"https://api.github.com/users/yanivbl6/events{/privacy}","received_events_url":"https://api.github.com/users/yanivbl6/received_events","type":"User","site_admin":false},"body":"This code should be compared with the upstream/expose_send_params branch\r\n\r\nThe goal of those commits is to allow the changes in the \"Expose send params\" branch to work without the required changes to the mlx5 driver. This is done by using direct verbs to extract driver info.\r\n\r\nThe method to bypass the driver is:\r\n1. Use Direct verbs to create the mlx5dv_qp object after qp creation, from which a pointer to the send queue and the send_queue size can be extracted.\r\n2. extend the gds qp context to hold additional fields: the send queue pointer and size extracted in the procedure above, and a counter that tracks the wqes being used in the send queue.\r\n3. The counter is to updated in each call to gds_post_send, by calling gds_report_post.\r\n4. gds_query_last_info() is implemented using the counter, pointer and size of send queue kept in the gds qp context. Only the last work-request can be probed. \r\n5. number of sges is determined by probing the opcode and ds from the last wqe.Number of BB consumed in the send queue can be checked in the same way but currently is fixed to 1.\r\n\r\n\r\nI haven't yet managed to run all the tests in gda successfully (with or without this patch), but I was able to run:\r\nRunning gds_kernel_latency, peersync, descriptors, RC\r\nRunning gds_kernel_latency, peersync, descriptors, GMEM buffers, RC\r\n\r\nwithout errors/hangs, after changing the hard coded number of batches in the test to 1.\r\n\r\n@haggaie @bureddy @drossetti @e-ago ","created_at":"2018-11-13T14:09:12Z","updated_at":"2018-11-15T08:19:10Z","closed_at":null,"merged_at":null,"merge_commit_sha":"a4b0cc0b94b5cddd839d45eb977218768e07775c","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/gpudirect/libgdsync/pulls/75/commits","review_comments_url":"https://api.github.com/repos/gpudirect/libgdsync/pulls/75/comments","review_comment_url":"https://api.github.com/repos/gpudirect/libgdsync/pulls/comments{/number}","comments_url":"https://api.github.com/repos/gpudirect/libgdsync/issues/75/comments","statuses_url":"https://api.github.com/repos/gpudirect/libgdsync/statuses/4f47911742f20abd7395c0e0ee5d13a7af6899d6","head":{"label":"yanivbl6:expose_wqe_params_driver_bypass","ref":"expose_wqe_params_driver_bypass","sha":"4f47911742f20abd7395c0e0ee5d13a7af6899d6","user":{"login":"yanivbl6","id":24679884,"node_id":"MDQ6VXNlcjI0Njc5ODg0","avatar_url":"https://avatars2.githubusercontent.com/u/24679884?v=4","gravatar_id":"","url":"https://api.github.com/users/yanivbl6","html_url":"https://github.com/yanivbl6","followers_url":"https://api.github.com/users/yanivbl6/followers","following_url":"https://api.github.com/users/yanivbl6/following{/other_user}","gists_url":"https://api.github.com/users/yanivbl6/gists{/gist_id}","starred_url":"https://api.github.com/users/yanivbl6/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yanivbl6/subscriptions","organizations_url":"https://api.github.com/users/yanivbl6/orgs","repos_url":"https://api.github.com/users/yanivbl6/repos","events_url":"https://api.github.com/users/yanivbl6/events{/privacy}","received_events_url":"https://api.github.com/users/yanivbl6/received_events","type":"User","site_admin":false},"repo":{"id":156671395,"node_id":"MDEwOlJlcG9zaXRvcnkxNTY2NzEzOTU=","name":"libgdsync","full_name":"yanivbl6/libgdsync","private":false,"owner":{"login":"yanivbl6","id":24679884,"node_id":"MDQ6VXNlcjI0Njc5ODg0","avatar_url":"https://avatars2.githubusercontent.com/u/24679884?v=4","gravatar_id":"","url":"https://api.github.com/users/yanivbl6","html_url":"https://github.com/yanivbl6","followers_url":"https://api.github.com/users/yanivbl6/followers","following_url":"https://api.github.com/users/yanivbl6/following{/other_user}","gists_url":"https://api.github.com/users/yanivbl6/gists{/gist_id}","starred_url":"https://api.github.com/users/yanivbl6/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yanivbl6/subscriptions","organizations_url":"https://api.github.com/users/yanivbl6/orgs","repos_url":"https://api.github.com/users/yanivbl6/repos","events_url":"https://api.github.com/users/yanivbl6/events{/privacy}","received_events_url":"https://api.github.com/users/yanivbl6/received_events","type":"User","site_admin":false},"html_url":"https://github.com/yanivbl6/libgdsync","description":"GPUDirect Async support for IB Verbs","fork":true,"url":"https://api.github.com/repos/yanivbl6/libgdsync","forks_url":"https://api.github.com/repos/yanivbl6/libgdsync/forks","keys_url":"https://api.github.com/repos/yanivbl6/libgdsync/keys{/key_id}","collaborators_url":"https://api.github.com/repos/yanivbl6/libgdsync/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/yanivbl6/libgdsync/teams","hooks_url":"https://api.github.com/repos/yanivbl6/libgdsync/hooks","issue_events_url":"https://api.github.com/repos/yanivbl6/libgdsync/issues/events{/number}","events_url":"https://api.github.com/repos/yanivbl6/libgdsync/events","assignees_url":"https://api.github.com/repos/yanivbl6/libgdsync/assignees{/user}","branches_url":"https://api.github.com/repos/yanivbl6/libgdsync/branches{/branch}","tags_url":"https://api.github.com/repos/yanivbl6/libgdsync/tags","blobs_url":"https://api.github.com/repos/yanivbl6/libgdsync/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/yanivbl6/libgdsync/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/yanivbl6/libgdsync/git/refs{/sha}","trees_url":"https://api.github.com/repos/yanivbl6/libgdsync/git/trees{/sha}","statuses_url":"https://api.github.com/repos/yanivbl6/libgdsync/statuses/{sha}","languages_url":"https://api.github.com/repos/yanivbl6/libgdsync/languages","stargazers_url":"https://api.github.com/repos/yanivbl6/libgdsync/stargazers","contributors_url":"https://api.github.com/repos/yanivbl6/libgdsync/contributors","subscribers_url":"https://api.github.com/repos/yanivbl6/libgdsync/subscribers","subscription_url":"https://api.github.com/repos/yanivbl6/libgdsync/subscription","commits_url":"https://api.github.com/repos/yanivbl6/libgdsync/commits{/sha}","git_commits_url":"https://api.github.com/repos/yanivbl6/libgdsync/git/commits{/sha}","comments_url":"https://api.github.com/repos/yanivbl6/libgdsync/comments{/number}","issue_comment_url":"https://api.github.com/repos/yanivbl6/libgdsync/issues/comments{/number}","contents_url":"https://api.github.com/repos/yanivbl6/libgdsync/contents/{+path}","compare_url":"https://api.github.com/repos/yanivbl6/libgdsync/compare/{base}...{head}","merges_url":"https://api.github.com/repos/yanivbl6/libgdsync/merges","archive_url":"https://api.github.com/repos/yanivbl6/libgdsync/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/yanivbl6/libgdsync/downloads","issues_url":"https://api.github.com/repos/yanivbl6/libgdsync/issues{/number}","pulls_url":"https://api.github.com/repos/yanivbl6/libgdsync/pulls{/number}","milestones_url":"https://api.github.com/repos/yanivbl6/libgdsync/milestones{/number}","notifications_url":"https://api.github.com/repos/yanivbl6/libgdsync/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/yanivbl6/libgdsync/labels{/name}","releases_url":"https://api.github.com/repos/yanivbl6/libgdsync/releases{/id}","deployments_url":"https://api.github.com/repos/yanivbl6/libgdsync/deployments","created_at":"2018-11-08T08:03:49Z","updated_at":"2018-11-08T08:03:51Z","pushed_at":"2018-11-13T13:48:42Z","git_url":"git://github.com/yanivbl6/libgdsync.git","ssh_url":"git@github.com:yanivbl6/libgdsync.git","clone_url":"https://github.com/yanivbl6/libgdsync.git","svn_url":"https://github.com/yanivbl6/libgdsync","homepage":null,"size":506,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":null,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"gpudirect:expose_send_params","ref":"expose_send_params","sha":"8c300c5e3e682072cb2cc82c05b2c7f5644d822d","user":{"login":"gpudirect","id":12426539,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEyNDI2NTM5","avatar_url":"https://avatars1.githubusercontent.com/u/12426539?v=4","gravatar_id":"","url":"https://api.github.com/users/gpudirect","html_url":"https://github.com/gpudirect","followers_url":"https://api.github.com/users/gpudirect/followers","following_url":"https://api.github.com/users/gpudirect/following{/other_user}","gists_url":"https://api.github.com/users/gpudirect/gists{/gist_id}","starred_url":"https://api.github.com/users/gpudirect/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gpudirect/subscriptions","organizations_url":"https://api.github.com/users/gpudirect/orgs","repos_url":"https://api.github.com/users/gpudirect/repos","events_url":"https://api.github.com/users/gpudirect/events{/privacy}","received_events_url":"https://api.github.com/users/gpudirect/received_events","type":"Organization","site_admin":false},"repo":{"id":70008475,"node_id":"MDEwOlJlcG9zaXRvcnk3MDAwODQ3NQ==","name":"libgdsync","full_name":"gpudirect/libgdsync","private":false,"owner":{"login":"gpudirect","id":12426539,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEyNDI2NTM5","avatar_url":"https://avatars1.githubusercontent.com/u/12426539?v=4","gravatar_id":"","url":"https://api.github.com/users/gpudirect","html_url":"https://github.com/gpudirect","followers_url":"https://api.github.com/users/gpudirect/followers","following_url":"https://api.github.com/users/gpudirect/following{/other_user}","gists_url":"https://api.github.com/users/gpudirect/gists{/gist_id}","starred_url":"https://api.github.com/users/gpudirect/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gpudirect/subscriptions","organizations_url":"https://api.github.com/users/gpudirect/orgs","repos_url":"https://api.github.com/users/gpudirect/repos","events_url":"https://api.github.com/users/gpudirect/events{/privacy}","received_events_url":"https://api.github.com/users/gpudirect/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/gpudirect/libgdsync","description":"GPUDirect Async support for IB Verbs","fork":false,"url":"https://api.github.com/repos/gpudirect/libgdsync","forks_url":"https://api.github.com/repos/gpudirect/libgdsync/forks","keys_url":"https://api.github.com/repos/gpudirect/libgdsync/keys{/key_id}","collaborators_url":"https://api.github.com/repos/gpudirect/libgdsync/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/gpudirect/libgdsync/teams","hooks_url":"https://api.github.com/repos/gpudirect/libgdsync/hooks","issue_events_url":"https://api.github.com/repos/gpudirect/libgdsync/issues/events{/number}","events_url":"https://api.github.com/repos/gpudirect/libgdsync/events","assignees_url":"https://api.github.com/repos/gpudirect/libgdsync/assignees{/user}","branches_url":"https://api.github.com/repos/gpudirect/libgdsync/branches{/branch}","tags_url":"https://api.github.com/repos/gpudirect/libgdsync/tags","blobs_url":"https://api.github.com/repos/gpudirect/libgdsync/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/gpudirect/libgdsync/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/gpudirect/libgdsync/git/refs{/sha}","trees_url":"https://api.github.com/repos/gpudirect/libgdsync/git/trees{/sha}","statuses_url":"https://api.github.com/repos/gpudirect/libgdsync/statuses/{sha}","languages_url":"https://api.github.com/repos/gpudirect/libgdsync/languages","stargazers_url":"https://api.github.com/repos/gpudirect/libgdsync/stargazers","contributors_url":"https://api.github.com/repos/gpudirect/libgdsync/contributors","subscribers_url":"https://api.github.com/repos/gpudirect/libgdsync/subscribers","subscription_url":"https://api.github.com/repos/gpudirect/libgdsync/subscription","commits_url":"https://api.github.com/repos/gpudirect/libgdsync/commits{/sha}","git_commits_url":"https://api.github.com/repos/gpudirect/libgdsync/git/commits{/sha}","comments_url":"https://api.github.com/repos/gpudirect/libgdsync/comments{/number}","issue_comment_url":"https://api.github.com/repos/gpudirect/libgdsync/issues/comments{/number}","contents_url":"https://api.github.com/repos/gpudirect/libgdsync/contents/{+path}","compare_url":"https://api.github.com/repos/gpudirect/libgdsync/compare/{base}...{head}","merges_url":"https://api.github.com/repos/gpudirect/libgdsync/merges","archive_url":"https://api.github.com/repos/gpudirect/libgdsync/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/gpudirect/libgdsync/downloads","issues_url":"https://api.github.com/repos/gpudirect/libgdsync/issues{/number}","pulls_url":"https://api.github.com/repos/gpudirect/libgdsync/pulls{/number}","milestones_url":"https://api.github.com/repos/gpudirect/libgdsync/milestones{/number}","notifications_url":"https://api.github.com/repos/gpudirect/libgdsync/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/gpudirect/libgdsync/labels{/name}","releases_url":"https://api.github.com/repos/gpudirect/libgdsync/releases{/id}","deployments_url":"https://api.github.com/repos/gpudirect/libgdsync/deployments","created_at":"2016-10-04T21:37:15Z","updated_at":"2018-11-09T04:48:48Z","pushed_at":"2018-11-13T14:09:13Z","git_url":"git://github.com/gpudirect/libgdsync.git","ssh_url":"git@github.com:gpudirect/libgdsync.git","clone_url":"https://github.com/gpudirect/libgdsync.git","svn_url":"https://github.com/gpudirect/libgdsync","homepage":null,"size":480,"stargazers_count":26,"watchers_count":26,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":9,"mirror_url":null,"archived":false,"open_issues_count":23,"license":null,"forks":9,"open_issues":23,"watchers":26,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/gpudirect/libgdsync/pulls/75"},"html":{"href":"https://github.com/gpudirect/libgdsync/pull/75"},"issue":{"href":"https://api.github.com/repos/gpudirect/libgdsync/issues/75"},"comments":{"href":"https://api.github.com/repos/gpudirect/libgdsync/issues/75/comments"},"review_comments":{"href":"https://api.github.com/repos/gpudirect/libgdsync/pulls/75/comments"},"review_comment":{"href":"https://api.github.com/repos/gpudirect/libgdsync/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/gpudirect/libgdsync/pulls/75/commits"},"statuses":{"href":"https://api.github.com/repos/gpudirect/libgdsync/statuses/4f47911742f20abd7395c0e0ee5d13a7af6899d6"}},"author_association":"NONE"}}
{"actor":{"display_login":"haggaie"}}