{"action":"created","comment":{"url":"https://api.github.com/repos/apache/flink/pulls/comments/197450435","pull_request_review_id":131193578,"id":197450435,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDE5NzQ1MDQzNQ==","diff_hunk":"@@ -54,18 +69,89 @@ public JobSubmitHandler(\n \n \t@Override\n \tprotected CompletableFuture<JobSubmitResponseBody> handleRequest(@Nonnull HandlerRequest<JobSubmitRequestBody, EmptyMessageParameters> request, @Nonnull DispatcherGateway gateway) throws RestHandlerException {\n-\t\tJobGraph jobGraph;\n-\t\ttry {\n-\t\t\tObjectInputStream objectIn = new ObjectInputStream(new ByteArrayInputStream(request.getRequestBody().serializedJobGraph));\n-\t\t\tjobGraph = (JobGraph) objectIn.readObject();\n-\t\t} catch (Exception e) {\n-\t\t\tthrow new RestHandlerException(\n-\t\t\t\t\"Failed to deserialize JobGraph.\",\n-\t\t\t\tHttpResponseStatus.BAD_REQUEST,\n-\t\t\t\te);\n+\t\tCollection<Path> uploadedFiles = request.getUploadedFiles();\n+\t\tMap<String, Path> nameToFile = uploadedFiles.stream().collect(Collectors.toMap(\n+\t\t\tpath -> path.getFileName().toString(),\n+\t\t\tentry -> entry\n+\t\t));\n+\n+\t\tJobSubmitRequestBody requestBody = request.getRequestBody();\n+\n+\t\tPath jobGraphFile = getPathAndAssertUpload(requestBody.jobGraphFileName, \"JobGraph\", nameToFile);\n+\n+\t\tCollection<org.apache.flink.core.fs.Path> jarFiles = new ArrayList<>(requestBody.jarFileNames.size());\n+\t\tfor (String jarFileName : requestBody.jarFileNames) {\n+\t\t\tPath jarFile = getPathAndAssertUpload(jarFileName, \"Jar\", nameToFile);","path":"flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/job/JobSubmitHandler.java","position":61,"original_position":62,"commit_id":"f47e899bf5f26b346cc7b3e637dd57722fcf2891","original_commit_id":"78b456eb861fd722a100ae6e153f1c0079649d8b","user":{"login":"tillrohrmann","id":5756858,"node_id":"MDQ6VXNlcjU3NTY4NTg=","avatar_url":"https://avatars1.githubusercontent.com/u/5756858?v=4","gravatar_id":"","url":"https://api.github.com/users/tillrohrmann","html_url":"https://github.com/tillrohrmann","followers_url":"https://api.github.com/users/tillrohrmann/followers","following_url":"https://api.github.com/users/tillrohrmann/following{/other_user}","gists_url":"https://api.github.com/users/tillrohrmann/gists{/gist_id}","starred_url":"https://api.github.com/users/tillrohrmann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tillrohrmann/subscriptions","organizations_url":"https://api.github.com/users/tillrohrmann/orgs","repos_url":"https://api.github.com/users/tillrohrmann/repos","events_url":"https://api.github.com/users/tillrohrmann/events{/privacy}","received_events_url":"https://api.github.com/users/tillrohrmann/received_events","type":"User","site_admin":false},"body":"introduce a constant for `\"Jar\"`?","created_at":"2018-06-22T13:47:29Z","updated_at":"2018-06-22T13:56:58Z","html_url":"https://github.com/apache/flink/pull/6203#discussion_r197450435","pull_request_url":"https://api.github.com/repos/apache/flink/pulls/6203","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/comments/197450435"},"html":{"href":"https://github.com/apache/flink/pull/6203#discussion_r197450435"},"pull_request":{"href":"https://api.github.com/repos/apache/flink/pulls/6203"}}},"pull_request":{"url":"https://api.github.com/repos/apache/flink/pulls/6203","id":196712713,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk2NzEyNzEz","html_url":"https://github.com/apache/flink/pull/6203","diff_url":"https://github.com/apache/flink/pull/6203.diff","patch_url":"https://github.com/apache/flink/pull/6203.patch","issue_url":"https://api.github.com/repos/apache/flink/issues/6203","number":6203,"state":"open","locked":false,"title":"[FLINK-9289][rest] Rework JobSubmitHandler to accept jar/artifact files","user":{"login":"zentol","id":5725237,"node_id":"MDQ6VXNlcjU3MjUyMzc=","avatar_url":"https://avatars1.githubusercontent.com/u/5725237?v=4","gravatar_id":"","url":"https://api.github.com/users/zentol","html_url":"https://github.com/zentol","followers_url":"https://api.github.com/users/zentol/followers","following_url":"https://api.github.com/users/zentol/following{/other_user}","gists_url":"https://api.github.com/users/zentol/gists{/gist_id}","starred_url":"https://api.github.com/users/zentol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zentol/subscriptions","organizations_url":"https://api.github.com/users/zentol/orgs","repos_url":"https://api.github.com/users/zentol/repos","events_url":"https://api.github.com/users/zentol/events{/privacy}","received_events_url":"https://api.github.com/users/zentol/received_events","type":"User","site_admin":false},"body":"Builds on #6199.\r\n\r\n## What is the purpose of the change\r\n\r\nThis PR reworks the `JobSubmitHandler` to accept jars/artifacts and the jobgraph in a single REST request, and changes the `RestClusterClient` to make use of this feature.\r\nAs a result the entire job submission now goes through the REST API.\r\n\r\nThere are still known issues in this PR but they won't impact the correctness; the core implementation is complete, works and is covered by tests.\r\n\r\n## Brief change log\r\n\r\n* remove blobserver port headers/handlers and usages\r\n* rework `JobSubmitRequestBody` to contain the names of the jobgraph, jar and artifact files, providing semantics to the collection of uploaded files\r\n* set visibility of utility methods in `ClientUtils` to `public`to allow passing jar/artifact collections manually\r\n* modified the `JobSubmitHandler` to\r\n  * read the jobgraph from an uploaded file\r\n  * upload uploaded jars/artifacts to the blob service and store keys in the jobgraph\r\n* modified the `RestClusterClient` to\r\n  * write jobgraph into a file for later upload\r\n  * make use of new `RestClient` API for multipart requests\r\n* extended ´RestConstants` for jar/binary `content-types`, and made the class an enum to get singleton properties\r\n* extended documentation of `JobSubmitHeaders`\r\n\r\n## Verifying this change\r\n\r\n* this PR only modifies existing tests to accommodate changes\r\n* the general functionality is covered by existing end-to-end tests\r\n\r\n## Does this pull request potentially affect one of the following parts:\r\n\r\n  - Dependencies (does it add or upgrade a dependency): (no)\r\n  - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes)\r\n  - The serializers: (no)\r\n  - The runtime per-record code paths (performance sensitive): (no)\r\n  - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)\r\n  - The S3 file system connector: (no)\r\n\r\n## Documentation\r\n\r\n  - Does this pull request introduce a new feature? (yes)\r\n  - If yes, how is the feature documented? (docs)\r\n","created_at":"2018-06-22T11:31:05Z","updated_at":"2018-06-22T13:56:58Z","closed_at":null,"merged_at":null,"merge_commit_sha":"65b89d7a348104f366bd70c42c30c59de7aef8e4","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/flink/pulls/6203/commits","review_comments_url":"https://api.github.com/repos/apache/flink/pulls/6203/comments","review_comment_url":"https://api.github.com/repos/apache/flink/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/flink/issues/6203/comments","statuses_url":"https://api.github.com/repos/apache/flink/statuses/f47e899bf5f26b346cc7b3e637dd57722fcf2891","head":{"label":"zentol:9280_epsilon","ref":"9280_epsilon","sha":"f47e899bf5f26b346cc7b3e637dd57722fcf2891","user":{"login":"zentol","id":5725237,"node_id":"MDQ6VXNlcjU3MjUyMzc=","avatar_url":"https://avatars1.githubusercontent.com/u/5725237?v=4","gravatar_id":"","url":"https://api.github.com/users/zentol","html_url":"https://github.com/zentol","followers_url":"https://api.github.com/users/zentol/followers","following_url":"https://api.github.com/users/zentol/following{/other_user}","gists_url":"https://api.github.com/users/zentol/gists{/gist_id}","starred_url":"https://api.github.com/users/zentol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zentol/subscriptions","organizations_url":"https://api.github.com/users/zentol/orgs","repos_url":"https://api.github.com/users/zentol/repos","events_url":"https://api.github.com/users/zentol/events{/privacy}","received_events_url":"https://api.github.com/users/zentol/received_events","type":"User","site_admin":false},"repo":{"id":20647396,"node_id":"MDEwOlJlcG9zaXRvcnkyMDY0NzM5Ng==","name":"flink","full_name":"zentol/flink","owner":{"login":"zentol","id":5725237,"node_id":"MDQ6VXNlcjU3MjUyMzc=","avatar_url":"https://avatars1.githubusercontent.com/u/5725237?v=4","gravatar_id":"","url":"https://api.github.com/users/zentol","html_url":"https://github.com/zentol","followers_url":"https://api.github.com/users/zentol/followers","following_url":"https://api.github.com/users/zentol/following{/other_user}","gists_url":"https://api.github.com/users/zentol/gists{/gist_id}","starred_url":"https://api.github.com/users/zentol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zentol/subscriptions","organizations_url":"https://api.github.com/users/zentol/orgs","repos_url":"https://api.github.com/users/zentol/repos","events_url":"https://api.github.com/users/zentol/events{/privacy}","received_events_url":"https://api.github.com/users/zentol/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/zentol/flink","description":"Mirror of Apache Flink","fork":true,"url":"https://api.github.com/repos/zentol/flink","forks_url":"https://api.github.com/repos/zentol/flink/forks","keys_url":"https://api.github.com/repos/zentol/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/zentol/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/zentol/flink/teams","hooks_url":"https://api.github.com/repos/zentol/flink/hooks","issue_events_url":"https://api.github.com/repos/zentol/flink/issues/events{/number}","events_url":"https://api.github.com/repos/zentol/flink/events","assignees_url":"https://api.github.com/repos/zentol/flink/assignees{/user}","branches_url":"https://api.github.com/repos/zentol/flink/branches{/branch}","tags_url":"https://api.github.com/repos/zentol/flink/tags","blobs_url":"https://api.github.com/repos/zentol/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/zentol/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/zentol/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/zentol/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/zentol/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/zentol/flink/languages","stargazers_url":"https://api.github.com/repos/zentol/flink/stargazers","contributors_url":"https://api.github.com/repos/zentol/flink/contributors","subscribers_url":"https://api.github.com/repos/zentol/flink/subscribers","subscription_url":"https://api.github.com/repos/zentol/flink/subscription","commits_url":"https://api.github.com/repos/zentol/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/zentol/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/zentol/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/zentol/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/zentol/flink/contents/{+path}","compare_url":"https://api.github.com/repos/zentol/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/zentol/flink/merges","archive_url":"https://api.github.com/repos/zentol/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/zentol/flink/downloads","issues_url":"https://api.github.com/repos/zentol/flink/issues{/number}","pulls_url":"https://api.github.com/repos/zentol/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/zentol/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/zentol/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/zentol/flink/labels{/name}","releases_url":"https://api.github.com/repos/zentol/flink/releases{/id}","deployments_url":"https://api.github.com/repos/zentol/flink/deployments","created_at":"2014-06-09T13:27:33Z","updated_at":"2018-05-22T07:24:57Z","pushed_at":"2018-06-22T11:47:48Z","git_url":"git://github.com/zentol/flink.git","ssh_url":"git@github.com:zentol/flink.git","clone_url":"https://github.com/zentol/flink.git","svn_url":"https://github.com/zentol/flink","homepage":null,"size":152285,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"2d6972d40541f953aa66fe556d5b27eb2fb536c6","user":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":20587599,"node_id":"MDEwOlJlcG9zaXRvcnkyMDU4NzU5OQ==","name":"flink","full_name":"apache/flink","owner":{"login":"apache","id":47359,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3MzU5","avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/flink","description":"Mirror of Apache Flink","fork":false,"url":"https://api.github.com/repos/apache/flink","forks_url":"https://api.github.com/repos/apache/flink/forks","keys_url":"https://api.github.com/repos/apache/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/flink/teams","hooks_url":"https://api.github.com/repos/apache/flink/hooks","issue_events_url":"https://api.github.com/repos/apache/flink/issues/events{/number}","events_url":"https://api.github.com/repos/apache/flink/events","assignees_url":"https://api.github.com/repos/apache/flink/assignees{/user}","branches_url":"https://api.github.com/repos/apache/flink/branches{/branch}","tags_url":"https://api.github.com/repos/apache/flink/tags","blobs_url":"https://api.github.com/repos/apache/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/flink/languages","stargazers_url":"https://api.github.com/repos/apache/flink/stargazers","contributors_url":"https://api.github.com/repos/apache/flink/contributors","subscribers_url":"https://api.github.com/repos/apache/flink/subscribers","subscription_url":"https://api.github.com/repos/apache/flink/subscription","commits_url":"https://api.github.com/repos/apache/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/flink/contents/{+path}","compare_url":"https://api.github.com/repos/apache/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/flink/merges","archive_url":"https://api.github.com/repos/apache/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/flink/downloads","issues_url":"https://api.github.com/repos/apache/flink/issues{/number}","pulls_url":"https://api.github.com/repos/apache/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/flink/labels{/name}","releases_url":"https://api.github.com/repos/apache/flink/releases{/id}","deployments_url":"https://api.github.com/repos/apache/flink/deployments","created_at":"2014-06-07T07:00:10Z","updated_at":"2018-06-22T12:08:04Z","pushed_at":"2018-06-22T12:38:49Z","git_url":"git://github.com/apache/flink.git","ssh_url":"git@github.com:apache/flink.git","clone_url":"https://github.com/apache/flink.git","svn_url":"https://github.com/apache/flink","homepage":null,"size":157554,"stargazers_count":3861,"watchers_count":3861,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":2610,"mirror_url":null,"archived":false,"open_issues_count":369,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":2610,"open_issues":369,"watchers":3861,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/6203"},"html":{"href":"https://github.com/apache/flink/pull/6203"},"issue":{"href":"https://api.github.com/repos/apache/flink/issues/6203"},"comments":{"href":"https://api.github.com/repos/apache/flink/issues/6203/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/flink/pulls/6203/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/flink/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/flink/pulls/6203/commits"},"statuses":{"href":"https://api.github.com/repos/apache/flink/statuses/f47e899bf5f26b346cc7b3e637dd57722fcf2891"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"tillrohrmann"}}