{"action":"created","comment":{"url":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/comments/168744155","pull_request_review_id":97163335,"id":168744155,"diff_hunk":"@@ -0,0 +1,76 @@\n+# Compute Engine comparison script\n+\n+## Description\n+An automatic mechanism to gather and compare results between Hadoop and Flink infrastructures.\n+ARGO serves availability/reliability results along with status timelines of the monitored elements (sites, services, endpoints, endpoint metrics). The current stable infrastructure is using Hadoop to compute availability and reliability and status results. Computation logic has been transferred to Flink also (currently running in a devel Flink cluster) and soon to be deployed in a Flink production cluster. It is essential for the switch to have a mechanism to compare results between the legacy and the new infrastructure.\n+\n+## How to use\n+`ce_compy.py -s <Output format> -t <Tenant> -c <Conf file> -th <Threshold> -sp <Report's save path>[Optional]`\n+\n+| argument | Description |\n+| --- | --- |\n+| -s <Output format> | Report's file format. Right now it supports csv, with the default being json. |\n+| -t <Tenant> | Tenant. |\n+| -c <Conf file> |  Path to the config file. |\n+| -th <Threshold> | Threshold availability and reliability results by a minimum threshold. |\n+| -sp <Save path> [Optional] | If not specified, the script will create a directory named **generated_reports** in the same directory with itself, and store the results there. |\n+\n+The report's file name will be: <tenant>@<date>_report.<output_format>\n+e.g. TENANTA@2018-02-15_report.csv\n+\n+### Configuration file\n+```\n+ [TENANTA]\n+hadoop: https://web-api-devel.argo.grnet.gr/api/v2/results/reportA/SERVICEGROUPS?start_time={start_date}T00%3A00%3A00Z&end_time={start_date}T23%3A53%3A00Z\n+\n+flink: https://web-api-devel.argo.grnet.gr/api/v2/results/reportB/SERVICEGROUPS?start_time={start_date}T00%3A00%3A00Z&end_time={start_date}T23%3A53%3A00Z\n+\n+token: *********************\n+\n+[TENANTB]\n+hadoop: https://web-api-devel.argo.grnet.gr/api/v2/results/reportA/SITES?start_time={start_date}T00%3A00%3A00Z&end_time={start_date}T23%3A53%3A00Z\n+\n+flink: https://web-api-devel.argo.grnet.gr/api/v2/results/reportB/SITES?start_time={start_date}T00%3A00%3A00Z&end_time={start_date}T23%3A53%3A00Z\n+\n+token: ***********************\n+\n+[PERIOD]\n+start_date: 2018-02-01\n+end_date: 2018-02-12\n+\n+[SaveLocation]\n+path: ./generated_reports/ \n+```\n+ - Each tenant is described by two urls,one for each infrastructure respectively, and a token, in order to access its data and compare.\n+ - The script supports functionality to calculate results for a period of time in a single execution. All of the results will be saved into the **[SaveLocation]** described by the tenant and the day.\n+\n+#### Results\n+The report produced by the script contains information about each site found under a group that belogs to the specified tenant.\n+e.g. The first row of the report for the  TENANTA.\n+\n+| Endpoint | A_prod | A_devel | R_prod | R_devel | D_a | D_r |\n+| --- | --- | --- | --- | --- | --- | -- |\n+|siteA@groupA|100|98|95|-1|2|na|\n+\n+For each endpoint, it shows the site it belogs to, the availability and reliability metrics coming from production and devel respectively. At the end it shows the calculated differences.\n+**Note:** If the availability or reliability metric for the day is -1, meaning that the owner stated scheduled downtime, it produced a non applicable value.\n+\n+Also during execution the script prints results in the command line regarding which points were missing from either compute engine, the average error and the thresholded endpoints sorted in descednding order.\n+\n+#### Dependencies needed for the script\n+`$ pip install requests pandas termcolor prettytable pytest`","path":"README.md","position":61,"original_position":61,"commit_id":"8058bebbdfe0c9ad6ade3abfd85f13a02b803dff","original_commit_id":"0e78b2145876685c34dd96dabcd4d6a90c243aca","user":{"login":"kkoumantaros","id":2379008,"avatar_url":"https://avatars2.githubusercontent.com/u/2379008?v=4","gravatar_id":"","url":"https://api.github.com/users/kkoumantaros","html_url":"https://github.com/kkoumantaros","followers_url":"https://api.github.com/users/kkoumantaros/followers","following_url":"https://api.github.com/users/kkoumantaros/following{/other_user}","gists_url":"https://api.github.com/users/kkoumantaros/gists{/gist_id}","starred_url":"https://api.github.com/users/kkoumantaros/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kkoumantaros/subscriptions","organizations_url":"https://api.github.com/users/kkoumantaros/orgs","repos_url":"https://api.github.com/users/kkoumantaros/repos","events_url":"https://api.github.com/users/kkoumantaros/events{/privacy}","received_events_url":"https://api.github.com/users/kkoumantaros/received_events","type":"User","site_admin":false},"body":"Please use the terminology we use for the compute_engines e.g. \r\nCompute Engine v1 (haddoop based) Compute Engine v2 (flink based)","created_at":"2018-02-16T12:30:11Z","updated_at":"2018-02-16T12:30:11Z","html_url":"https://github.com/ARGOeu/ce_comp/pull/1#discussion_r168744155","pull_request_url":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/1","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/comments/168744155"},"html":{"href":"https://github.com/ARGOeu/ce_comp/pull/1#discussion_r168744155"},"pull_request":{"href":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/1"}},"in_reply_to_id":168739743},"pull_request":{"url":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/1","id":169385737,"html_url":"https://github.com/ARGOeu/ce_comp/pull/1","diff_url":"https://github.com/ARGOeu/ce_comp/pull/1.diff","patch_url":"https://github.com/ARGOeu/ce_comp/pull/1.patch","issue_url":"https://api.github.com/repos/ARGOeu/ce_comp/issues/1","number":1,"state":"open","locked":false,"title":"ARGO-998 Mechanism to compare results between hadoop - flink infrastructures","user":{"login":"agelostsal","id":15677505,"avatar_url":"https://avatars1.githubusercontent.com/u/15677505?v=4","gravatar_id":"","url":"https://api.github.com/users/agelostsal","html_url":"https://github.com/agelostsal","followers_url":"https://api.github.com/users/agelostsal/followers","following_url":"https://api.github.com/users/agelostsal/following{/other_user}","gists_url":"https://api.github.com/users/agelostsal/gists{/gist_id}","starred_url":"https://api.github.com/users/agelostsal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/agelostsal/subscriptions","organizations_url":"https://api.github.com/users/agelostsal/orgs","repos_url":"https://api.github.com/users/agelostsal/repos","events_url":"https://api.github.com/users/agelostsal/events{/privacy}","received_events_url":"https://api.github.com/users/agelostsal/received_events","type":"User","site_admin":false},"body":"Implementation of the compute engine comparison script. An automatic mechanism to gather and compare results between Hadoop and Flink infrastructures.\r\n- Functionality ready\r\n- Documentation ready\r\n- Unit tests ready\r\n\r\n","created_at":"2018-02-15T14:27:31Z","updated_at":"2018-02-16T12:30:11Z","closed_at":null,"merged_at":null,"merge_commit_sha":"34a942bd211e93c58769e4a0abdc8f7873f2c30b","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/1/commits","review_comments_url":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/1/comments","review_comment_url":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ARGOeu/ce_comp/issues/1/comments","statuses_url":"https://api.github.com/repos/ARGOeu/ce_comp/statuses/8058bebbdfe0c9ad6ade3abfd85f13a02b803dff","head":{"label":"agelostsal:comparison_script","ref":"comparison_script","sha":"8058bebbdfe0c9ad6ade3abfd85f13a02b803dff","user":{"login":"agelostsal","id":15677505,"avatar_url":"https://avatars1.githubusercontent.com/u/15677505?v=4","gravatar_id":"","url":"https://api.github.com/users/agelostsal","html_url":"https://github.com/agelostsal","followers_url":"https://api.github.com/users/agelostsal/followers","following_url":"https://api.github.com/users/agelostsal/following{/other_user}","gists_url":"https://api.github.com/users/agelostsal/gists{/gist_id}","starred_url":"https://api.github.com/users/agelostsal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/agelostsal/subscriptions","organizations_url":"https://api.github.com/users/agelostsal/orgs","repos_url":"https://api.github.com/users/agelostsal/repos","events_url":"https://api.github.com/users/agelostsal/events{/privacy}","received_events_url":"https://api.github.com/users/agelostsal/received_events","type":"User","site_admin":false},"repo":{"id":121622846,"name":"ce_comp","full_name":"agelostsal/ce_comp","owner":{"login":"agelostsal","id":15677505,"avatar_url":"https://avatars1.githubusercontent.com/u/15677505?v=4","gravatar_id":"","url":"https://api.github.com/users/agelostsal","html_url":"https://github.com/agelostsal","followers_url":"https://api.github.com/users/agelostsal/followers","following_url":"https://api.github.com/users/agelostsal/following{/other_user}","gists_url":"https://api.github.com/users/agelostsal/gists{/gist_id}","starred_url":"https://api.github.com/users/agelostsal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/agelostsal/subscriptions","organizations_url":"https://api.github.com/users/agelostsal/orgs","repos_url":"https://api.github.com/users/agelostsal/repos","events_url":"https://api.github.com/users/agelostsal/events{/privacy}","received_events_url":"https://api.github.com/users/agelostsal/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/agelostsal/ce_comp","description":"Argo Compute Engines comparison script","fork":true,"url":"https://api.github.com/repos/agelostsal/ce_comp","forks_url":"https://api.github.com/repos/agelostsal/ce_comp/forks","keys_url":"https://api.github.com/repos/agelostsal/ce_comp/keys{/key_id}","collaborators_url":"https://api.github.com/repos/agelostsal/ce_comp/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/agelostsal/ce_comp/teams","hooks_url":"https://api.github.com/repos/agelostsal/ce_comp/hooks","issue_events_url":"https://api.github.com/repos/agelostsal/ce_comp/issues/events{/number}","events_url":"https://api.github.com/repos/agelostsal/ce_comp/events","assignees_url":"https://api.github.com/repos/agelostsal/ce_comp/assignees{/user}","branches_url":"https://api.github.com/repos/agelostsal/ce_comp/branches{/branch}","tags_url":"https://api.github.com/repos/agelostsal/ce_comp/tags","blobs_url":"https://api.github.com/repos/agelostsal/ce_comp/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/agelostsal/ce_comp/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/agelostsal/ce_comp/git/refs{/sha}","trees_url":"https://api.github.com/repos/agelostsal/ce_comp/git/trees{/sha}","statuses_url":"https://api.github.com/repos/agelostsal/ce_comp/statuses/{sha}","languages_url":"https://api.github.com/repos/agelostsal/ce_comp/languages","stargazers_url":"https://api.github.com/repos/agelostsal/ce_comp/stargazers","contributors_url":"https://api.github.com/repos/agelostsal/ce_comp/contributors","subscribers_url":"https://api.github.com/repos/agelostsal/ce_comp/subscribers","subscription_url":"https://api.github.com/repos/agelostsal/ce_comp/subscription","commits_url":"https://api.github.com/repos/agelostsal/ce_comp/commits{/sha}","git_commits_url":"https://api.github.com/repos/agelostsal/ce_comp/git/commits{/sha}","comments_url":"https://api.github.com/repos/agelostsal/ce_comp/comments{/number}","issue_comment_url":"https://api.github.com/repos/agelostsal/ce_comp/issues/comments{/number}","contents_url":"https://api.github.com/repos/agelostsal/ce_comp/contents/{+path}","compare_url":"https://api.github.com/repos/agelostsal/ce_comp/compare/{base}...{head}","merges_url":"https://api.github.com/repos/agelostsal/ce_comp/merges","archive_url":"https://api.github.com/repos/agelostsal/ce_comp/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/agelostsal/ce_comp/downloads","issues_url":"https://api.github.com/repos/agelostsal/ce_comp/issues{/number}","pulls_url":"https://api.github.com/repos/agelostsal/ce_comp/pulls{/number}","milestones_url":"https://api.github.com/repos/agelostsal/ce_comp/milestones{/number}","notifications_url":"https://api.github.com/repos/agelostsal/ce_comp/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/agelostsal/ce_comp/labels{/name}","releases_url":"https://api.github.com/repos/agelostsal/ce_comp/releases{/id}","deployments_url":"https://api.github.com/repos/agelostsal/ce_comp/deployments","created_at":"2018-02-15T11:39:03Z","updated_at":"2018-02-14T15:23:45Z","pushed_at":"2018-02-16T11:29:44Z","git_url":"git://github.com/agelostsal/ce_comp.git","ssh_url":"git@github.com:agelostsal/ce_comp.git","clone_url":"https://github.com/agelostsal/ce_comp.git","svn_url":"https://github.com/agelostsal/ce_comp","homepage":null,"size":11,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":null,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"ARGOeu:devel","ref":"devel","sha":"7bf1e366685947c543ebe6a6481d940ce3cca611","user":{"login":"ARGOeu","id":6882712,"avatar_url":"https://avatars2.githubusercontent.com/u/6882712?v=4","gravatar_id":"","url":"https://api.github.com/users/ARGOeu","html_url":"https://github.com/ARGOeu","followers_url":"https://api.github.com/users/ARGOeu/followers","following_url":"https://api.github.com/users/ARGOeu/following{/other_user}","gists_url":"https://api.github.com/users/ARGOeu/gists{/gist_id}","starred_url":"https://api.github.com/users/ARGOeu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ARGOeu/subscriptions","organizations_url":"https://api.github.com/users/ARGOeu/orgs","repos_url":"https://api.github.com/users/ARGOeu/repos","events_url":"https://api.github.com/users/ARGOeu/events{/privacy}","received_events_url":"https://api.github.com/users/ARGOeu/received_events","type":"Organization","site_admin":false},"repo":{"id":121526030,"name":"ce_comp","full_name":"ARGOeu/ce_comp","owner":{"login":"ARGOeu","id":6882712,"avatar_url":"https://avatars2.githubusercontent.com/u/6882712?v=4","gravatar_id":"","url":"https://api.github.com/users/ARGOeu","html_url":"https://github.com/ARGOeu","followers_url":"https://api.github.com/users/ARGOeu/followers","following_url":"https://api.github.com/users/ARGOeu/following{/other_user}","gists_url":"https://api.github.com/users/ARGOeu/gists{/gist_id}","starred_url":"https://api.github.com/users/ARGOeu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ARGOeu/subscriptions","organizations_url":"https://api.github.com/users/ARGOeu/orgs","repos_url":"https://api.github.com/users/ARGOeu/repos","events_url":"https://api.github.com/users/ARGOeu/events{/privacy}","received_events_url":"https://api.github.com/users/ARGOeu/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ARGOeu/ce_comp","description":"Argo Compute Engines comparison script","fork":false,"url":"https://api.github.com/repos/ARGOeu/ce_comp","forks_url":"https://api.github.com/repos/ARGOeu/ce_comp/forks","keys_url":"https://api.github.com/repos/ARGOeu/ce_comp/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ARGOeu/ce_comp/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ARGOeu/ce_comp/teams","hooks_url":"https://api.github.com/repos/ARGOeu/ce_comp/hooks","issue_events_url":"https://api.github.com/repos/ARGOeu/ce_comp/issues/events{/number}","events_url":"https://api.github.com/repos/ARGOeu/ce_comp/events","assignees_url":"https://api.github.com/repos/ARGOeu/ce_comp/assignees{/user}","branches_url":"https://api.github.com/repos/ARGOeu/ce_comp/branches{/branch}","tags_url":"https://api.github.com/repos/ARGOeu/ce_comp/tags","blobs_url":"https://api.github.com/repos/ARGOeu/ce_comp/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ARGOeu/ce_comp/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ARGOeu/ce_comp/git/refs{/sha}","trees_url":"https://api.github.com/repos/ARGOeu/ce_comp/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ARGOeu/ce_comp/statuses/{sha}","languages_url":"https://api.github.com/repos/ARGOeu/ce_comp/languages","stargazers_url":"https://api.github.com/repos/ARGOeu/ce_comp/stargazers","contributors_url":"https://api.github.com/repos/ARGOeu/ce_comp/contributors","subscribers_url":"https://api.github.com/repos/ARGOeu/ce_comp/subscribers","subscription_url":"https://api.github.com/repos/ARGOeu/ce_comp/subscription","commits_url":"https://api.github.com/repos/ARGOeu/ce_comp/commits{/sha}","git_commits_url":"https://api.github.com/repos/ARGOeu/ce_comp/git/commits{/sha}","comments_url":"https://api.github.com/repos/ARGOeu/ce_comp/comments{/number}","issue_comment_url":"https://api.github.com/repos/ARGOeu/ce_comp/issues/comments{/number}","contents_url":"https://api.github.com/repos/ARGOeu/ce_comp/contents/{+path}","compare_url":"https://api.github.com/repos/ARGOeu/ce_comp/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ARGOeu/ce_comp/merges","archive_url":"https://api.github.com/repos/ARGOeu/ce_comp/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ARGOeu/ce_comp/downloads","issues_url":"https://api.github.com/repos/ARGOeu/ce_comp/issues{/number}","pulls_url":"https://api.github.com/repos/ARGOeu/ce_comp/pulls{/number}","milestones_url":"https://api.github.com/repos/ARGOeu/ce_comp/milestones{/number}","notifications_url":"https://api.github.com/repos/ARGOeu/ce_comp/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ARGOeu/ce_comp/labels{/name}","releases_url":"https://api.github.com/repos/ARGOeu/ce_comp/releases{/id}","deployments_url":"https://api.github.com/repos/ARGOeu/ce_comp/deployments","created_at":"2018-02-14T15:23:45Z","updated_at":"2018-02-14T15:23:45Z","pushed_at":"2018-02-16T11:29:45Z","git_url":"git://github.com/ARGOeu/ce_comp.git","ssh_url":"git@github.com:ARGOeu/ce_comp.git","clone_url":"https://github.com/ARGOeu/ce_comp.git","svn_url":"https://github.com/ARGOeu/ce_comp","homepage":null,"size":4,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":2,"mirror_url":null,"archived":false,"open_issues_count":1,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":2,"open_issues":1,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/1"},"html":{"href":"https://github.com/ARGOeu/ce_comp/pull/1"},"issue":{"href":"https://api.github.com/repos/ARGOeu/ce_comp/issues/1"},"comments":{"href":"https://api.github.com/repos/ARGOeu/ce_comp/issues/1/comments"},"review_comments":{"href":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/1/comments"},"review_comment":{"href":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ARGOeu/ce_comp/pulls/1/commits"},"statuses":{"href":"https://api.github.com/repos/ARGOeu/ce_comp/statuses/8058bebbdfe0c9ad6ade3abfd85f13a02b803dff"}},"author_association":"NONE"}}
{"actor":{"display_login":"kkoumantaros"}}