{"action":"created","issue":{"url":"https://api.github.com/repos/openminted/Open-Call-Discussions/issues/38","repository_url":"https://api.github.com/repos/openminted/Open-Call-Discussions","labels_url":"https://api.github.com/repos/openminted/Open-Call-Discussions/issues/38/labels{/name}","comments_url":"https://api.github.com/repos/openminted/Open-Call-Discussions/issues/38/comments","events_url":"https://api.github.com/repos/openminted/Open-Call-Discussions/issues/38/events","html_url":"https://github.com/openminted/Open-Call-Discussions/issues/38","id":317138288,"number":38,"title":"Docker image size restritions","user":{"login":"saxtouri","id":3098971,"avatar_url":"https://avatars3.githubusercontent.com/u/3098971?v=4","gravatar_id":"","url":"https://api.github.com/users/saxtouri","html_url":"https://github.com/saxtouri","followers_url":"https://api.github.com/users/saxtouri/followers","following_url":"https://api.github.com/users/saxtouri/following{/other_user}","gists_url":"https://api.github.com/users/saxtouri/gists{/gist_id}","starred_url":"https://api.github.com/users/saxtouri/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/saxtouri/subscriptions","organizations_url":"https://api.github.com/users/saxtouri/orgs","repos_url":"https://api.github.com/users/saxtouri/repos","events_url":"https://api.github.com/users/saxtouri/events{/privacy}","received_events_url":"https://api.github.com/users/saxtouri/received_events","type":"User","site_admin":false},"labels":[{"id":888685500,"url":"https://api.github.com/repos/openminted/Open-Call-Discussions/labels/bug","name":"bug","color":"d73a4a","default":true}],"state":"open","locked":false,"assignee":{"login":"antleb","id":191451,"avatar_url":"https://avatars0.githubusercontent.com/u/191451?v=4","gravatar_id":"","url":"https://api.github.com/users/antleb","html_url":"https://github.com/antleb","followers_url":"https://api.github.com/users/antleb/followers","following_url":"https://api.github.com/users/antleb/following{/other_user}","gists_url":"https://api.github.com/users/antleb/gists{/gist_id}","starred_url":"https://api.github.com/users/antleb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/antleb/subscriptions","organizations_url":"https://api.github.com/users/antleb/orgs","repos_url":"https://api.github.com/users/antleb/repos","events_url":"https://api.github.com/users/antleb/events{/privacy}","received_events_url":"https://api.github.com/users/antleb/received_events","type":"User","site_admin":false},"assignees":[{"login":"antleb","id":191451,"avatar_url":"https://avatars0.githubusercontent.com/u/191451?v=4","gravatar_id":"","url":"https://api.github.com/users/antleb","html_url":"https://github.com/antleb","followers_url":"https://api.github.com/users/antleb/followers","following_url":"https://api.github.com/users/antleb/following{/other_user}","gists_url":"https://api.github.com/users/antleb/gists{/gist_id}","starred_url":"https://api.github.com/users/antleb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/antleb/subscriptions","organizations_url":"https://api.github.com/users/antleb/orgs","repos_url":"https://api.github.com/users/antleb/repos","events_url":"https://api.github.com/users/antleb/events{/privacy}","received_events_url":"https://api.github.com/users/antleb/received_events","type":"User","site_admin":false},{"login":"greenwoodma","id":3025272,"avatar_url":"https://avatars1.githubusercontent.com/u/3025272?v=4","gravatar_id":"","url":"https://api.github.com/users/greenwoodma","html_url":"https://github.com/greenwoodma","followers_url":"https://api.github.com/users/greenwoodma/followers","following_url":"https://api.github.com/users/greenwoodma/following{/other_user}","gists_url":"https://api.github.com/users/greenwoodma/gists{/gist_id}","starred_url":"https://api.github.com/users/greenwoodma/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/greenwoodma/subscriptions","organizations_url":"https://api.github.com/users/greenwoodma/orgs","repos_url":"https://api.github.com/users/greenwoodma/repos","events_url":"https://api.github.com/users/greenwoodma/events{/privacy}","received_events_url":"https://api.github.com/users/greenwoodma/received_events","type":"User","site_admin":false},{"login":"saxtouri","id":3098971,"avatar_url":"https://avatars3.githubusercontent.com/u/3098971?v=4","gravatar_id":"","url":"https://api.github.com/users/saxtouri","html_url":"https://github.com/saxtouri","followers_url":"https://api.github.com/users/saxtouri/followers","following_url":"https://api.github.com/users/saxtouri/following{/other_user}","gists_url":"https://api.github.com/users/saxtouri/gists{/gist_id}","starred_url":"https://api.github.com/users/saxtouri/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/saxtouri/subscriptions","organizations_url":"https://api.github.com/users/saxtouri/orgs","repos_url":"https://api.github.com/users/saxtouri/repos","events_url":"https://api.github.com/users/saxtouri/events{/privacy}","received_events_url":"https://api.github.com/users/saxtouri/received_events","type":"User","site_admin":false},{"login":"Jodee90","id":6078140,"avatar_url":"https://avatars2.githubusercontent.com/u/6078140?v=4","gravatar_id":"","url":"https://api.github.com/users/Jodee90","html_url":"https://github.com/Jodee90","followers_url":"https://api.github.com/users/Jodee90/followers","following_url":"https://api.github.com/users/Jodee90/following{/other_user}","gists_url":"https://api.github.com/users/Jodee90/gists{/gist_id}","starred_url":"https://api.github.com/users/Jodee90/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Jodee90/subscriptions","organizations_url":"https://api.github.com/users/Jodee90/orgs","repos_url":"https://api.github.com/users/Jodee90/repos","events_url":"https://api.github.com/users/Jodee90/events{/privacy}","received_events_url":"https://api.github.com/users/Jodee90/received_events","type":"User","site_admin":false},{"login":"courado","id":6794160,"avatar_url":"https://avatars1.githubusercontent.com/u/6794160?v=4","gravatar_id":"","url":"https://api.github.com/users/courado","html_url":"https://github.com/courado","followers_url":"https://api.github.com/users/courado/followers","following_url":"https://api.github.com/users/courado/following{/other_user}","gists_url":"https://api.github.com/users/courado/gists{/gist_id}","starred_url":"https://api.github.com/users/courado/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/courado/subscriptions","organizations_url":"https://api.github.com/users/courado/orgs","repos_url":"https://api.github.com/users/courado/repos","events_url":"https://api.github.com/users/courado/events{/privacy}","received_events_url":"https://api.github.com/users/courado/received_events","type":"User","site_admin":false},{"login":"gkirtzou","id":8426223,"avatar_url":"https://avatars3.githubusercontent.com/u/8426223?v=4","gravatar_id":"","url":"https://api.github.com/users/gkirtzou","html_url":"https://github.com/gkirtzou","followers_url":"https://api.github.com/users/gkirtzou/followers","following_url":"https://api.github.com/users/gkirtzou/following{/other_user}","gists_url":"https://api.github.com/users/gkirtzou/gists{/gist_id}","starred_url":"https://api.github.com/users/gkirtzou/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gkirtzou/subscriptions","organizations_url":"https://api.github.com/users/gkirtzou/orgs","repos_url":"https://api.github.com/users/gkirtzou/repos","events_url":"https://api.github.com/users/gkirtzou/events{/privacy}","received_events_url":"https://api.github.com/users/gkirtzou/received_events","type":"User","site_admin":false},{"login":"galanisd","id":10739759,"avatar_url":"https://avatars0.githubusercontent.com/u/10739759?v=4","gravatar_id":"","url":"https://api.github.com/users/galanisd","html_url":"https://github.com/galanisd","followers_url":"https://api.github.com/users/galanisd/followers","following_url":"https://api.github.com/users/galanisd/following{/other_user}","gists_url":"https://api.github.com/users/galanisd/gists{/gist_id}","starred_url":"https://api.github.com/users/galanisd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/galanisd/subscriptions","organizations_url":"https://api.github.com/users/galanisd/orgs","repos_url":"https://api.github.com/users/galanisd/repos","events_url":"https://api.github.com/users/galanisd/events{/privacy}","received_events_url":"https://api.github.com/users/galanisd/received_events","type":"User","site_admin":false}],"milestone":null,"comments":16,"created_at":"2018-04-24T09:15:16Z","updated_at":"2018-04-24T11:53:46Z","closed_at":null,"author_association":"COLLABORATOR","body":"This issue is critical from an infrastructure perspective. Please, let me know what you think:\r\n\r\nSome of the docker images are unexpectedly large (e.g., 7-8 GBs or more). Because of that:\r\n1. the system fails to execute these jobs due to timeout\r\n2. even if we override the timeout issue by pre-downloading the huge image, the cluster nodes get full very quickly, rendering the system unusable\r\n\r\nI think we should reconsider our policy on the docker image sizes and set size restrictions. If we don't, we will experience a lot of failures and our storage resources will get consumes really fast. Also, if we run the system on a commercial cluster, the cost of the infrastructure will rise too high very quickly and it will be nearly impossible to scale.\r\n\r\nI wonder how images get so big. What kind of software code requires so many GBs, considering that desktop OSs overloaded with heavy apps can be bundled in just 5 to 10 GBs. Is it just software or do we allow people to load images with data as well?\r\n\r\nPutting any kind of data inside the images is a highly inefficient practice because it makes our system too expensive to run (I can explain this further, if you like).\r\n\r\nWhat do you think we should do?"},"comment":{"url":"https://api.github.com/repos/openminted/Open-Call-Discussions/issues/comments/383903369","html_url":"https://github.com/openminted/Open-Call-Discussions/issues/38#issuecomment-383903369","issue_url":"https://api.github.com/repos/openminted/Open-Call-Discussions/issues/38","id":383903369,"user":{"login":"gkirtzou","id":8426223,"avatar_url":"https://avatars3.githubusercontent.com/u/8426223?v=4","gravatar_id":"","url":"https://api.github.com/users/gkirtzou","html_url":"https://github.com/gkirtzou","followers_url":"https://api.github.com/users/gkirtzou/followers","following_url":"https://api.github.com/users/gkirtzou/following{/other_user}","gists_url":"https://api.github.com/users/gkirtzou/gists{/gist_id}","starred_url":"https://api.github.com/users/gkirtzou/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gkirtzou/subscriptions","organizations_url":"https://api.github.com/users/gkirtzou/orgs","repos_url":"https://api.github.com/users/gkirtzou/repos","events_url":"https://api.github.com/users/gkirtzou/events{/privacy}","received_events_url":"https://api.github.com/users/gkirtzou/received_events","type":"User","site_admin":false},"created_at":"2018-04-24T11:53:46Z","updated_at":"2018-04-24T11:53:46Z","author_association":"NONE","body":"To get an idea of the docker images sizes that come from Open Call II\r\n\r\n- IXA_pipes (ragerri/de-ixa-pipes-omtd:0.1.0) :  400 MB. IXA has 8 different images one per language\r\n- Alex (abravp/openminted_scisumservices:1.0.0) **26.1GB** \r\n- BOLSTM (andrelamurias/bolstm:omtd)  8.66 GB \r\n- Jones - Pubrunner hackathon (jakelever/pubrunner:0.3.1), 1.3GB\r\n- Taln - Freeling (taln/openminted_freeling:1.0)  6.77GB \r\n- Taln - BabelNetExtractor (taln/openminted_babelnet:1.0) **26.6GB** \r\n- UPFMT (dumitrescustefan/upfmt)  2.46 GB\r\n- FIMDA (erechtheus/fimda:0.2.2)  110 MB "}}
{"actor":{"display_login":"gkirtzou"}}