{"action":"created","issue":{"url":"https://api.github.com/repos/apache/cloudstack/issues/2820","repository_url":"https://api.github.com/repos/apache/cloudstack","labels_url":"https://api.github.com/repos/apache/cloudstack/issues/2820/labels{/name}","comments_url":"https://api.github.com/repos/apache/cloudstack/issues/2820/comments","events_url":"https://api.github.com/repos/apache/cloudstack/issues/2820/events","html_url":"https://github.com/apache/cloudstack/issues/2820","id":353426002,"node_id":"MDU6SXNzdWUzNTM0MjYwMDI=","number":2820,"title":"template downloading fails","user":{"login":"izenk","id":6417697,"node_id":"MDQ6VXNlcjY0MTc2OTc=","avatar_url":"https://avatars2.githubusercontent.com/u/6417697?v=4","gravatar_id":"","url":"https://api.github.com/users/izenk","html_url":"https://github.com/izenk","followers_url":"https://api.github.com/users/izenk/followers","following_url":"https://api.github.com/users/izenk/following{/other_user}","gists_url":"https://api.github.com/users/izenk/gists{/gist_id}","starred_url":"https://api.github.com/users/izenk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/izenk/subscriptions","organizations_url":"https://api.github.com/users/izenk/orgs","repos_url":"https://api.github.com/users/izenk/repos","events_url":"https://api.github.com/users/izenk/events{/privacy}","received_events_url":"https://api.github.com/users/izenk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2018-08-23T15:07:09Z","updated_at":"2018-11-20T13:09:13Z","closed_at":null,"author_association":"NONE","body":"<!--\r\nVerify first that your issue/request is not already reported on GitHub.\r\nAlso test if the latest release and master branch are affected too.\r\nAlways add information AFTER of these HTML comments, but no need to delete the comments.\r\n-->\r\n\r\n##### ISSUE TYPE\r\n<!-- Pick one below and delete the rest -->\r\n * Bug Report\r\n\r\n##### COMPONENT NAME\r\n<!--\r\nCategorize the issue, e.g. API, VR, VPN, UI, etc.\r\n-->\r\n~~~\r\nTEMPLATE\r\n~~~\r\n\r\n##### CLOUDSTACK VERSION\r\n<!--\r\nNew line separated list of affected versions, commit ID for issues on master branch.\r\n-->\r\n~~~\r\n4.11.0\r\n4.11.1\r\n~~~\r\n\r\n##### CONFIGURATION\r\n<!--\r\nInformation about the configuration if relevant, e.g. basic network, advanced networking, etc.  N/A otherwise\r\n-->\r\nADVANCED NETWORKING\r\n\r\n##### OS / ENVIRONMENT\r\n<!--\r\nInformation about the environment if relevant, N/A otherwise\r\n-->\r\nCentOS Linux release 7.4.1708\r\n\r\n##### SUMMARY\r\n<!-- Explain the problem/feature briefly -->\r\nWhen trying to add template from UI, uploading process fails with error: Stream closed\r\n\r\nMay be important:\r\ntemplate size is 104 GB\r\ntemplate format is RAW\r\n\r\n##### STEPS TO REPRODUCE\r\n<!--\r\nFor bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate.\r\n-->\r\n**UI**\r\nTemplates -> Add Template -> fill form (downloading from http)\r\n\r\n\r\n##### EXPECTED RESULTS\r\n<!-- What did you expect to happen when running the steps above? -->\r\n~~~\r\nTemplate is downloaded and registered\r\n~~~\r\n\r\n##### ACTUAL RESULTS\r\n<!-- What actually happened? -->\r\nDownloading fails with error: Stream closed\r\n\r\n<!-- Paste verbatim command output between quotes below -->\r\n~~~\r\n2018-08-23 12:14:34,078 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-2:null) (logid:) SeqA 5-577195: Sending Seq 5-577195:  { An\r\ns: , MgmtId: 90520739393856, via: 5, Ver: v1, Flags: 100010, [{\"com.cloud.agent.api.AgentControlAnswer\":{\"result\":true,\"wait\":0}}] }\r\n2018-08-23 12:14:35,813 DEBUG [c.c.a.ApiServlet] (qtp745160567-15:ctx-4e0d3cf8) (logid:f2420e15) ===START===  x.x.x.x -- GET  comman\r\nd=registerTemplate&response=json&projectid=62b2e0d2-5f81-4a6d-8d8b-591283f10e4a&name=test-oracledb-tmpl5&displayText=test-oracledb-tmpl5&u\r\nrl=http%3A%2F%2Ftemplates-host%2Fcscoracle-db.raw&zoneids=4a2e5664-edc7-4a0d-ae4a-fd56e1dc4314&format=RAW&isextractable=false&pass\r\nwordEnabled=false&isdynamicallyscalable=false&osTypeId=b70da464-54f4-11e8-8821-525400843140&hypervisor=KVM&ispublic=true&requireshvm=false\r\n&isfeatured=false&isrouting=false&details%5B0%5D.hypervisortoolsversion=xenserver56&directdownload=false&checksum=&_=1535019275786\r\n2018-08-23 12:14:35,819 DEBUG [c.c.a.ApiServer] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) CIDRs from which account 'Acc\r\nt[projectid]' is allowed to perform API calls: 0.0.0.0/0\r\n2018-08-23 12:14:35,838 DEBUG [c.c.r.ResourceLimitManagerImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Checking if am\r\nount of resources of Type = 'template' for Project Name = projectname in Domain Id = 5 is exceeded: Account Resource Limit = 20, Current A\r\nccount Resource Amount = 6, Requested Resource Amount = 1.\r\n2018-08-23 12:14:35,841 DEBUG [c.c.r.ResourceLimitManagerImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Checking if do\r\nmain resource limits of Type 'template' for Domain Id = 5 is exceeded: Domain Resource Limit = -1, Current Domain Resource Amount = 6, Req\r\nuested Resource Amount = 1.\r\n2018-08-23 12:14:35,842 DEBUG [c.c.r.ResourceLimitManagerImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Checking if domain resource limits of Type 'template' for Domain Id = 2 is exceeded: Domain Resource Limit = -1, Current Domain Resource Amount = 6, Requested Resource Amount = 1.\r\n2018-08-23 12:14:35,886 DEBUG [c.c.r.ResourceLimitManagerImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Checking if amount of resources of Type = 'secondary_storage' for Project Name = projectname in Domain Id = 5 is exceeded: Account Resource Limit = 429496729600, Current Account Resource Amount = 217944424448, Requested Resource Amount = 108972212224.\r\n2018-08-23 12:14:35,889 DEBUG [c.c.r.ResourceLimitManagerImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Checking if domain resource limits of Type 'secondary_storage' for Domain Id = 5 is exceeded: Domain Resource Limit = -1, Current Domain Resource Amount = 217944424448, Requested Resource Amount = 108972212224.\r\n2018-08-23 12:14:35,890 DEBUG [c.c.r.ResourceLimitManagerImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Checking if domain resource limits of Type 'secondary_storage' for Domain Id = 2 is exceeded: Domain Resource Limit = -1, Current Domain Resource Amount = 217944424448, Requested Resource Amount = 108972212224.\r\n2018-08-23 12:14:35,919 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) template 247 is not in store:1, type:Image\r\n2018-08-23 12:14:35,942 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) template 247 is already in store:1, type:Image\r\n2018-08-23 12:14:35,959 DEBUG [o.a.c.s.i.BaseImageStoreDriverImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Downloading template to data store 1\r\n2018-08-23 12:14:35,970 DEBUG [o.a.c.s.i.BaseImageStoreDriverImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Performing image store createTemplate async callback\r\n2018-08-23 12:14:36,003 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) getCommandHostDelegation: class org.apache.cloudstack.storage.command.DownloadCommand\r\n2018-08-23 12:14:36,003 DEBUG [c.c.h.XenServerGuru] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) We are returning the default host to execute commands because the command is not of Copy type.\r\n2018-08-23 12:14:36,004 DEBUG [o.a.c.s.RemoteHostEndPoint] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Sending command org.apache.cloudstack.storage.command.DownloadCommand to host: 6\r\n2018-08-23 12:14:36,006 DEBUG [c.c.a.t.Request] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Seq 6-1837468647967162405: Sending  { Cmd , MgmtId: 90520739393856, via: 6(s-2-VM), Ver: v1, Flags: 100011, [{\"org.apache.cloudstack.storage.command.DownloadCommand\":{\"hvm\":false,\"description\":\"test-oracledb-tmpl5\",\"checksum\":\"\",\"maxDownloadSizeInBytes\":137438953472,\"id\":247,\"resourceType\":\"TEMPLATE\",\"installPath\":\"template/tmpl/47/247\",\"_store\":{\"com.cloud.agent.api.to.NfsTO\":{\"_url\":\"nfs://secondary-storage\",\"_role\":\"Image\"}},\"url\":\"http://link to template\",\"format\":\"RAW\",\"accountId\":47,\"name\":\"247-47-8efe5f29-c3ee-3016-acd5-e4a4fb76bcfe\",\"secUrl\":\"nfs://secondary-storage\",\"wait\":0}}] }\r\n2018-08-23 12:14:36,006 DEBUG [c.c.r.ResourceLimitManagerImpl] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) Updating resource Type = template count for Account = 47 Operation = increasing Amount = 1\r\n2018-08-23 12:14:36,057 DEBUG [c.c.a.ApiServlet] (qtp745160567-15:ctx-4e0d3cf8 ctx-194618fb) (logid:f2420e15) ===END===  x.x.x.x -- GET  command=registerTemplate&response=json&projectid=62b2e0d2-5f81-4a6d-8d8b-591283f10e4a&name=test-oracledb-tmpl5&displayText=test-oracledb-tmpl5&url=http%3A%2F%2Ftemplate-host%2Fcscoracle-db.raw&zoneids=4a2e5664-edc7-4a0d-ae4a-fd56e1dc4314&format=RAW&isextractable=false&passwordEnabled=false&isdynamicallyscalable=false&osTypeId=b70da464-54f4-11e8-8821-525400843140&hypervisor=KVM&ispublic=true&requireshvm=false&isfeatured=false&isrouting=false&details%5B0%5D.hypervisortoolsversion=xenserver56&directdownload=false&checksum=&_=1535019275786\r\n2018-08-23 12:14:39,155 DEBUG [c.c.a.t.Request] (AgentManager-Handler-13:null) (logid:) Seq 6-1837468647967162405: Processing:  { Ans: , MgmtId: 90520739393856, via: 6, Ver: v1, Flags: 10, [{\"com.cloud.agent.api.storage.DownloadAnswer\":{\"jobId\":\"ba4f5e6d-3466-45d7-8eb4-381d128422d7\",\"downloadPct\":0,\"errorString\":\"Stream closed\",\"downloadStatus\":\"DOWNLOAD_ERROR\",\"downloadPath\":\"/mnt/SecStorage/447bd664-f78c-31d8-86c6-0bba5bfa390e/template/tmpl/47/247/dnld5981961215788364666tmp_\",\"installPath\":\"template/tmpl/47/247\",\"templateSize\":0,\"templatePhySicalSize\":0,\"checkSum\":\"\",\"result\":true,\"details\":\"Stream closed\",\"wait\":0}}] }\r\n2018-08-23 12:14:39,156 DEBUG [o.a.c.s.i.BaseImageStoreDriverImpl] (RemoteHostEndPoint-4:ctx-58623d61) (logid:6a92f38a) Performing image store createTemplate async callback\r\n2018-08-23 12:14:39,213 WARN  [o.a.c.alerts] (RemoteHostEndPoint-4:ctx-58623d61) (logid:6a92f38a) AlertType:: 28 | dataCenterId:: 1 | podId:: null | clusterId:: null | message:: Failed to register template: d039d030-322c-49b3-8fa6-871edee2fb4b with error: Stream closed\r\n2018-08-23 12:14:39,235 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl] (RemoteHostEndPoint-4:ctx-58623d61) (logid:6a92f38a) Failed to register template: d039d030-322c-49b3-8fa6-871edee2fb4b with error: Stream closed\r\n~~~\r\n"},"comment":{"url":"https://api.github.com/repos/apache/cloudstack/issues/comments/440267496","html_url":"https://github.com/apache/cloudstack/issues/2820#issuecomment-440267496","issue_url":"https://api.github.com/repos/apache/cloudstack/issues/2820","id":440267496,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0MDI2NzQ5Ng==","user":{"login":"DaanHoogland","id":2486961,"node_id":"MDQ6VXNlcjI0ODY5NjE=","avatar_url":"https://avatars1.githubusercontent.com/u/2486961?v=4","gravatar_id":"","url":"https://api.github.com/users/DaanHoogland","html_url":"https://github.com/DaanHoogland","followers_url":"https://api.github.com/users/DaanHoogland/followers","following_url":"https://api.github.com/users/DaanHoogland/following{/other_user}","gists_url":"https://api.github.com/users/DaanHoogland/gists{/gist_id}","starred_url":"https://api.github.com/users/DaanHoogland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DaanHoogland/subscriptions","organizations_url":"https://api.github.com/users/DaanHoogland/orgs","repos_url":"https://api.github.com/users/DaanHoogland/repos","events_url":"https://api.github.com/users/DaanHoogland/events{/privacy}","received_events_url":"https://api.github.com/users/DaanHoogland/received_events","type":"User","site_admin":false},"created_at":"2018-11-20T13:09:12Z","updated_at":"2018-11-20T13:09:12Z","author_association":"CONTRIBUTOR","body":"top of head guess: file server, network and vm are the suspects, @DennisKonrad. as you hav already pproven the isolated code of course. So my first step would be to prove it against the fileserver (testing the network is unfortunately implied on that path). If it fails go for another (type of) fileserver over the same networkpath. As a side track, invewtigating the iptables and logging on the SSVM might shed some light as well."}}
{"actor":{"display_login":"DaanHoogland"}}