{"action":"created","issue":{"url":"https://api.github.com/repos/Storj/bridge-client-javascript/issues/25","repository_url":"https://api.github.com/repos/Storj/bridge-client-javascript","labels_url":"https://api.github.com/repos/Storj/bridge-client-javascript/issues/25/labels{/name}","comments_url":"https://api.github.com/repos/Storj/bridge-client-javascript/issues/25/comments","events_url":"https://api.github.com/repos/Storj/bridge-client-javascript/issues/25/events","html_url":"https://github.com/Storj/bridge-client-javascript/issues/25","id":156810357,"number":25,"title":"Upload filesize limited to 8MB ?","user":{"login":"Th3Van","id":18688983,"avatar_url":"https://avatars.githubusercontent.com/u/18688983?v=3","gravatar_id":"","url":"https://api.github.com/users/Th3Van","html_url":"https://github.com/Th3Van","followers_url":"https://api.github.com/users/Th3Van/followers","following_url":"https://api.github.com/users/Th3Van/following{/other_user}","gists_url":"https://api.github.com/users/Th3Van/gists{/gist_id}","starred_url":"https://api.github.com/users/Th3Van/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Th3Van/subscriptions","organizations_url":"https://api.github.com/users/Th3Van/orgs","repos_url":"https://api.github.com/users/Th3Van/repos","events_url":"https://api.github.com/users/Th3Van/events{/privacy}","received_events_url":"https://api.github.com/users/Th3Van/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Storj/bridge-client-javascript/labels/bug","name":"bug","color":"EE3F46"},{"url":"https://api.github.com/repos/Storj/bridge-client-javascript/labels/in%20progress","name":"in progress","color":"ededed"}],"state":"open","locked":false,"assignee":{"login":"niahmiah","id":653102,"avatar_url":"https://avatars.githubusercontent.com/u/653102?v=3","gravatar_id":"","url":"https://api.github.com/users/niahmiah","html_url":"https://github.com/niahmiah","followers_url":"https://api.github.com/users/niahmiah/followers","following_url":"https://api.github.com/users/niahmiah/following{/other_user}","gists_url":"https://api.github.com/users/niahmiah/gists{/gist_id}","starred_url":"https://api.github.com/users/niahmiah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/niahmiah/subscriptions","organizations_url":"https://api.github.com/users/niahmiah/orgs","repos_url":"https://api.github.com/users/niahmiah/repos","events_url":"https://api.github.com/users/niahmiah/events{/privacy}","received_events_url":"https://api.github.com/users/niahmiah/received_events","type":"User","site_admin":false},"milestone":null,"comments":5,"created_at":"2016-05-25T17:46:35Z","updated_at":"2016-05-27T02:38:04Z","closed_at":null,"body":"I'm unable to upload files bigger than 8MB, using storjcli with the \"uploadfile\" option.\r\nAnything larger than that is returning with a [ERROR] and times out.\r\n\r\nI've been running some tests using storj.dk and api.storj.io bridge servers, but with same result.\r\nEvery \"accepting storage contract / CONSIGN\" are going out just fine, but the last one is always missing, if file size are greater than 8MB.\r\n\r\n**8MB file -> Storj.dk bridge :** \r\n<pre><code>Storj.dk - storjcli client - 8MB file : \r\n root@Server014:~/storjcli/script# storjcli -k VaNkEy16 uploadfile 574224fc5a3ce4e2697c30a2 ./Storj.dk-8MB-File.tmp\r\n [info]   Generating encryption key...\r\n [info]   Encrypting file \"./Storj.dk-8MB-File.tmp\"\r\n [info]   Encryption complete!\r\n [info]   Creating storage token...\r\n [info]   Storing file, hang tight!\r\n [info]   Encryption key saved to keyring.\r\n [info]   File successfully stored in bucket.\r\n [info]   Name: Storj.dk-8MB-File.tmp, Type: application/octet-stream, Size: 8388624 bytes, ID: 57459d65785767c21b431d01\r\n\r\nStorj.dk - Storj-brige server - 8MB file :\r\n {\"level\":\"info\",\"message\":\"creating PUSH token for 574224fc5a3ce4e2697c30a2\",\"timestamp\":\"2016-05-25T12:37:05.635Z\"}\r\n {\"level\":\"info\",\"message\":\"publishing message on topic \\\"0f01020202\\\"\",\"timestamp\":\"2016-05-25T12:37:05.776Z\"}\r\n {\"level\":\"info\",\"message\":\"publishing message on topic \\\"0f01020202\\\"\",\"timestamp\":\"2016-05-25T12:37:06.042Z\"}\r\n {\"level\":\"info\",\"message\":\"accepting storage contract offer from d7a1147c7398413bed31ab9a3145ac842e1b50b2\",\"timestamp\":\"2016-05-25T12:37:08.627Z\"}\r\n {\"level\":\"info\",\"message\":\"sending CONSIGN message to {\\\"protocol\\\":\\\"0.6.15\\\",\\\"address\\\":\\\"a.storj.eu\\\",\\\"port\\\":8290,\\\"nodeID\\\":\\\"d7a1147c7398413bed31ab9a3145ac842e1b50b2\\\",\\\"lastSeen\\\":1464179828655}\",\"timestamp\":\"2016-05-25T12:37:08.655Z\"}\r\n {\"level\":\"info\",\"message\":\"accepting storage contract offer from d7a1147c7398413bed31ab9a3145ac842e1b50b2\",\"timestamp\":\"2016-05-25T12:37:08.977Z\"} \r\n {\"level\":\"info\",\"message\":\"sending CONSIGN message to {\\\"protocol\\\":\\\"0.6.15\\\",\\\"address\\\":\\\"a.storj.eu\\\",\\\"port\\\":8290,\\\"nodeID\\\":\\\"d7a1147c7398413bed31ab9a3145ac842e1b50b2\\\",\\\"lastSeen\\\":1464179828994}\",\"timestamp\":\"2016-05-25T12:37:08.994Z\"}\r\n\r\n(2 x publishing message - 2 x CONSIGN message) </code></pre>\r\n\r\n**9MB file -> Storj.dk bridge :** \r\n<pre><code>Storj.dk - storjcli client - 9MB file : \r\n root@Server014:~/storjcli/script# storjcli -k VaNkEy16 uploadfile 574224fc5a3ce4e2697c30a2 ./Storj.dk-9MB-File.tmp\r\n [info]   Generating encryption key...\r\n [info]   Encrypting file \"./Storj.dk-9MB-File.tmp\"\r\n [info]   Encryption complete!\r\n [info]   Creating storage token...\r\n [info]   Storing file, hang tight!\r\n[error]  socket hang up \r\n\r\nStorj.dk - Storj-brige server - 9MB file :\r\n {\"level\":\"info\",\"message\":\"creating PUSH token for 574224fc5a3ce4e2697c30a2\",\"timestamp\":\"2016-05-25T12:41:32.181Z\"}\r\n {\"level\":\"info\",\"message\":\"publishing message on topic \\\"0f01020202\\\"\",\"timestamp\":\"2016-05-25T12:41:32.304Z\"}\r\n {\"level\":\"info\",\"message\":\"publishing message on topic \\\"0f01020202\\\"\",\"timestamp\":\"2016-05-25T12:41:32.336Z\"}\r\n {\"level\":\"info\",\"message\":\"publishing message on topic \\\"0f01020202\\\"\",\"timestamp\":\"2016-05-25T12:41:32.572Z\"}\r\n {\"level\":\"info\",\"message\":\"accepting storage contract offer from d7a1147c7398413bed31ab9a3145ac842e1b50b2\",\"timestamp\":\"2016-05-25T12:41:35.630Z\"}\r\n {\"level\":\"info\",\"message\":\"sending CONSIGN message to {\\\"protocol\\\":\\\"0.6.15\\\",\\\"address\\\":\\\"a.storj.eu\\\",\\\"port\\\":8290,\\\"nodeID\\\":\\\"d7a1147c7398413bed31ab9a3145ac842e1b50b2\\\",\\\"lastSeen\\\":1464180095648}\",\"timestamp\":\"2016-05-25T12:41:35.648Z\"}\r\n {\"level\":\"info\",\"message\":\"accepting storage contract offer from d7a1147c7398413bed31ab9a3145ac842e1b50b2\",\"timestamp\":\"2016-05-25T12:41:35.778Z\"}\r\n {\"level\":\"info\",\"message\":\"sending CONSIGN message to {\\\"protocol\\\":\\\"0.6.15\\\",\\\"address\\\":\\\"a.storj.eu\\\",\\\"port\\\":8290,\\\"nodeID\\\":\\\"d7a1147c7398413bed31ab9a3145ac842e1b50b2\\\",\\\"lastSeen\\\":1464180095840}\",\"timestamp\":\"2016-05-25T12:41:35.840Z\"}\r\n\r\n(3 x publishing message - but only 2 x CONSIGN message)\r\n</code></pre>\r\n\r\n**32MB file -> Storj.dk bridge :**\r\n<pre><code>\r\nStorj.dk - storjcli client - 32MB file : \r\n root@Server014:~/storjcli/script# storjcli -k VaNkEy16 uploadfile 574224fc5a3ce4e2697c30a2 ./Storj.dk-32MB-File.tmp\r\n [info]   Generating encryption key...\r\n [info]   Encrypting file \"./Storj.dk-32MB-File.tmp\"\r\n [info]   Encryption complete!\r\n [info]   Creating storage token...\r\n [info]   Storing file, hang tight!\r\n [error]  socket hang up\r\n\r\nStorj.dk - Storj-brige server - 32MB file :\r\n {\"level\":\"info\",\"message\":\"publishing message on topic \\\"0f01020202\\\"\",\"timestamp\":\"2016-05-25T12:48:41.872Z\"}\r\n {\"level\":\"info\",\"message\":\"publishing message on topic \\\"0f01020202\\\"\",\"timestamp\":\"2016-05-25T12:48:42.024Z\"}\r\n {\"level\":\"info\",\"message\":\"publishing message on topic \\\"0f01020202\\\"\",\"timestamp\":\"2016-05-25T12:48:42.047Z\"}\r\n {\"level\":\"info\",\"message\":\"publishing message on topic \\\"0f01020202\\\"\",\"timestamp\":\"2016-05-25T12:48:42.097Z\"}\r\n {\"level\":\"info\",\"message\":\"accepting storage contract offer from d7a1147c7398413bed31ab9a3145ac842e1b50b2\",\"timestamp\":\"2016-05-25T12:48:47.566Z\"}\r\n {\"level\":\"info\",\"message\":\"accepting storage contract offer from d7a1147c7398413bed31ab9a3145ac842e1b50b2\",\"timestamp\":\"2016-05-25T12:48:47.584Z\"}\r\n {\"level\":\"info\",\"message\":\"accepting storage contract offer from d7a1147c7398413bed31ab9a3145ac842e1b50b2\",\"timestamp\":\"2016-05-25T12:48:47.596Z\"}\r\n {\"level\":\"info\",\"message\":\"sending CONSIGN message to {\\\"protocol\\\":\\\"0.6.15\\\",\\\"address\\\":\\\"a.storj.eu\\\",\\\"port\\\":8290,\\\"nodeID\\\":\\\"d7a1147c7398413bed31ab9a3145ac842e1b50b2\\\",\\\"lastSeen\\\":1464180527623}\",\"timestamp\":\"2016-05-25T12:48:47.623Z\"}\r\n {\"level\":\"info\",\"message\":\"sending CONSIGN message to {\\\"protocol\\\":\\\"0.6.15\\\",\\\"address\\\":\\\"a.storj.eu\\\",\\\"port\\\":8290,\\\"nodeID\\\":\\\"d7a1147c7398413bed31ab9a3145ac842e1b50b2\\\",\\\"lastSeen\\\":1464180527643}\",\"timestamp\":\"2016-05-25T12:48:47.643Z\"}\r\n {\"level\":\"info\",\"message\":\"sending CONSIGN message to {\\\"protocol\\\":\\\"0.6.15\\\",\\\"address\\\":\\\"a.storj.eu\\\",\\\"port\\\":8290,\\\"nodeID\\\":\\\"d7a1147c7398413bed31ab9a3145ac842e1b50b2\\\",\\\"lastSeen\\\":1464180527664}\",\"timestamp\":\"2016-05-25T12:48:47.664Z\"}\r\n\r\n(4 x publishing message - but only 3 x CONSIGN message)\r\n</code></pre>\r\n\r\n**8/9/32MB file -> api.storj.io bridge :**\r\n<pre><code>\r\napi.storj.io - storjcli client - 8MB file : \r\n root@Server022:~/bridge-client-javascript/script# storjcli -k VaNkEy16 uploadfile 57449f473142a39c6f541eda ./Storj.dk-8MB-File.tmp\r\n [info]   Generating encryption key...\r\n [info]   Encrypting file \"./Storj.dk-8MB-File.tmp\"\r\n [info]   Encryption complete!\r\n [info]   Creating storage token...\r\n [info]   Storing file, hang tight!\r\n [info]   Encryption key saved to keyring.\r\n [info]   File successfully stored in bucket.\r\n [info]   Name: Storj.dk-8MB-File.tmp, Type: application/octet-stream, Size: 8388624 bytes, ID: 574585603142a39c6f5433a9\r\n\r\napi.storj.io - storjcli client - 9MB file : \r\n root@Server022:~/bridge-client-javascript/script# storjcli -k VaNkEy16 uploadfile 57449f473142a39c6f541eda ./Storj.dk-9MB-File.tmp\r\n [info]   Generating encryption key...\r\n [info]   Encrypting file \"./Storj.dk-9MB-File.tmp\"\r\n [info]   Encryption complete!\r\n [info]   Creating storage token...\r\n [info]   Storing file, hang tight!\r\n [error]  E11000 duplicate key error collection: bridge.usernonces index: user_1_nonce_1 dup key: { : \"****removed******\", : \"09c3373b-25b7-4180-85aa-e177551d1f70\" }\r\n\r\napi.storj.io - storjcli client - 32MB file : \r\n root@Server022:~/bridge-client-javascript/script# storjcli -k VaNkEy16 uploadfile 57449f473142a39c6f541eda ./Storj.dk-10MB-File.tmp\r\n [info]   Generating encryption key...\r\n [info]   Encrypting file \"./Storj.dk-10MB-File.tmp\"\r\n [info]   Encryption complete!\r\n [info]   Creating storage token...\r\n [info]   Storing file, hang tight!\r\n [error]  E11000 duplicate key error collection: bridge.usernonces index: user_1_nonce_1 dup key: { : \"****removed******\", : \"02911fab-ebc3-4d97-b65a-3097c5abf31a\" }\r\n</code></pre>\r\n\r\n[Storjcli-bridge upload file test - server014.storj.dk.txt](https://github.com/Storj/bridge-client-javascript/files/282534/Storjcli-bridge.upload.file.test.-.server014.storj.dk.txt)\r\n[Storjcli-bridge upload file test - api.storj.io.txt](https://github.com/Storj/bridge-client-javascript/files/282533/Storjcli-bridge.upload.file.test.-.api.storj.io.txt)\r\n\r\n[Storj.dk-bridge-log-files-Th3Van.zip](https://github.com/Storj/bridge-client-javascript/files/282547/Storj.dk-bridge-log-files-Th3Van.zip)\r\n\r\nRegards,\r\nTh3Van\r\nstorj.dk"},"comment":{"url":"https://api.github.com/repos/Storj/bridge-client-javascript/issues/comments/222046654","html_url":"https://github.com/Storj/bridge-client-javascript/issues/25#issuecomment-222046654","issue_url":"https://api.github.com/repos/Storj/bridge-client-javascript/issues/25","id":222046654,"user":{"login":"gordonwritescode","id":1188875,"avatar_url":"https://avatars.githubusercontent.com/u/1188875?v=3","gravatar_id":"","url":"https://api.github.com/users/gordonwritescode","html_url":"https://github.com/gordonwritescode","followers_url":"https://api.github.com/users/gordonwritescode/followers","following_url":"https://api.github.com/users/gordonwritescode/following{/other_user}","gists_url":"https://api.github.com/users/gordonwritescode/gists{/gist_id}","starred_url":"https://api.github.com/users/gordonwritescode/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gordonwritescode/subscriptions","organizations_url":"https://api.github.com/users/gordonwritescode/orgs","repos_url":"https://api.github.com/users/gordonwritescode/repos","events_url":"https://api.github.com/users/gordonwritescode/events{/privacy}","received_events_url":"https://api.github.com/users/gordonwritescode/received_events","type":"User","site_admin":false},"created_at":"2016-05-27T02:38:04Z","updated_at":"2016-05-27T02:38:04Z","body":"@phutchins @niahmiah if load is the issue, then that happened sooner than I thought. Since the API and the network interface run in the same process, the network chatter hogs the event loop, which likely causes these requests to the API to get back seated, which causes the enormous latency and the resulting nginx timeouts. \n\nI think that we need to pull the `RenterInterface` usage in bridge out into a micro service so we can run a little swarm of those bad boys and bridge can select one during an API request. This would also alleviate some of the issues with using the API host as a network seed for farmers. \n\nI wanted to write a [dnode](https://github.com/substack/dnode) plugin for core's network module anyway, so this would be a good reason to do so. How does that sound to you guys, assuming that this is in fact the issue?"}}
{"actor":{"display_login":"gordonwritescode"}}