{"action":"created","comment":{"url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/comments/63512683","id":63512683,"diff_hunk":"@@ -556,6 +552,18 @@ C#\n .. literalinclude:: _static/crawlera-proxy.cs\n     :language: csharp\n \n+Best practices\n+==============\n+We have prepared several best practices when using crawlera smart proxy.\n+\n+Set download timeout\n+--------------------\n+One of the most common problems our users have is too low download timeout in their scrapping application. Handling one request in crawlera can take a long time. This happens due to `throtling <http://doc.scrapinghub.com/crawlera.html#why-are-requests-slower-through-crawlera-as-opposed-to-using-proxies-directly>`_. Crawlera will try several times to process your request with different slaves and different delay time. Recommended value is **300s**. If you are using scrapy please check our `example configuration <http://doc.scrapinghub.com/crawlera.html#using-crawlera-with-scrapy>`_.","path":"crawlera.rst","position":39,"original_position":39,"commit_id":"a806867aa6caa33f975a346a4f2f9ab5f4a8f54b","original_commit_id":"a806867aa6caa33f975a346a4f2f9ab5f4a8f54b","user":{"login":"oltarasenko","id":48589,"avatar_url":"https://avatars.githubusercontent.com/u/48589?v=3","gravatar_id":"","url":"https://api.github.com/users/oltarasenko","html_url":"https://github.com/oltarasenko","followers_url":"https://api.github.com/users/oltarasenko/followers","following_url":"https://api.github.com/users/oltarasenko/following{/other_user}","gists_url":"https://api.github.com/users/oltarasenko/gists{/gist_id}","starred_url":"https://api.github.com/users/oltarasenko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oltarasenko/subscriptions","organizations_url":"https://api.github.com/users/oltarasenko/orgs","repos_url":"https://api.github.com/users/oltarasenko/repos","events_url":"https://api.github.com/users/oltarasenko/events{/privacy}","received_events_url":"https://api.github.com/users/oltarasenko/received_events","type":"User","site_admin":false},"body":"I want to double check wording here, e.g. `Crawlera will try several times to process your request with different slaves and different delay time` sounds incorrect to me, and I would suggest `Crawlera will try to process your request with different slaves and delay time` ","created_at":"2016-05-17T12:30:32Z","updated_at":"2016-05-17T12:30:32Z","html_url":"https://github.com/scrapinghub/doc.scrapinghub.com/pull/137#discussion_r63512683","pull_request_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/137","_links":{"self":{"href":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/comments/63512683"},"html":{"href":"https://github.com/scrapinghub/doc.scrapinghub.com/pull/137#discussion_r63512683"},"pull_request":{"href":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/137"}}},"pull_request":{"url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/137","id":70022085,"html_url":"https://github.com/scrapinghub/doc.scrapinghub.com/pull/137","diff_url":"https://github.com/scrapinghub/doc.scrapinghub.com/pull/137.diff","patch_url":"https://github.com/scrapinghub/doc.scrapinghub.com/pull/137.patch","issue_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues/137","number":137,"state":"open","locked":false,"title":"Add best practices chapter and fix some default values","user":{"login":"amleczko","id":224414,"avatar_url":"https://avatars.githubusercontent.com/u/224414?v=3","gravatar_id":"","url":"https://api.github.com/users/amleczko","html_url":"https://github.com/amleczko","followers_url":"https://api.github.com/users/amleczko/followers","following_url":"https://api.github.com/users/amleczko/following{/other_user}","gists_url":"https://api.github.com/users/amleczko/gists{/gist_id}","starred_url":"https://api.github.com/users/amleczko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amleczko/subscriptions","organizations_url":"https://api.github.com/users/amleczko/orgs","repos_url":"https://api.github.com/users/amleczko/repos","events_url":"https://api.github.com/users/amleczko/events{/privacy}","received_events_url":"https://api.github.com/users/amleczko/received_events","type":"User","site_admin":false},"body":"","created_at":"2016-05-13T16:20:22Z","updated_at":"2016-05-17T12:30:32Z","closed_at":null,"merged_at":null,"merge_commit_sha":"7bb9d2b7fb23ef189b9f3200b7291560b09ee669","assignee":{"login":"oltarasenko","id":48589,"avatar_url":"https://avatars.githubusercontent.com/u/48589?v=3","gravatar_id":"","url":"https://api.github.com/users/oltarasenko","html_url":"https://github.com/oltarasenko","followers_url":"https://api.github.com/users/oltarasenko/followers","following_url":"https://api.github.com/users/oltarasenko/following{/other_user}","gists_url":"https://api.github.com/users/oltarasenko/gists{/gist_id}","starred_url":"https://api.github.com/users/oltarasenko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oltarasenko/subscriptions","organizations_url":"https://api.github.com/users/oltarasenko/orgs","repos_url":"https://api.github.com/users/oltarasenko/repos","events_url":"https://api.github.com/users/oltarasenko/events{/privacy}","received_events_url":"https://api.github.com/users/oltarasenko/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/137/commits","review_comments_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/137/comments","review_comment_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/comments{/number}","comments_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues/137/comments","statuses_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/statuses/a806867aa6caa33f975a346a4f2f9ab5f4a8f54b","head":{"label":"scrapinghub:crawlera-bestpractices","ref":"crawlera-bestpractices","sha":"a806867aa6caa33f975a346a4f2f9ab5f4a8f54b","user":{"login":"scrapinghub","id":699596,"avatar_url":"https://avatars.githubusercontent.com/u/699596?v=3","gravatar_id":"","url":"https://api.github.com/users/scrapinghub","html_url":"https://github.com/scrapinghub","followers_url":"https://api.github.com/users/scrapinghub/followers","following_url":"https://api.github.com/users/scrapinghub/following{/other_user}","gists_url":"https://api.github.com/users/scrapinghub/gists{/gist_id}","starred_url":"https://api.github.com/users/scrapinghub/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scrapinghub/subscriptions","organizations_url":"https://api.github.com/users/scrapinghub/orgs","repos_url":"https://api.github.com/users/scrapinghub/repos","events_url":"https://api.github.com/users/scrapinghub/events{/privacy}","received_events_url":"https://api.github.com/users/scrapinghub/received_events","type":"Organization","site_admin":false},"repo":{"id":12142360,"name":"doc.scrapinghub.com","full_name":"scrapinghub/doc.scrapinghub.com","owner":{"login":"scrapinghub","id":699596,"avatar_url":"https://avatars.githubusercontent.com/u/699596?v=3","gravatar_id":"","url":"https://api.github.com/users/scrapinghub","html_url":"https://github.com/scrapinghub","followers_url":"https://api.github.com/users/scrapinghub/followers","following_url":"https://api.github.com/users/scrapinghub/following{/other_user}","gists_url":"https://api.github.com/users/scrapinghub/gists{/gist_id}","starred_url":"https://api.github.com/users/scrapinghub/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scrapinghub/subscriptions","organizations_url":"https://api.github.com/users/scrapinghub/orgs","repos_url":"https://api.github.com/users/scrapinghub/repos","events_url":"https://api.github.com/users/scrapinghub/events{/privacy}","received_events_url":"https://api.github.com/users/scrapinghub/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/scrapinghub/doc.scrapinghub.com","description":"Scrapinghub Documentation","fork":false,"url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com","forks_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/forks","keys_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/keys{/key_id}","collaborators_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/teams","hooks_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/hooks","issue_events_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues/events{/number}","events_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/events","assignees_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/assignees{/user}","branches_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/branches{/branch}","tags_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/tags","blobs_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/refs{/sha}","trees_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/trees{/sha}","statuses_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/statuses/{sha}","languages_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/languages","stargazers_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/stargazers","contributors_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/contributors","subscribers_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/subscribers","subscription_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/subscription","commits_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/commits{/sha}","git_commits_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/commits{/sha}","comments_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/comments{/number}","issue_comment_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues/comments{/number}","contents_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/contents/{+path}","compare_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/compare/{base}...{head}","merges_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/merges","archive_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/downloads","issues_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues{/number}","pulls_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls{/number}","milestones_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/milestones{/number}","notifications_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/labels{/name}","releases_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/releases{/id}","deployments_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/deployments","created_at":"2013-08-15T19:48:50Z","updated_at":"2016-03-12T01:22:43Z","pushed_at":"2016-05-17T12:28:30Z","git_url":"git://github.com/scrapinghub/doc.scrapinghub.com.git","ssh_url":"git@github.com:scrapinghub/doc.scrapinghub.com.git","clone_url":"https://github.com/scrapinghub/doc.scrapinghub.com.git","svn_url":"https://github.com/scrapinghub/doc.scrapinghub.com","homepage":"http://doc.scrapinghub.com","size":12381,"stargazers_count":2,"watchers_count":2,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":6,"mirror_url":null,"open_issues_count":3,"forks":6,"open_issues":3,"watchers":2,"default_branch":"master"}},"base":{"label":"scrapinghub:master","ref":"master","sha":"ff0d916d62f3b3f109948c28f1f67bca278b481f","user":{"login":"scrapinghub","id":699596,"avatar_url":"https://avatars.githubusercontent.com/u/699596?v=3","gravatar_id":"","url":"https://api.github.com/users/scrapinghub","html_url":"https://github.com/scrapinghub","followers_url":"https://api.github.com/users/scrapinghub/followers","following_url":"https://api.github.com/users/scrapinghub/following{/other_user}","gists_url":"https://api.github.com/users/scrapinghub/gists{/gist_id}","starred_url":"https://api.github.com/users/scrapinghub/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scrapinghub/subscriptions","organizations_url":"https://api.github.com/users/scrapinghub/orgs","repos_url":"https://api.github.com/users/scrapinghub/repos","events_url":"https://api.github.com/users/scrapinghub/events{/privacy}","received_events_url":"https://api.github.com/users/scrapinghub/received_events","type":"Organization","site_admin":false},"repo":{"id":12142360,"name":"doc.scrapinghub.com","full_name":"scrapinghub/doc.scrapinghub.com","owner":{"login":"scrapinghub","id":699596,"avatar_url":"https://avatars.githubusercontent.com/u/699596?v=3","gravatar_id":"","url":"https://api.github.com/users/scrapinghub","html_url":"https://github.com/scrapinghub","followers_url":"https://api.github.com/users/scrapinghub/followers","following_url":"https://api.github.com/users/scrapinghub/following{/other_user}","gists_url":"https://api.github.com/users/scrapinghub/gists{/gist_id}","starred_url":"https://api.github.com/users/scrapinghub/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scrapinghub/subscriptions","organizations_url":"https://api.github.com/users/scrapinghub/orgs","repos_url":"https://api.github.com/users/scrapinghub/repos","events_url":"https://api.github.com/users/scrapinghub/events{/privacy}","received_events_url":"https://api.github.com/users/scrapinghub/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/scrapinghub/doc.scrapinghub.com","description":"Scrapinghub Documentation","fork":false,"url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com","forks_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/forks","keys_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/keys{/key_id}","collaborators_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/teams","hooks_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/hooks","issue_events_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues/events{/number}","events_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/events","assignees_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/assignees{/user}","branches_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/branches{/branch}","tags_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/tags","blobs_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/refs{/sha}","trees_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/trees{/sha}","statuses_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/statuses/{sha}","languages_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/languages","stargazers_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/stargazers","contributors_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/contributors","subscribers_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/subscribers","subscription_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/subscription","commits_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/commits{/sha}","git_commits_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/git/commits{/sha}","comments_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/comments{/number}","issue_comment_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues/comments{/number}","contents_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/contents/{+path}","compare_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/compare/{base}...{head}","merges_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/merges","archive_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/downloads","issues_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues{/number}","pulls_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls{/number}","milestones_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/milestones{/number}","notifications_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/labels{/name}","releases_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/releases{/id}","deployments_url":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/deployments","created_at":"2013-08-15T19:48:50Z","updated_at":"2016-03-12T01:22:43Z","pushed_at":"2016-05-17T12:28:30Z","git_url":"git://github.com/scrapinghub/doc.scrapinghub.com.git","ssh_url":"git@github.com:scrapinghub/doc.scrapinghub.com.git","clone_url":"https://github.com/scrapinghub/doc.scrapinghub.com.git","svn_url":"https://github.com/scrapinghub/doc.scrapinghub.com","homepage":"http://doc.scrapinghub.com","size":12381,"stargazers_count":2,"watchers_count":2,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":6,"mirror_url":null,"open_issues_count":3,"forks":6,"open_issues":3,"watchers":2,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/137"},"html":{"href":"https://github.com/scrapinghub/doc.scrapinghub.com/pull/137"},"issue":{"href":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues/137"},"comments":{"href":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/issues/137/comments"},"review_comments":{"href":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/137/comments"},"review_comment":{"href":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/pulls/137/commits"},"statuses":{"href":"https://api.github.com/repos/scrapinghub/doc.scrapinghub.com/statuses/a806867aa6caa33f975a346a4f2f9ab5f4a8f54b"}}}}