{"action":"created","issue":{"url":"https://api.github.com/repos/code4craft/webmagic/issues/265","repository_url":"https://api.github.com/repos/code4craft/webmagic","labels_url":"https://api.github.com/repos/code4craft/webmagic/issues/265/labels{/name}","comments_url":"https://api.github.com/repos/code4craft/webmagic/issues/265/comments","events_url":"https://api.github.com/repos/code4craft/webmagic/issues/265/events","html_url":"https://github.com/code4craft/webmagic/issues/265","id":137862362,"number":265,"title":"目前的sample都是一个类一个爬虫，想多个爬虫协作怎么处理","user":{"login":"tankilo","id":10413284,"avatar_url":"https://avatars.githubusercontent.com/u/10413284?v=3","gravatar_id":"","url":"https://api.github.com/users/tankilo","html_url":"https://github.com/tankilo","followers_url":"https://api.github.com/users/tankilo/followers","following_url":"https://api.github.com/users/tankilo/following{/other_user}","gists_url":"https://api.github.com/users/tankilo/gists{/gist_id}","starred_url":"https://api.github.com/users/tankilo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tankilo/subscriptions","organizations_url":"https://api.github.com/users/tankilo/orgs","repos_url":"https://api.github.com/users/tankilo/repos","events_url":"https://api.github.com/users/tankilo/events{/privacy}","received_events_url":"https://api.github.com/users/tankilo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-03-02T12:31:58Z","updated_at":"2016-03-02T14:10:56Z","closed_at":null,"body":"文档的列表详情示例，爬新浪微博的，运行了下，自己改了下，发现总是爬一个列表页，然后把列表里的每项爬到，然后再爬下一个列表页。想一个线程爬列表页，然后爬出的列表给给另一个线程爬详情页。生产者消费者的意思。"},"comment":{"url":"https://api.github.com/repos/code4craft/webmagic/issues/comments/191250804","html_url":"https://github.com/code4craft/webmagic/issues/265#issuecomment-191250804","issue_url":"https://api.github.com/repos/code4craft/webmagic/issues/265","id":191250804,"user":{"login":"tankilo","id":10413284,"avatar_url":"https://avatars.githubusercontent.com/u/10413284?v=3","gravatar_id":"","url":"https://api.github.com/users/tankilo","html_url":"https://github.com/tankilo","followers_url":"https://api.github.com/users/tankilo/followers","following_url":"https://api.github.com/users/tankilo/following{/other_user}","gists_url":"https://api.github.com/users/tankilo/gists{/gist_id}","starred_url":"https://api.github.com/users/tankilo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tankilo/subscriptions","organizations_url":"https://api.github.com/users/tankilo/orgs","repos_url":"https://api.github.com/users/tankilo/repos","events_url":"https://api.github.com/users/tankilo/events{/privacy}","received_events_url":"https://api.github.com/users/tankilo/received_events","type":"User","site_admin":false},"created_at":"2016-03-02T14:10:56Z","updated_at":"2016-03-02T14:10:56Z","body":"知道了，用LinkedBlockingQueue来在两个线程间传递列表url\r\nus.codecraft.webmagic.Spider.addUrl(String...)\r\nus.codecraft.webmagic.Spider.setExitWhenComplete(boolean)"}}