{"action":"created","comment":{"url":"https://api.github.com/repos/ethereum/go-ethereum/pulls/comments/75748969","id":75748969,"diff_hunk":"@@ -68,30 +71,38 @@ type TxPool struct {\n \tevents       event.Subscription\n \tlocalTx      *txSet\n \tmu           sync.RWMutex\n-\tpending      map[common.Hash]*types.Transaction // processable transactions\n-\tqueue        map[common.Address]map[common.Hash]*types.Transaction\n \n-\twg sync.WaitGroup // for shutdown sync\n+\tpending map[common.Address]*txList         // All currently processable transactions\n+\tqueue   map[common.Address]*txList         // Queued but non-processable transactions\n+\tall     map[common.Hash]*types.Transaction // All transactions to allow lookups\n+\tbeats   map[common.Address]time.Time       // Last heatbeat from each known account","path":"core/tx_pool.go","position":25,"original_position":25,"commit_id":"9266bc57bc688dbf53d14ac2970188738742cb60","original_commit_id":"9266bc57bc688dbf53d14ac2970188738742cb60","user":{"login":"awishformore","id":3630494,"avatar_url":"https://avatars.githubusercontent.com/u/3630494?v=3","gravatar_id":"","url":"https://api.github.com/users/awishformore","html_url":"https://github.com/awishformore","followers_url":"https://api.github.com/users/awishformore/followers","following_url":"https://api.github.com/users/awishformore/following{/other_user}","gists_url":"https://api.github.com/users/awishformore/gists{/gist_id}","starred_url":"https://api.github.com/users/awishformore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/awishformore/subscriptions","organizations_url":"https://api.github.com/users/awishformore/orgs","repos_url":"https://api.github.com/users/awishformore/repos","events_url":"https://api.github.com/users/awishformore/events{/privacy}","received_events_url":"https://api.github.com/users/awishformore/received_events","type":"User","site_admin":false},"body":"`heatbeat` -> `heartbeat`","created_at":"2016-08-22T20:11:06Z","updated_at":"2016-08-22T20:11:06Z","html_url":"https://github.com/ethereum/go-ethereum/pull/2742#discussion_r75748969","pull_request_url":"https://api.github.com/repos/ethereum/go-ethereum/pulls/2742","_links":{"self":{"href":"https://api.github.com/repos/ethereum/go-ethereum/pulls/comments/75748969"},"html":{"href":"https://github.com/ethereum/go-ethereum/pull/2742#discussion_r75748969"},"pull_request":{"href":"https://api.github.com/repos/ethereum/go-ethereum/pulls/2742"}}},"pull_request":{"url":"https://api.github.com/repos/ethereum/go-ethereum/pulls/2742","id":75332126,"html_url":"https://github.com/ethereum/go-ethereum/pull/2742","diff_url":"https://github.com/ethereum/go-ethereum/pull/2742.diff","patch_url":"https://github.com/ethereum/go-ethereum/pull/2742.patch","issue_url":"https://api.github.com/repos/ethereum/go-ethereum/issues/2742","number":2742,"state":"open","locked":false,"title":"Transaction pool optimizations","user":{"login":"karalabe","id":129561,"avatar_url":"https://avatars.githubusercontent.com/u/129561?v=3","gravatar_id":"","url":"https://api.github.com/users/karalabe","html_url":"https://github.com/karalabe","followers_url":"https://api.github.com/users/karalabe/followers","following_url":"https://api.github.com/users/karalabe/following{/other_user}","gists_url":"https://api.github.com/users/karalabe/gists{/gist_id}","starred_url":"https://api.github.com/users/karalabe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/karalabe/subscriptions","organizations_url":"https://api.github.com/users/karalabe/orgs","repos_url":"https://api.github.com/users/karalabe/repos","events_url":"https://api.github.com/users/karalabe/events{/privacy}","received_events_url":"https://api.github.com/users/karalabe/received_events","type":"User","site_admin":false},"body":"##### Commit `core, eth, miner: only retain 1 tx/nonce, remove bad ones`\r\n\r\nOur transaction pool currently maintains arbitrarily many transactions with a given nonce. This isn't the best because only the highest paying transaction will be executed either way, so everything else is just wasting resources. This PR reworks the txpool internals so that instead of using a set of transactions (that we cannot do specialized checks on) it uses per-account nonce-indexed transaction list maintenance. This allows any newly added transaction to be immediately compared to existing ones and only the best retained.\r\n\r\nThe PR also fixes another bug in the worker whereby invalid transactions were not removed from the pool.\r\n\r\n##### Commit `core, eth, internal, miner: optimize txpool for quick ops`\r\n\r\nEven though the previous commit changed the internal representation of pending/queued transactions from flat sets to per-account nonce-sorted lists, most of the transaction pool algorithms still operated on them as if they were unsorted, loosing precious performance.\r\n\r\nThis commit reworks the actual operational implementation of the transaction pool so that it takes advantage of the already sorted representation of the transactions. It further enhances the nonce-gapped queued transactions with a heap-based nonce index to allow quickly locating processable transactions instead of needing to always verify every transaction individually.\r\n\r\nThe PR also caches sorted versions of the pending/future queues so that multiple consecutive queries without actual data mutations are fast. Similarly, to avoid the costly balance filtering (dropping txs which cannot be funded by the sender) during every cycle, each accounts' transaction list also caches the cost of the highest transaction. This ensures that if the account balance is higher that the cost of the most expensive transaction, there's no need to individually check all of them.\r\n\r\n##### Commit `core/types, miner: switch over to the grouped tx sets`\r\n\r\nThe miner currently first sorted the entire list of pending transactions, then iterated it and incorporated them into the blocks one by one. If however a transaction from a specific account could not be executed (failed, out of gas, low price), all subsequent transactions needed to be checked against various ignore lists and dropped.\r\n\r\nSince this PR already maintains and delivers transactions on a per-account-nonce-sorted basis, its much more optimal to do the sorting on the fly, pulling the best transaction for incorporation but only inserting the next from the account if all went well. In the case where a transaction fails for some reason, it is simply dropped from the current queue, preventing any other txs from the same account ever entering in its place and wasting time (nonce error).\r\n\r\n##### Commit `core: add upper bound on the queued transactions`\r\n\r\nCurrently there is no upper limit on the total number of queued (non executable) transaction in the transaction pool, only a per account limit. This could lead to nasty stuff if someone were to create thousands of accounts to spam transactions with.\r\n\r\nThis commit places an upper limit of 64K transactions that's queueable, after which transactions from the account with the least recent activity (i.e. least recently executed transaction) are starting to be dropped. Further there's also a periodic (1 minute) check that iterates over all queued transactions and drops the stale ones (older than 3 hours) to ensure that we don't accumulate junk over time.","created_at":"2016-06-27T15:33:27Z","updated_at":"2016-08-22T20:11:06Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3e4405361a5925d7650b60bb0cd1ed4039dc2c5f","assignee":{"login":"fjl","id":6915,"avatar_url":"https://avatars.githubusercontent.com/u/6915?v=3","gravatar_id":"","url":"https://api.github.com/users/fjl","html_url":"https://github.com/fjl","followers_url":"https://api.github.com/users/fjl/followers","following_url":"https://api.github.com/users/fjl/following{/other_user}","gists_url":"https://api.github.com/users/fjl/gists{/gist_id}","starred_url":"https://api.github.com/users/fjl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fjl/subscriptions","organizations_url":"https://api.github.com/users/fjl/orgs","repos_url":"https://api.github.com/users/fjl/repos","events_url":"https://api.github.com/users/fjl/events{/privacy}","received_events_url":"https://api.github.com/users/fjl/received_events","type":"User","site_admin":false},"assignees":[{"login":"fjl","id":6915,"avatar_url":"https://avatars.githubusercontent.com/u/6915?v=3","gravatar_id":"","url":"https://api.github.com/users/fjl","html_url":"https://github.com/fjl","followers_url":"https://api.github.com/users/fjl/followers","following_url":"https://api.github.com/users/fjl/following{/other_user}","gists_url":"https://api.github.com/users/fjl/gists{/gist_id}","starred_url":"https://api.github.com/users/fjl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fjl/subscriptions","organizations_url":"https://api.github.com/users/fjl/orgs","repos_url":"https://api.github.com/users/fjl/repos","events_url":"https://api.github.com/users/fjl/events{/privacy}","received_events_url":"https://api.github.com/users/fjl/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/ethereum/go-ethereum/milestones/26","html_url":"https://github.com/ethereum/go-ethereum/milestones/1.5.0","labels_url":"https://api.github.com/repos/ethereum/go-ethereum/milestones/26/labels","id":1593424,"number":26,"title":"1.5.0","description":"","creator":{"login":"karalabe","id":129561,"avatar_url":"https://avatars.githubusercontent.com/u/129561?v=3","gravatar_id":"","url":"https://api.github.com/users/karalabe","html_url":"https://github.com/karalabe","followers_url":"https://api.github.com/users/karalabe/followers","following_url":"https://api.github.com/users/karalabe/following{/other_user}","gists_url":"https://api.github.com/users/karalabe/gists{/gist_id}","starred_url":"https://api.github.com/users/karalabe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/karalabe/subscriptions","organizations_url":"https://api.github.com/users/karalabe/orgs","repos_url":"https://api.github.com/users/karalabe/repos","events_url":"https://api.github.com/users/karalabe/events{/privacy}","received_events_url":"https://api.github.com/users/karalabe/received_events","type":"User","site_admin":false},"open_issues":29,"closed_issues":23,"state":"open","created_at":"2016-02-19T15:55:33Z","updated_at":"2016-08-22T13:33:00Z","due_on":null,"closed_at":null},"commits_url":"https://api.github.com/repos/ethereum/go-ethereum/pulls/2742/commits","review_comments_url":"https://api.github.com/repos/ethereum/go-ethereum/pulls/2742/comments","review_comment_url":"https://api.github.com/repos/ethereum/go-ethereum/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ethereum/go-ethereum/issues/2742/comments","statuses_url":"https://api.github.com/repos/ethereum/go-ethereum/statuses/9266bc57bc688dbf53d14ac2970188738742cb60","head":{"label":"karalabe:tx-spam-protection","ref":"tx-spam-protection","sha":"9266bc57bc688dbf53d14ac2970188738742cb60","user":{"login":"karalabe","id":129561,"avatar_url":"https://avatars.githubusercontent.com/u/129561?v=3","gravatar_id":"","url":"https://api.github.com/users/karalabe","html_url":"https://github.com/karalabe","followers_url":"https://api.github.com/users/karalabe/followers","following_url":"https://api.github.com/users/karalabe/following{/other_user}","gists_url":"https://api.github.com/users/karalabe/gists{/gist_id}","starred_url":"https://api.github.com/users/karalabe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/karalabe/subscriptions","organizations_url":"https://api.github.com/users/karalabe/orgs","repos_url":"https://api.github.com/users/karalabe/repos","events_url":"https://api.github.com/users/karalabe/events{/privacy}","received_events_url":"https://api.github.com/users/karalabe/received_events","type":"User","site_admin":false},"repo":{"id":33668567,"name":"go-ethereum","full_name":"karalabe/go-ethereum","owner":{"login":"karalabe","id":129561,"avatar_url":"https://avatars.githubusercontent.com/u/129561?v=3","gravatar_id":"","url":"https://api.github.com/users/karalabe","html_url":"https://github.com/karalabe","followers_url":"https://api.github.com/users/karalabe/followers","following_url":"https://api.github.com/users/karalabe/following{/other_user}","gists_url":"https://api.github.com/users/karalabe/gists{/gist_id}","starred_url":"https://api.github.com/users/karalabe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/karalabe/subscriptions","organizations_url":"https://api.github.com/users/karalabe/orgs","repos_url":"https://api.github.com/users/karalabe/repos","events_url":"https://api.github.com/users/karalabe/events{/privacy}","received_events_url":"https://api.github.com/users/karalabe/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/karalabe/go-ethereum","description":"Jeffrey Wilcke's Go implementation of the Ethereum y/w paper","fork":true,"url":"https://api.github.com/repos/karalabe/go-ethereum","forks_url":"https://api.github.com/repos/karalabe/go-ethereum/forks","keys_url":"https://api.github.com/repos/karalabe/go-ethereum/keys{/key_id}","collaborators_url":"https://api.github.com/repos/karalabe/go-ethereum/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/karalabe/go-ethereum/teams","hooks_url":"https://api.github.com/repos/karalabe/go-ethereum/hooks","issue_events_url":"https://api.github.com/repos/karalabe/go-ethereum/issues/events{/number}","events_url":"https://api.github.com/repos/karalabe/go-ethereum/events","assignees_url":"https://api.github.com/repos/karalabe/go-ethereum/assignees{/user}","branches_url":"https://api.github.com/repos/karalabe/go-ethereum/branches{/branch}","tags_url":"https://api.github.com/repos/karalabe/go-ethereum/tags","blobs_url":"https://api.github.com/repos/karalabe/go-ethereum/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/karalabe/go-ethereum/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/karalabe/go-ethereum/git/refs{/sha}","trees_url":"https://api.github.com/repos/karalabe/go-ethereum/git/trees{/sha}","statuses_url":"https://api.github.com/repos/karalabe/go-ethereum/statuses/{sha}","languages_url":"https://api.github.com/repos/karalabe/go-ethereum/languages","stargazers_url":"https://api.github.com/repos/karalabe/go-ethereum/stargazers","contributors_url":"https://api.github.com/repos/karalabe/go-ethereum/contributors","subscribers_url":"https://api.github.com/repos/karalabe/go-ethereum/subscribers","subscription_url":"https://api.github.com/repos/karalabe/go-ethereum/subscription","commits_url":"https://api.github.com/repos/karalabe/go-ethereum/commits{/sha}","git_commits_url":"https://api.github.com/repos/karalabe/go-ethereum/git/commits{/sha}","comments_url":"https://api.github.com/repos/karalabe/go-ethereum/comments{/number}","issue_comment_url":"https://api.github.com/repos/karalabe/go-ethereum/issues/comments{/number}","contents_url":"https://api.github.com/repos/karalabe/go-ethereum/contents/{+path}","compare_url":"https://api.github.com/repos/karalabe/go-ethereum/compare/{base}...{head}","merges_url":"https://api.github.com/repos/karalabe/go-ethereum/merges","archive_url":"https://api.github.com/repos/karalabe/go-ethereum/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/karalabe/go-ethereum/downloads","issues_url":"https://api.github.com/repos/karalabe/go-ethereum/issues{/number}","pulls_url":"https://api.github.com/repos/karalabe/go-ethereum/pulls{/number}","milestones_url":"https://api.github.com/repos/karalabe/go-ethereum/milestones{/number}","notifications_url":"https://api.github.com/repos/karalabe/go-ethereum/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/karalabe/go-ethereum/labels{/name}","releases_url":"https://api.github.com/repos/karalabe/go-ethereum/releases{/id}","deployments_url":"https://api.github.com/repos/karalabe/go-ethereum/deployments","created_at":"2015-04-09T12:51:45Z","updated_at":"2016-02-23T11:27:22Z","pushed_at":"2016-08-19T11:04:28Z","git_url":"git://github.com/karalabe/go-ethereum.git","ssh_url":"git@github.com:karalabe/go-ethereum.git","clone_url":"https://github.com/karalabe/go-ethereum.git","svn_url":"https://github.com/karalabe/go-ethereum","homepage":"","size":60811,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":2,"mirror_url":null,"open_issues_count":1,"forks":2,"open_issues":1,"watchers":0,"default_branch":"develop"}},"base":{"label":"ethereum:develop","ref":"develop","sha":"3b39d4d1c15df2697284c3d7a61564f98ab45c70","user":{"login":"ethereum","id":6250754,"avatar_url":"https://avatars.githubusercontent.com/u/6250754?v=3","gravatar_id":"","url":"https://api.github.com/users/ethereum","html_url":"https://github.com/ethereum","followers_url":"https://api.github.com/users/ethereum/followers","following_url":"https://api.github.com/users/ethereum/following{/other_user}","gists_url":"https://api.github.com/users/ethereum/gists{/gist_id}","starred_url":"https://api.github.com/users/ethereum/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ethereum/subscriptions","organizations_url":"https://api.github.com/users/ethereum/orgs","repos_url":"https://api.github.com/users/ethereum/repos","events_url":"https://api.github.com/users/ethereum/events{/privacy}","received_events_url":"https://api.github.com/users/ethereum/received_events","type":"Organization","site_admin":false},"repo":{"id":15452919,"name":"go-ethereum","full_name":"ethereum/go-ethereum","owner":{"login":"ethereum","id":6250754,"avatar_url":"https://avatars.githubusercontent.com/u/6250754?v=3","gravatar_id":"","url":"https://api.github.com/users/ethereum","html_url":"https://github.com/ethereum","followers_url":"https://api.github.com/users/ethereum/followers","following_url":"https://api.github.com/users/ethereum/following{/other_user}","gists_url":"https://api.github.com/users/ethereum/gists{/gist_id}","starred_url":"https://api.github.com/users/ethereum/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ethereum/subscriptions","organizations_url":"https://api.github.com/users/ethereum/orgs","repos_url":"https://api.github.com/users/ethereum/repos","events_url":"https://api.github.com/users/ethereum/events{/privacy}","received_events_url":"https://api.github.com/users/ethereum/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ethereum/go-ethereum","description":"Official golang implementation of the Ethereum protocol","fork":false,"url":"https://api.github.com/repos/ethereum/go-ethereum","forks_url":"https://api.github.com/repos/ethereum/go-ethereum/forks","keys_url":"https://api.github.com/repos/ethereum/go-ethereum/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ethereum/go-ethereum/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ethereum/go-ethereum/teams","hooks_url":"https://api.github.com/repos/ethereum/go-ethereum/hooks","issue_events_url":"https://api.github.com/repos/ethereum/go-ethereum/issues/events{/number}","events_url":"https://api.github.com/repos/ethereum/go-ethereum/events","assignees_url":"https://api.github.com/repos/ethereum/go-ethereum/assignees{/user}","branches_url":"https://api.github.com/repos/ethereum/go-ethereum/branches{/branch}","tags_url":"https://api.github.com/repos/ethereum/go-ethereum/tags","blobs_url":"https://api.github.com/repos/ethereum/go-ethereum/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ethereum/go-ethereum/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ethereum/go-ethereum/git/refs{/sha}","trees_url":"https://api.github.com/repos/ethereum/go-ethereum/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ethereum/go-ethereum/statuses/{sha}","languages_url":"https://api.github.com/repos/ethereum/go-ethereum/languages","stargazers_url":"https://api.github.com/repos/ethereum/go-ethereum/stargazers","contributors_url":"https://api.github.com/repos/ethereum/go-ethereum/contributors","subscribers_url":"https://api.github.com/repos/ethereum/go-ethereum/subscribers","subscription_url":"https://api.github.com/repos/ethereum/go-ethereum/subscription","commits_url":"https://api.github.com/repos/ethereum/go-ethereum/commits{/sha}","git_commits_url":"https://api.github.com/repos/ethereum/go-ethereum/git/commits{/sha}","comments_url":"https://api.github.com/repos/ethereum/go-ethereum/comments{/number}","issue_comment_url":"https://api.github.com/repos/ethereum/go-ethereum/issues/comments{/number}","contents_url":"https://api.github.com/repos/ethereum/go-ethereum/contents/{+path}","compare_url":"https://api.github.com/repos/ethereum/go-ethereum/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ethereum/go-ethereum/merges","archive_url":"https://api.github.com/repos/ethereum/go-ethereum/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ethereum/go-ethereum/downloads","issues_url":"https://api.github.com/repos/ethereum/go-ethereum/issues{/number}","pulls_url":"https://api.github.com/repos/ethereum/go-ethereum/pulls{/number}","milestones_url":"https://api.github.com/repos/ethereum/go-ethereum/milestones{/number}","notifications_url":"https://api.github.com/repos/ethereum/go-ethereum/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ethereum/go-ethereum/labels{/name}","releases_url":"https://api.github.com/repos/ethereum/go-ethereum/releases{/id}","deployments_url":"https://api.github.com/repos/ethereum/go-ethereum/deployments","created_at":"2013-12-26T13:05:46Z","updated_at":"2016-08-22T19:52:01Z","pushed_at":"2016-08-22T16:44:07Z","git_url":"git://github.com/ethereum/go-ethereum.git","ssh_url":"git@github.com:ethereum/go-ethereum.git","clone_url":"https://github.com/ethereum/go-ethereum.git","svn_url":"https://github.com/ethereum/go-ethereum","homepage":"http://ethereum.github.io/go-ethereum/","size":167817,"stargazers_count":2020,"watchers_count":2020,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":665,"mirror_url":null,"open_issues_count":280,"forks":665,"open_issues":280,"watchers":2020,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ethereum/go-ethereum/pulls/2742"},"html":{"href":"https://github.com/ethereum/go-ethereum/pull/2742"},"issue":{"href":"https://api.github.com/repos/ethereum/go-ethereum/issues/2742"},"comments":{"href":"https://api.github.com/repos/ethereum/go-ethereum/issues/2742/comments"},"review_comments":{"href":"https://api.github.com/repos/ethereum/go-ethereum/pulls/2742/comments"},"review_comment":{"href":"https://api.github.com/repos/ethereum/go-ethereum/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ethereum/go-ethereum/pulls/2742/commits"},"statuses":{"href":"https://api.github.com/repos/ethereum/go-ethereum/statuses/9266bc57bc688dbf53d14ac2970188738742cb60"}}}}
{"actor":{"display_login":"awishformore"}}