{"action":"created","issue":{"url":"https://api.github.com/repos/bitcoin/bitcoin/issues/9106","repository_url":"https://api.github.com/repos/bitcoin/bitcoin","labels_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/9106/labels{/name}","comments_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/9106/comments","events_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/9106/events","html_url":"https://github.com/bitcoin/bitcoin/issues/9106","id":188099126,"number":9106,"title":"'estimatefee 1' is currently failing","user":{"login":"dooglus","id":573356,"avatar_url":"https://avatars.githubusercontent.com/u/573356?v=3","gravatar_id":"","url":"https://api.github.com/users/dooglus","html_url":"https://github.com/dooglus","followers_url":"https://api.github.com/users/dooglus/followers","following_url":"https://api.github.com/users/dooglus/following{/other_user}","gists_url":"https://api.github.com/users/dooglus/gists{/gist_id}","starred_url":"https://api.github.com/users/dooglus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dooglus/subscriptions","organizations_url":"https://api.github.com/users/dooglus/orgs","repos_url":"https://api.github.com/users/dooglus/repos","events_url":"https://api.github.com/users/dooglus/events{/privacy}","received_events_url":"https://api.github.com/users/dooglus/received_events","type":"User","site_admin":false},"labels":[{"id":98298007,"url":"https://api.github.com/repos/bitcoin/bitcoin/labels/P2P","name":"P2P","color":"006b75","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-11-08T20:46:40Z","updated_at":"2016-11-10T20:17:44Z","closed_at":null,"body":"When I run `estimatefee 1` right now, I get `-1` as the reply, meaning that it isn't possible to get a 95% chance of inclusion in the next block no matter what fee I include, but based on the collected mempool stats that isn't true.\r\n\r\nI traced the `TxConfirmStats::EstimateMedianVal()` call to see what was happening. It went like this:\r\n\r\nIn order to get \"enough transaction data points in this range of buckets\", it first grouped buckets 97 through 75 inclusive, found that their average confirm probability was 94.45% (below the 95% threshold), and so hit the `break` and stopped looking further.\r\n\r\nIt turns out that if it had looked at bucket 74 it would have had enough data points on its own, and would have had a probability of 95.51%, and so we could have returned a value of around 0.01 BTC.\r\n\r\nSo I'm seeing a 94.45% chance of confirmation in the next block if the fee is 0.01156269 or more per kB, and a 95.51% chance if the fee is in the range 0.01051153 - 0.01156269 per kB.\r\n\r\nIt appears that the code makes the assumption that higher fees always result in a higher probability, but that isn't actually the case.\r\n\r\nWouldn't it be better to start at the cheapest bucket and work upwards, stopping at the first one that has a high enough probability, rather than stopping as soon as we find an expensive bucket with too low a probability? Otherwise it is possible for a small miner to sabotage fee estimation by targeting just one bucket. If he has 5% of the hash rate and refuses to confirm transactions in a particular expensive (0.01 BTC per kB) bucket he will ensure that the search stops at that bucket for all `estimatefee 1` calls. He can create enough such transactions to make the bucket statistically significant for less than 1 BTC per day which causing almost 'unlimited' damage to the usefulness of `estimatefee 1`.\r\n\r\nAlso, isn't 95% too high of a threshold when we see almost that many empty or nearly empty blocks? If I lower the threshold to 90% I see a much more reasonable result, of around 0.0006 BTC per kB for next-block confirmation. In [his original email](http://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg06405.html) Alex wrote:\r\n\r\n> Since even the highest fee transactions are confirmed within the first block only 90-93% of the time, I decided to use 80% as my cutoff.\r\n\r\nThat seems like a reasonable argument. Why was the number [later changed from 85% to 95%](https://github.com/bitcoin/bitcoin/commit/f22ac4)?"},"comment":{"url":"https://api.github.com/repos/bitcoin/bitcoin/issues/comments/259796016","html_url":"https://github.com/bitcoin/bitcoin/issues/9106#issuecomment-259796016","issue_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/9106","id":259796016,"user":{"login":"dooglus","id":573356,"avatar_url":"https://avatars.githubusercontent.com/u/573356?v=3","gravatar_id":"","url":"https://api.github.com/users/dooglus","html_url":"https://github.com/dooglus","followers_url":"https://api.github.com/users/dooglus/followers","following_url":"https://api.github.com/users/dooglus/following{/other_user}","gists_url":"https://api.github.com/users/dooglus/gists{/gist_id}","starred_url":"https://api.github.com/users/dooglus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dooglus/subscriptions","organizations_url":"https://api.github.com/users/dooglus/orgs","repos_url":"https://api.github.com/users/dooglus/repos","events_url":"https://api.github.com/users/dooglus/events{/privacy}","received_events_url":"https://api.github.com/users/dooglus/received_events","type":"User","site_admin":false},"created_at":"2016-11-10T20:17:44Z","updated_at":"2016-11-10T20:17:44Z","body":"> The change from 85% to 95% was motivated by a desire to respond more quickly to changes in market conditions\r\n\r\nHow about if there were a 2nd, optional parameter to the `estimate*fee` calls giving the percentage. It could default to 95, to give the same behavior as the current code by default, but I could `estimatefee 1 90` to see how much fee I need to include to have a 90% chance of getting into the next block.\r\n\r\nWould just a pull request likely be merged?"}}
{"actor":{"display_login":"dooglus"}}