{"action":"created","issue":{"url":"https://api.github.com/repos/decred/dcrd/issues/971","repository_url":"https://api.github.com/repos/decred/dcrd","labels_url":"https://api.github.com/repos/decred/dcrd/issues/971/labels{/name}","comments_url":"https://api.github.com/repos/decred/dcrd/issues/971/comments","events_url":"https://api.github.com/repos/decred/dcrd/issues/971/events","html_url":"https://github.com/decred/dcrd/issues/971","id":288380912,"number":971,"title":"Block Header Commitments Discussion","user":{"login":"davecgh","id":2115102,"avatar_url":"https://avatars3.githubusercontent.com/u/2115102?v=4","gravatar_id":"","url":"https://api.github.com/users/davecgh","html_url":"https://github.com/davecgh","followers_url":"https://api.github.com/users/davecgh/followers","following_url":"https://api.github.com/users/davecgh/following{/other_user}","gists_url":"https://api.github.com/users/davecgh/gists{/gist_id}","starred_url":"https://api.github.com/users/davecgh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davecgh/subscriptions","organizations_url":"https://api.github.com/users/davecgh/orgs","repos_url":"https://api.github.com/users/davecgh/repos","events_url":"https://api.github.com/users/davecgh/events{/privacy}","received_events_url":"https://api.github.com/users/davecgh/received_events","type":"User","site_admin":false},"labels":[{"id":804097765,"url":"https://api.github.com/repos/decred/dcrd/labels/consensus","name":"consensus","color":"d4c5f9","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-01-14T04:06:55Z","updated_at":"2018-05-13T19:28:30Z","closed_at":null,"author_association":"OWNER","body":"I'd like to open community discussion and request feedback on potential hard-forking changes to the Decred block header.  Obviously, a full blown [DCP](https://github.com/decred/dcps/) with additional details such as a description of fraud and inclusion proofs, a full specification versus the informal one provided here, discussion of other designs considered, deployment specifics, and test vectors will be required before any type of vote on this could happen.  The intention here is to focus solely on the nuts and bolts of the proposed changes as preliminary work toward that end.\r\n\r\n## Abstract\r\n\r\nThis proposes modifications to the block header to provide an extensible framework, via future consensus-voted hard forks, for adding consensus-enforced commitments to the header without requiring any changes to its size or mining-related field offsets.\r\n\r\n## Motivation\r\n\r\nCurrently, the only fully supported trustless security model in Decred is the full-node model where every node and wallet is required to have access to the entire blockchain to verify every block faithfully follows all of consensus validation rules.\r\n\r\nWhile this model provides the highest security guarantees and is required by the consensus daemon to provide an ordered and timestamped public ledger that protects against double spends and modification of existing transactions, it is highly desirable to support other security models where applications can make varying levels of security tradeoffs according to their domain in exchange for no longer requiring access to whole blocks or the entire blockchain.\r\n\r\n### Simplified Payment Verification (SPV)\r\n\r\nOne well-known alternative security model that was discussed in the [original Bitcoin whitepaper](https://bitcoin.org/bitcoin.pdf) is SPV.  This model supports verifying payments without requiring access to the entire blockchain in exchange for trusting proof-of-work miners to verify the history and a stronger non-partitioning assumption.  However, as the whitepaper notes, this means that if an attacker is able to overpower the network, it can trick SPV nodes into believing fabricated transactions are valid for as long as the attacker overpowers the network.\r\n\r\nIn addition to all potential avenues for fraud typical to pure proof-of-work systems, Decred's hybrid proof-of-work and proof-of-stake model adds additional considerations such as consensus voting results, ticket selection semantics, and block invalidation.  Another complication with this model that was not discussed in the paper is that it relies on the ability to identify and retrieve relevant transactions which can be challenging to do without leaking information regarding coin ownership, which is not ideal from a privacy perspective.\r\n\r\nNevertheless, it is possible to bring the SPV security level very close to that of full nodes through a combination of committed fraud proofs, inclusion proofs, and filters so long as there is at least one honest fully-validating node on the network to sound the alarm.  This is the case because once it is possible to generate a compact fraud proof which SPV nodes can verify quickly, they can securely reject the associated invalid block(s) accordingly.\r\n\r\nCompare this model to the completely trust-based model many lightweight and mobile wallets use today where they communicate with a centralized server controlled by the wallet vendor and simply trust everything the service reports.\r\n\r\n### Alternative Security Models\r\n\r\nThere are also a variety of other security models with varying levels of tradeoffs that can be achieved by making commitments which allow compact proofs that can be verified quickly readily available.  A couple of examples are history pruning with full validation starting from the first non-pruned block, and selective proofing.\r\n\r\nWith careful design, applications can make use of relevant committed proofs to offer strong cryptographic guarantees of certain aspects without needing to fully validate the entire chain themselves.  Such models are typically highly preferable to the fully trust-based models that most applications use at the current time.\r\n\r\n## Informal Specification\r\n\r\n* Repurpose the `StakeRoot` field to house a commitment root and rename it to `CommitmentRoot` accordingly\r\n* Calculate the new `CommitmentRoot` field as the root of a Merkle tree whose leaves are the individual hashes for each desired commitment\r\n  * In order to avoid complicating this proposal, the field MUST be set to the zero hash (32-bytes of zeros) to indicate no specific commitments to include are defined\r\n* Modify the existing `MerkleRoot` field to include the stake transactions in addition to the regular transactions to ensure the header still commits to them by calculating it as `BLAKE256(regular_tx_tree_merkle_root || stake_tx_tree_merkle_root)`\r\n  * Note that this is simply itself a two-leaf Merkle tree that consists of the two Merkle roots of the respective transaction trees.\r\n\r\n## Rationale\r\n\r\nThe most efficient and convenient location to store consensus-enforced commitments which allow compact proofs to be generated is the block header.  Additionally, most software which works with the blockchain nearly always already requires access to the block headers in order to follow the chain with the most proof-of-work and prove transactions are included in the associated block by making use of the committed Merkle root.  Further, since headers are already an integral part of the system, there is already significant tooling for acquiring and working with them.\r\n\r\nThe proposed changes are intentionally engineered to avoid changing the overall size of the header and field offsets of all mining-related fields to prevent breaking existing mining hardware and infrastructure that secures the network.\r\n\r\nThe `StakeRoot` field was repurposed to house the new commitment scheme since it is already the correct size for a commitment merkle root hash and the stake transaction tree merkle root can be rolled into the existing `MerkleRoot` field elegantly without requiring significant modifications.\r\n\r\nMaking use of a Merkle tree for calculating the new `CommitmentRoot` field allows efficient log-sized inclusion proofs of the commitments to be generated and requested from untrusted sources.  Further, each leaf is itself intended to be a commitment to some arbitrary data which will typically be some structure that employs techniques that also allow compact fraud and inclusion proofs to be constructed such as Fenwick trees, Golomb-Rice filters, and additional Merkle trees.\r\n\r\nThis approach provides an extensible and efficient method of consensus-enforced proof chaining which allows the block header to commit to an arbitrary number of short proofs which thin clients can very quickly verify."},"comment":{"url":"https://api.github.com/repos/decred/dcrd/issues/comments/388650155","html_url":"https://github.com/decred/dcrd/issues/971#issuecomment-388650155","issue_url":"https://api.github.com/repos/decred/dcrd/issues/971","id":388650155,"user":{"login":"xaur","id":24484727,"avatar_url":"https://avatars0.githubusercontent.com/u/24484727?v=4","gravatar_id":"","url":"https://api.github.com/users/xaur","html_url":"https://github.com/xaur","followers_url":"https://api.github.com/users/xaur/followers","following_url":"https://api.github.com/users/xaur/following{/other_user}","gists_url":"https://api.github.com/users/xaur/gists{/gist_id}","starred_url":"https://api.github.com/users/xaur/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xaur/subscriptions","organizations_url":"https://api.github.com/users/xaur/orgs","repos_url":"https://api.github.com/users/xaur/repos","events_url":"https://api.github.com/users/xaur/events{/privacy}","received_events_url":"https://api.github.com/users/xaur/received_events","type":"User","site_admin":false},"created_at":"2018-05-13T19:28:30Z","updated_at":"2018-05-13T19:28:30Z","author_association":"NONE","body":"One application of commitments was noted in [floating fees discussion](https://www.reddit.com/r/decred/comments/8ixay1/decred_investment_thesis_by_placeholder_ventures/dyx64yd/):\r\n\r\n> in order to support lightweight wallets, they too would need the ability to trustlessly obtain the necessary information for the current fee rate, which implies commitments"}}
{"actor":{"display_login":"xaur"}}