What is the motivation behind Russell Yanofsky's work to separate Bitcoin Core into independent node, wallet and GUI processes? What are the advantages to process separation?


There are benefits to both users and developers to having Bitcoin Core split into separate node, wallet and GUI processes.

As Alyssa Hertig outlines here the benefit to users will be being able to run the Bitcoin Core node on a different machine to the Bitcoin Core wallet rather than being forced to run them on the same machine. A user could leave a node running continuously in the background but start and stop the wallet and the GUI as needed. It also opens up the prospect of using a different (i.e. not the Bitcoin Core) GUI or wallet with the Bitcoin Core node.

For Bitcoin Core developers, Yanofsky highlights maintainability and security as the key advantages.

Process separation will make Bitcoin Core more easily maintainable as it defines interfaces at process boundaries. Different parts of the code can interact by calling each other instead of sharing state. This helps code review by making it easier to identify dependencies between parts of the code. Defining boundaries in the codebase will also make code review more scalable as reviewers will just need to understand part of the codebase well rather than needing to understand interdependencies across the whole codebase.

From a security perspective, the wallet and node code could run with different privileges and vulnerabilities should be harder to exploit given they will be limited to a single process. Inter-process communication (IPC) makes new debugging tools available such as the IPC_DEBUG environment variable to log all IPC calls.

There are some potential disadvantages that Yanofsky highlights. Inter-process communication is generally slower. IPC code can be tricky to write and may have bugs. Bad interfaces and unnecessary layers of abstraction can make it harder to implement new features. Features such as SPV (Simplified Payment Verification) that cross process boundaries will likely be more difficult to build.

[edit: ariard contends that the ZeroMQ library makes it relatively easy to write IPC code. He also argues that clean interfaces don't make it harder to implement new features (e.g. SPV) compared to bloated code as a single process.]

Overall it seems clear the advantages outweigh the disadvantages. At the time of writing (August 2020) there are four remaining PRs to be reviewed and merged into Bitcoin Core and then Bitcoin Core should be multiprocess!

For more details on the process separation project see here.

You can install multiprocess Bitcoin Core using these instructions with debugging guidance here.

  • its often a bad idea to separate a complete working system around abstraction lines that can be blurred.. wallet - gui - core is there not already and api? is the official bitcoin core making such a drastic change?
    – jaybny
    Sep 14 '20 at 23:19
  • 1
    Yes Bitcoin Core (to the extent that it is "official") has been moving in this direction for a number of years thanks mainly to the work of Russell Yanofsky. There are interfaces in response to your concern on blurred abstraction lines. (github.com/ryanofsky/bitcoin/tree/ipc-export/src/interfaces) And there are RPCs (developer.bitcoin.org/reference/rpc/index.html) in response to your question on an API. But bitcoind has been single process until Yanofsky's work. Sep 15 '20 at 8:20
  • thanks! there are other ways of doing this, namely using things like nanomsg - which abstracts the "socket" calls where they can be in-memory, inter-thread, inter-process, or tcp - i'm just raising a warning flag on this - i know I am late to the party, but have relevant experience here. hope the option exists for bitcoin-qt to access state from memory directly.
    – jaybny
    Sep 15 '20 at 15:39

Your Answer

