11institutetext: Cyber Security Group, Delft University of Technology, The
Netherlands 11email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>22institutetext: Ubitech Ltd, Digital Security &
Trusted Computing Group, Athens, Greece 22email<EMAIL_ADDRESS>
# ID-based self-encryption via Hyperledger Fabric based smart contract

Ilya Grishkov 11 Roland Kromes 11 Thanassis Giannetsos 22 Kaitai Liang 11

###### Abstract

This paper offers a prototype of a Hyperledger Fabric-IPFS based network
architecture including a smart contract based encryption scheme that meant to
improve the security of user’s data that is being uploaded to the distributed
ledger. A new extension to the self-encryption scheme was deployed by
integrating data owner’s identity into the encryption process. Such
integration allows to permanently preserve ownership of the original file and
link it to the person/entity who originally uploaded it. Moreover, self-
encryption provides strong security guarantees that decryption of a file is
computationally not feasible under the condition that the encrypted file and
the key are safely stored.

###### Keywords:

Blockchain IPFS Self-Encryption Security Hyperledger Fabric

##  1 Introduction

The modern world is increasingly adopting blockchain technology. The first
major market adoption of blockchain happened in 2009 when Bitcoin was
introduced [12]. Interest in blockchain solutions grew over the years and lead
to the invention of Ethereum - Bitcoin peer but with support for smart
contracts which are digital codes enabling the description of complete
business logic. [1]. The introduction of smart contracts leads to further
development in the field of blockchain and created demand for more industry-
friendly solutions that allow to identify users of the system (Know-Your-
Customer, Anti-Money-Laundering). Hyperledger Fabric was then introduced as a
highly modular permissioned blockchain that allows great customization to suit
particular industrial needs [3]. Given its customizability and modularity,
Hyperledger Fabric (HLF) is a perfect platform for extending it with various
trust and privacy preservation solutions.

According to Huang et al. [7] the main component of a blockchain that is being
attack the most is a smart contract. High frequency of attack on a component
designed to handle user private data suggests a need for an alternative
approach to handling sensitive information other than just sending it raw to
the ledger. An local data encryption prior to sending data to the smart
contract could be a solution.

Self-encryption was introduced as a mean of encrypting files that ”requires no
user intervention or passwords” [9]. This algorithm can be used for local
encryption of files, encrypted chunks of which will be later uploaded to a
cloud-based storage or to a distributed file system (e.g., IPFS
111https://ipfs.io/) Pointers to the encrypted chunks are then sent to the
ledger. It can be noted that storing only the hash values of the encrypted
data chunks on blockchain ledger is vital when the data size is significant.
The authors in [5] point out that sending data to a blockchain frequently,
when the data size is large, can cause the entire blockchain network to crash.
Sending only the hash values of the given data is more optimal as a hash value
is usually 32 bytes long. While this solution allows to keep file content
private, the file itself is not linked in any way to its owner. A variant of
identity based encryption can tackle this problem. If a file is self-encrypted
with owners identity used during the encryption process, this file remains
linked to the person who initially uploaded it to the blockchain. This way
original ownership can be preserved.

This paper aims at exploring trust and privacy preserving solutions in
Hyperledger Fabric blockchain. More specifically the goal is to further
investigate the utility of a combination of identity based encryption and
self-encryption as means of improving security of the data in the HLF; extend
the previously done research by [13] and implement ID-based self-encryption
via Hyperledger Fabric smart contract. Hence the main research question is:
”How can security of Hyperledger Fabric smart contracts be improved using ID-
based self-encryption?”

Within this paper an approach of integrating ID-based self-encryption is
presented. Moreover a detailed description of prototype implementation is
given. In addition to this implementation of ID-based self-encryption, a
practical fully decentralized network architecture for storing encrypted data
has also been deployed. In this proposed network, the data owner can use ID-
based self-encryption to store encrypted data in a decentralized and secure
manner. The encrypted data chunks are stored in an InterPlanetary File System
(IPFS) which is a decentralized systems for file storage. To store the
references (hash values) of the encrypted data chunks, the Hyperledger Fabric
blockchain was used.

This work is structured as follows. Section 2 describes related works used to
achieve the goal. Section 3 gives a background about the the implementation of
ID-based self-encryption. Section 4 discusses the inner workings and the
proposed implantation of ID-based self-encryption. The performance and
security analysis of the proposed implementation is presented in section 5.
Finally, the work is discussed in section 6, and concluded in section 7.

##  2 State of the Art

Blockchain is a distributed ledger technology that provides immutability and
transparency of data to members of the blockchain network [19]. The blockchain
is also a peer-to-peer network in which participants are known to each other.
Authentication of participants is ensured using elliptic curve cryptography.
Today’s blockchains enable the deployment of complete business logic in a
seamless manner. These digital business logic are also known as smart
contracts.

Blockchain technology is used in several use cases such as smart city, smart
agriculture, vehicle networks [10] and also healthcare [15]. In the latter two
cases, the privacy and ownership of data transmitted from a driver and medical
patient is particularly important, as this data may contain privacy sensitive
information. It can be noted that in these latter use case using a private
blockchain such as Hyperledger Fabric can be a more optimal choice as they can
provide higher security and privacy level.

The topic of security and privacy of the Hyperledger Fabric has been
thoroughly studied [18], [4], [16]. Moreover a research has been conducted
this year by a student of Delft University of Technology [13] addressing
similar issue of improving HLF security using self-encryption.

The concept of self-encryption was introduced by Yu Chen [2]. The approach of
the original paper involves converting a file into a bit stream, extract the
key by randomly selecting bits from the stream and then doing the encryption
using that key. After the encryption the key and the encrypted file should be
stored separetely, e.g. the key can be stored locally, while the encrypted
file can be sent to a server.

The original encryption scheme was also extended by Moch Rezky Debby Rahardjo
[14]. According to the paper, ”The modification is located in dividing the
plaintext and ciphertext into 1024-bit chunks at XOR process and using the
date when encryption process starts as a seed. The modification also adds the
database for the key management function”. Storing the key and the encrypted
chunks in separate places makes in computationally not feasible to get the
original data.

The later industrial adaptation of the self-encryption scheme happened when a
team lead by David Irvine made self-encryption the core of his company’s
(MaidSafe) product - SAFE Network [9]. Irvine’s implementation of the self-
encryption scheme will be the basis of this work, hence a more detailed
explanation of the implementation of the algorithm will be given.

Figure 1 shows the encryption process. First, the original file is getting
split into minimum of 3 file chunks. After the file is split into chunks the
algorithm creates a data map, where the key needed for decryption will be
stored. Each chunk is then hashed and those hashes are written to the data
map. Parts of those hashes are used as a key and initialization vector for AES
128 algorithm that encrypts each file chunk. When encryption is done, each
encrypted chunk is obfuscated with the previously computed hash values by
applying a XoR function. At the end of the process the encryption scheme
returns a data map that is going to be later used for decryption, and the
encrypted file chunks.

Figure 1: Self-encryption process, adopted from [11]

##  3 Background

Original implementation of the self-encryption schema by David Irvine [11] was
modified and used for this research. The use of rayon library (which adds
parallelization to the code) was removed from the algorithm, due to the fact
that the compilation target (WebAssembly) only supports single-threaded code.
Additionally the code base was modified to include an interface for
communication with the external code. Changes were also done to the Cargo.toml
to make the code compatible with the target. Modified self-encryption
algorithm was compiled to WebAssembly and run in a virtual machine (VM) and
invoked from the code of the developed local application (which allows the
interaction with the Hyperledger Fabric Smart Contract). A more detailed
description of the process will be given in the section 4. The benchmarks of
this implementation will be provided in section 5.

Hyperledger Fabric test network v2.4.3 was used. Test network was deployed to
Docker based on the tutorials provided by Hyperledger Fabric 222Usage of the
command requires navigating to the root directory of the test-network,
provided by the Hyperledger Fabric [8] .

Smart contract was then deployed (detailed in section 4).

Encrypted file storage is handled by the IPFS, which is a distributed Torrent
database, which uses hashes of files to address its content. IPFS node was
also deployed to Docker. For IPFS deployment two directories (staging and
data) were mounted on the host file system to persist the stored data, when
the container is stopped. Hyperledger Fabric provides official software
development kit (SDK) for 3 languages: Go, Java, Javascript. Go was chosen for
implementation of the project, due to ease of integration with both
Hyperledger Fabric and the IPFS. Encryption library is written in Rust and is
compiled to WebAssembly, hence a way to call WebAssembly was needed. Go also
provides support for Wasmer library that allows to call WebAssembly function
directly from Go code.

##  4 ID-based self-encryption

###  4.1 Integrating identity into the encryption

This paper offers an extension to the algorithm proposed by Irvine [9].
Encryption step in the original algorithm is modified to include identity of a
person who is running the algorithm into the encryption process. Instead of
using part of the chunk hash as a key for AES 128, the result of XoR of the
hashed identity and the chunk hash is used as a key. The identity can be any
string of any length. If the length of this string is shorter than the length
of the key, then the cycle function is applied to the string, which repeats
the iterator of a string. The hashing function SipHash 1-3 is used to hash the
identity of a user, before passing it to the XoR function. Figure 2
demonstrates the process of encrypting a file using the modified version of
self-encryption with identity integrated into the encryption process.

Figure 2: ID-based self-encryption process

Decryption of the file, that was encrypted using ID-based self-encryption, is
similar to that of a regular self-encryption, with the key for AES 128 being
the only different part. The decryption calculates the key the same way the
encryption does it by applying XoR function to the hash of identity and the
chunk hash from the data map.

The implementation of the encryption scheme can be found on GitHub
333https://github.com/ilyagrishkov/ib-self-encryption-rust

###  4.2 Connecting the encryption algorithm and the local application

The implementation of the identity-based self-encryption is written purely in
Rust, while the client application is written in Go. This creates a demand for
a way to integrate Rust library into Go code. Among the solutions to tackle
the problem are:

  1. 1.

Use Go tools to assemble the Go code and compile Rust code into a static
library. Then link compiled code using additional assembly ”glue-code” [17].

  2. 2.

Compile Rust to a static library and call it from the Go code using Go build-
in pseudo-library C for interacting with native interfaces.

  3. 3.

Compile Rust to WebAssembly (WASM) code and call it from Go using Wasmer
library 444https://wasmer.io/.

All of the methods have been successfully tried. The first two methods do not
allow cross compilation, because both of them require compiling Rust to a
static library, which is platform-specific. Additionally, the first methods
requires the use of assembly language, which is different on different
processor architectures and operating systems. The second method also uses C
pseudo-library, which does not allow cross-compilation of the Go code.
Overall, both methods are very _platform-specific_ , which makes them less
preferable choice.

The third method was chosen for connecting Rust library to Go code. Compiling
Rust to WASM to use as a standalone application or a library, can be done
using the following command:

⬇

$ cargo build –target=[chosen_target]

where chosen_target is a WebAssembly target that can be either wasm32-unknown-
unknown or wasm32-wasi. The latter was used, because it compile using WASI API
555https://wasi.dev/, which is a system API that provides access to multiple
operating system functionalities, such as access to the file system.

The resulting WASM file is then placed in a hidden folder in the home
directory of a user, so it can later be loaded by the Go code. As the WASM
code is used within a virtual machine (VM), it’s independent from the
operating system it will run on, so requires compiling only for one target.

####  4.2.1 Calling WASM from Go using Wasmer

In order to call WASM code a VM needs to be used. Wasmer library provides such
VM, that can also be initialized from within Go code. The process of calling
WASM code from the Go application is demonstrated in the Figure 3. This
process consists of the following steps:

  1. 1.

Loading WASM code into a Wasmer VM

    1. (a)

A directory on the host operating system, that will be accessible in the VM,
need to be specified

    2. (b)

Optionally, standard output of the WASM library can be inherited.

  2. 2.

Invoking a function by specifying its name and the return type and passing
arguments to it

Figure 3: Connection of the WASM encryption library to the client application
via the Wasmer library and the developed wrapper

The communication between the Go code and WASM library and passing arguments
for function invocation is happening using C types, which means that types
like strings are not supported directly and need to be converted to
corresponding C types. In case of a string being passed as an argument, it
needs to be written to memory and end with a zero byte. The pointer to the
first byte of this string is then passed to the invoked function as an
argument.

As the host operating system memory is inaccessible for the VM, allocation and
deallocation of memory need to happen within the VM itself. In order to
facilitate the allocation and deallocation two dedicated Rust functions were
developed as a part of id-based encryption library interface: allocate and
deallocate.

In case the called function requires a string as an argument, the allocation
needs to be performed before passing the pointer to that string. The allocate
function has to be called to allocate memory inside the VM. The memory is then
accessed from the Go code and each byte of the string argument is written to
the newly allocated memory. The pointer to the memory and the length need to
be preserved in order to deallocate the memory, before the program terminates.
The pointer to the first memory cell containing the string argument is then
passed as an argument to the function that is being called.

####  4.2.2 Wrapper code for Wasmer calls

A wrapper code has been written to simplify invocation of WASM functions. The
major simplification that this code provides is the ability to pass Go native-
type argument to the wrapper, which then performs all the necessary processing
and allocation, if needed. The pointers to string or array types as well as
their lengths are stored, so when the program terminates, the memory is
getting deallocated.

Moreover, the developed wrapper code allows to pass simple numerical Go types
(integers, floats, bytes, etc.) as pointers to the WASM library, so the
changes that are happening to them when WASM functions run are also reflected
in Go code, without the need to return anything.

Additionally, the wrapper requires return type parameter argument (which is
represented as an enumerator), when calling the invocation function through
the wrapper. It uses the return type to case the return of WASM function to
corresponding Go type. In cases when a pointer to a string is returned, the
wrapper reads bytes from the VM memory until the zero byte and creates a Go
string from it. The return type of the wrapper’s invocation function is a
generic _interface{}_, which requires additional type casting. For example, in
case the called function returned a pointer to a string, a Go string will be
built from the pointer, but a user will still have to dynamically convert the
returned value as it will be _interface{}_.

###  4.3 Smart contract

The smart contract in Hyperledger Fabric allows to define assets that will be
on the ledger. This paper defines an asset containing three fields: ID, Owner
and CID. The code below shows the definition of an asset written in Go.

⬇

type Asset struct {

ID string ‘json:”ID”‘

Owner string ‘json:”Owner”‘

CID []string ‘json:”CID”‘

}

Listing 1: The struct representing an asset on the Hyperledger Fabric ledger

The ID is a universally unique identifier (UUID) that is generated, when the
new asset is created. The Owner is a string of hexadecimal numbers
representing a public key of a user, who created the asset. The CID is an
array if unique identifier that reference encrypted file chunks saved in IPFS.
The references to the encrypted data chunks remain immutable, and can also be
used for verifying if encrypted data chunks were manipulated (the hash of an
encrypted data chunk is a unique value).

Additionally, the smart contract defines a list of functions for creating,
deleting and updating assets. The implementation can be found on
GitHub666https://github.com/ilyagrishkov/ib-self-encryption-smart-contract.

###  4.4 Self-Encryption work flow in a blockchain-IPFS based network

The encryption and decryption process as well as interactions with the IPFS
and the Hyperledger Fabric are orchestrated by a local application, which is a
command line interface (CLI) tool written in Go. The prototype of the tool is
accessible from GitHub777https://github.com/ilyagrishkov/ib-self-encryption

Execution of any command starts with creating a new instance of a WASM wrapper
and loading of the encryption library. When the command requires interaction
with the Hyperledger Fabric, presence of the wallet, containing identity
(which is necessary to enable interaction with the smart contract), is being
checked. If the wallet is missing it’s getting populated based on the
certificates and keys of a user. When this preparation is done, the execution
of the command starts.

At the end of the program execution the wrapper iterates over all allocated
memory pointers and individually deallocates them.

Figure 4: Work flow in the blockchain-IPFS based network when using self-
encryption

There are two major parts of the system - encryption and decryption. Figure 3
demonstrates the workflow of both of them.

####  4.4.1 Encryption

The first part, encryption, that deals with encrypting a file and uploading
data to the Hyperledger Fabric starts when the following command is called:

⬇

$ ibse add [file] [key_output_path]

where ibse is the name of the local app, file is the absolute path to the file
that needs to be encrypted, and key_output_path is the absolute path to
location where the key will be stored.

The original file is is getting uploaded to the directory that was mapped
during the VM initialization. From there it can be read by the WASM code. The
encryption function is then called and the output is written to a new
directory inside the mapped one. The output consists of multiple encrypted
file chunks and a data map. The data map is moved to the location specified by
the user and can later be shared via a secure channel. Each encrypted file
chunk is being put into a zip archive to preserve their names, when uploading
to the IPFS, and sent to the IPFS. The unique identifier, corresponding to
each chunk (Content Identifiers or CIDs which are the hash values of the
files) is returned. A smart contract function is then called that creates a
new asset with all CIDs.

####  4.4.2 Decryption

The second part of the system, decryption, is invoked using the following
command:

⬇

$ ibse get [block] [key] [destination]

where block is the UUID of an asset in HLF blockchain that contains CIDs of
encrypted chunks, key is the absolute path to the data map, and destination is
the absolute path to location where decrypted file should be written.

The UUID allows to identify an asset containing CIDs of encrypted file chunks.
Each of the chunks is downloaded from the IPFS, unarchived, and written to the
directory that is accessible from the VM. The data map is then copied to the
same directory. After collecting all the necessary files for decryption, the
decryption function is called and the restored file is written to a user-
specified destination.

##  5 Results

###  5.1 Performance analysis

Benchmarking of the system was done on the iMac 2019, 3,6 GHz 8-Core Intel
Core i9 with 32 GB of memory running on MacOS 12.3.1.

Benchmarking of the implemented id-based self-encryption scheme was done. As
the encryption itself is not implemented in the same language as the rest of
the project (the encryption is implemented in Rust and the rest of the project
is in Go), the execution time can differ when Rust functions are called from
Go compared to pure Rust execution time.

Files of sizes 100-, 250-, 500-, 750 kilobytes, 1 megabyte, 10-, 25-, 50-,
75-, and 100 megabytes were created for benchmarking both the pure Rust
implementation as well as the WASM + Go implementations. Moreover, for this
benchmark both the Rust code and the WASM library were optimized using maximum
level of optimization provided by the Rust compiler.

Figure 5: Dependence of the execution time of id-based self-encryption
algorithm in pure Rust from the file size

The initial benchmark was performed on the encryption function only and was
measuring execution time of the pure Rust implementation. Figure 4 shows the
results of the benchmarking.

The chart show near-linear dependence between the size of the file and the
time it takes to encrypt it. This dependence can be explained by the fact that
the most demanding computational is the AES 128 encryption process and with
the increase of the file size, then number of chunks it is split to increases.
Each chunk of the original file needs to be individually encrypted, hence the
computation time grows linearly with the size of the file.

As the encryption function execution time grows linearly due to the
computational demand of the AES 128 and the hashing algorithms, the decryption
process will be identical, because it uses the same algorithms for decryption.

In order to achieve more objective benchmark results, file of each size has
been encrypted 100 times and the average calculated. In order to visualize
execution time a chart in Python using MatPlotLib888https://matplotlib.org/
was created. The chart contains a 25-bin histogram, each representing density
of a particular measurement. Following central limit theorem the distribution
of the execution time measurements was assumed normal, so the mean and the
standard deviation were calculated and the distribution plotted over the
histogram. Figure 5 shows an example of combined charts for pure Rust and WASM
+ Go execution times, when encrypting 50 megabytes file. The blue histogram on
the left-hand side shows results of the 100 measurements of the execution time
of the Rust implementation; on the right-hand side - of the WASM + Go
implementation.

Figure 6: Run time distribution for 50MB file encryption using pure Rust and
WebAssembly + Go implementations

The results of execution time measurements have for various file sizes are
summarized in Table 1. The execution time shown in the table is the average
number of seconds it takes a corresponding implementation to encrypt a file of
a corresponding size. In addition to the average execution time, the overhead
of the WASM + Go implementation is calculated for every pair of measurements.

File size (Byte) | Average execution time (Sec) |  Overhead (%)
---|---|---
Rust | WASM + Go
100KB | 0.0024 | 0.0042 | 75.89
250KB | 0.0038 | 0.007 | 84.51
500KB | 0.005 | 0.0088 | 75.48
750KB | 0.0069 | 0.0117 | 71.4
1MB | 0.0081 | 0.0139 | 71.29
10MB | 0.0747 | 0.117 | 56.55
25MB | 0.1885 | 0.2851 | 51.22
50MB | 0.3686 | 0.5644 | 53.15
75MB | 0.5492 | 0.8447 | 53.81
100MB | 0.7317 | 1.1201 | 53.08
Table 1:  Average execution time and overhead when encrypting files of
different sizes using id-based self-encryption

It is visible from the table that the overhead has a clear downwards trend
(except the spike, when encrypting 250KB file). When the execution time of a
WASM + Go encryption implementation is less than 0.01 seconds, the overhead
falls in the range between 70% and 85%. When the execution time is longer than
0.1 seconds, the overhead goes down to 50% - 55% and stays in that range when
the file size increases. Figure 6 demonstrates the overhead of WASM + Go
encryption of file of different sizes.

Figure 7: WebAssembly + Go implementation overhead measurements over for files
of different sizes compared to pure Rust implementation

Such decrease in the overhead, when the execution time becomes longer is
explained by the presence of the Wasmer library invocation overhead, which
occurs every time a call is made to the Wasmer VM. When execution time itself
is less than 0.01 seconds the invocation overhead is significant compared to
the execution time. At the same time, when the execution time becomes longer,
the overhead from invocation becomes insignificant, and measurements start to
approximate real WASM VM overhead, which is around 50% - 55%.

###  5.2 Security analysis

The designed app has multiple surfaces of attack. The IPFS nodes, where the
encrypted file chunks are stored can attacked. Also, an adversary can be gain
unauthorized access to the ledger with references to file on the IPFS. Both of
those possibilities are analyzed below.

The security of IPFS nodes (assuming the encrypted file chunks were stored
individually on multiple nodes) can be compromised, in which case encrypted
files will be leaked to the malicious user. As encrypted file chunks have been
stored on different nodes, the probability that all of them being compromised
is negligible and should not be considered. Additionally, individual files do
not have any link to each other, so matching multiple encrypted chunks, that
are needed for successful decryption, is not computationally feasible. The
data map containing the keys for the decryption, was stored locally by the
user, who encrypted the file. Without the original keys, the decryption of the
self-encrypted data is computationally not feasible [9]. Moreover, as the
proposed self-encryption is also related to the data owner or user identity,
the decryption cannot be done until the identity is not provided. Thus the
data ownership is also provided by the id-based implantation.

##  6 Discussion

The results show high security guarantees of the id-based self-encryption
scheme, when used for encrypting data, stored on the Hyperledger Fabric
blockchain. This allows to use the implemented prototype as a secure medium
for saving a retrieving information from the ledger.

In future works full Go implementation of self-encryption should be compared
with the design proposed in this paper.

It was also beyond the scope of this study to create a standardized
benchmarking for WASM and Rust libraries. It can be done by using multiple
sample programs that test specific properties of the programming language
(e.g. efficacy of memory allocation and deallocation) or very computationally
intensive programs [6]. The objective could be running a containerized version
of both libraries against a set of such programs and analyzing the run time.

Moreover, the study can be expanded by analyzing and comparing CPU and memory
load of WASM and Rust libraries. Such benchmark could be also done using
sample programs mentioned in the previous paragraph.

##  7 Conclusion

In this study a new approach to storage of files on the Hyperledger Fabric
blockchain was presented. The demonstrated approach allows for secure storage
of data in a decentralized way, with ability to preserve the original file
ownership and also information about the person, who encrypted it. This
approach can be used where high security and trust in the integrity of data
stored on the ledger is need. The prototype uses Rust implementation of id-
based self-encryption that is compiled to WebAssembly and invoked from Go
code.

Additionally, the study demonstrates relatively low overhead (the overhead is
around 55%) and high performance level of WebAssembly library integration with
the Go code base, compared to the pure Rust implementation. Relatively low
overhead of WASM creates possibilities for developers to use WASM integration
with Go and other languages that support Wasmer library as a cross-platform
solution that allows to achieve high degrees of performance, while also being
deterministic.

The wrapper proposed in this work can also be used in Golang-based back-end
applications that aim to use the cryptographic libraries deployed in Rust
providing a more memory-safe execution.

## References

  * [1] Buterin, V.: Ethereum: A next-generation smart contract and decentralized application platform (2014), https://github.com/ethereum/wiki/wiki/White-Paper, accessed: 2016-08-22
  * [2] Chen, Y., Ku, W.S.: Self-encryption Scheme For Data Security in Mobile Devices. In: 2009 6th IEEE Consumer Communications and Networking Conference. pp. 1–5. IEEE (2009)
  * [3] Christian, C.: Architecture of the Hyperledger Blockchain Fabric (2016), https://www.zurich.ibm.com/dccl/papers/cachin$_/$dccl.pdf, accessed: 2016-08-10
  * [4] Dabholkar, A., Saraswat, V.: Ripping the Fabric: Attacks and Mitigations on Hyperledger Fabric. In: International Conference on Applications and Techniques in Information Security. pp. 300–311. Springer (2019)
  * [5] Gerrits, L., Kromes, R., Verdier, F.: A True Decentralized Implementation Based on IoT and Blockchain: a Vehicle Accident Use Case. In: 2020 International Conference on Omni-layer Intelligent Systems (COINS). pp. 1–6 (2020). https://doi.org/10.1109/COINS49042.2020.9191405
  * [6] Gouy, I.: Toy benchmark programs, https://benchmarksgame-team.pages.debian.net/benchmarksgame/why-measure-toy-benchmark-programs.html
  * [7] Huang, Y., Bian, Y., Li, R., Zhao, J.L., Shi, P.: Smart Contract Security: A Software Lifecycle Perspective. IEEE Access 7, 150184–150202 (2019). https://doi.org/10.1109/ACCESS.2019.2946988
  * [8] Hyperledger: Using the Fabric test network (2020), https://hyperledger-fabric.readthedocs.io/en/release-2.2/test$_$network.html, accessed: 2022-03-19
  * [9] Irvine, D.: Self encrypting data (2010), unpublished Manuscript
  * [10] Kromes, R., Gerrits, L., Verdier, F.: Adaptation of an embedded architecture to run Hyperledger Sawtooth Application. In: 2019 IEEE 10th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON). pp. 0409–0415 (2019). https://doi.org/10.1109/IEMCON.2019.8936264
  * [11] Maidsafe: self_encryption. https://github.com/maidsafe/self$_/$encryption (2022)
  * [12] Nakamoto, S.: Bitcoin: A Peer-to-Peer Electronic Cash System (Dec 2008), https://bitcoin.org/bitcoin.pdf, accessed: 2015-07-01
  * [13] Park, C.: Using Self-Encryption to safeguard data security in Fabric’s smart contract. Bachelor’s thesis (2022), https://repository.tudelft.nl/islandora/object/uuid:15c5eee3-0be6-4d71-bf67-3ea5e576aa05?collection=education
  * [14] Rahardjo, M.R.D., Shidik, G.F.: Design and implementation of self encryption method on file security. In: 2017 International Seminar on Application for Technology of Information and Communication (iSemantic). pp. 181–186. IEEE (2017)
  * [15] Ray, P.P., Dash, D., Salah, K., Kumar, N.: Blockchain for IoT-Based Healthcare: Background, Consensus, Platforms, and Use Cases. IEEE Systems Journal 15(1), 85–94 (2021). https://doi.org/10.1109/JSYST.2020.2963840
  * [16] Stamatellis, C., Papadopoulos, P., Pitropakis, N., Katsikas, S., Buchanan, W.J.: A Privacy-Preserving Healthcare Framework Using Hyperledger Fabric. Sensors 20(22), 6587 (2020)
  * [17] Valsorda, F.: Rustgo: Calling rust from go with near-zero overhead. https://words.filippo.io/rustgo/ (2022)
  * [18] Yamashita, K., Nomura, Y., Zhou, E., Pi, B., Jun, S.: Potential Risks of Hyperledger Fabric Smart Contracts. In: 2019 IEEE International Workshop on Blockchain Oriented Software Engineering (IWBOSE). pp. 1–10. IEEE (2019)
  * [19] Zibin, Z., Xie, S., Dai, H.N., Chen, X., Wang, H.: Blockchain challenges and opportunities: A survey. International Journal of Web and Grid Services 4, 352–375 (2018). https://doi.org/https://doi.org/10.1504/IJWGS.2018.095647, https://www.sciencedirect.com/science/article/pii/S0167739X17329205


and MDM-2015-0509, some of which include ERDF funds from the European Union.
IFAE is partially funded by the CERCA program of the Generalitat de Catalunya.
Research leading to these results has received funding from the European
Research Council under the European Union’s Seventh Framework Program
(FP7/2007-2013) including ERC grant agreements 240672, 291329, and 306478. We
acknowledge support from the Brazilian Instituto Nacional de Ciência e
Tecnologia (INCT) do e-Universo (CNPq grant 465376/2014-2).

This manuscript has been authored by Fermi Research Alliance, LLC under
Contract No. DE-AC02-07CH11359 with the U.S. Department of Energy, Office of
Science, Office of High Energy Physics.

Software: SExtractor (Bertin & Arnouts, 1996), PSFEx (Bertin, 2011), SCAMP
(Bertin, 2006), SWarp (Bertin et al., 2002; Bertin, 2010)

## Data Availability

The data underlying this article and that were used to build the processed
data products were accessed from NOAO (https://www.noao.edu/). The derived
data will be released publicly through the DES collaboration at
https://des.ncsa.illinois.edu/releases/ upon the completion of the cosmology
analysis for which they are used.

## References

  * Abbott et al. (2018) Abbott T. M. C., et al. 2018, ApJS, 239, 18
  * Abolfathi et al. (2018) Abolfathi B., et al. 2018, ApJS, 235, 42
  * Aihara et al. (2018) Aihara H., et al. 2018, PASJ, 70, S4
  * Aihara et al. (2019) Aihara H., et al. 2019, PASJ, 71, 114
  * Alarcon et al. (2020) Alarcon A., et al. 2020, arXiv e-prints, p. arXiv:2007.11132
  * Baldry et al. (2018) Baldry I. K., et al. 2018, MNRAS, 474, 3875
  * Bazin et al. (2011) Bazin G., et al. 2011, A&A, 534, A43
  * Benítez (2000) Benítez N., 2000, ApJ, 536, 571
  * Bernstein & Armstrong (2014a) Bernstein G. M., Armstrong R., 2014a, MNRAS, 438, 1880
  * Bernstein & Armstrong (2014b) Bernstein G. M., Armstrong R., 2014b, MNRAS, 438, 1880
  * Bernstein et al. (2016) Bernstein G. M., et al. 2016, MNRAS, 459, 4467
  * Bertin (2006) Bertin E., 2006, in Gabriel C., et al. eds, Astronomical Society of the Pacific Conference Series Vol. 351, Astronomical Data Analysis Software and Systems XV. p. 112
  * Bertin (2010) Bertin E., 2010, SWarp: Resampling and Co-adding FITS Images Together, Astrophysics Source Code Library (ascl:1010.068)
  * Bertin (2011) Bertin E., 2011, in Evans I. N., et al. eds, Astronomical Society of the Pacific Conference Series Vol. 442, Astronomical Data Analysis Software and Systems XX. p. 435
  * Bertin & Arnouts (1996) Bertin E., Arnouts S., 1996, A&AS, 117, 393
  * Bertin et al. (2002) Bertin E., et al. 2002, in Bohlender D. A., Durand D., Handley T. H., eds, Astronomical Society of the Pacific Conference Series Vol. 281, Astronomical Data Analysis Software and Systems XI. p. 228
  * Bonnett et al. (2016) Bonnett C., et al. 2016, Phys. Rev. D, 94, 042005
  * Bordoloi et al. (2010) Bordoloi R., Lilly S. J., Amara A., 2010, MNRAS, 406, 881
  * Bouwens et al. (2007) Bouwens R. J., et al. 2007, ApJ, 670, 928
  * Bowler et al. (2020) Bowler R. A. A., et al. 2020, MNRAS, 493, 2059
  * Bradshaw et al. (2013) Bradshaw E. J., et al. 2013, MNRAS, 433, 194
  * Brammer et al. (2008) Brammer G. B., van Dokkum P. G., Coppi P., 2008, ApJ, 686, 1503
  * Brown et al. (2014) Brown M. J. I., et al. 2014, ApJS, 212, 18
  * Buchs et al. (2019) Buchs R., et al. 2019, MNRAS, 489, 820
  * Burke et al. (2018) Burke D. L., et al. 2018, AJ, 155, 41
  * Chambers et al. (2016) Chambers K. C., et al. 2016, arXiv e-prints, p. arXiv:1612.05560
  * Cohen (2002) Cohen J. G., 2002, ApJ, 567, 672
  * Coil et al. (2004) Coil A. L., et al. 2004, ApJ, 609, 525
  * Coil et al. (2011) Coil A. L., et al. 2011, ApJ, 741, 8
  * Colless et al. (2001) Colless M., et al. 2001, MNRAS, 328, 1039
  * Conroy & Gunn (2010) Conroy C., Gunn J. E., 2010, FSPS: Flexible Stellar Population Synthesis (ascl:1010.043)
  * Cool et al. (2013) Cool R. J., et al. 2013, ApJ, 767, 118
  * Cooper et al. (2012) Cooper M. C., et al. 2012, MNRAS, 425, 2116
  * Coppin et al. (2006) Coppin K., et al. 2006, MNRAS, 372, 1621
  * Czekaj et al. (2014) Czekaj M. A., et al. 2014, A&A, 564, A102
  * D’Andrea et al. (2018) D’Andrea C. B., et al. 2018, preprint, p. arXiv:1811.09565 (arXiv:1811.09565)
  * DES Collaboration (2016) DES Collaboration 2016, MNRAS, 460, 1270
  * DES Collaboration et al. (2020) DES Collaboration et al., 2020, To be submitted to
  * Daddi et al. (2004) Daddi E., et al. 2004, ApJ, 617, 746
  * Dahlen et al. (2013) Dahlen T., et al. 2013, ApJ, 775, 93
  * Dalton et al. (2006) Dalton G. B., et al. 2006, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series. p. 62690X, doi:10.1117/12.670018
  * Davidzon et al. (2017) Davidzon I., et al. 2017, A&A, 605, A70
  * Dawid (1984) Dawid A. P., 1984, Journal of the Royal Statistical Society. Series A (General), 147, 278
  * Drlica-Wagner et al. (2018) Drlica-Wagner A., et al. 2018, ApJS, 235
  * Elmegreen et al. (2007) Elmegreen D. M., et al. 2007, ApJ, 658, 763
  * Emerson et al. (2004) Emerson J. P., et al. 2004, The Messenger, 117, 27
  * Etherington et al. (2017) Etherington J., et al. 2017, MNRAS, 466, 228
  * Euclid Collaboration (2020) Euclid Collaboration 2020, A&A, 644, A31
  * Everett et al. (2020) Everett S., et al., 2020, To be submitted to MNRAS
  * Fitzpatrick (1999) Fitzpatrick E. L., 1999, PASP, 111, 63
  * Flaugher (2005) Flaugher B., 2005, International Journal of Modern Physics A, 20, 3121
  * Flaugher et al. (2015) Flaugher B., et al. 2015, AJ, 150, 150
  * Foucaud et al. (2007) Foucaud S., et al. 2007, MNRAS, 376, L20
  * Furusawa et al. (2008) Furusawa H., et al. 2008, ApJS, 176, 1
  * Gaia Collaboration (2016) Gaia Collaboration 2016, A&A, 595, A1
  * Galametz et al. (2013) Galametz A., et al. 2013, ApJS, 206, 10
  * Galametz et al. (2018) Galametz A., et al. 2018, MNRAS, 475, 4148
  * Garilli et al. (2014) Garilli B., et al. 2014, A&A, 562, A23
  * Giavalisco et al. (2004) Giavalisco M., et al. 2004, ApJ, 600, L93
  * González-Fernández et al. (2018) González-Fernández C., et al. 2018, MNRAS, 474, 5459
  * Guo et al. (2013) Guo Y., et al. 2013, ApJS, 207, 24
  * Hartley et al. (2013) Hartley W. G., et al. 2013, MNRAS, 431, 3045
  * Hartley et al. (2015) Hartley W. G., et al. 2015, MNRAS, 451, 1613
  * Hartley et al. (2020) Hartley W. G., et al. 2020, MNRAS, 496, 4769
  * Hasinger et al. (2018) Hasinger G., et al. 2018, ApJ, 858, 77
  * Herenz et al. (2017) Herenz E. C., et al. 2017, A&A, 606, A12
  * Hoyle et al. (2018) Hoyle B., et al. 2018, MNRAS, 478, 592
  * Ilbert et al. (2013) Ilbert O., et al. 2013, A&A, 556, A55
  * Jarvis et al. (2013) Jarvis M. J., et al. 2013, MNRAS, 428, 1281
  * Jarvis et al. (2016) Jarvis M., et al. 2016, MNRAS, 460, 2245
  * Jones et al. (2009) Jones D. H., et al. 2009, MNRAS, 399, 683
  * Joye & Mandel (2003) Joye W. A., Mandel E., 2003, New Features of SAOImage DS9. p. 489
  * Kashikawa et al. (2004) Kashikawa N., et al. 2004, PASJ, 56, 1011
  * Kim et al. (2011) Kim J. W., et al. 2011, MNRAS, 410, 241
  * Koekemoer et al. (2011) Koekemoer A. M., et al. 2011, ApJS, 197, 36
  * Kohonen (1982) Kohonen T., 1982, Biological Cybernetics, 43
  * Kovač et al. (2010) Kovač K., et al. 2010, ApJ, 718, 86
  * Kriek et al. (2015) Kriek M., et al. 2015, ApJS, 218, 15
  * Kron (1980) Kron R. G., 1980, ApJS, 43, 305
  * Kuijken et al. (2019) Kuijken K., et al. 2019, A&A, 625, A2
  * Lahav et al. (2020) Lahav O., et al. 2020, The Dark Energy Survey: The Story of a Cosmological Experiment. World Scientific
  * Laigle et al. (2016) Laigle C., et al. 2016, ApJS, 224, 24
  * Lane et al. (2007) Lane K. P., et al. 2007, MNRAS, 379, L25
  * Lang et al. (2016) Lang D., Hogg D. W., Mykytyn D., 2016, The Tractor: Probabilistic astronomical source detection and measurement (ascl:1604.008)
  * Lawrence et al. (2007) Lawrence A., et al. 2007, MNRAS, 379, 1599
  * Le Fèvre et al. (2005) Le Fèvre O., et al. 2005, A&A, 439, 845
  * Le Fèvre et al. (2013) Le Fèvre O., et al. 2013, A&A, 559, A14
  * Le Fèvre et al. (2015) Le Fèvre O., et al. 2015, A&A, 576, A79
  * Leauthaud et al. (2007) Leauthaud A., et al. 2007, ApJS, 172, 219
  * Lee et al. (2013) Lee B., et al. 2013, ApJ, 774, 47
  * Lidman et al. (2020) Lidman C., et al. 2020, MNRAS, 496, 19
  * Lilly et al. (2007) Lilly S. J., et al. 2007, ApJS, 172, 70
  * Lilly et al. (2009) Lilly S. J., et al. 2009, ApJS, 184, 218
  * Lonsdale et al. (2003) Lonsdale C. J., et al. 2003, PASP, 115, 897
  * Lotz et al. (2006) Lotz J. M., et al. 2006, ApJ, 636, 592
  * MacCrann et al. (2020) MacCrann N., et al. 2020, arXiv e-prints, p. arXiv:2012.08567
  * Maltby et al. (2019) Maltby D. T., et al. 2019, MNRAS, 489, 1139
  * Malz & Hogg (2020) Malz A. I., Hogg D. W., 2020, arXiv e-prints, p. arXiv:2007.12178
  * Masters et al. (2015) Masters D., et al. 2015, ApJ, 813, 53
  * Masters et al. (2017) Masters D. C., et al. 2017, ApJ, 841, 111
  * Masters et al. (2019) Masters D. C., et al. 2019, ApJ, 877, 81
  * McCracken et al. (2012) McCracken H. J., et al. 2012, A&A, 544, A156
  * McLure et al. (2013) McLure R. J., et al. 2013, MNRAS, 428, 1088
  * McLure et al. (2018) McLure R. J., et al. 2018, MNRAS, 479, 25
  * Merlin et al. (2015) Merlin E., et al. 2015, A&A, 582, A15
  * Merlin et al. (2016) Merlin E., et al. 2016, A&A, 595, A97
  * Momcheva et al. (2016) Momcheva I. G., et al. 2016, ApJS, 225, 27
  * Momose et al. (2020) Momose R., et al. 2020, arXiv e-prints, p. arXiv:2002.07335
  * Morganson et al. (2018) Morganson E., et al. 2018, PASP, 130, 074501
  * Mortlock et al. (2015) Mortlock A., et al. 2015, MNRAS, 447, 2
  * Mucesh et al. (2020) Mucesh S., et al. 2020, arXiv e-prints, p. arXiv:2012.05928
  * Muzzin et al. (2012) Muzzin A., et al. 2012, ApJ, 746, 188
  * Muzzin et al. (2013) Muzzin A., et al. 2013, ApJS, 206, 8
  * Myles et al. (2020) Myles J., et al. 2020, arXiv e-prints, p. arXiv:2012.08566
  * Neilsen et al. (2016) Neilsen Jr. E. H., et al. 2016, doi:10.2172/1250877
  * Neilsen et al. (2019) Neilsen Eric H. J., et al. 2019, doi:10.2172/1574836
  * Quadri et al. (2008) Quadri R. F., et al. 2008, ApJ, 685, L1
  * Rasmussen & Williams (2005) Rasmussen C. E., Williams C. K. I., 2005, Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press
  * Rest et al. (2014) Rest A., et al. 2014, ApJ, 795, 44
  * Robin & Creze (1986) Robin A., Creze M., 1986, A&A, 157, 71
  * Rowe et al. (2015) Rowe B. T. P., et al. 2015, Astronomy and Computing, 10, 121
  * Rudnick et al. (2003) Rudnick G., et al. 2003, ApJ, 599, 847
  * Sánchez & Bernstein (2019) Sánchez C., Bernstein G. M., 2019, MNRAS, 483, 2801
  * Sánchez et al. (2014) Sánchez C., et al. 2014, MNRAS, 445, 1482
  * Schlafly & Finkbeiner (2011) Schlafly E. F., Finkbeiner D. P., 2011, ApJ, 737, 103
  * Schlegel et al. (1998) Schlegel D. J., Finkbeiner D. P., Davis M., 1998, ApJ, 500, 525
  * Schmidt et al. (2020) Schmidt S. J., et al. 2020, MNRAS,
  * Scoville et al. (2007) Scoville N., et al. 2007, ApJS, 172, 1
  * Seitz & Schneider (1997) Seitz C., Schneider P., 1997, A&A, 318, 687
  * Sevilla-Noarbe et al. (2020) Sevilla-Noarbe I., et al. 2020, arXiv e-prints, p. arXiv:2011.03407
  * Sherman et al. (2020) Sherman S., et al. 2020, MNRAS, 491, 3318
  * Silverman et al. (2015) Silverman J. D., et al. 2015, ApJS, 220, 12
  * Skrutskie et al. (2006a) Skrutskie M. F., et al. 2006a, AJ, 131, 1163
  * Skrutskie et al. (2006b) Skrutskie M. F., et al. 2006b, AJ, 131, 1163
  * Somerville et al. (2004) Somerville R. S., et al. 2004, ApJ, 600, L171
  * Straatman et al. (2018) Straatman C. M. S., et al. 2018, ApJS, 239, 27
  * Suchyta et al. (2016) Suchyta E., et al. 2016, MNRAS, 457, 786
  * Tanaka et al. (2018) Tanaka M., et al. 2018, PASJ, 70, S9
  * Tasca et al. (2017) Tasca L. A. M., et al. 2017, A&A, 600, A110
  * Wake et al. (2011) Wake D. A., et al. 2011, ApJ, 728, 46
  * Williams et al. (1996) Williams R. E., et al. 1996, AJ, 112, 1335
  * Williams et al. (2009) Williams R. J., et al. 2009, ApJ, 691, 1879
  * Wisnioski et al. (2019) Wisnioski E., et al. 2019, ApJ, 886, 124
  * Wittman et al. (2002) Wittman D. M., et al. 2002, in Tyson J. A., Wolff S., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 4836, Survey and Other Telescope Technologies and Discoveries. pp 73–82 (arXiv:astro-ph/0210118), doi:10.1117/12.457348
  * Wright et al. (2019) Wright A. H., et al. 2019, A&A, 632, A34
  * Zhan & Knox (2006) Zhan H., Knox L., 2006, ApJ, 644, 663
  * Zuntz et al. (2018) Zuntz J., et al. 2018, MNRAS, 481, 1149

##  Appendix A Astrometry test with Gaussian Process

At the start of this project, UltraVISTA data from the DR3
release181818http://www.eso.org/sci/observing/phase3/data_releases/uvista_dr3.pdf
was used. When comparing matched source positions between the UltraVISTA DR3
data set and the DECam images of the COSMOS field, significant offsets were
found, as can be seen in the upper panels of Fig. 22. In order to place both
data sets into a common astrometric reference frame, we correct the offsets
using a Gaussian Process (GP) model (a supervised machine learning method, see
e.g., Rasmussen & Williams, 2005). We select bright stars from the matched
UltraVISTA-DECam catalogue with the condition:

$\displaystyle\tt{spread\\_model}+\frac{5}{3}\tt{spreaderr\\_model}<0.002,$
(9)

where spread_model and spreaderr_model are the corresponding columns from the
DECam SExtractor catalogue. We then take eighty per cent of these sources and
train a two dimensional Gaussian process for each RA offset and Declination
offset between the sources’ positions in the DECam and UltraVISTA images. The
result of applying the correction from this Gaussian Process model for the two
dimensional surfaces in RA and Declination offset are shown in the left hand
column of Fig. 22. The centre column of Fig. 22 also shows the offsets for the
remaining twenty per cent of the star sources which were reserved as a test
set (i.e., which were not used in training the GP). As can be seen from the
residual offsets, after correction with the GP, (centre row), the correction
has significantly reduced the systematic offsets between the UltraVISTA and
DECam positions. This can also be seen in the galaxy sample, in the right most
column of Fig. 22. The GP used was implemented using
GPy191919https://github.com/SheffieldML/GPy, with a Matern 5/2 kernel.
Separate GPs were trained in the stripes in RA defined by the UltraVISTA deep
and ultra-deep regions. We implemented a number of improvements on this
baseline configuration, including adding a white noise kernel to allow for
stellar motions; including bright, compact galaxies as extra training points;
and a number of different appropriate kernels. None of these led to
significant improvements in the residual offsets after correction. In
particular, tuning of kernels to allow for smaller scale variations in the
offset correction surface led to over-training, with smaller residuals for
training sources, but larger residuals for the test sources. A summary of the
improvements in the astrometry can be seen in the lower left panel of Fig. 22.

Upon the release of UltraVISTA
DR4202020http://ultravista.org/release4/dr4_release.pdf, the initial offsets
between UltraVISTA and DECam data were reduced to the same level as the post-
correction residuals for DR3. We therefore proceed with the DR4 data without
GP correction.

Figure 22: Residuals between UltraVISTA DR3 positions and DECam positions in
the COSMOS field. _Top_ row shows pre-correction offsets projected in the RA
direction, _middle_ row shows residuals following correction with the Gaussian
Process model, _lower_ row shows one dimensional histograms of the offsets
both pre- and post-correction. _Left_ column shows offsets for the sample of
stars used to train the Gaussian Process, _centre_ column shows offsets for
the reserved test stars, and _right_ column for the galaxy sources.

##  Appendix B Additional data

We expand on the description of the data in Sec. 2.1 by summarising the
quantities related to depth and image quality for the full set of 10 deep
fields in Table LABEL:tab:full_table.

In Fig. 23 we visualise the magnitude limits for the DECam data over each of
the four fields chosen for the cosmology catalogue and for each of the filters
$ugrizY$.

In Fig. 24 we plot the difference between the Deep-Fields catalogue PSF
magnitudes in $griz$ and the same objects found in HSC for SN-X3 and COSMOS.
In order to make the conversion between the DECam filters and the HSC filters,
we used the Laigle et al. (2016) catalogue and Brown et al. (2014) templates
to perform a guided interpolation from Subaru broad bands to each of DES and
HSC (see the appendix of Rudnick et al. 2003 for a description of the method
used). With the magnitude differences and DECam colours per object, we fit a
first-order polynomial to convert DECam bands to their HSC counterparts. The
r-band conversion was poorly represented by a simple polynomial, with
considerable dispersion at fixed colour.

For stellar sources, the $g,r$ and $z$ bands match within the expected
uncertainties ($0.5\%$ for the Deep-Fields catalogue and $1\%$ for HSC). The
For the $i$-band, the COSMOS field show a $4\%$ difference between the two
catalogues, the origin of which is unclear. There is a known issue in the HSC
Cosmos photometry (Aihara et al., 2018), though this is not expected to result
in an offset larger than $1.5\%$. We use PSF magnitudes for this comparison
and so do not expect the magnitudes of galaxies to match.

Figure 23: Magnitude limits calculated across each field. The fields (top to
bottom) are SN-C3, SN-E2, SN-X3, and COSMOS, respectively. The range shown for
each panel is set to show variations (within 1 magnitude) of the maximum depth
for a given field/band combination (the value for the maximum depth is given
in the upper right corner of each panel).

Figure 24: Comparison of PSF magnitudes between DES Deep Fields and HSC. DES
magnitudes have been converted to the HSC filter system.

##  Appendix C Catalog Release Columns

Table 8: Selected Deep Field Catalogue Columns. Note: names ending “*” show that the quantity is available for UGRIZJHKs filter bands; names ending “**” show that the quantity is available for redshift values from $z=0.01$ to $z=8$. Full details will be available upon release at https://des.ncsa.illinois.edu/releases.  Catalogue Column | Units | Description
---|---|---
ID | - | Object identifier
TILENAME | - | DECam field and chip number string
RA | Degrees | Equatorial Coordinates (J2000)
DEC | Degrees | Equatorial Coordinates (J2000)
BDF_FLUX_* | 3.63 nJy | Bulge+disk model flux with AB zero-point$=30$
BDF_FLUX_ERR_* | 3.63 nJy | Bulge+disk model flux uncertainty
BDF_FLUX_DERED_CALIB_* | 3.63 nJy | Corrected bulge+disk model flux with AB zero-point$=30$
BDF_FLUX_ERR_DERED_CALIB_* | 3.63 nJy | Corrected bulge+disk model flux uncertainty
BDF_MAG_DERED_CALIB_* | AB mag | Corrected bulge+disk model magnitude
BDF_MAG_ERR_DERED_CALIB_* | AB mag | Corrected bulge+disk model magnitude uncertainty
BDF_T | arcsec | Pre-seeing bulge+disk model size
BDF_T_RATIO | - | Bulge+disk model axis ratio
BDF_FRACDEV | - | Fraction of light contained in the bulge model component
PSF_FLUX_* | 3.63 nJy | PSF model flux with AB zero-point$=30$
PSF_FLUX_ERR_* | 3.63 nJy | PSF model flux uncertainty
PSF_MAG_* | AB mag | PSF model magnitude
PSF_MAG_ERR_* | AB mag | PSF model magnitude uncertainty
PSF_MAG_DERED_CALIB_* | AB mag | Corrected PSF model magnitude
PSF_MAG_ERR_DERED_CALIB_* | AB mag | Corrected PSF model magnitude uncertainty
PSF_T | arcsec | Model PSF size
GAP_FLUX_* | 3.63 nJy | Gaussian aperture flux with AB zero-point$=30$
GAP_FLUX_ERR_* | 3.63 nJy | Gaussian aperture flux uncertainty
EBV_SFD98 | - |  $E(B-V)$ value from Schlegel et al. (1998)
MASK_FLAGS | - | Flag containing mask information for DECam data: Unmasked sources=0
MASK_FLAGS_NIR | - | Flag containing mask information for VIRCam data: Unmasked sources=0
FLAGS | - | Flag identifier for DECam data
FLAGSTR | - | Flag information string for DECam data
FLAGS_NIR | - | Flag identifier for VIRCam data
FLAGSTR_NIR | - | Flag information string for VIRCam data
FOF_ID | - | Friends-of-friends group identifier
FOF_SIZE | - | Number of objects in friends-of-friends group
Z_P | - | Maximum posterior redshift value
Z_PEAK | - | Mean redshift of the most probable redshift peak
Z_MC | - | Random draw from probability distribution function
PDZ_** | - | Redshift probability density at $z=**$
KNN_CLASS | - | Star-galaxy classification: Galaxy=1, Star=2, Ambiguous=3, Unclassified=0

Table LABEL:tab:DF_catalog_columns provides descriptions of selected columns
from our catalogue which will appear alongside other data products released as
part of the DES Y3 cosmology analysis at
https://des.ncsa.illinois.edu/releases.

## Affiliations

1 Department of Astronomy, University of Geneva, ch. d’Ecogia 16, CH-1290
Versoix, Switzerland
2 Center for Cosmology and Astro-Particle Physics, The Ohio State University,
Columbus, OH 43210, USA
3 Kavli Institute for Particle Astrophysics & Cosmology, P. O. Box 2450,
Stanford University, Stanford, CA 94305, USA
4 Department of Astronomy, University of Illinois at Urbana-Champaign, 1002 W.
Green Street, Urbana, IL 61801, USA
5 National Center for Supercomputing Applications, 1205 West Clark St.,
Urbana, IL 61801, USA
6 Brookhaven National Laboratory, Bldg 510, Upton, NY 11973, USA
7 Department of Physics, University of Oxford, Denys Wilkinson Building, Keble
Road, Oxford OX1 3RH, UK
8 Jodrell Bank Center for Astrophysics, School of Physics and Astronomy,
University of Manchester, Oxford Road, Manchester, M13 9PL, UK
9 Department of Physics and Astronomy, University of Pennsylvania,
Philadelphia, PA 19104, USA
10 Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas
(CIEMAT), Madrid, Spain
11 Fermi National Accelerator Laboratory, P. O. Box 500, Batavia, IL 60510,
USA
12 Argonne National Laboratory, 9700 South Cass Avenue, Lemont, IL 60439, USA
13 Institute of Astronomy, University of Cambridge, Madingley Road, Cambridge
CB3 0HA, UK
14 Kavli Institute for Cosmology, University of Cambridge, Madingley Road,
Cambridge CB3 0HA, UK
15 Physics Department, 2320 Chamberlin Hall, University of Wisconsin-Madison,
1150 University Avenue Madison, WI 53706-1390
16 SLAC National Accelerator Laboratory, Menlo Park, CA 94025, USA
17 George P. and Cynthia Woods Mitchell Institute for Fundamental Physics and
Astronomy, and Department of Physics and Astronomy, Texas A&M University,
College Station, TX 77843, USA
18 University of Nottingham, School of Physics and Astronomy, Nottingham NG7
2RD, UK
19 School of Mathematics and Physics, University of Queensland, Brisbane, QLD
4072, Australia
20 Department of Physics, Carnegie Mellon University, Pittsburgh, Pennsylvania
15312, USA
21 Department of Astronomy and Astrophysics, University of Chicago, Chicago,
IL 60637, USA
22 Kavli Institute for Cosmological Physics, University of Chicago, Chicago,
IL 60637, USA
23 Santa Cruz Institute for Particle Physics, Santa Cruz, CA 95064, USA
24 Jet Propulsion Laboratory, California Institute of Technology, 4800 Oak
Grove Dr., Pasadena, CA 91109, USA
25 Department of Physics, Stanford University, 382 Via Pueblo Mall, Stanford,
CA 94305, USA
26 Department of Physics, The Ohio State University, Columbus, OH 43210, USA
27 Department of Applied Mathematics and Theoretical Physics, University of
Cambridge, Cambridge CB3 0WA, UK
28 Department of Physics, University of Michigan, Ann Arbor, MI 48109, USA
29 Department of Astrophysical Sciences, Princeton University, Peyton Hall,
Princeton, NJ 08544, USA
30 Department of Physics, ETH Zurich, Wolfgang-Pauli-Strasse 16, CH-8093
Zurich, Switzerland
31 Department of Physics, Duke University Durham, NC 27708, USA
32 Cerro Tololo Inter-American Observatory, NSF’s National Optical-Infrared
Astronomy Research Laboratory, Casilla 603, La Serena, Chile
33 Departamento de Física Matemática, Instituto de Física, Universidade de São
Paulo, CP 66318, São Paulo, SP, 05314-970, Brazil
34 Laboratório Interinstitucional de e-Astronomia - LIneA, Rua Gal. José
Cristino 77, Rio de Janeiro, RJ - 20921-400, Brazil
35 Institute of Cosmology and Gravitation, University of Portsmouth,
Portsmouth, PO1 3FX, UK
36 CNRS, UMR 7095, Institut d’Astrophysique de Paris, F-75014, Paris, France
37 Sorbonne Universités, UPMC Univ Paris 06, UMR 7095, Institut
d’Astrophysique de Paris, F-75014, Paris, France
38 Department of Physics and Astronomy, Pevensey Building, University of
Sussex, Brighton, BN1 9QH, UK
39 Department of Physics & Astronomy, University College London, Gower Street,
London, WC1E 6BT, UK
40 Instituto de Astrofisica de Canarias, E-38205 La Laguna, Tenerife, Spain
41 Universidad de La Laguna, Dpto. Astrofisica, E-38206 La Laguna, Tenerife,
Spain
42 Institut de Física d’Altes Energies (IFAE), The Barcelona Institute of
Science and Technology, Campus UAB, 08193 Bellaterra (Barcelona) Spain
43 Institut d’Estudis Espacials de Catalunya (IEEC), 08034 Barcelona, Spain
44 Institute of Space Sciences (ICE, CSIC), Campus UAB, Carrer de Can Magrans,
s/n, 08193 Barcelona, Spain
45 INAF-Osservatorio Astronomico di Trieste, via G. B. Tiepolo 11, I-34143
Trieste, Italy
46 Institute for Fundamental Physics of the Universe, Via Beirut 2, 34014
Trieste, Italy
47 Observatório Nacional, Rua Gal. José Cristino 77, Rio de Janeiro, RJ -
20921-400, Brazil
48 Department of Astronomy, University of California, Berkeley, 501 Campbell
Hall, Berkeley, CA 94720, USA
49 Department of Physics, IIT Hyderabad, Kandi, Telangana 502285, India
50 Faculty of Physics, Ludwig-Maximilians-Universität, Scheinerstr. 1, 81679
Munich, Germany
51 Department of Astronomy/Steward Observatory, University of Arizona, 933
North Cherry Avenue, Tucson, AZ 85721-0065, USA
52 Institute of Theoretical Astrophysics, University of Oslo. P.O. Box 1029
Blindern, NO-0315 Oslo, Norway
53 Instituto de Fisica Teorica UAM/CSIC, Universidad Autonoma de Madrid, 28049
Madrid, Spain
54 Department of Astronomy, University of Michigan, Ann Arbor, MI 48109, USA
55 Center for Astrophysics $|$ Harvard & Smithsonian, 60 Garden Street,
Cambridge, MA 02138, USA
56 Australian Astronomical Optics, Macquarie University, North Ryde, NSW 2113,
Australia
57 Lowell Observatory, 1400 Mars Hill Rd, Flagstaff, AZ 86001, USA
58 Department of Astronomy, The Ohio State University, Columbus, OH 43210, USA
59 Radcliffe Institute for Advanced Study, Harvard University, Cambridge, MA
02138
60 Institució Catalana de Recerca i Estudis Avançats, E-08010 Barcelona, Spain
61 Max Planck Institute for Extraterrestrial Physics, Giessenbachstrasse,
85748 Garching, Germany
62 School of Physics and Astronomy, University of Southampton, Southampton,
SO17 1BJ, UK
63 Computer Science and Mathematics Division, Oak Ridge National Laboratory,
Oak Ridge, TN 37831
64 Universitäts-Sternwarte, Fakultät für Physik, Ludwig-Maximilians
Universität München, Scheinerstr. 1, 81679 München, Germany
65 Institute for Astronomy, University of Edinburgh, Edinburgh EH9 3HJ, UK