# From Easy to Hard: Two-stage Selector and Reader
for Multi-hop Question Answering

Xin-Yi Li, Wei-Jun Lei, Yu-Bin Yang
State Key Laboratory for Novel Software Technology
Nanjing University, Nanjing 21023, China
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Multi-hop question answering (QA) is a challenging task requiring QA systems
to perform complex reasoning over multiple documents and provide supporting
facts together with the exact answer. Existing works tend to utilize graph-
based reasoning and question decomposition to obtain the reasoning chain,
which inevitably introduces additional complexity and cumulative error to the
system. To address the above issue, we propose a simple yet effective novel
framework, From Easy to Hard (FE2H), to remove distracting information and
obtain better contextual representations for the multi-hop QA task. Inspired
by the iterative document selection process and the progressive learning
custom of humans, FE2H divides both the document selector and reader into two
stages following an easy-to-hard manner. Specifically, we first select the
document most relevant to the question and then utilize the question together
with this document to select other pertinent documents. As for the QA phase,
our reader is first trained on a single-hop QA dataset and then transferred
into the multi-hop QA task. We comprehensively evaluate our model on the
popular multi-hop QA benchmark HotpotQA. Experimental results demonstrate that
our method ourperforms all other methods in the leaderboard of HotpotQA
(distractor setting).

##  1 Introduction

Figure 1: An example of multi-hop QA from HotpotQA benchmark. The illustrated
reasoning process echoes our two-stage document selection that performed from
easy to hard.

Question answering (QA) is a popular benchmark task in natural language
understanding that aims to teach the machine to answer questions according to
the given context. Earlier QA datasets such as SQuAD Rajpurkar et al. (2016a)
mainly focus on evaluating the ability of finding answers with a single
document per question. Subsequently, more challenging datasets that provide
each question with multiple documents appear Joshi et al. (2017); Dunn et al.
(2017). However, most questions of these datasets could still be solved using
only one document. Recently, many datasets Welbl et al. (2018); Talmor and
Berant (2018); Yang et al. (2018) that require multi-hop reasoning over
multiple documents are constructed. In this paper, we focus on HotpotQA Yang
et al. (2018), which is one of the representative multi-hop QA benchmarks.
Each question in the HotpotQA Yang et al. (2018) is provided with a set of
documents that includes both relevant and irrelevant documents. In addition to
the exact answer spans, HotpotQA Yang et al. (2018) also offers supporting
sentences that served as explanations to train and evaluate the QA systems.

Following the current mainstream pipeline for multi-hop QA Fang et al. (2020);
Tu et al. (2020); Wu et al. (2021), our proposed framework also consists of a
selector to minimize distracting information and a reader to provide exact
answers and explanations. As for the design of the selector, most earlier
works ignore the multi-hop nature of the task and treat the candidate
documents or sentences independently. To address this problem, recent works
include SAE Tu et al. (2020) and S2G Wu et al. (2021) apply a multi-head self-
attention (MHSA) layer to encourage inter-document interactions. However, they
still use a pre-trained language model (PLM) to encode each document
separately before this MHSA layer, which cannot fully exploit the strong
modeling and natural language understanding capacity of the PLMs and thus
results in limited interactions. Notice that although S2G further adopts
cascaded retrieval that fed with the text of the coarse-selected documents
compared to SAE, these selectors aim to find all of the relevant documents at
one time, ignoring the sequential nature of the relevant documents brought by
the multi-hop reasoning.

Different from the above-mentioned selectors, our proposed selector consists
of two stages, where one document is selected at a stage. Specifically, we
first select the document that is most relevant to the question and then
select the other one based on the question and the previously selected
document. Figure 1 illustrates an example of this two-stage document selection
process when we are answering multi-hop questions, which suggests that our
two-stage selector shows great consistency to the reasoning process of humans.
Concretely, we first arrive at document $D_{10}$ according to the question,
which indicates that “Seven Brief Lessons on Physics is written by Carlo
Rovelli”. Next, we identify the document containing “Carlo Rovelli has worked
in France since 2000” using the question together with document $D_{10}$. Our
experiments demonstrate that with the help of the two-stage easy-to-hard
manner, even simply adding a binary classification layer on a PLM could
achieve strong performance.

After document selection, the filtered context is passed to the question
answering module, namely the reader. Many existing works focus on guiding the
readers to imitate the human reasoning process with elaborately designed
pipelines, which can be roughly divided into two categories. One category is
graph-based approaches, including SAE Tu et al. (2020) that treats sentences
as graph nodes and HGN Fang et al. (2020) that builds a hierarchical graph
with nodes of different granularities. However, these methods require
additional techniques, including named entity recognition and graph modeling,
which inevitably lead to the growth of complexity and error accumulation. Due
to these disadvantages, there have been works demonstrating that graph
modeling may not be indispensable Shao et al. (2020). Consequently, we also do
not exploit any graph-based reasoning methods for the sake of simplicity.
Another category is question decomposition methods, such as DecompRC Min et
al. (2019) and ONUS Perez et al. (2020) that map one multi-hop, hard question
to many single-hop, easy sub-questions. Although we appreciate the idea of
decomposing the hard task into easy ones, the difficulty and complexity
brought by the sub-question generation cannot be ignored.

Inspired by the question decomposition-based methods and the experience that
children are usually learned from the very easy questions, we pose the
question that whether it is too hard and confusing for models to directly
learn to answer multi-hop questions from scratch. Therefore, our reader is
firstly trained on a simpler single-hop dataset SQuAD Rajpurkar et al. (2016b)
before performing the harder, multi-hop QA task. We assume that training on
the single-hop dataset could initialize the model with better parameters and
make it easier for the model to answer multi-hop questions. In addition, the
process of learning from easy to hard is consistent with the widely accepted
concept that there supposed to be a gradual progression of skill acquisition
Liu et al. (2017); Roads et al. (2018); Song et al. (2019). Our experiments
illustrate that the two-stage reader effectively boosts the performance, which
verifies our assumption.

The main contributions of our work are summarized as follows:

  * •

We propose a simple yet effective multi-hop QA framework called FE2H, where
both the document selector and reader are divided into two stages following
the easy-to-hard manner.

  * •

We introduce a novel document selection module that iteratively performs
binary classification tasks to select relevant documents by simply adding a
prediction layer on a PLM.

  * •

The proposed reader is first trained on a single-hop QA dataset and then
transferred into the multi-hop QA task, which is inspired by the progressive
learning process of humans.

  * •

We comprehensively evaluate our model on the popular multi-hop QA benchmark
HotpotQA. Experimental results show that the proposed FE2H can surpass the
previous state-of-the-art by a large margin across all metrics.

##  2 Related Work

#### Selector for QA

Sentence or document selector aims to minimize noisy information, which has
been proved effective for improving the performance and efficiency of the QA
system. The study in Min et al. (2018) proposes a sentence selector for SQuAD
Rajpurkar et al. (2016b) since questions can be answered within a single
sentence. As for multi-hop QA, Wang et al. (2019) and Groeneveld et al. (2020)
simply treat the documents and sentences separately, ignoring the interaction
between them. Therefore, SAE Tu et al. (2020) and S2G Wu et al. (2021) append
an MHSA layer to encourage document interaction. To acquire the finer-grained
result, S2G Wu et al. (2021) further adopt another cascaded paragraph
retrieval module, which takes the text of the selected documents as input and
subsequently explore deeper relationships between them. Since SAE and S2G
attempt to locate all of the relevant documents at the same time, a naive
binary classifier could not perform well. Therefore, SAE Tu et al. (2020) and
S2G Wu et al. (2021) reformulate the classification objective into a ranking
and scoring objective respectively to conform with the ranking nature of the
selector. Different from the above selectors, we present a novel two-stage
selector that neither selects the documents independently nor simultaneously.

#### Reader for multi-hop QA

Existing multi-hop QA readers can be roughly divided into two categories:
reasoning based on intermediate results and reasoning based on graph. The
first category can be further grouped by whether the intermediate result is
represented in hidden state or free form text. As for the former group, QFE
Nishida et al. (2019) identifies relevant sentences by updating an RNN hidden
state vector in each step, others update the representation vector of query
and context iteratively to obtain the final answer Das et al. (2018); Feldman
and El-Yaniv (2019). The representative works of the latter group mainly
acquire the intermediate textual result by firstly decomposing the original
question or generating a sub-question and then adopting off-the-shelf readers
Qi et al. (2019); Min et al. (2019); Perez et al. (2020). The second category
exploits graph neural networks to reason over the constructed graph. DFGN Qiu
et al. (2019) and CogQA Ding et al. (2019) focus on the entity graph while SAE
Tu et al. (2020) treats sentences as graph nodes to identify supporting
sentences. Furthermore, Tu et al. (2019), HGN Fang et al. (2020) and AMGN Li
et al. (2021) reasoning over the heterogeneous graph with different types of
nodes and edges. However, there are recent works that question whether the
graph structure is necessary. Quark Groeneveld et al. (2020), C2F Reader Shao
et al. (2020) and S2G Wu et al. (2021) demonstrate that the performances of
the simple graph-free readers are on par with the graph-based reasoning
methods. C2F Reader Shao et al. (2020) further claims that graph-attention can
be considered as a special case of self-attention. In this paper, we propose a
reader that simply follows a two-stage pipeline and free of any recurrent
state updating and graph modeling.

##  3 Proposed Framework

Figure 2: Overview of our proposed framework FE2H. The left part shows our
whole pipeline, including a first stage selector to find the most relevant
document to the question, a second stage selector to find the other relevant
document according to the question and the previously selected document, and a
question answering module to extract answers and supporting facts based on the
selected documents and question. The right part illustrates the detailed
computation process of our cross attention enhanced reading comprehension
model.

We choose the distractor setting of HotpotQA Yang et al. (2018) as our
testbed, where each question is equipped with 2 relevant documents and 8
distractors. A list of supporting sentences and an answer that could be an
exact text span or “Yes/No” are also provided for evaluation. The overview of
our proposed framework is shown in Figure 2. Following the traditional
pipeline of multi-hop QA system, We first utilize a document selection module
to construct almost noise-free context and then pass them to the downstream
question answering module to extract supporting facts and answer span
simultaneously.

###  3.1 Document Selection Module

At this phase, we aim to filter the distracting information and generate high-
quality context, namely almost noise-free context, for the following question
answering module. Note that although there are many works that perform
recurrent reasoning using graph modeling and question decomposition when
extracting supporting sentences and answers, existing selectors still extract
relevant documents independently or simultaneously, which has been discussed
in Section 1 and Section 2. Therefore, we assume that the recurrent reasoning
manner should also be applied in a larger granularity, namely the document
level. Recall that when we answer a multi-hop question, in most cases, we
first select the document most relevant to the question and use this document
together with the question to find another. Inspired by this insight, we
divide the document selection into two steps, one document at a step, which is
different from the existing methods.

#### The first stage

In this stage, we aim to select the document that is most relevant to the
question. We feed the sequence “[CLS] + question + [SEP] + document + [SEP]”
to a PLM ELECTRA Clark et al. (2020) and project the output of token “[CLS]”
to calculate the relevant score $P(d|q)$ for each candidate document $d$ and
question $q$. We label all the documents containing supporting facts as
relevant and calculate the binary cross-entropy loss as:

$\displaystyle\mathcal{L}_{S_{1}}=-\sum_{i=1}^{N}\sum_{j=1}^{M}(t_{ij}\log
P(d_{ij}|q_{i})$ (1) $\displaystyle+(1-t_{ij})\log(1-P(d_{ij}|q_{i})))$

where $q_{i}$ is the $i$th question in the dataset, $d_{ij}$ is the $j$th
candidate document of $q_{i}$, $t_{ij}$ is the label of the question-document
pair $(q_{i},d_{ij})$, $N$ is the number of questions and $M$ is the size of
each candidate document set, which is 10 for HotpotQA. We select the document
with the highest relevance score and denote the chosen document as $p_{1}$.
That is, given question $q_{i}$,
$p_{i1}=\mathop{\arg\max}\limits_{j}P(d_{ij}|q_{i})$.

#### The second stage

The aim of this stage is to find the other relevant document based on the
previous result. Therefore, we generate the input as “[CLS] + question + [SEP]
+ $\rm{document}_{1}$ \+ [SEP] + $\rm{document}_{2}$ \+ [SEP]”, where
$\rm{document}_{1}$ is the document selected at the first step and
$\rm{document}_{2}$ is another candidate document. We calculate the relevance
score of the candidate document as the previous step, which is denoted as
$P(d|q,p)$. The binary cross-entropy loss is computed as:

$\displaystyle\mathcal{L}_{S_{2}}=-\sum_{i=1}^{N}\sum_{j=1}^{M}\mathbb{I}(d_{ij}\neq
p_{i1})(t_{ij}\log P(d_{ij}|q_{i},p_{i1})$ (2)
$\displaystyle+(1-t_{ij})\log(1-P(d_{ij}|q_{i},p_{i1})))$

where $\mathbb{I}$ is the indicator function. Likewise, we select the document
with the highest relevance score of this step and denote it as $p_{2}$.

After the two-stage retrieval, the selected documents for each question are
concatenated and passed to the following question answering module to serve as
the high-quality context.

###  3.2 Question Answering Module

#### Multi-task Model

In this phase, we implement a multi-task model to extract the answers and the
supporting facts of the multi-hop questions, which is illustrated in Figure 2.
Firstly, we concatenate the context and the question as a sequence and use a
PLM called ELECTRA Clark et al. (2020) to obtain the contextual
representations of the input text. Subsequently, We exploit cross attention to
enhance the interaction between the context and the question to acquire better
context embeddings. Finally, a multi-task learning approach is adopted to
jointly extract the answer and supporting sentences by simply adding a linear
prediction layer.

Firstly, we are supposed to construct the input sequence as the input for
ELECTRA Clark et al. (2020). Considering the “Yes/No” answer case, “yes” and
“no” tokens are added to the beginning of the sequence. Besides, we add a
“[UNK]” before each sentence in the context, whose contextual representation
is used for supporting sentence classification. Therefore, our input sequence
can be formulated as “[CLS] + yes + no + context + [SEP] + question + [SEP]”.
We denote the ELECTRA output of the input as
$H=[{{h}_{1}},\cdots,{{h}_{L}}]\in{{\mathbb{R}}^{L\times{d}}}$ , the context
output as
${{H}_{c}}=[{{h}_{1}},\cdots,{{h}_{S}}]\in{{\mathbb{R}}^{S\times{d}}}$ and the
question output as
${{H}_{q}}=[{{h}_{S+1}},\cdots,{{h}_{L}}]\in{{\mathbb{R}}^{T\times{d}}}$ ,
where $L$ is the length of the input sequence, $S$ is the length of the
context, $T=L-S$ is the length of the query, and ${d}$ is the hidden vector
dimension of ELECTRA.

To improve the QA performance, we further add a cross attention interaction
module to enhance the interaction between the context and query
representations. At the same time, in order to obtain a better result and make
the convergence faster during training, we use two cross attention blocks,
including one with layer normalization and the other without layer
normalization. The cross attention is computed as:

$\displaystyle{Q}$ $\displaystyle=H_{c}W^{Q}\\!$ (3) $\displaystyle{K}$
$\displaystyle=H_{q}W^{K}\\!$ (4) $\displaystyle{V}$
$\displaystyle=H_{q}W^{V}\\!$ (5) $\displaystyle\operatorname{CrossAttention}$
$\displaystyle=\operatorname{Attention}(Q,K,V)$ (6)
$\displaystyle\operatorname{Attention}(Q,K,V)$
$\displaystyle=\operatorname{softmax}(\frac{Q{{K}^{\mathsf{T}}}}{\sqrt{{{d}_{k}}}})V$
(7)

To extract the answer spans, we utilize a linear prediction layer on the
context representations to identify the start and end position of the answer.
The corresponding loss items are denoted as ${\mathcal{L}_{start}}$ and
${\mathcal{L}_{end}}$ respectively. As for the supporting facts prediction, we
utilize a linear binary classifier on the output of the “[UNK]” that added
before each sentence to decide whether a sentence is relevant or not. The
classification objective of supporting facts is denoted as
${\mathcal{L}_{sf}}$. Finally, we jointly optimize the above objectives as:

$\displaystyle\mathcal{L}={{\gamma}_{1}}{\mathcal{L}_{start}}+{{\gamma}_{2}}{\mathcal{L}_{end}}+{{\gamma}_{3}}{\mathcal{L}_{sf}}$
(8)

#### Two-stage Reader

As discussed in Section 1, we assume that directly fine-tuned a PLM on a
multi-hop QA task may be too hard and confusing for models. As a result, the
model may answer the multi-hop questions with the single-hop shortcut, namely
word-matching the question with a single sentence Jiang and Bansal (2019).
Therefore, we first train our model on a single-hop QA dataset SQuAD Rajpurkar
et al. (2016b) and then transfer the model to the multi-hop QA task. We
suppose that the parameters that optimized by the single-hop QA task are much
better than the random initialized parameters. Consequently, after learning
single-hop QA task, the model tends to converge faster and achieve higher
performance. Note that the model structure of the single-hop and multi-hop
stage is the same.

##  4 Experiments

Model | Ans | Sup | Joint
---|---|---|---
EM | F1 | EM | F1 | EM | F1
Baseline Model Yang et al. (2018) | 45.60 | 59.02 | 20.32 | 64.49 | 10.83 | 40.16
QFE Nishida et al. (2019) | 53.86 | 68.06 | 57.75 | 84.49 | 34.63 | 59.61
DFGN Qiu et al. (2019) | 56.31 | 69.69 | 51.50 | 81.62 | 33.62 | 59.82
SAE-large Tu et al. (2020) | 66.92 | 79.62 | 61.53 | 86.86 | 45.36 | 71.45
C2F Reader Shao et al. (2020) | 67.98 | 81.24 | 60.81 | 87.63 | 44.67 | 72.73
HGN-large Fang et al. (2020) | 69.22 | 82.19 | 62.76 | 88.47 | 47.11 | 74.21
AMGN+ Li et al. (2021) | 70.53 | 83.37 | 63.57 | 88.83 | 47.77 | 75.24
S2G+EGA Wu et al. (2021) | 70.92 | 83.44 | 63.86 | 88.68 | 48.76 | 75.47
SAE+1 | 70.74 | 83.61 | 63.70 | 88.95 | 48.15 | 75.72
FE2H on ELECTRA(ours) | 69.54 | 82.69 | 64.78 | 88.71 | 48.46 | 74.90
FE2H on ALBERT(ours) | 71.89 | 84.44 | 64.98 | 89.14 | 50.04 | 76.54
Table 1:  Comparison of the results on the test set of HotpotQA in the
distractor setting. FE2H model obtains the overall competitive performance and
achieves the state-of-the-art results in all metrics on the leaderboard among
all works including unpublished methods such as SAE+1, demonstrating the
strong multi-hop reasoning ability of our model.

###  4.1 Dataset

We evaluate our model on the distractor setting of HotpotQA Yang et al.
(2018). HotpotQA consists of 113k Wikipedia-based and crowdsourced question-
answer pairs, including 90K training examples, 7.4K development examples and
7.4K test examples.

In the distractor setting, each question in the dataset is provided with a
small set of documents that contains two supporting documents and eight
irrelevant documents. HotpotQA Yang et al. (2018) also provides a list of
supporting sentences for each question, which encourages model to explain the
predictions. The Exact Match (EM) and F1 scores are used to evaluate models
for both tasks and joint EM and F1 scores are used in the evaluation of the
overall performance. The performance of our model on the test set is obtained
by submitting our best model on the development set. Since the test set is not
available, we report the performance of our comprehensive experiments on the
development set.

###  4.2 Implementation Details

Our model is implemented based on the transformers of Hugging Face Wolf et al.
(2020) and we adopt ELECTRA and ALBERT as our PLM to obtain the contextual
embeddings of the questions and context. Our experiments are all performed on
2 or 3 NVIDIA A100 GPUs. The implementation details of our two-stage selector
and reader are listed as follows.

#### Two-stage Selector

We train two document selectors with a base and a large version of ELECTRA,
respectively. Due to resource limitation, the ablations on document selection
is only performed on ELECTRA-base. The number of epochs is set to 3 and batch
size is set to 12. We use BERT-Adam with learning rate of 2e-5 for the
optimization. The hyper-parameters of our two selectors of different stages
are same while the model parameters are different and trained sequentially.

#### Two-stage Reader

As for the question answering phase, we train the FE2H on ELECTRA model with
batch size of 16 with 2 NVIDIA A100 GPUs, learning rate of 1.5e-5, warm-up
rate of 0.1, and L2 weight decay of 0.01. Besides, we use ELECTRA-large as the
PLM of this phase. It takes about 2 hours per epoch and 6.5 hours in total to
train a reader of a single stage. we train the FE2H on ALBERT model with batch
size of 12 with 3 NVIDIA A100 GPUs, learning rate of 1.0e-5, warm-up rate of
0.1, and L2 weight decay of 0.01. Besides, we use Albert-xxlarge-v2 as the PLM
of this phase. It takes about 25 hours in total to train a reader of a single
stage.

###  4.3 Experimental Results

#### Results on HotpotQA

Table 1 shows the performance of published advanced methods on the test set of
HotpotQA in the distractor setting. Despite the simplicity of our model, Our
FE2H on ALBERT is the state-of-the-art model on the leaderboard111Leaderboard
available at https://hotpotqa.github.io. Our model achieves the best result in
all metrics, demonstrating the strong multi-hop reasoning ability of our
model. The Joint EM improvement is 1.28% and the Joint F1 improvement is 1.07%
among all the published methods. And the Joint F1 improvement is 0.82% among
all the submitted methods including SAE+1.

Model | EM | F1
---|---|---
$\mathrm{SAE}_{large}$ | 91.98 | 95.76
HGN | - | 94.53
$\mathrm{S2G}_{large}$ | 95.77 | 97.82
$\mathrm{FE2H}_{base}$ (Ours) | 95.53 | 97.59
$\mathrm{FE2H}_{large}$ (Ours) | 96.32 | 98.02
Table 2:  Document selection results on the dev set of HotpotQA. “-” means the
data is not available.Bold ones indicate state of the art, and underlined ones
indicate the second best result.

#### Document Selection

We compare our selector with the ones of three published advanced works, that
is, HGN, SAE and S2G, which also apply carefully designed selectors to
retrieve relevant documents for the downstream readers. We use EM and F1 to
evaluate the performance of the document selectors. As shown in Table 2, our
selector outperforms these three strong baselines. Even the base version of
our selector achieves competitive performance compared to the previous best
selector.

###  4.4 Ablation

Setting | EM | F1
---|---|---
FE2H | 95.53 | 97.59
1st Stage + Top 2 | 89.10 | 94.41
1st Stage + THOLD | 85.10 | 87.01
1st Stage (AW) + Top 2 | 84.17 | 91.78
1st Stage (AW) + 2nd Stage | 92.72 | 96.09
Table 3:  Ablation on two-stage selector on dev set of HotpotQA. works.
“THOLD” means using a threshold to select documents. “AW” means training with
assigned weights.

#### Two-stage Selector Ablation

The results of the two-stage selector ablation are shown in Table 3. To prove
the validity of out two-stage selector, we remove the second stage and
evaluate the results. Results illustrate that, given the output of the first
stage, no matter we select the top 2 documents or select the documents whose
scores above a certain threshold, both EM and F1 score drop significantly.
Note that both SAE and S2G believe that the documents containing the answer
span (i.e., gold documents) are more important for downstream tasks and thus
assign these documents with higher score. However, we assume that relevant
documents are of equal importance, especially for the first stage of our
document selection. Since we do not reformulate the binary classification
objective to ranking-related objectives like SAE and S2G, we assign greater
weights to the gold documents to verify this assumption. Results illustrate
that performance drops significantly after reassigning the weights of the
relevant documents as above at the first stage. It is worth pointing that the
performance of single-stage setting degrades more compared to the two-stage
setting, indicating that directly guiding the selector to identify the gold
documents is infeasible, thus both relevant documents are equally important.

Setting | Joint F1 | Joint EM
---|---|---
Ours | 75.30 | 48.55
\- Cross Attention w/ LN | 75.14 | 48.37
\- Cross Attention w/o LN | 75.07 | 48.37
\- Cross Attention Block | 75.20 | 48.19
Two Cross Attention w/ LN | 74.98 | 48.31
Two Cross Attention w/o LN | 75.18 | 48.29
Feed-Forward Network | 75.05 | 48.00
Self-Attention | 74.85 | 48.00
Bi-Attention | 74.47 | 47.29
Co-Attention | 74.10 | 46.92

Table 4: Ablation results on components of the question answering module.

#### QA Model Ablation

Due to the computing resource limitations, we only use a single-stage reader,
i.e. the reader without training on the single-hop QA dataset, to perform the
ablation of QA model structure, which is shown in Table 4. Specifically, we
investigate the effect of different interaction layers between the question
and the context on the performance of extracting supporting facts and answers.

Recall that our QA model exploits a cross attention block that consists of a
cross attention layer with layer normalization and one without. We first
ablate the cross attention block of our QA model. As shown in Table 4, both
the joint F1 score and joint EM drops whether one or two cross attention
layers are removed. Furthermore, in order to explore the effect of the
asymmetric structure of our cross attention block, we replace it with two same
cross attention layers with and without layer normalization, respectively.
Experimental results show that our cross attention block outperforms the above
two settings. We hypothesis that this is because the one with layer
normalization tends to converge faster Ba et al. (2016) and the one without
layer normalization tends to explore a better global solution due to the
randomness of the layer. As a result, our cross attention block combines the
advantages of both and achieve better performance.

In addition to the cross attention layer, we attempt to explore the effect of
other layers on enhancing the interaction between the question and the
context, which includes self-attention, co-attention, bi-attention, and feed-
forward layers. These layers are directly added to the PLM during experiments.
Note that our QA model without the cross attention block is equivalent to
directly predicting results based on the output of PLM, namely the baseline QA
model, while our experiments show that adding these layers all lead to
performance degradation compared to the baseline. For these degradations, our
intuitive explanations are listed as follows. Since the feed-forward network
only takes the context embeddings as input, thus prohibits the interaction
between the question and context, which probably results in performance
degradation. As for self-attention, recall that the PLM has already contained
many self-attention layers, thus learning an extra self-attention layer from
scratch may not help. Moreover, because the interactions modeled by bi-
attention Seo et al. (2017) and co-attention Xiong et al. (2017) are more
complicated compared to the ones of self-attention and cross-attention, we
suppose that it is hard to train a model with many pre-trained self-attention
layers and a random-initialized bi-attention or co-attention layer.

In summary, however, we come to the conclusion that though our cross attention
block improves the performance, the improvement is not significant and many
other interaction mechanisms even negatively affect the performance. That is
to say, without any additional computations, directly fine-tuned PLMs could
achieve competitive performance on the downstream tasks.

PLM | First Stage | Second Stage | Joint F1 | Joint EM
---|---|---|---|---
BERT | SHD | MHD | 68.58 | 41.90
| MHD | - | 67.13 | 39.86
| SHD+MHD | - | 68.37 | 41.51
| SHD | SHD+MHD | 68.23 | 41.89
ELECTRA | SHD | MHD | 75.73 | 49.37
| MHD | - | 75.30 | 48.55
| SHD+MHD | - | 75.43 | 48.87
| SHD | SHD+MHD | 75.55 | 48.48

Table 5: Ablation results on training strategies of the two-stage reader. SHD
means single-hop dataset SQuAD and MHD means multi-hop dataset HotpotQA.

#### Two-stage Reader Ablation

To demonstrate the effectiveness of the two-stage manner of our reader, we
perform our ablation studies using a smaller PLM BERT-base Devlin et al.
(2019) and a larger PLM ELECTRA-large Clark et al. (2020). The results are
illustrated in Table 5, which demonstrate that the two-stage manner brings
more significant performance improvement on BERT Devlin et al. (2019) than
ELECTRA. Specifically, joint F1 drops 1.45% and joint EM drops 2.04% when only
training on the multi-hop QA dataset. We hypothesis that this is because
larger PLM like ELECTRA has learned much more common knowledge and obtained
stronger natural language understanding ability, thus being further fine-tuned
on simple tasks like single-hop QA may be of limited help. While as for
smaller PLM like BERT Devlin et al. (2019), single-hop QA datasets could offer
more task-specific knowledge to the PLM and effectively bridge the gap between
the PLM and the multi-hop QA task. Therefore, we recommend the two-stage
reader, especially under the circumstances that resources are limited and only
small PLMs are permitted.

Subsequently, we perform experiments using both single-hop and multi-hop QA
datasets but with different training strategies. As illustrated in Table 5,
performance drops when we perform single-hop and multi-hop QA tasks
simultaneously, which indicates that the performance improvement of our two-
stage reader is not only brought by the use of more labeled QA pairs, but also
brought by the easy-to-hard two-stage manner. Notice that only using the
single-hop dataset at the first stage and using both datasets at the second
stage also leads to performance degradation. We presume that this is because
once the model has learned the easy task, feeding it with the datasets of both
tasks in the next step is of little use and even probably negatively impact
the training results.

##  5 Conclusion

We propose FE2H, a simple yet effective framework for multi-hop QA that
divides both the document selection and question answering into two stages
following an easy-to-hard manner. Experimental results demonstrate that since
we cannot feed all of the candidate documents to the PLMs at a time due to the
input length limitation, taking the multi-hop reasoning nature into
consideration at the document selection phase significantly improves the
overall performance. As for the subsequent QA phase, thanks to the great
natural language understanding ability of the PLMs, the performance of our
simple two-stage reader is better than the state-of-the-art approaches without
any graph structure and explicit reasoning chains. We hope this work could
facilitate more simple yet powerful multi-hop QA approaches with the help of
the advanced PLMs.

## Acknowledgements

We thank many colleagues at Nanjing University for their help, particularly
Qing-Long Zhang, for useful discussion.

## References

  * Ba et al. (2016) Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016.  Layer normalization.  _arXiv:1607.06450_.
  * Clark et al. (2020) Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020.  ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators.  In _ICLR_.
  * Das et al. (2018) Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, and Andrew McCallum. 2018.  Multi-step retriever-reader interaction for scalable open-domain question answering.  In _ICLR_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _NAACL-HLT_.
  * Ding et al. (2019) Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang. 2019.  Cognitive graph for multi-hop reading comprehension at scale.  In _ACL_.
  * Dunn et al. (2017) Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Güney, Volkan Cirik, and Kyunghyun Cho. 2017.  Searchqa: A new q&a dataset augmented with context from a search engine.  _CoRR_ , abs/1704.05179.
  * Fang et al. (2020) Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, and Jingjing Liu. 2020\.  Hierarchical graph network for multi-hop question answering.  In _EMNLP_.
  * Feldman and El-Yaniv (2019) Yair Feldman and Ran El-Yaniv. 2019.  Multi-hop paragraph retrieval for open-domain question answering.  In _ACL_.
  * Groeneveld et al. (2020) Dirk Groeneveld, Tushar Khot, Mausam, and Ashish Sabharwal. 2020.  A simple yet strong pipeline for HotpotQA.  In _EMNLP_.
  * Jiang and Bansal (2019) Yichen Jiang and Mohit Bansal. 2019.  Avoiding reasoning shortcuts: Adversarial evaluation, training, and model development for multi-hop QA.  In _ACL_.
  * Joshi et al. (2017) Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017.  TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension.  In _ACL_.
  * Li et al. (2021) Ronghan Li, Lifang Wang, Shengli Wang, and Zejun Jiang. 2021.  Asynchronous multi-grained graph network for interpretable multi-hop reading comprehension.  In _IJCAI_.
  * Liu et al. (2017) Weiwei Liu, Ivor W Tsang, and Klaus-Robert Müller. 2017.  An easy-to-hard learning paradigm for multiple classes and multiple labels.  _The Journal of Machine Learning Research_ , 18.
  * Min et al. (2018) Sewon Min, Victor Zhong, Richard Socher, and Caiming Xiong. 2018.  Efficient and robust question answering from minimal context over documents.  In _ACL_.
  * Min et al. (2019) Sewon Min, Victor Zhong, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2019.  Multi-hop reading comprehension through question decomposition and rescoring.  In _ACL_.
  * Nishida et al. (2019) Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, and Junji Tomita. 2019.  Answering while summarizing: Multi-task learning for multi-hop QA with evidence extraction.  In _ACL_.
  * Perez et al. (2020) Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho, and Douwe Kiela. 2020.  Unsupervised question decomposition for question answering.  In _EMNLP_.
  * Qi et al. (2019) Peng Qi, Xiaowen Lin, Leo Mehr, Zijian Wang, and Christopher D. Manning. 2019.  Answering complex open-domain questions through iterative query generation.  In _EMNLP-IJCNLP_.
  * Qiu et al. (2019) Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, and Yong Yu. 2019\.  Dynamically fused graph network for multi-hop reasoning.  In _ACL_.
  * Rajpurkar et al. (2016a) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016a.  SQuAD: 100,000+ questions for machine comprehension of text.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 2383–2392, Austin, Texas. Association for Computational Linguistics.
  * Rajpurkar et al. (2016b) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016b.  SQuAD: 100,000+ questions for machine comprehension of text.  In _EMNLP_.
  * Roads et al. (2018) Brett D Roads, Buyun Xu, June K Robinson, and James W Tanaka. 2018.  The easy-to-hard training advantage with real-world medical images.  _Cognitive Research: Principles and Implications_ , 3(1):1–13.
  * Seo et al. (2017) Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017.  Bidirectional attention flow for machine comprehension.  _ICLR_.
  * Shao et al. (2020) Nan Shao, Yiming Cui, Ting Liu, Shijin Wang, and Guoping Hu. 2020.  Is Graph Structure Necessary for Multi-hop Question Answering? In _EMNLP_.
  * Song et al. (2019) Shaoyue Song, Hongkai Yu, Zhenjiang Miao, Dazhou Guo, Wei Ke, Cong Ma, and Song Wang. 2019.  An easy-to-hard learning strategy for within-image co-saliency detection.  _Neurocomputing_ , 358:166–176.
  * Talmor and Berant (2018) Alon Talmor and Jonathan Berant. 2018.  The web as a knowledge-base for answering complex questions.  In _NAACL-HLT_.
  * Tu et al. (2019) Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xiaodong He, and Bowen Zhou. 2019\.  Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs.  In _ACL_.
  * Tu et al. (2020) Ming Tu, Bowen Zhou, et al. 2020.  Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents.  In _AAAI_.
  * Wang et al. (2019) Zhiguo Wang, Patrick Ng, Xiaofei Ma, Ramesh Nallapati, and Bing Xiang. 2019.  Multi-passage BERT: A globally normalized BERT model for open-domain question answering.  In _EMNLP-IJCNLP_.
  * Welbl et al. (2018) Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018.  Constructing datasets for multi-hop reading comprehension across documents.  _TACL_ , 6.
  * Wolf et al. (2020) Thomas Wolf, Alexander M. Rush, et al. 2020.  Transformers: State-of-the-art natural language processing.  In _EMNLP_.
  * Wu et al. (2021) Bohong Wu, Zhuosheng Zhang, and Hai Zhao. 2021.  Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy.  _arXiv:2107.11823_.
  * Xiong et al. (2017) Caiming Xiong, Victor Zhong, and Richard Socher. 2017.  Dynamic coattention networks for question answering.  _ICLR_.
  * Yang et al. (2018) Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018.  HotpotQA: A dataset for diverse, explainable multi-hop question answering.  In _EMNLP_.


# Lomas: A Platform for Confidential Analysis of Private Data

Damien Aymon111Equal contribution. 1, Dan-Thuy Lam††footnotemark:  1,
Lancelot Marti††footnotemark:  1, Pauline Maury-Laribière††footnotemark:  1,
Christine Choirat1,2, and Raphaël de Fondeville1
1 Federal Statistical Office, Neuchâtel, Switzerland
2 University of Geneva, Geneva, Switzerland

###### Abstract

Public services collect massive volumes of data to fulfill their missions.
These data fuel the generation of regional, national, and international
statistics across various sectors. However, their immense potential remains
largely untapped due to strict and legitimate privacy regulations. In this
context, Lomas is a novel open-source platform designed to realize the full
potential of the data held by public administrations. It enables authorized
users, such as approved researchers and government analysts, to execute
algorithms on confidential datasets without directly accessing the data. The
Lomas platform is designed to operate within a trusted computing environment,
such as governmental IT infrastructure. Authorized users access the platform
remotely to submit their algorithms for execution on private datasets. Lomas
executes these algorithms without revealing the data to the user and returns
the results protected by Differential Privacy, a framework that introduces
controlled noise to the results, rendering any attempt to extract identifiable
information unreliable. Differential Privacy allows for the mathematical
quantification and control of the risk of disclosure while allowing for a
complete transparency regarding how data is protected and utilized. The
contributions of this project will significantly transform how data held by
public services are used, unlocking valuable insights from previously
inaccessible data. Lomas empowers research, informing policy development,
e.g., public health interventions, and driving innovation across sectors, all
while upholding the highest data confidentiality standards.

##  1 Introduction

Public services possess a vast amount of data that are indispensable for
carrying out their public-interest missions. States invest significant time
and resources in gathering, sorting, and consolidating these datasets, which
serve as a unique and dependable source of information. However, despite their
inherent value, their full utilization for the betterment of public welfare
remains incomplete. These data are strictly earmarked for specific purposes,
adhering to data protection regulations such as the General Data Protection
Framework (GDPR) [19] in the European Union (EU) or the Data Protection Act
[20] in Switzerland. Public entities such has National Statistical Offices
(NSOs) and statistical agencies are constrained by even stricter laws such as
Title 13 [13] for the US Census Bureau or the regulation on statistical
confidentiality [18] in the EU. These offices are indeed required to protect
the confidentiality of the data they collect, which is legally different than
privacy [8, p. 197,222], and requires a higher degree of protection to ensure
their secrecy, and cannot use the data for any other purposes than the
publication of national statistics. Exceptions exists, such as for academic
research [17] in the European Union, for which any request for data sharing
undergoes a thorough review to evaluate its legitimacy and the risk of
identifiable data disclosure. Typically, the administrative process for
accessing such data in these instances is lengthy and intricate.

These oases of data remains unreachable and the full potential of the data
collected by public services can only be realized through their ”secondary
utilization”, i.e., using these data for purposes other than those for which
they were collected:

  * •

Other governmental offices can exploit these data to effectively support the
public policy-making process based on objective indicators. For instance, a
National Statistical Office can furnish accurate demographic and economic data
to inform policy decisions regarding public health or education.

  * •

Research laboratories can employ this data to advance knowledge and enhance a
country’s competitiveness through innovation. For example, university
researchers could analyze data held by governmental agencies to devise
innovative solutions to address poverty or school dropout rates.

  * •

The private sector can exploit these data to eventually bolster national
economic vitality. For instance, companies could utilize data collected by
public services to motivate strategic decisions regarding investment or
product development.

  * •

These data can also made available to other states to better address global
crises such as the COVID-19 epidemic. For instance, a country could use
epidemiological data detained by other nations to facilitate coordinating an
effective global response to the pandemic.

However, it is imperative that this data reuse is politically and socially
accepted, ensuring that the benefits it brings do not come at the expense of
citizens, households, and businesses. To uphold trust in the state, it is
therefore crucial to control the risk of disclosing identifiable data when
they are used both within and outside the public sector. Lomas attempts to
answer this societal challenge by offering a service enabling ”eyes-off data
science”, i.e., a practice of data science where private data is never
accessed directly by its practitioner. The platform facilitates the
realization of the full potential of private data while formally managing the
disclosure risk of identifiable data, thereby safeguarding organizational
reputation and mitigating adverse consequences for contributing individuals.

The Lomas platform follows the pioneering experiment initiated by the UN Pet
Lab hackathon [22] with the objective of demonstrating that the secondary
utilization of data collected by a NGO was possible thanks to Privacy-
Enhancing-Technologies (PETs). The hackathon’s platform had been developed by
the company Oblivious 222https://www.oblivious.com/, who open sourced the
competition’s code serving as Lomas’ starting point. The platform serves as a
hub where authorized entities seeking to repurpose data can connect,
generating data products, i.e., the results of any data processing algorithm
such as descriptive statistics or trained machine learning models, while
controlling disclosure risks and so safeguarding the original data at the
confidential unit level from direct exposure to users.

Lomas is a ”remote access” service, however, it adds upon conventional
solutions in two key aspects: Firstly, it guarantees that users never directly
access identifiable data and secondly, it eliminates the need for human
intervention in conducting output checks on the algorithms executed by the
user. The latter typically necessitates rigorous scrutiny of algorithmic
results by experts in disclosure risk, a process that is both expensive and
time-intensive. Employing PETs enables the automation of the entire process
while maintaining formal oversight of the risk of disclosure.

Consequently, the platform unlocks the value of data held by public services
through their secondary usage, which may have been deemed so far too sensitive
or prohibited under the current legislation. Leveraging PETs enables precise
management of data accesses and disclosure risks regardless of the nature of
private data usage, so that the risk of harmful consequences for contributing
individuals is controlled when their data is re-used: A sufficient mitigation
of this risk contributes to safeguard public trust. Alternatively, the
platform can also be used to expedite the development of data analysis
algorithms while awaiting formal data sharing procedures prescribed by law,
when they exist.

To stress the platform’s added-value, let us consider the utilization of data
for research as an illustrative example. Presently, processes for sharing data
from public services are characterized by their slowness and complexity
necessitating the establishment of a formal legal agreement and heavy data
pre-processing; tasks that both require the direct intervention of public
servants at all hierarchical levels. In addition to its considerable
administrative overhead, this procedure markedly diminishes the pace of
research—a circumstance not always compatible with international crises such
as the Covid-19 pandemic. For research, Lomas holds significant relevance in
the following scenarios:

  1. 1.

Facilitation and/or preliminary work: This approach aims to expedite data
provision to researchers, enabling them to commence their work promptly; they
can then run feasibility assessment of analyses, prepare and test algorithms
while awaiting the conclusion of the administrative process for obtaining the
original data. The advantages are twofold:

     * •

For the researcher: Acceleration of the research project.

     * •

For the public administration: Resource savings in cases where analyses are
deemed to be unfeasible.

  2. 2.

Enabling Accessibility: This case involves providing access to data that was
previously considered too sensitive, under the condition that these are never
disclosed in their original form and the risk of disclosure is meticulously
managed. The advantages are outlined as follows:

     * •

For the user: Analyze to data that would typically be inaccessible.

     * •

For the government: Achieving the full potential inherent in its collected
data. Moreover, there is a favorable reputational outcome, as the financial
commitment invested in gathering and consolidating datasets extends beyond the
originating governmental office.

To ensure data confidentiality on the platform, algorithmic outputs undergo
random perturbation through Differential Privacy (DP) techniques. For high
level of perturbations, access to the platform can be granted with minimal
contractual obligations as the disclosure risk can be mathematically proved
very low. While the obtained results may exhibit limitations in usefulness,
this opportunity remains valuable as it allows for instance researchers to
swiftly commence work by testing their code and experimental protocols.

To the best of our knowledge, Lomas is the first open-source platform of its
kind. Beyond its innovative design, it stands out as the only platform
developed by the public sector for the public sector, targeting public
servants and researchers. Although Lomas primarily serves this audience, it
also hold the potential to facilitate national and international
collaborations with the private sector and NGOs. To ensure broad availability
and ease of deployment and maintenance, we have partnered with INSEE, the
French NSO, and integrated Lomas into their datalab platform, Onyxia. Through
this partnership, we aim to make Lomas widely accessible, lowering barriers to
the adoption of PETs and creating a valuable public good.

The article is structured as follows: In Section 2, we start by discussing
PETs and how trust relationships help decide which technology to use. Section
3 delves deeper into the dynamics of trust relationships for public
administrations, concurrently introducing Differential Privacy as a means to
mitigate the risk of data disclosure on the platform. Subsequently, Section 4
elucidates the foundational design principles guiding Lomas’ development,
underscoring the pivotal role of metadata for the platform’s functionalities
and presents the service’s features from the user’s perspective. Lastly, we
address the challenges inherent in such a platform and outline our plans for
future enhancements.

##  2 Overview of Privacy-Enhancing Technologies

Privacy-Enhancing Technologies safeguard data confidentiality throughout the
analysis and dissemination of sensitive information. Their utilization allows
public services such as NSOs to realize the full benefits of the data they
collect while simultaneously minimizing privacy risks for those entrusting
their data. Specifically, PETs address the lack of trust among the parties
involved in the data value chain, namely

  * •

An identifiable person, often referred to as a privacy unit, pertains to any
natural or legal entity, or a collection thereof, such as a household,
producing information not publicly available and necessitating protection. In
the instance of natural persons, an identifiable individual is also called a
data subject. Consequently, ”identifiable data” encompasses all information
gathered regarding an identifiable person. Within the framework of data
protection legislations like the Swiss Data Protection Act or the GDPR, which
primarily concern natural persons, the term ”personal data” is employed.

  * •

Input party(ies): An entity responsible for the collection of identifiable
data and tasked with its value enhancement through the creation of a data
product. Examples include a governmental offices, hospitals, schools and
National Statistial Offices which generate statistics such as a poverty rates.

  * •

Computing Party(ies): An entity responsible for providing the computational
resources essential for developing the data product. Computations commonly
occur on remote servers, usually through a cloud infrastructure. This
infrastructure may be managed either by a trusted organization, such as a
governmental agency, or by a private enterprise offering access via the public
Internet to servers utilized possibly by multiple clients, thus often referred
as public cloud services.

  * •

Result Party(ies): The group of individuals to whom the data product is
disseminated. For instance, in the case of a public authority, any Open
Government Data product is distributed without constraints to the public,
representing the broadest distribution circle where confidentiality controls
must be stringent due to the heightened risk of extracting identifiable data.
Alternatively, the dissemination of a product may be confined to narrower
circles, such as public administration employees, all of whom are bound by
official, or statistical, secrecy, thereby reducing the risk of attempting to
extract identifiable data compared to distribution to the general public.

The trust relationship among the aforementioned parties dictates the level of
confidentiality that must be upheld throughout the data value chain. In this
context, ”trust” encompasses any rationale justifying the non-disclosure of
identifiable data to another party. This rationale may include, for instance,
legal prohibitions against the exchange of identifiable data between parties,
distrust from data subjects toward the data-providing entity, or the lack of
legal frameworks or organizational measures to mitigate attempts to extract
identifiable data. Overcoming the absence of trust between multiple
stakeholders can be achieved through the adoption of privacy-enhancing
technologies, which can be broadly categorized into two categories:

  * •

Input Privacy technologies are designed to enable two or more stakeholders
described above to contribute data for computation without any other party
accessing the original data in clear text, i.e., unencrypted or unprotected.
These technologies typically rely on cryptographic methods, or dedicated
hardware, and are thus integral components of data security frameworks.

  * •

Output Privacy technologies are specifically engineered to restrict the
extraction of identifiable data from a data product. It is noteworthy that
within this framework, the data product is voluntarily released by the data
provider. This procedure is also known as (statistical) disclosure control, a
domain well-established in NSO, which have honed this practice over numerous
years.

Thus, contingent upon the trust relationships among the diverse stakeholders
within the data value chain, the entity responsible for ensuring
confidentiality, typically the input party, will use a combination of PETs to
safeguard the confidentiality of the inputs and outputs.

##  3 Lomas Trust Schema and Differential Privacy

In this section, we start by describing trust relationships in public
administrations to motivate the choice of the PETs used in Lomas and explain
in detail why Differential Privacy is most natural in this context.

###  3.1 Trust Relationships in Public Services

The architecture of the platform is structured in accordance with the trust
schema depicted in Figure 1, aiming to make extensive use of relationships
specificities within public administrations:

  * •

Confidentiality units transmit their data to the input party, typically a
governmental office. The trust relationship is commonly enforced through
established legal frameworks.

  * •

Public agencies often have access to a trusted computing provider, either
through their own infrastructure or a centralized provider such as a
Governmental Office for Information Technology, Systems, and
Telecommunication. This arrangement obviates the necessity for input privacy-
enhancing-technologies. The provider assumes responsibility for data security,
including implementing requisite measures to prevent unauthorized data access
on the platform. Additionally, the provider oversees user identification and
authentication.

  * •

The eligible Results Parties may vary, encompassing governmental employees
from entities other than the data-collecting office to the general public. In
all scenarios, it is presumed that no inherent trust relationship exists
between the input party (the office providing the data) and the results party.
Consequently, the deployment of output privacy technologies, notably
differential privacy, becomes imperative to mitigate the risk of disclosing
confidential information.

This trust model is prevalent in public administrations, which have access to
trusted computing resources directly managed by the state or other certified
providers.

Figure 1: Trust schema for Lomas, a platform for public services to enable
analysis of their private data to other parties while controlling the risk of
disclosure. This schema is common for public administrations which can rely on
trusted IT infrastructures.

###  3.2 Disclosure Control Mechanisms

From a strictly technical standpoint, recent scientific advancements
underscore the inability to ensure complete anonymization, i.e., irreversibly
eliminate the risk of identifiable data disclosure: the publication of any
data product, regardless of its nature, inherently carries a non-zero risk of
identifiable data exposure. Nonetheless, this risk can be managed through
technical means. These frameworks facilitate the management of what is
commonly referred to as the privacy-utility trade-off: essentially, this
concept elucidates the challenge of achieving both high levels of
confidentiality and data product utility simultaneously. Consequently,
significantly mitigating the risk of disclosure often necessitates sacrificing
product utility, and vice versa. The output privacy-enhancing technologies
enable the quantification and informed management of this trade-off.
Conversely, determining an acceptable level of disclosure risk and/or minimum
utility is a complex socio-political matter that cannot be solely addressed
through technological means.

In the absence of control over auxiliary sources of information available to
potential attackers, un-protected data products become susceptible to
reconstruction [2], linkage [16], and membership [10] attacks. Through the
process of randomizing algorithm outputs, Differential Privacy (DP) [5]
renders any attempts to extract identifiable data from the data product
unreliable, thereby mitigating its disclosure risk. The latter offers several
key properties that contribute to its effectiveness in safeguarding data
privacy:

  * •

Firstly, DP provides a robust privacy guarantee, ensuring precise control over
the risk of identifiable data disclosure. This guarantee remains steadfast
regardless of the auxiliary information and technical capabilities available
to potential attackers.

  * •

Secondly, DP’s post-processing property ensures resilience against any form of
data transformation. This robustness allows for full transparency regarding
the randomization mechanism employed for data protection, thereby enhancing
trust and accountability in the privacy-preserving process.

  * •

Lastly, DP’s composition property enables the quantification of disclosure
risks across multiple data products. By tracking risks independently of the
nature of the data products, DP facilitates the comprehensive assessment of
disclosure risks at the confidential unit level, thus enhancing overall
privacy protection.

The quantification of this risk is achieved through a parameter known as the
privacy-loss budget, the selection of which is contingent upon both the
sensitivity of the data and the intended utilization of the safeguarded data
product.

In order for a data analysis to be sanctioned for execution on the Lomas
platform, it necessitates the inclusion of a disclosure control mechanism,
specifically a randomization mechanism that adheres to the conditions of
differential privacy. The project does not aim to develop these algorithms
internally, but rather relies on established open-source libraries such as
OpenDP [21] and SmartNoiseSQL [14] for data manipulation and transformation,
SmartNoise-Synth [15] for generation of syntetic datasets as well as
DiffPrivLib [9] for machine learning and artificial intelligence tasks.

##  4 Platform Design and Features

This section first outlines the principles guiding the development of Lomas,
with the objective of developing the platform as a portable, open-source,
community-driven standard product for public services. Then, we provide a
detailed description of the workflow from the user’s perspective, illustrating
the processes and interactions involved in utilizing Lomas. Details of the
Lomas architecture and deployment can be found in Appendix A. Finally, we
emphasize the importance of metadata as a crucial element for the automated
application of DP. We dedicate a portion of this document to explain the
reasons behind this emphasis and its implications for the effectiveness of
Lomas.

###  4.1 Design Principles

We start by presenting the principles underpinning the architectural design of
Lomas:

#### Open-source

Building upon the code developed for the 2022 UN Pet Lab Hackathon, the Lomas
platform is open-source and available under a MIT license. The latter gives
the rights without limitation to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software. The code is
available on the GitHub of the Data Science Competence Center
333https://github.com/dscc-admin-ch/lomas of the Swiss Federal Statistical
Office. The project is open to contributions by the community to incorporate
new functionalities.

#### Modular

The objective of the service is not to furnish customized implementations of
existing services and/or libraries for which trusted and validated open-source
implementations already exist. Instead, it depends on the incorporation of
established open-source libraries such as ”openDP,” ”SmartNoise-SQL”,
SmartNoise-Synth”, and ”DiffPrivLib.” The inclusion of libraries not yet
provided by the service can be facilitated either through a proposal to the
development team or via direct contribution to the project on GitHub3. The
service also relies on established data management and storage services for
which integrators for most popular solutions, e.g., tabular data files, S3,
…are, or will be, offered.

#### Portable

With the aim of positioning Lomas as a standard platform provided by public
services, it has been designed to maximize its portability. Consequently, its
components have been either containerized or integrated into a library,
enabling deployment or installation on standard infrastructure.

#### Easy to use

The platform’s value proposition revolves around streamlining access to
differentially private data science. This simplification manifests in two key
aspects:

  * •

The deployment of the service, from both the operator’s and the user’s
perspectives, is prioritized to be as straightforward and automated as
possible. For instance, we offer Helm charts for the automatic deployment of
Lomas’ containers; furthermore, the service is available for testing as a
”click-and-deploy” service on the data science platform Onyxia
444https://datalab.sspcloud.fr/, developed by the INSEE, the french NSO.

  * •

the usage of libraries implementing differential privacy is simplified as much
as possible, for instance by automatically populating parameters obtainable
from the metadata catalog.

#### Scalable

The containerized architecture of the service inherently enables the platform
to scale alongside the computing resources accessible on the infrastructure
utilized for deployment. Regarding scalability of dataset sizes, the service’s
capability to manage large datasets hinges on the implementation of DP
libraries: certain services may necessitate loading the entire dataset into
memory, whereas others facilitate streaming or distributed computations. The
modular design of Lomas is pivotal in ensuring the scalability of the service.

#### Secure

From a security standpoint, the Lomas platform is not yet production-ready and
currently lacks the implementation of security protocols for authentication
and connections. These aspects will require further adaptations during a
potential deployment in production. Nonetheless, Lomas employs input query
verification using Pydantic [1] and entrusts the evaluation of DP-protected
pipelines to the respective libraries. Additionally, certain measures to
mitigate timing attacks have been implemented.

#### Client-server architecture

The Lomas platform is structured around two primary components: a client
library and an HTTP server. The former offers a set of commands allowing an
accredited user to send DP-protected algorithms to the server, while the
latter serves as a gate-keeper between the user and the private dataset by
processing the client’s requests and managing an administration database
storing for instance user identities, budgets, and DP-algorithms archives. The
organization aiming to offer the service is responsible for its deployment as
well as managing users and available datasets by adding, modifying or deleting
information in the administration database. Following the modularity
principle, the Lomas service is not intended as a substitute for existing data
storage infrastructure. Instead, it relies on pre-existing services to store
and administer private datasets. The service’s purpose is solely to furnish
the requisite tools for computing differentially private algorithms on
designated datasets, while managing their access paths in the administration
database along with their metadata.

###  4.2 Lomas Service’s Description

Figure 2: Schematic diagram of Lomas, a platform for public services to enable
analysis of their own private data to other parties while controlling the risk
of disclosure. Numbers in black squares refer to the user’s step-by-step
workflow whose description can be found in Section 4.2.

The service’s step-by-step workflow is depicted in Figure 2, elucidating the
process by which data is analyzed by authorized users. From the user’s
perspective, the service operates along the following steps that matches the
numbers in Figure 2:

  1. 1.

The user establishes a remote connection to the platform, gaining access to a
virtual environment tailored for executing Python programming language
scripts. Extending compatibility to the R programming environment will be
considered in future developments.

  2. 2.

Utilizing the ”`lomas-client`” library, users can review a catalog of datasets
that have been made available to them. This catalog includes comprehensive
information regarding the nature of the datasets, their representation
(metadata), and the currently allocated privacy-loss budget.

  3. 3.

Upon selecting a dataset, users can download a fully simulated dummy dataset
into their computational environment, faithfully replicating the metadata
attributes of the original dataset. This dummy dataset serves as a means to
design and test data analysis algorithms without spending their privacy-loss
budget.

  4. 4.

Subsequently, users transmit their finalized algorithms to a remote server
managed by the public administration. Prior to execution, the algorithm
undergoes scrutiny to ensure the presence of a confidentiality control
mechanism, specifically a differentially private mechanism safeguarding the
algorithm’s output. If such a mechanism is absent, the algorithm’s execution
is declined. Additionally, the server verifies the user’s dataset access
rights and the adequacy of their privacy-loss budget.

  5. 5.

If the access and confidentiality requirements are met, the secure server
retrieves the private data from the secure storage space.

  6. 6.

The algorithm is autonomously applied to the original dataset, with the
resultant calculation being protected via differential privacy.

  7. 7.

The protected result is saved and transmitted back to the user. Logging
algorithms and their DP-protected results enables authorities to monitor user
activity and datasets’ utilization.

This approach empowers users to execute algorithms without direct access to
private data, while simultaneously assuring the public agency providing the
data that the risk of disclosing confidential information is minimized.

###  4.3 Metadata Catalog

Metadata pertains to descriptive information concerning data, commonly
referred to as ’data about data’. It furnishes supplementary details that
characterize a piece of information, distinct from the information’s content
itself. For instance, within the context of a dataset table, the data refers
to the actual cell values, whereas metadata encompasses attributes such as
column names and types. In the pursuit of ”eyes-off data science”, as enabled
by Lomas, metadata bears critical importance:

  * •

Users depend on metadata to comprehend the available data and request
allocation of a privacy-loss budget.

  * •

Metadata facilitates the service’s capacity to dynamically generate realistic
dummy datasets, enabling the design and testing of algorithms without
depleting the privacy-loss budget.

  * •

The service relies on metadata for the autonomous execution of differentially
private pipelines without requiring user intervention. For instance, knowledge
of the range of possible values for an attribute is indispensable for applying
the appropriate level of noise to ensure the desired privacy guarantee.

Due to these considerations, metadata in Lomas are assumed to be publicly
accessible, as is typically the case for data held by public services, or at
least freely available to the platform’s users.

With the increasing adoption by public administrations of the ”once only
principle” [11, 6, 4], i.e., that data from citizens and businesses are
collected by public services only once, the platform benefits from initiatives
aimed at cataloging and harmonizing the datasets maintained by public
entities, particularly focusing on documenting their metadata.
Interoperability standards, such as the DCAT Application Profile for data
portals in Europe [7] and data catalogs such as the I14Y platform in
Switzerland 555https://www.i14y.admin.ch/en/home , simplify the generation of
dummy datasets for assessing user-designed data analysis algorithms. The
direct integration of these standards into Lomas would enable the platform to
leverage these initiatives synergistically, thereby fully exploiting the value
of state-held data.

##  5 Challenges and Future Developments

Lomas capitalizes on the distinctive attributes inherent in trust
relationships within public administrations. Through collaboration with a
trusted computing party, the platform’s focus is placed only on ensuring the
confidentiality of the generated results. This objective is realized through
the implementation of differential privacy, whose primary challenge pertains
to the determination of the privacy-loss budget. Indeed, the DP framework
enables precise control over the balance between data utility and
confidentiality by allocating a maximum privacy-loss budget for each user. The
selection of this budget entails a consideration that has political
implications and can be articulated as follows: ”What is the maximal
acceptable level of risk to which the dataset’s confidentiality units can be
exposed?” or ”What is the minimal intended purpose for which the data product
must reliably be used?”. These inquiries align with the ”confidentiality
first” and ”utility first” principles, respectively, allowing an organization
to delineate its priorities and judiciously determine the privacy-loss budget
through informed decision-making.

To illustrate this compromise and decision-making process, let us consider
again the utilization of data in research as an illustrative example. The two
scenarios described in Section 1, namely ”Facilitation and/or preliminary
work” and ”Enabling Accessibility”, both call for a small privacy-loss budget,
i.e., large random perturbations, to compensate for the high sensitivity of
the project, whether due to the sensitivity of the data or the short time
frame: The high risk yielded by either the sensitive nature of the data, or
the urgency of the request, is mitigated thanks to an adequate privacy-loss
budget policy that reflects the context in which the data value enhancement
takes place.

Lomas thus hinges upon the establishment of a policy governing the
authorizations of data product’s creation and the selection of an adequate
privacy-loss budget. Such policy cannot be comprehensively prescribed solely
through a technological lens; it must consider the broader ecosystem,
including the nature of the project, sensitivity of the data, the circle of
individuals accessing the data product, the computing environment, the
existing legal framework, and, notably, the objective assessment of the
resultant disclosure risk. Its establishment thus necessitates a data
governance framework such as the ”Five Safes” [3] encompassing all dimensions
of the data value enhancement process, namely Project, Data, People, Setting,
and Output, to mediate discussions among privacy professionals, decision-
makers, and citizens. Confidentiality alone does not ensure social acceptance
of data secondary usage: the project’s alignment with the rule of law and its
public benefits also comes into play and for those, an objective
quantification using a technological solution is not possible.

There are very active ongoing efforts to help bridge the gap between
technology and policy. The recent US Executive Order on Artificial
Intelligence [23] emphasizes the importance of Privacy-Enhancing Technologies
(PETs) like differential privacy – the core technology behind Lomas. This
Executive Order requests federal agencies to integrate AI technologies in a
manner that ensures the protection of individual privacy and national
security. To this end, the National Institute of Standards and Technology
(NIST) has been tasked with developing standards, guidelines and certification
mechanisms, which aim to simplify the implementation of differential privacy
[12]. These initiatives aim to eliminate the need for users to grasp the
intricacies of differential privacy guarantees, thus fostering broader
adoption and trust in these frameworks.

Currently, Lomas integrates open-source libraries that implement differential
privacy for SQL queries and classical data analytics. We plan to extend its
functionalities to support the training of machine learning algorithms and the
creation of synthetic datasets. Thanks to the metadata catalog, which we plan
to align on international standards by extending frameworks such as DCAT to
meet differential privacy requirements, future developments will include
improved automatic parameters filling in differential privacy algorithms,
improving and simplifying the platform’s usability. Additionally, from a
system engineering perspective, the platform will implement standard security
protocols to ensure secure connections between its components, manage users
authentication and broaden the range of dataset storage types handled by the
service.

## Acknowledgment

The Lomas development team would like to sincerely thank the company Oblivious
for sharing and open-sourcing the code behind the 2022 UN Pet Lab’s Hackathon
and the Onyxia development team at INSEE for their support in deploying Lomas
on their platform.

## References

  * [1] Pydantic, 2024.
  * [2] John M. Abowd, Tamara Adams, Robert Ashmead, David Darais, Sourya Dey, Simson L. Garfinkel, Nathan Goldschlag, Daniel Kifer, Philip Leclerc, Ethan Lew, Scott Moore, Rolando A. Rodríguez, Ramy N. Tadros, and Lars Vilhuber.  The 2010 census confidentiality protections failed, here’s how and why.  arXiv:2312.11283, 12 2023.
  * [3] Tanvi Desai, Felix Ritchie, and Richard Welpton.  Five safes: designing data access for research.  2016\.
  * [4] Digital Public Services Switzerland.  National eGovernment study 2022, 2022.
  * [5] Cynthia Dwork.  Differential privacy.  pages 1–12. Springer Berlin, 2006.
  * [6] European Commission.  Eu eGovernment action plan 2016-2020, 4 2016.
  * [7] European Commission.  DCAT application profile for data portals in europe, 2 2024.
  * [8] Federal Committee on Statistical Methodology .  Statistical policy working paper 22: Report on statistical disclosure limitation methodology, 1994.
  * [9] Naoise Holohan, Stefano Braghin, Pól Mac Aonghusa, and Killian Levacher.  Diffprivlib: The ibm differential privacy library.  7 2019.
  * [10] Nils Homer, Szabolcs Szelinger, Margot Redman, David Duggan, Waibhav Tembe, Jill Muehling, John V. Pearson, Dietrich A. Stephan, Stanley F. Nelson, and David W. Craig.  Resolving individuals contributing trace amounts of dna to highly complex mixtures using high-density snp genotyping microarrays.  PLoS Genetics, 4, 8 2008.
  * [11] R. D. Kush, D. Warzel, M. A. Kush, A. Sherman, E. A. Navarro, R. Fitzmartin, F. Pétavy, J. Galvez, L. B. Becnel, F. L. Zhou, N. Harmon, B. Jauregui, T. Jackson, and L. Hudson.  Fair data sharing: The roles of common data elements and harmonization, 7 2020.
  * [12] Joseph P. Near and David Darais.  Guidelines for Evaluating Differential Privacy Guarantees (National Institute of Standards and Technology, Gaithersburg, MD), 2023.
  * [13] Senate and House of Representatives of the United States of America.  Title 13 - Census, 8 1954.
  * [14] SmartNoise SDK Team.  Smartnoise SQL, 2023.
  * [15] SmartNoise SDK Team.  Smartnoise Synth, 2023.
  * [16] Latanya Sweeney.  Simple demographics often identify people uniquely.  2000\.
  * [17] The European Parliament and the European Council.  Commission regulation (eu) no 557/2013 - Access to confidential data for scientific purposes, 2009.
  * [18] The European Parliament and the European Council.  Regulation (ec) no 223/2009 - Statistical confidentiality, 2009.
  * [19] The European Parliament and the European Council.  Regulation (eu) 2016/679 - General Data Protection Regulation, 2016.
  * [20] The Federal Assembly of the Swiss Confederation.  Sr 235.1 - Federal Act of 25 september 2020 on data protection, 2020.
  * [21] The OpenDP team.  The OpenDP white paper, 2020.
  * [22] The UN PET Lab.  The un pet lab’s hackathon, 2022.
  * [23] United States, Executive Office of the President Joseph Biden.  Executive Order 14110 - Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence.  Federal Register, 88, 11 2023.

##  Appendix A Detailed Service’s Architecture and Deployment

###  A.1 Client Library: Functionalities and Workflow

Submitting requests to the server is done through the Python library `lomas-
client`666https://pypi.org/project/lomas-client/, which can be installed using
`pip install lomas-client`. This ensures synchronization of library versions
between the client and server, a crucial requirement for the platform’s
seamless operation. The library provides the following functionalities:

  * •

Algorithm Serialization: the library transforms DP-protected algorithms into
JSON objects that can be de-serialized by the server. The serialization is
tailored for each available DP-library.

  * •

Communication with the server: serialized DP-algorithms and corresponding DP-
protected results are transmitted between the server and the client through a
single REST API call.

  * •

De-serialization: the library checks the validity of the server’s response
and, in case of validity, proceeds to its de-serialization in order to deliver
the DP-algorithm’s result to the user.

We now outline the functionalities of the library, depicting the step-by-step
process followed by a user who possesses the authorization to query a
sensitive dataset containing personal information about penguins inhabiting
diverse islands. The user’s goal is to compute the average bill length of the
penguin population whose data has been collected. A code example demonstrating
all these steps is provided in Figure 3.

#### Client object creation

The user begins by instantiating a client object, utilizing the server’s URL,
along with the user’s and dataset’s names. This client object oversees the
requests dispatched to the server and necessitates invocation each time the
user intends to submit an algorithm for remote execution.

⬇

from lomas_client import Client

import polars as pl

# Client object creation

client = Client(

url = "https://lomas_server.ch",

user_name = "Dr. Antartica",

dataset_name = "PENGUIN"

)

# Available privacy-loss budget retrieval

my_initial_buget = client.get_initial_budget()

my_total_spent_buget = client.get_total_spent_budget()

my_remaining_budget = client.get_remaining_budget()

# Metadata retrieval

penguin_metadata = client.get_metadata()

# Dummy dataset generation

dummy_df = client.get_dummy_dataset(seed = 42, nb_rows=100)

# Algorithm design (SQL query using polars)

QUERY = "SELECT AVG(bill_length) FROM df"

with pl.SQLContext(df=pl.from_pandas(dummy_df)) as ctx:

local_bill_length = ctx.execute(QUERY, eager=True)

# DP-Algorithm Execution Verification on Server

## Remote execution verification

server_bill_length = client.smartnoise_query(

query = QUERY, epsilon = 100, delta = 0.99, dummy = True, seed = 42

)

## Remote execution consistency check

assert abs(local_bill_length - server_bill_length) < 0.01

# Estimation of privacy-loss budget expenditure

client.estimate_smartnoise_cost(

query = QUERY, epsilon = 0.1, delta = 0.00001

)

# Query execution on private dataset

avg_bill_length = client.smartnoise_query(

query = QUERY, epsilon = 0.1, delta = 0.00001

)

# Archives consultation

previous_queries = client.get_previous_queries()

Figure 3: Example of Python code using Lomas to compute the differential
private average bill size of a population of penguins.

#### Available privacy-loss budget retrieval

Before sending any algorithm for execution, users can determine the available
privacy-loss budget for the dataset being analyzed through the following
functionalities:

  * •

`get_initial_budget()` retrieves the initial privacy-loss budget that was
allocated to the user by the administrator.

  * •

`get_total_spent_budget()` provides the total budget spent at the time.

  * •

`get_remaining_budget()` returns the remaining available privacy-loss budget.

Each of these functionalities provides values for both privacy-loss parameters
$\epsilon$ and $\delta$.

#### Metadata retrieval

To attain a thorough comprehension of the dataset and its contents, users can
retrieve the dataset’s metadata from the catalog utilizing the
`get_dataset_metadata()` function. In Lomas, each dataset is accompanied by a
metadata file containing crucial information for the appropriate utilization
of Differential Privacy. This includes details such as the maximum number of
contributions by privacy unit, column types, and the range of potential values
assumed by the variables. An illustration of the function output is provided
in Figure 4.

island | bill_length
---|---
A | 55.1
B | 46.1
A | 50.7
A | 35.7
B | 47.0
B | 51.5
(a) Private dataset

(b) Metadata

island | bill_length
---|---
B | 34.8
A | 36.4
B | 34.8
B | 60.8
B | 61.2
A | 39.3
(c) Dummy dataset

Figure 4: Example of private data, metadata, and dummy data for the analysis
of the penguin population.

#### Dummy dataset generation

For a more concrete comprehension of the dataset’s structure, as well as for
designing and testing algorithms, users may request the generation of a dummy
dataset using the `get_dummy_dataset(nb_rows, seed)` function. The latter
produces a randomly generated dataset satisfying the constraints imposed by
the metadata. The user is allowed to specify a seed for random number
generation and the desired number of rows.The dummy dataset, an example of
which is provided in Figure 4, serves as a means to locally design and test
algorithms before attempting their remote execution on the server.

#### Algorithm design

The user is now prepared to start the design of the algorithm intended for
execution on the private data. To prevent unintentional expenditure of the
privacy-loss budget at this stage, it is strongly advised to design and test
the algorithm locally on the dummy dataset using conventional libraries for
data manipulation. For example, `polars` facilitates the testing of SQL
queries or data analysis pipelines on the local dummy dataframe. Its
compatibility with existing libraries implementing Differential Privacy
minimizes the effort required for integrating a differentially private
mechanism.

#### DP-Algorithm execution verification on server

The user can now proceed to test the execution of the algorithm on the server.
To accomplish this, the algorithm must first be modified to incorporate a
differentially private mechanism. This can be achieved either by utilizing a
DP-library, such as SmartnoiseSQL with `smartnoise_query(query, epsilon,
delta, dummy, seed)`, or by integrating a randomization mechanism, for
instance, using OpenDP. Firstly, the user may validate the DP-algorithm by
requesting the server to execute it on the ”dummy” dataset. This approach
allows the user to verify the syntax of the algorithm without incurring any
budgetary expenditure.

After confirming the correctness of the syntax, users can validate the
algorithm’s result on the server by ensuring consistency between the results
of remote execution and those obtained from the local non-DP algorithm. As
dummy datasets do not consume budget during the execution of DP-algorithms,
users can select privacy parameters resulting in negligible noise. This
enables users to compare the results of both non-protected and DP-protected
algorithms, thereby evaluating the accuracy of remote execution based on the
proximity of their outcomes.

#### Estimation of privacy-loss budget expenditure

The privacy-loss parameters utilized for the execution of a DP-algorithm may
not directly correspond to the actual consumed budget in practice; typically,
it tends to be higher contingent upon the algorithm’s complexity.
Consequently, it is crucial to estimate the expenditure of the privacy-loss
budget prior to dispatching the algorithm for processing on the server. For
instance, the functions `estimate_smartnoise_cost()` and
`estimate_opendp_cost()` provide the accurate budget expenditure for a given
request. Accordingly, users have the flexibility to experiment with various
values of privacy parameters until they identify those aligning with the
privacy-loss budget they are prepared to allocate. It should be emphasized
once again that any request submitted to the server for testing on the dummy
dataset does not deplete the user’s available privacy-loss budget.

#### Query execution on original dataset

Once the privacy parameters have been chosen and the algorithm’s execution has
been rigorously validated, the user can proceed to apply the algorithm to the
private data by simply omitting the `dummy = True` parameter. For example,
using `smartnoise_query(query, epsilon, delta)` for ”smartnoise-sql” and
`opendp_query(opendp_pipeline)` for ”opendp” will yield the DP-protected
result on the private dataset.

#### Archives consultation

All executed requests that deplete the privacy-loss budget are archived for
subsequent reference by both the user and the platform’s administrator. This
functionality facilitates the retrieval of past DP-algorithms and their
corresponding responses, thereby mitigating the risk of duplicate queries that
could exhaust the privacy-loss budget.

Additional information regarding the library’s functions can be accessed in
the GitHub documentation pages, alongside Jupyter Notebooks providing further
detailed examples. Presently, the client is exclusively accessible in Python.
However, it could potentially be ported to alternative programming languages
such as R, with the primary constraint being the availability of libraries
implementing differential privacy in the respective language.

###  A.2 Server: Components and Functionalities

The server comprises two principal components:

  * •

A client-facing HTTP server responsible for processing and validating user
requests, as well as executing diverse algorithms on the private and dummy
data. Its primary function is to efficiently manage incoming requests from
clients and execute authorized DP-algorithms.

  * •

An administration database, typically a MongoDB by default, which serves as a
repository for user and dataset information. User-related data includes access
permissions to specific datasets, allocated privacy-loss budgets, remaining
budgets, and archives of executed DP-algorithms. Dataset-related data
encompasses details such as dataset names, information and credentials for
dataset access, along with corresponding metadata.

As previously stated, dataset storage is not managed by Lomas. Instead, the
platform interfaces with external databases, typically operated by a data
provider, to retrieve datasets prior to query execution. Presently, the
service includes adapters for S3, HTTP files server, and files stored on
locally mounted volumes.

Interaction with the client library is brought by a FastAPI application
through various HTTP Rest API calls. The functions outlined below furnish the
means to administer the server and automate the execution of DP-algorithms:

#### Service management

Lomas provides a command line tool for managing the administration database
which enables seamless addition, modification, and deletion of users, query
archives, and dataset metadata either individually or in batches, provided the
correct security credentials are provided. An illustration of these functions
can be found in Figure 5

⬇

# Service deployment in a Kubernetes cluster

# \-----------------------------------------------------------------------

# Download Helm chart

helm repo add lomas https://dscc-admin-ch.github.io/helm-charts

# Adapt values.yaml file and run the following to deploy the chart

helm install -f values.yaml lomas-server lomas/lomas-server

# Service administration through CLI tool

# \-----------------------------------------------------------------------

# Install script and requirements

git clone https://github.com/dscc-admin-ch/lomas.git

python -m venv venv

./venv/bin/activate

cd ./lomas/server

pip install -r requirements.txt

cd ./src

# Add a user

python mongodb_admin.py \--add_user_with_budget \--user "Dr. Antartica"\

\--dataset PENGUIN \--epsilon 10 \--delta 0.005

# Add a dataset

python mongodb_admin.py \--add_dataset \--dataset PENGUIN \

\--database_type PATH_DB \--dataset_path \

"https://raw.githubusercontent.com/mwaskom/seaborn-data/master/penguins.csv"\

\--metadata_database_type PATH_DB \

\--metadata_path ../data/collections/metadata/penguin_metadata.yaml

# Check everything was correctly configured

python mongodg_admin.py \--show_collection users

python mongodg_admin.py \--show_collection datasets

python mongodg_admin.py \--show_collection metadata

Figure 5: Example of code to manage Lomas’ services.

#### Sensitive data requests

The server handles a typical request from a user, wherein the user requests
the execution of a DP-algorithm on a private dataset, as follow:

  1. 1.

The user initiates a request containing the desired DP-algorithm via the
client library.

  2. 2.

The server validates the user’s access to the dataset and verifies the absence
of any ongoing algorithms. Should an algorithm already be in progress, the
query is not processed, and an error message is returned to the user.

  3. 3.

Subsequently, the DP-algorithm is extracted and reconstructed from the user’s
request.

  4. 4.

A validation process is initiated to assess the compatibility of the algorithm
with the corresponding Differential Privacy library. The procedure involves
verifying that the necessary protection mechanisms prescribed by the DP
library are satisfied. For example, in the case of the OpenDP library, the
algorithm must incorporate a measurement indicating the inclusion of a
randomization mechanism to safeguard the algorithm’s result. If the pipeline
fails to pass the validation process, an error message is returned to the
user.

  5. 5.

The privacy-loss budget of the algorithm is computed and compared to the
user’s remaining budget.

  6. 6.

If the remaining budget permits, the algorithm is executed and the expended
budget is subtracted from the user’s allowance.

  7. 7.

The algorithm and its associated result are stored in the administration
database, facilitating subsequent retrieval, thereby allowing users to re-
access their findings as necessary.

  8. 8.

Upon successful validation of the preceding checks and updating of the budget
in the administrative database, the result is transmitted to the user.

#### Dummy data requests

The user can apply DP-algorithms on dummy datasets for testing purposes
without incurring budgetary expenditures. The procedure follows the subsequent
steps:

  1. 1.

The server retrieves the metadata associated with the dataset stored in the
administrative database. Subsequently, the server generates a dataset randomly
following the metadata structure (including column types, bounds, names,
etc.).

  2. 2.

The server reconstructs the DP-algorithm and verifies its validity, employing
a process similar to algorithms applied on private data.

  3. 3.

The server executes the differentially private algorithm on the dummy data,
and the output is transmitted directly to the user through the client library.

For algorithms applied to dummy datasets, the checks and updates pertaining to
privacy-loss budget are not conducted, and the algorithms and their results
are not archived.

#### Privacy-loss budget request

As presented in Section A.1, the server implements functions to compute the
budget expenditure of their DP-algorithm. The implementation directly
leverages the tools provided by the integrated DP libraries.

#### Administration database requests

A collection of functions provides the users with the relevant data stored in
the administration database, encompassing metadata, archives, initial, and
expended budget details. Therefore, upon reception of such a request along
with the associated parameters, the server queries the administration
database, post-processes the result if required, and transmits the relevant
information back to the client.