11institutetext: Fraunhofer AISEC
11email<EMAIL_ADDRESS>
# Application-Oriented Selection of Privacy Enhancing Technologies††thanks:
The final publication is available at link.springer.com
(https://doi.org/10.1007/978-3-031-07315-1_5)

Immanuel Kunz  Andreas Binder

###### Abstract

To create privacy-friendly software designs, architects need comprehensive
knowledge of existing privacy-enhancing technologies (PETs) and their
properties. Existing works that systemize PETs, however, are outdated or focus
on comparison criteria rather than providing guidance for their practical
selection. In this short paper we present an enhanced classification of PETs
that is more application-oriented than previous proposals. It integrates
existing criteria like the privacy protection goal, and also considers
practical criteria like the functional context, a technology’s maturity, and
its impact on various non-functional requirements. We expect that our
classification simpliefies the selection of PETs for experts and non-experts.

###### Keywords:

Privacy Engineering Privacy Enhancing Technologies Privacy By Design.

##  1 Introduction

A decisive activity in privacy engineering is the selection of appropriate
Privacy Enhancing Technologies (PETs), for example to fulfill requirements or
mitigate risks in goal-based or risk-based engineering methods respectively.
While this step is highly application-specific, it can be approached
systematically, since common decision criteria for PET-selection exist. For
example, one criterion that can guide engineers in their design decisions is
the privacy goal that is targeted by a PET, such as anonymity or
undetectability.

Existing works have proposed different systematizations of PETs in the past.
The LINDDUN methodology [11], for instance, categorizes PETs using their
privacy protection goal, and differentiates between PETs that target
transactional and contextual data [44]. Heurix et al. [22] categorize PETs,
e.g., regarding the trust scenario they target and their involvement of a
trusted third party. Yet, these systematizations do not sufficiently take into
account the practical context in which PETs are selected: they often omit the
PET’s functional context it can be applied in, as well as other practical
criteria. Also, they are partly outdated.

In this short paper, we develop a new PET classification that is more
application-oriented. Our classification builds upon previous proposals,
integrating some of their criteria [11, 22, 1] like the technology’s targeted
privacy protection goal and its impact on other non-functional requirements.
It furthermore includes the PET’s functional context, as well as
prioritization criteria, like the maturity of the technology. We present a
classification of 30 PETs that we have done according to the proposed
criteria.

To demonstrate the effectiveness of our classification, we compare it to the
one proposed by LINDDUN based on a use case. We expect that our work can
support engineers in selecting appropriate PETs, and motivate PET developers
to evaluate their technologies according to our criteria, making them more
comparable and usable for architects and engineers.

##  2 Classification

Various approaches to selecting appropriate PETs have been proposed in the
past, but the adequate application of PETs can be even more challenging than
their selection, for example due to the development effort involved in
applying a PET in a certain environment. Our goal is therefore to develop an
application-centered classification of PETs that anticipates the challenges
that a PET can cause when it is applied. In the following, we describe and
motivate the criteria we have included in our classification.

###  2.1 Motivating Example

To motivate the choice of criteria, we lay out the following scenario of an
engineer developing an architecture for a generic privacy-friendly cloud
service, assuming that it is representative for the decisions architects and
developers have to make when implementing privacy requirements. This generic
service allows users to register, authenticate, provide personal data to the
service (e.g. names, addresses), interact with other users of the service
(e.g. sending messages), and to retrieve data that is stored in the service’s
storages.

An engineer creates a threat model for an initial design of the service which
reveals a detectability threat for the service’s users: since their messages
to other users are observable by the service and possibly external attackers,
their relationships can be identified. A requirement is therefore elicited
which states that users’ messages should be undetectable. A PET that mitigates
this threat has to match several criteria:

First, it has to fit the functional context of the service. This includes the
targeted privacy protection goal, such as anonymity or undetectability. To
ensure suitability for the scenario, however, the targeted privacy protection
goal is not sufficient; also, the PET has to fit the functional requirements
of the application. In a messaging scenario, for example, different PETs are
applicable than in an information retrieval or computing scenario. Ideally,
the PET should furthermore not only target the correct privacy goal, but
should also be measurable in the achieved privacy gain. This way, a more
comparable and reproducible selection is facilitated.

Second, various non-functional properties can play a role. For example, the
PET should be mature enough to be applied and maintained by the development
team: applying a certain technology can otherwise imply large efforts for
configuration and further development of the technology. To be able to weigh
different usable PETs against each other, also further properties are of
interest: PETs can have an impact on the overall architecture, on the utility
of transactional data, and the performance of the respective interaction.

In the following we describe the criteria we use to cover these
considerations, and classify a list of PETs accordingly.

###  2.2 Criteria

####  2.2.1 Privacy Protection Goal

One of the most meaningful criteria for selection is the targeted privacy
protection goal. To that end we use the goals proposed by the LINDDUN
methodology [11], most of which have originally been proposed by Pfitzmann and
Hansen [32]: Unlinkability, anonymity, plausible deniability, undetectability,
confidentiality, awareness, and policy compliance.

Every PET targets one or more of these goals. Often, it is the case that
several goals are targeted by one PET, since they partly overlap or imply each
other. For example, it is hardly possible to achieve undetectability without
anonymity. In our classification, however, we usually only present one privacy
protection goal which we consider to be primarily targeted.

####  2.2.2 Privacy Metric

A technology’s suitability and effectiveness need to be measurable to evaluate
its added privacy gain and monitor the system over time. Note that the broader
problem of creating a metric suite that comprehensively covers the notion of
privacy is a research problem out of scope for this paper (see for example
Wagner and Yevseyeva [43]). A number of privacy metrics are reviewed in [42].

####  2.2.3 Functional Scenario

The functional context the PET shall be applied in is highly application-
specific. Still, some categories of functional scenarios can be identified and
can be used as a selection filter. We identify the following generic
interactions:Computation, Messaging, Retrieval, Release, Authentication, as
well as Authorization. We use the motivating scenario above to clarify these
values: a computation means that data is processed, e.g. a virtual machine
processes user data to create recommendations for users; Release is the
release of data to another party, e.g. a user sends location data to the
social network; Messaging, is a point-to-point interaction between two users
via $n$ other parties, e.g. two users of the same social network communicate
with each other via the service; Authentication is the process of determining
the identity of a user, while Authorization is the process of determining the
rights of an identified user.

####  2.2.4 Maturity

When selecting a PET, cost factors play an important role, e.g. in the set-up
of the PET and in its continuous maintenance. In this paper we use the
technology’s maturity as an indicator for set-up and maintenance costs, since
a technology that is less mature will likely have more defects and will likely
imply a more laborious set-up.

For this criterion we loosely base the possible values on the Technology
Readiness Level (TRL) which describes a technology’s maturity on a scale from
1 to 9 [31]. We generalize this scale to 3 levels as follows: level 1 is a
level often achieved in scientific work, which describes a concept and may
already prove feasibility in a proof of concept; level 2 can be seen as the
development and testing stage, i.e. adopting such a technology still would
require considerable development effort if it is applied to a specific use
case; finally, level 3 means that the technology is readily available and
field-tested, but may still require some set-up cost for the adaption.

####  2.2.5 Performance Impact

The performance of processes and interactions can be impacted by the use of
PETs. Evaluating the performance of a certain PET, however, is not trivial,
especially in comparison to other technologies. We therefore evaluate a PET’s
performance in a simplified manner as follows. We first generically describe
the performance requirements in a certain functional scenario, and then assess
if the use of a PET is expected to significantly impact the performance
requirements or not.

We consider Computation and Retrieval scenarios generally to have high
performance requirements: In these scenarios the user waits for the result of
the interaction and will probably notice also small delays. In contrast,
Authentication, Authorization, and Release scenarios generally have low
performance requirements since they are usually one-time actions for which
performance impacts are more acceptable. Also, we consider Messaging to be a
scenario of asynchronous communication where small increases in latency are
not noticed by the users.

####  2.2.6 Architectural Impact

An impact on the architecture is given if the PET requires a dedicated
architectural component or modifications to the architecture, e.g. setting up
a mix net requires a separate mix server. This is an important selection
criterion, since the selection of a PET with this property needs to be
considered early on in the design process.

####  2.2.7 Utility

A utility impact is given if a PET reduces the quality of transactional data,
e.g. by distorting or filtering it—and thus decreasing the data’s utility.

Table 1: Classification of Privacy Enhancing Technologies. A black square
indicates that the PET addresses the respective goal, while a triangle
indicates a negative impact on the respective criterion.

Name |  Linkability |  Identifiability |  Non-Repudiation |  Detectability |  Disclosure |  Unawareness |  Non-Compliance |  Metrics |  Functional Scenario |  Maturity |  Performance |  Architecture |  Utility
---|---|---|---|---|---|---|---|---|---|---|---|---|---
k-anonymity, l-diversity, t-closeness | $\blacksquare$ |  |  |  |  |  |  |  Data similarity | Release | 3 [33] |  |  | $\blacktriangleleft$
Suppression | $\blacksquare$ | $\blacksquare$ |  |  |  |  |  |  Data similarity | Release | 3 [33] |  |  | $\blacktriangleleft$
Recoding | $\blacksquare$ |  |  |  |  |  |  |  Data similarity | Release | 3 [33] |  |  | $\blacktriangleleft$
Aggregation | $\blacksquare$ |  |  |  |  |  |  |  Data similarity | Release | 3 [33] |  | $\blacktriangleleft$ | $\blacktriangleleft$
Swapping | $\blacksquare$ |  |  |  |  |  |  |  Data similarity | Release | 1 [23] |  |  |
Noise masking | $\blacksquare$ |  |  |  |  |  |  |  Data similarity | Release | 3 [30] |  |  | $\blacktriangleleft$
PRAM | $\blacksquare$ |  |  |  |  |  |  |  Data similarity | Release | 2 [40] |  |  | $\blacktriangleleft$
Synthetic data | $\blacksquare$ |  |  |  |  |  |  |  Data similarity | Release |  |  |  | $\blacktriangleleft$
Mix Network | $\blacksquare$ | $\blacksquare$ |  | $\blacksquare$ |  |  |  |  | Messaging | 3 [38] | $\blacktriangleleft$ | $\blacktriangleleft$ |
Group Signatures | $\blacksquare$ | $\blacksquare$ | $\blacksquare$ |  |  |  |  |  Cryptographic Games | Release | 2 [24] |  | $\blacktriangleleft$ |
Anonymous Credentials | $\blacksquare$ | $\blacksquare$ |  |  |  |  |  |  Cryptographic Games | AuthN | 2 [8] |  |  |
Zero Knowledge Proofs | $\blacksquare$ | $\blacksquare$ |  |  |  |  |  |  Cryptographic Games | AuthN, AuthZ | 2 [5] | $\blacktriangleleft$ |  |
Pseudonymization |  | $\blacksquare$ |  |  |  |  |  |  Entropy |  AuthN, Release | 3 [14] |  |  |
Deniable Authentication |  |  | $\blacksquare$ |  |  |  |  |  Cryptographic Games | Messaging | 3 [17] |  |  |
Deniable encryption |  |  | $\blacksquare$ |  | $\blacksquare$ |  |  |  Cryptographic Games | Messaging | 3 [17] |  |  |
Searchable Encryption |  |  |  |  | $\blacksquare$ |  |  |  Cryptographic Games | Retrieval | 3 [6] |  |  |
Private Information Retrieval |  |  |  |  | $\blacksquare$ |  |  |  Cryptographic Games | Retrieval | 2 [28] | $\blacktriangleleft$ | $\blacktriangleleft$ |
Oblivious Transfer |  |  |  |  | $\blacksquare$ |  |  |  Cryptographic Games | Retrieval | 2 [7] | $\blacktriangleleft$ | $\blacktriangleleft$ |
Proxy Re-Encryption |  |  |  |  | $\blacksquare$ |  |  |  Cryptographic Games | Messaging | 2 [16] | $\blacktriangleleft$ | $\blacktriangleleft$ |
Homomorphic Encryption |  |  |  |  | $\blacksquare$ |  |  |  Cryptographic Games | Computation | 2 [18] | $\blacktriangleleft$ |  |
Trusted Execution Environment |  |  |  |  | $\blacksquare$ |  |  |  Cryptographic Games | Computation | 3 [25, 3] |  |  |
(A)Symmetric Encryption |  |  |  |  | $\blacksquare$ |  |  |  Cryptographic Games |  Messaging, Release | 3 |  |  |
Dummy traffic |  |  |  | $\blacksquare$ |  |  |  |  Data similarity | Messaging |  |  |  |
Steganography |  |  |  | $\blacksquare$ | $\blacksquare$ |  |  |  Entropy | Messaging | 2 [9] |  |  |
MPC |  |  |  |  | $\blacksquare$ |  |  |  Cryptographic Games | Computation | 3 [4] | $\blacktriangleleft$ |  |
Local Differential Privacy |  |  | $\blacksquare$ |  |  |  |  |  Indistinguishability | Release | 2 [41] |  |  | $\blacktriangleleft$
Global Differential Privacy | $\blacksquare$ |  |  |  |  |  |  |  Indistinguishability | Release | 2 [41] |  |  | $\blacktriangleleft$
Attribute-based encr. |  |  |  |  | $\blacksquare$ |  |  |  Cryptographic Games | AuthN, AuthZ | 2 [39] |  |  |
Federated Learning |  |  |  |  | $\blacksquare$ |  |  |  Attacker Success Probability | Release | 2 [37] |  | $\blacktriangleleft$ |

###  2.3 Classifying PETs

Table 1 shows our classification proposal according to the criteria defined
above. Note that our classification only includes the so-called hard privacy
goals [11], i.e. Anonymity, Unlinkability, Plausible Deniability, and
Undetectability. The soft privacy goals Awareness and Policy Compliance111As
soft privacy goals, some works also use the goals Intervenability and
Transparency [21]. are usually targeted by more generic design patterns (see
e.g. [1]). There can, however, be overlaps between technologies and patterns:
For instance, onion routing can be seen both as a design pattern and a PET.

Note that regarding the targeted privacy protection goal, we always assign the
goal that is targeted primarily. For example, Zero Knowledge Proofs (ZKP)
primarily address the threats linkability and identifiability. While they
could theoretically also be used to secretly release information, an engineer
would not choose ZKP to achieve confidentiality.

Note also that throughout the paper, we use k-anonymity [36] as a placeholder
also for other related ones like l-diversity [29], t-closeness [27], etc.

##  3 Use Case and Discussion

###  3.1 Use Case

In this section, we compare our classification to the LINDDUN classification
which was first proposed by Deng et al. [11] and has since been updated on the
LINDDUN website [44]222Note that we do not compare our approach to Heurix et
al. [22], since they partly use different privacy protection goals and
generally provide few selection criteria.. Note that the LINDDUN
classification also includes a selection methodology. This methodology is
based on the LINDDUN threat types which are connected to general mitigation
strategies. These strategies in turn are mapped to applicable PETs. For
instance, a linkability threat concerning a data flow may be mapped to the
mitigation strategy protect transactional data which in turn yields the PETs
multi-party computation, encryption, and others.

We use again the motivating example introduced earlier, and extend and detail
it as follows: our example cloud service is a social network that allows to
add friends, exchange private messages with each other, as well as make public
posts. Furthermore, the service offers a location-based feature where users
can provide their location data and are then offered possible contacts in
their proximity they can message. Note that the example used in the original
LINDDUN approach is a social network application as well, making it a well-
suited basis for a comparison.

We use three example threats from different LINDDUN categories, i.e. a
linkability, an identifiability, and a disclosure threat, to demonstrate the
effectiveness of our classification in comparison to the LINDDUN
classification. Note that these threats have also been identified (on a more
high level) in an example analysis conducted by the LINDDUN authors for their
social network running example, see [10]. The threats and results from the
PET-selection of both approaches are described in the following. Table 2
summarizes the results.

####  3.1.1 Identifiability

This threat describes the possibility that the server can identify the user
via the transmitted transactional data, e.g. due to identifiers like name or
address.

  * •

Applying LINDDUN in this scenario we obtain 7 PETs. With our classification we
obtain: Suppression, Recoding, Aggregation, Swapping, Noise Masking, PRAM,
Synthetic Data, Mix Network, Group Signatures, Global Differential Privacy.

  * •

This is the only case in which our classification outputs more PETs than the
LINDDUN methodology.

####  3.1.2 Linkability

This threat concerns potential linkability of different types of transactional
data which is released by the user to the server. Applying the LINDDUN method
results in a set of 10 PETs as summarized in Table 2.

In comparison we can see that our classification correctly omits several PETs
since they are not usable to mitigate a linkability threat, i.e. Multi-Party
Computation, (A)Symmetric Encryption, and Homomorphic Encryption. Furthermore,
it does provide more applicable ones, e.g. Aggregation and Noise Masking which
can obfuscate the relation between data.

####  3.1.3 Disclosure

This threat concerns the disclosure of transmitted data due to insecure
connections. Applying the LINDDUN method results in a set of 12 PETs, see
Table 2. Our approach results in 4 PETs: Proxy Re-Encryption, Deniable
Encryption, (A)symmetric Encryption, and Steganography.

  * •

On the basis of this comparison, we expect that our classification better
supports engineers in the selection of PETs than existing classifications and
selection approaches.

  * •

This is achieved mainly by filtering through the functional scenario, but also
the more targeted mapping of PETs to protection goals.

  * •

Consider also that using our classification, a user can further prioritize the
results using the maturity, utility, and architecture impact criteria. For
example, the (a)symmetric encryption may be prioritized in the last example
because it has high maturity and no impact on architecture or utility.

Table 2: Comparison of the results of applying the LINDDUN PET Selection
method and our classification. Those PETs that are included in both
classifications are written in bold. For example, for the linkability threat
LINDDUN suggests Multi-Party Computation as a possible mitigation. This PET is
also included in our classification, which, however, has not suggested it for
this threat. Verifiable Encryption is also suggested by LINDDUN, but it has
not been considered in our classification at all.

Threat |  LINDDUN Result |  Our Classification
---|---|---
Linkability |  k-anonymity Multi-Party Computation (A)symmetric encryption Homomorphic Encryption Deniable Encryption Anonymous Buyerseller Watermarking Prot. Verifiable Encryption |  k-anonymity and l-diversity Suppression Recoding Aggregation Swapping Noise Masking PRAM Synthetic Data Group Signatures Global Differential privacy
Disclosure (Release) |  k-Anonymity (A)symmetric Encryption Homomorphic Encryption Private Information Retrieval Oblivious Transfer Searchable Encryption Deniable Encryption Verifiable Encryption Context-Based Access Control Privacy-Aware Access Control Privacy-Preserving Data Mining Private Search |  (A)Symmetric Encryption Federated Learning
Disclosure (Computation) |  Homomorphic Encryption k-Anonymity (A)symmetric Encryption Private Information Retrieval Oblivious Transfer Searchable Encryption Deniable Encryption Verifiable Encryption Context-Based Access Control Privacy-Aware Access Control Privacy-Preserving Data Mining Private Search |  Homomorphic encryption Multi-Party Computation Trusted Execution Environment
Identifiability (Release) |  Multi-Party Computation (A)symmetric encryption Homomorphic encryption Deniable Encryption Anonymous Buyerseller Watermarking Prot. Verifiable Encryption |  k-anonymity and l-diversity Suppression Pseudonymization Group Signatures Global Differential Privacy

###  3.2 Discussion

####  3.2.1 Limitations

One limitation of our approach is its coverage: while it is an extensive
collection of 32 PETs, it is not complete and should be extended and
maintained in the future. Especially the evaluation of the maturity criterion
may become outdated soon, e.g. if current research proposals are developed
further.

Furthermore, the criteria we propose are deduced from a use case. Therefore,
their effectiveness still has to be validated in real-world studies. Some
relevant criteria, e.g. regarding other non-functional requirements, could
also be missing.

Still, we expect our classification to improve the systematic selection of
PETs, and the evaluation of software architectures. For instance, design
decisions in software architectures can be linked to our classifications and
systematically evaluated.

Our classification could be biased since the use case and the threats the
LINDDUN analysis identifies were known to the authors before the
classification was finished. We assume, however, that the bias is low since it
was developed in discussion with multiple domain experts who did not know the
LINDDUN analysis. Also, we would argue that it is evident from that both the
general criteria as well as the classification itself are independently
applicable from the social network example.

####  3.2.2 Criteria

Evidently, it is not guaranteed that the criteria we propose are comprehensive
and that they capture what engineers require as selection criteria in
practice. On the basis of the case example above, however, we expect that it
works better than existing approaches also in other applications.

In comparison to Al-Momani et al. [1], we do not include criteria that
indicate an impact on security and complexity, because we would argue that
they are redundant. Complexity is always increased by a PET to some degree,
while the actual degree of complexity is too difficult to measure. With
regards to the security impact, there is one privacy protection goal that
directly contradicts a security goal, i.e. plausible deniability contradicts
non-repudiation. Thus, any PET that targets plausible deniability also
counteracts said security goal which can therefore directly be derived from
our classification.

In the following we also compare our criteria with Heurix et al. [22]:

  * •

The Aim dimension is similar to our privacy protection goal.

  * •

The Scenario dimension is not in scope for us, since we focus on client-server
interactions where the server is untrusted.

  * •

The Aspect dimension is similar to the mitigation strategies in the LINDDUN
method [44]. We do not consider these because they are implied in the
functional scenario: for instance, a PET that targets the authentication
scenario addresses protection of ID.

  * •

The Foundation and Data dimensions not relevant for the selection in practice.

  * •

The Trusted Third Party dimension is covered by our criterion of architectural
impact.

  * •

The Reversibility dimension is largely the same criterion as our utility
criterion, since the distortion or deletion of a value is usually non-
reversible.

Furthermore, previous selection methodologies do not provide means for
prioritizing PETs [26, 11]. In a set of potentially applicable PETs, however,
we would argue it is important to have prioritization factors, such as their
maturity, as we propose in this paper.

##  4 Related Work

###  4.1 Privacy By Design

Generally, our classification can be seen as a tool that supports privacy by
design. As such, it is complementary to other privacy engineering methods
which often assume a PET-selection without further detailing this step [19,
20, 35]. One such approach is proposed by Alshammari and Simpson [2] who
develop an engineering process that devises architectural strategies, i.e.
combinations of tactics, patterns, and PETs, to fulfill privacy goals. In
their approach, the set of usable PETs is determined by the chosen design
pattern. The concrete selection of a PET, however, is not specified in their
work. As such, our classification could be integrated into their methodology.

###  4.2 Systematization of PETs

Also, further works have investigated the selection and systematization of
PETs. Al-Momani et al. [1] follow a similar approach as we do but focus on
privacy patterns rather than concrete technologies. As explained above,
patterns rather target soft privacy goals. They use the following criteria to
classify patterns: applicability scope , privacy objective, qualities (e.g.
performance impact), data focus, and LINDDUN GO hotspot. In this paper, we
have partly used similar criteria; many criteria, however, are different since
the selection of concrete technologies requires other seleciton criteria than
patterns, e.g. maturity. Note also, that our classification of the targeted
privacy objective (called privacy goal in this paper) differs in some cases
from theirs. Kunz et al. [26] also propose a selection method for PETs but
their approach is limited to PETs that manipulate transactional data, e.g.
generalization or filter. A systematization of PETs is proposed by Heurix et
al. [22]. They identify the dimensions Scenario, Aspect, Aim, Foundation,
Data, trusted third party, and Reversibility. We would argue, however, that
these dimensions are not directly helpful for engineers who have to select a
concrete PET. Rubio et al. [34] review 10 PETs regarding their efficiency for
smart grids. Since their analysis is focused on smart grids, they also use
respective classification criteria, like suitability for billing or monitoring
purposes. Their work is thus complementary to ours since we do not include
smart grids as a functional scenario.

ENISA has previously promoted a prototype of a PET maturity repository [13];
to the best of the authors’ knowledge, however, ENISA has not continued this
repository.

There is furthermore an ENISA publication about Privacy and Data Protection by
Design which classifies PETs into several categories which, however, are not
intended as selection criteria [12].

Another recent ENISA publication [15] proposes a categorization of PETs
regarding the categories truth-preserving, intelligibility-preserving, and
operable technology.

##  5 Conclusions

The selection of privacy-enhancing technologies is a task that is difficult to
address systematically. In this paper we have proposed application-oriented
criteria that allow such a systematic selection, and have classified a number
of PETs according to these criteria, e.g. their functional scenario and
applicable metrics.

One open issue is the performance evaluation of PETs, since their performances
are usually not easily comparable. In future work, we therefore plan to
propose an evaluation framework for the measurement of the performance of
PETs. We also want to extend our classification with more PETs, and connect
them with other concepts, such as design patterns. Unifying these in, e.g., a
comprehensive ontological description of privacy concepts may represent a
valuable support tool for engineers. Furthermore, existing threat modeling
tools can be extended with suggestions for mitigation based on our
classification. Future work also needs to show the effectiveness of the
proposed classification in real-world applications.

## References

  * [1] Al-Momani, A., Wuyts, K., Sion, L., Kargl, F., Joosen, W., Erb, B., Bösch, C.: Land of the lost: privacy patterns’ forgotten properties: enhancing selection-support for privacy patterns. In: Proceedings of the 36th Annual ACM Symposium on Applied Computing. pp. 1217–1225 (2021)
  * [2] Alshammari, M., Simpson, A.: Privacy architectural strategies: an approach for achieving various levels of privacy protection. In: Proceedings of the 2018 Workshop on Privacy in the Electronic Society. pp. 143–154 (2018)
  * [3] AMD: Amd sev. https://developer.amd.com/sev/
  * [4] Bab, K., Flockhart, P., Qin, L., Micheal, W.M., Issa, R., Lapets, A., Manzelli, R., Howe, W., Jansen, F.: Jiff. https://github.com/multiparty/jiff (2021)
  * [5] Bloemen, R., Vienhage, P.: Openzkp. https://github.com/0xProject/OpenZKP (2020)
  * [6] Bost, R.: Open symmetric searchable encryption (opensse). https://github.com/OpenSSE (2021)
  * [7] Centelles, A., Diehl, S.: 1-out-of-2 oblivious transfer. https://github.com/adjoint-io/oblivious-transfer (2020)
  * [8] Claßen, Philip and Grabowski, Kamil and Modras, Krzysztof: Anonymous credentials. https://github.com/whotracksme/anonymous-credentials (2020)
  * [9] David, R., Sison, J., Vickery, J., Bundoo, K.A., Ahmed, S., Sybil-E: Lsb-steganography. https://github.com/RobinDavid/LSB-Steganography (2020)
  * [10] Deng, M., Wuyts, K., Scandariato, R., Preneel, B., Joosen, W.: Linddun: running example - social network 2.0 -. https://www.linddun.org/downloads, Social network 2.0 – running example https://www.linddun.org/downloads,Socialnetwork2.0–runningexample, accessed on 14.02.2022
  * [11] Deng, M., Wuyts, K., Scandariato, R., Preneel, B., Joosen, W.: A privacy threat analysis framework: supporting the elicitation and fulfillment of privacy requirements. Requirements Engineering 16(1), 3–32 (2011)
  * [12] European Union Agency for Cybersecurity (ENISA): Privacy and data protection by design (2015), https://www.enisa.europa.eu/publications/privacy-and-data-protection-by-design
  * [13] European Union Agency for Cybersecurity (ENISA): Pets maturity assessment repository (2019), https://www.enisa.europa.eu/publications/enisa2019s-pets-maturity-assessment-repository
  * [14] European Union Agency for Cybersecurity (ENISA): Pseudonymisation techniques and best practices—recommendations on shaping technology according to data protection and privacy provisions (2019), https://www.enisa.europa.eu/publications/pseudonymisation-techniques-and-best-practices
  * [15] European Union Agency for Cybersecurity (ENISA)): Data protection engineering (2022), https://www.enisa.europa.eu/publications/data-protection-engineering
  * [16] Frederickson, C.: recrypt. https://github.com/IronCoreLabs/recrypt-rs (2022)
  * [17] Goldberg, I.: Off-the-record messaging. https://otr.cypherpunks.ca/
  * [18] Google: Fully homomorphic encryption (fhe). https://github.com/google/fully-homomorphic-encryption
  * [19] Gürses, S., Troncoso, C., Diaz, C.: Engineering privacy by design. Computers, Privacy & Data Protection 14(3), 25 (2011)
  * [20] Gürses, S., Troncoso, C., Diaz, C.: Engineering privacy by design reloaded. In: Amsterdam Privacy Conference. pp. 1–21 (2015)
  * [21] Hansen, M., Jensen, M., Rost, M.: Protection goals for privacy engineering. In: 2015 IEEE Security and Privacy Workshops. pp. 159–166. IEEE (2015)
  * [22] Heurix, J., Zimmermann, P., Neubauer, T., Fenz, S.: A taxonomy for privacy enhancing technologies. Computers & Security 53, 1–17 (2015)
  * [23] Hundepool, A., Domingo-Ferrer, J., Franconi, L., Giessing, S., Nordholt, E.S., Spicer, K., De Wolf, P.P.: Statistical disclosure control, vol. 2. Wiley New York (2012)
  * [24] IBM: libgroupsig. https://github.com/IBM/libgroupsig (2021)
  * [25] Intel: Intel sgx. https://www.intel.com/content/www/us/en/architecture-and-technology/software-guard-extensions.html
  * [26] Kunz, I., Banse, C., Stephanow, P.: Selecting privacy enhancing technologies for iot-based services. In: International Conference on Security and Privacy in Communication Systems. pp. 455–474. Springer (2020)
  * [27] Li, N., Li, T., Venkatasubramanian, S.: t-closeness: Privacy beyond k-anonymity and l-diversity. In: 2007 IEEE 23rd international conference on data engineering. pp. 106–115. IEEE (2007)
  * [28] Liones, E., Langille, D.: Muchpir demo. https://github.com/ReverseControl/MuchPIR (2021)
  * [29] Machanavajjhala, A., Kifer, D., Gehrke, J., Venkitasubramaniam, M.: l-diversity: Privacy beyond k-anonymity. ACM Transactions on Knowledge Discovery from Data (TKDD) 1(1), 3–es (2007)
  * [30] Mivule, K.: Utilizing noise addition for data privacy, an overview. arXiv preprint arXiv:1309.3958 (2013)
  * [31] NASA: Technology readiness level definitions. https://www.nasa.gov/directorates/heo/scan/engineering/technology/technology_readiness_level
  * [32] Pfitzmann, A., Hansen, M.: A terminology for talking about privacy by data minimization: Anonymity, unlinkability, undetectability, unobservability, pseudonymity, and identity management (2010)
  * [33] Prasser, F., Kohlmayer, F., Babioch, K., Vujosevic, I., Bild, R.: Arx data anonymization tool. https://arx.deidentifier.org/
  * [34] Rubio, J.E., Alcaraz, C., Lopez, J.: Selecting privacy solutions to prioritise control in smart metering systems. In: International Conference on Critical Information Infrastructures Security. pp. 176–188. Springer (2016)
  * [35] Spiekermann, S., Cranor, L.F.: Engineering privacy. IEEE Transactions on software engineering 35(1), 67–82 (2008)
  * [36] Sweeney, L.: k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 10(05), 557–570 (2002)
  * [37] The TensorFlow Federated Authors: TensorFlow Federated (12 2018), https://github.com/tensorflow/federated
  * [38] The TOR Project: Tor browser. https://www.torproject.org/
  * [39] Unknown authors: Openabe. https://github.com/zeutro/openabe (2021)
  * [40] Unknown Authors: Python implementation of post-randomisation method for disclosure control. https://github.com/JiscDACT/pram (2021)
  * [41] Unknown authors: Differential privacy. https://github.com/google/differential-privacy (2022)
  * [42] Wagner, I., Eckhoff, D.: Technical privacy metrics: a systematic survey. ACM Computing Surveys (CSUR) 51(3), 1–38 (2018)
  * [43] Wagner, I., Yevseyeva, I.: Designing strong privacy metrics suites using evolutionary optimization. ACM Transactions on Privacy and Security (TOPS) 24(2), 1–35 (2021)
  * [44] Wuyts, K., Van Landuyt, D., Sions, L., Wouter, J.: Linddun: Mitigation strategies and solutions. https://www.linddun.org/mitigation-strategies-and-solutions, accessed: 2021-07-30


# Will your Doorbell Camera still recognize you as you grow old?

Wang Yao  School of Engineering, University of Galway, Ireland.  Muhammad Ali
Farooq  School of Engineering, University of Galway, Ireland.  Joseph Lemley
Xperi Corporation, Galway.  Peter Corcoran  School of Engineering, University
of Galway, Ireland.

###### Abstract

Robust authentication for low-power consumer devices such as doorbell cameras
poses a valuable and unique challenge. This work explores the effect of age
and aging on the performance of facial authentication methods. Two public age
datasets, AgeDB and Morph-II have been used as baselines in this work. A
photo-realistic age transformation method has been employed to augment a set
of high-quality facial images with various age effects. Then the effect of
these synthetic aging data on the high-performance deep-learning-based face
recognition model is quantified by using various metrics including Receiver
Operating Characteristic (ROC) curves and match score distributions.
Experimental results demonstrate that long-term age effects are still a
significant challenge for the state-of-the-art facial authentication method.

Keywords: Synthetic Data, GAN, Age Effect, Face Recognition, Deep Learning

##  1 Introduction

Figure 1: Example of an individual facial variation during the aging process
(Images from [Moschoglou et al., 2017]).

Automatic face authentication/recognition is one of the active and long-
standing research topics in the field of computer vision. Studies [Taskiran et
al., 2020] have shown that factors such as age, lighting, and pose can have a
significant impact on the performance of face recognition algorithms. Recent
work [Yao et al., 2022] has studied the main factors such as illumination and
pose that affect facial authentication, which shows the feasibility of
implementing a robust authentication method for low-power consumer devices. In
this work, we are focusing on another challenging factor, age, i.e., how age
bias affects the state-of-the-art Face Recognition (FR) method.

Age progression as a basic demographic can be used for real-time biometric
authentication applications such as doorbell cameras and FaceID for cell
phones. It is necessary to obtain sufficient aging data from diverse
identities to study the robustness of biometric authentication systems to age
effects. Figure 1 shows an example of the aging effects of a celebrity.
However, collecting data on a person from birth to old age is difficult, and
existing age datasets rarely have large amounts of face data from multiple
identities at different stages of age. In this work, a generative adversarial
network (GAN) based aging technique has been used to create a synthetic aging
dataset, which could generate different target ages of an identity. In this
way, the robustness of the FR model to aging can be verified without a need to
collect data from human subjects over long periods of time. The main
contributions of this paper are as follows.

  * (1)

This work quantifies the long-term aging effect on state-of-the-art neural
face recognition algorithm.

  * (2)

This work explores the feasibility of using synthetic aging data to augment
real-world age data.

  * (3)

Experimental results show that large age intervals cause obvious degradation
in the FR algorithm.

##  2 Background

The development of face authentication technology in recent years has resulted
in the emergence of many state-of-the-art FR algorithms based on deep
learning. These algorithms have achieved remarkable performance improvements
on large-scale data sets by optimizing deep neural networks and loss
functions. Studies have shown that the performance of face recognition is
affected by demographics including age, race, and gender which can produce
bias. For example, [Deb et al., 2017] present a longitudinal study of FR and
find a significant decrease in the accuracy of COTS-A and COTS-B face matchers
over a time interval of 8.5 years. [Boussaad and Boucetta, 2021] find that the
recognition rate becomes significantly low when age intervals are greater than
20 years. These studies always adopted feature extraction-based FR methods.
This raises the following question in this study. How robust is the recent
deep learning-based FR method to age bias?

##  3 Methodology

#### Datasets:

Two publicly available age datasets AgeDB and Morph II are adopted in this
work to study the age effect on the FR algorithm. (a) AgeDB [Moschoglou et
al., 2017] contains 568 subjects with 16488 images. The average number of
images per subject is 29. (b) Morph II [Ricanek and Tesafaye, 2006] contains
55134 images with 13618 identities taken from 2003 to 2007. Each subject has
an average of 4 images.

#### Synthetic Aging Method:

SAM [Alaluf et al., 2021], as one of the state-of-the-art synthetic aging
techniques is adopted in this work to generate more aging samples. SAM treats
the aging process as an image-to-image transformation problem by pairing a
pre-trained fixed StyleGAN generator with an encoder. The task of the encoder
is to encode real face images directly into the StyleGAN latent space to
obtain the faces with expected age change. Some synthetic aging samples are
shown in Figure 2.

Figure 2: Samples of age variations using the synthetic aging method.

#### Face Recognition Method:

ArcFace [Deng et al., 2019] is selected as the FR model to verify the validity
of synthetic aging faces, which uses additive angular margin loss to obtain
good intra-class compactness and inter-class dispersion. In this work, MTCNN
is employed to detect and crop the faces. Then, these faces are fed into the
FR network to obtain facial features. The identity similarity score is derived
by calculating the cosine similarity of the two facial features.

#### Evaluation Metrics:

Two evaluation metrics including the receiver operating characteristic (ROC)
curve and match score distribution have been adopted in our experiment to
quantify the effect of synthetic aging data on the high-performance deep-
learning-based FR model.

##  4 Experiment

Figure 3: Experimental Process.

#### Experimental Setting:

The process of this experiment is shown in Figure 3. First, all detectable
face images from 20 to 30 years old are selected for pre-processing to obtain
$256\times 256$ images. Then, these face images are taken as the original
images to synthesize faces of different ages, and here we synthesize face
images of 20, 30, 40, 50, 60, 70, and 80 years old respectively. Moreover,
positive-identity pairs (PPs) and negative-identity pairs (NPs) are formed by
using the original images. A positive-identity pair means an image pair from
the same identity. A negative-identity pair means an image pair from different
identities. For different age intervals, one of the images from PPs/NPs is
replaced separately with the images of the target age. Finally, the images
from PPs/NPs are fed into the FR model to calculate the similarity scores,
which are used for evaluation.

###  4.1 ROC Comparison

Figure 4: The ROC curves for ArcFace of AgeDB and Morph II.

The ROC curves in Figure 4 quantify the effect of different age intervals on
the FR algorithm. Age intervals within 20 years have a weak effect on the
performance of the state-of-the-art FR model. There is a slight degradation of
the FR model performance for a 30-year time interval. Longer time intervals
still cause significant degradation in the performance of the FR model.

###  4.2 Match Score Comparison

Figure 5 shows the impostor and genuine distributions of different age
intervals. The impostor distributions are similar across different age
intervals, while the genuine distributions with a noticeable shift to the left
as the age intervals get larger. As the age interval increases, the area where
the impostor distribution intersects with the true distribution increases, and
the recognition performance decreases. This indicates that the genuine
distribution is the main reason leading to the degradation of the performance
of the FR algorithm.

Figure 5: The match score distributions for ArcFace of AgeDB and Morph II.

##  5 Conclusion

This work qualifies the impact of age intervals on a SOTA FR algorithm and
illustrates the potential value of synthetic age data in analyzing the
robustness of face authentication systems. Initial experiments associated with
synthetic age data have shown that it is valid to use synthetic data to
enlarge the dataset for analysis of how aging affects FR models. The
evaluation results from ROC curves and match score comparison show that age
differences within 20 years do not have a noticeable impact on face
recognition accuracy, while long-term age differences remain a significant
challenge for the current facial authentication method. In future work, we
will explore the quality of synthetic age data, and do a comparison of the
real age data and the synthetic age data. We plan to expand this work to
create a synthetic age dataset and to implement a privacy-secured robust face
authentication solution on low-power neural accelerators.

## Acknowledgments

This research is supported by (i) Irish Research Council Enterprise
Partnership Ph.D. Scheme (Project ID: EPSPG/2020/40), (ii) Xperi Corporation,
Ireland, and (iii) the Data-Center Audio/Visual Intelligence on-Device (DAVID)
Project (2020–2023) funded by the Disruptive Technologies Innovation Fund
(DTIF).

## References

  * [Alaluf et al., 2021] Alaluf, Y., Patashnik, O., and Cohen-Or, D. (2021).  Only a matter of style: Age transformation using a style-based regression model.  ACM Trans. Graph., 40(4).
  * [Boussaad and Boucetta, 2021] Boussaad, L. and Boucetta, A. (2021).  The aging effects on face recognition algorithms: the accuracy according to age groups and age gaps.  In 2021 International Conference on Artificial Intelligence for Cyber Security Systems and Privacy (AI-CSP), pages 1–6.
  * [Deb et al., 2017] Deb, D., Best-Rowden, L., and Jain, A. K. (2017).  Face recognition performance under aging.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 46–54.
  * [Deng et al., 2019] Deng, J., Guo, J., Xue, N., and Zafeiriou, S. (2019).  Arcface: Additive angular margin loss for deep face recognition.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699.
  * [Moschoglou et al., 2017] Moschoglou, S., Papaioannou, A., Sagonas, C., Deng, J., Kotsia, I., and Zafeiriou, S. (2017).  Agedb: The first manually collected, in-the-wild age database.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
  * [Ricanek and Tesafaye, 2006] Ricanek, K. and Tesafaye, T. (2006).  Morph: a longitudinal image database of normal adult age-progression.  In 7th International Conference on Automatic Face and Gesture Recognition (FGR06), pages 341–345.
  * [Taskiran et al., 2020] Taskiran, M., Kahraman, N., and Erdem, C. E. (2020).  Face recognition: Past, present and future (a review).  Digital Signal Processing, 106:102809.
  * [Yao et al., 2022] Yao, W., Varkarakis, V., Costache, G., Lemley, J., and Corcoran, P. (2022).  Toward robust facial authentication for low-power edge-ai consumer devices.  IEEE Access, 10:123661–123678.