# Deep Privacy Funnel Model:
From a Discriminative to a Generative Approach
with an Application to Face Recognition

Behrooz Razeghi $\\!{\href https://orcid.org/0000-0001-9568-4166}$
$\\!{}^{\ast}$, Parsa Rahimi $\\!{\href
https://orcid.org/0000-0001-7927-268X}$ $\\!{}^{\ast}\,{}^{\dagger}$,
Sébastien Marcel $\\!{\href https://orcid.org/0000-0002-2497-9140}$
$\\!{}^{\ast}\,{}^{\ddagger}$
${}^{\ast}\\!$ Idiap Research Institute, Switzerland
${}^{\dagger}\\!$ École Polytechnique Fédérale de Lausanne (EPFL)
‡ Université de Lausanne (UNIL)

###### Abstract

In this study, we apply the information-theoretic Privacy Funnel (PF) model to
the domain of face recognition, developing a novel method for privacy-
preserving representation learning within an end-to-end training framework.
Our approach addresses the trade-off between obfuscation and utility in data
protection, quantified through logarithmic loss, also known as self-
information loss. This research provides a foundational exploration into the
integration of information-theoretic privacy principles with representation
learning, focusing specifically on the face recognition systems. We
particularly highlight the adaptability of our framework with recent
advancements in face recognition networks, such as AdaFace and ArcFace. In
addition, we introduce the Generative Privacy Funnel ($\mathsf{GenPF}$) model,
a paradigm that extends beyond the traditional scope of the PF model, referred
to as the Discriminative Privacy Funnel ($\mathsf{DisPF}$). This
$\mathsf{GenPF}$ model brings new perspectives on data generation methods with
estimation-theoretic and information-theoretic privacy guarantees.
Complementing these developments, we also present the deep variational PF
(DVPF) model. This model proposes a tractable variational bound for measuring
information leakage, enhancing the understanding of privacy preservation
challenges in deep representation learning. The DVPF model, associated with
both $\mathsf{DisPF}$ and $\mathsf{GenPF}$ models, sheds light on connections
with various generative models such as Variational Autoencoders (VAEs),
Generative Adversarial Networks (GANs), and Diffusion models. Complementing
our theoretical contributions, we release a reproducible PyTorch package,
facilitating further exploration and application of these privacy-preserving
methodologies in face recognition systems. The source code will be made
available upon acceptance at: https://gitlab.idiap.ch/biometric/deep-
variational-privacy-funnel.

Keywords: Information-theoretic privacy, statistical inference, privacy funnel
models, information obfuscation, face recognition systems, representation
learning.

Corresponding Author: Behrooz Razeghi

###### Contents

  1. 1 Introduction
    1. 1.1 Motivation
    2. 1.2 Key Contributions
    3. 1.3 Outline
    4. 1.4 Notations
  2. 2 Navigating the Data Privacy Paradigm
    1. 2.1 Lunch with Turing and Shannon333This section is inspired by the insightful work of (P. Calmon, 2015; Hsu et al., 2021) and adapted from (Razeghi, 2023).
    2. 2.2 Identification, Quantification, and Mitigation of Privacy Risks
    3. 2.3 Privacy-Enhancing Technologies
    4. 2.4 Prior-Dependent vs. Prior-Independent Mechanisms in PETs
    5. 2.5 Challenges in Data-Driven Privacy Preservation Mechanisms
    6. 2.6 Threats to PETs
    7. 2.7 Biometric PETs
    8. 2.8 Related Works
  3. 3 Preliminaries
    1. 3.1 General Loss Functions for Positive Measures
    2. 3.2 Measuring Privacy Leakage and Utility Performance
  4. 4 Privacy Funnel Model: Discriminative and Generative Paradigms
    1. 4.1 Discriminative Privacy Funnel ($\mathsf{DisPF}$) Method
    2. 4.2 Generative Privacy Funnel ($\mathsf{GenPF}$) Method
    3. 4.3 Threat Model
  5. 5 Deep Variational Privacy Funnel
    1. 5.1 Information Leakage Approximation
    2. 5.2 Information Utility Approximation
    3. 5.3 Deep Variational Privacy Funnel Objectives
    4. 5.4 Learning Framework
    5. 5.5 Role of Information Complexity in Privacy Leakage
  6. 6 Face Recognition Experiments
    1. 6.1 Face Recognition Leading Models and Their Core Mechanisms
    2. 6.2 Backbone Architectures for Feature Extraction
    3. 6.3 Datasets for Training and Validation
    4. 6.4 Metrics Used to Evaluate Face Recognition Model Performance
    5. 6.5 Experiments Setup
    6. 6.6 Experiments Results
    7. 6.7 Discussions and Future Directions
  7. 7 Conclusion
  8. A Connecting the Privacy Funnel Method with Other Models
    1. A.1 Connection with Information Bottleneck Model
    2. A.2 Connection with Complexity-Leakage-Utility Bottleneck Model
    3. A.3 Connection with Image-to-Image Transition Models
  9. B Estimation of Mutual Information via MINE
  10. C Training Details
    1. C.1 The Role of Randomness in DVPF Training
    2. C.2 Alpha Scheduler
    3. C.3 Uncertainty Decoder (Conditional Generator):
  11. D Deep Private Feature Extraction/Generation Experiment

##  1 Introduction

###  1.1 Motivation

(a)

(b)

Figure 1: High-level schematic comparison of privacy funnel models: (a)
discriminative (classical) paradigm; (b) generative paradigm.

In the evolving landscape of face recognition technology, a critical issue has
emerged: the need to balance privacy preservation with the utility of data.
This challenge is particularly acute in the context of representation
learning, where protecting individual privacy often conflicts with the demand
for high-quality data analysis. Existing solutions in privacy-preserving
representation learning face recognition approaches do not address the
inherent information-theoretic trade-off between privacy and utility. This gap
in the existing approaches underscores the importance of exploring new
methodologies that can pave the way to understanding, quantifying and
mitigating privacy risks for face recognition systems.

Our research introduces a novel approach to this problem by applying the
information-theoretic Privacy Funnel (PF) model to face recognition
systems111This manuscript extends our paper accepted at the 2024 IEEE
International Conference on Acoustics, Speech, and Signal Processing
(ICASSP’24) (Razeghi et al., 2024).. We propose a new method for privacy-
preserving representation learning within an end-to-end training framework.
This approach quantifies the trade-off between obfuscation and utility through
logarithmic loss, generalizable to other loss functions for positive measures,
effectively integrating the principles of information-theoretic privacy with
face recognition technology. The adaptability of our framework with recent
advancements in face recognition networks, including AdaFace and ArcFace,
highlights its relevance and applicability in current technological contexts.

Distinguishing our approach from conventional methods, we introduce the
Generative Privacy Funnel ($\mathsf{GenPF}$) model and the Deep Variational
Privacy Funnel (DVPF) framework. The $\mathsf{GenPF}$ model extends beyond the
traditional scope of Privacy Funnel analysis, offering new perspectives on
data generation with privacy guarantees. Concurrently, the DVPF model provides
a variational bound for measuring information leakage, enhancing the
understanding of privacy challenges in deep representation learning.
Additionally, the proposed methodology can also be combined with prior-
independent privacy-enhancing techniques like differential privacy, marrying
both prior-dependent and prior-independent mechanisms.

###  1.2 Key Contributions

Our research makes the following contributions to the field:

  * •

Overview of Data Privacy Paradigm: Our research presents an overview of the
current landscape in data privacy, articulating a clear understanding of
privacy risks and the methodologies for their identification, quantification,
and mitigation. It methodically categorizes Privacy-Enhancing Technologies
(PETs), distinguishing between prior-dependent and prior-independent
mechanisms and their significance in the broader context of privacy
protection. Part of this work serves as a brief, but foundational review,
offering a synthesized perspective on the privacy risk management in digital
environments, while also addressing the implications for biometric PETs.

  * •

First Study of Privacy Funnel Analysis in Face Recognition Systems: To the
best of our knowledge, our work is the first to apply privacy funnel analysis
to modern face recognition systems. This study bridges the gap between
information-theoretic privacy principles and the practical implementation of
privacy-preserving representation learning, focusing specifically on advanced
face recognition technologies. Our framework is also designed to be easily
integrated with recently developed face recognition networks like AdaFace and
ArcFace in a plug-and-play fashion.

  * •

Proposition of the Generative Privacy Funnel Model: We introduce the
Generative Privacy Funnel ($\mathsf{GenPF}$) model, setting it apart from the
traditional Privacy Funnel, which we now term the Discriminative Privacy
Funnel ($\mathsf{DisPF}$) model. The $\mathsf{GenPF}$ model offers new
insights into the generation of synthetic data with estimation-theoretic and
information-theoretic privacy guarantees. We further explore a specific
formulation of the $\mathsf{GenPF}$ model, demonstrating its potential and
effectiveness in privacy preservation within face recognition systems.

  * •

Introduction of the Deep Variational Privacy Funnel Model: Our work also
includes the development of the deep variational PF (DVPF) model. We present a
tight variational bound for quantifying information leakage, which provides a
deeper understanding of the complexities involved in privacy preservation
during deep variational PF learning. The DVPF model sheds light on connections
with various generative models, such as the VAE family, GAN family, and
Diffusion models. Furthermore, we have applied the DVPF model to the advanced
face recognition systems.

  * •

Versatile Data Processing Capabilities: Our model can process both raw image
samples and embeddings derived from facial images.

###  1.3 Outline

We explore the data privacy paradigm in Sec. 2. In Sec. 3, we briefly
introduce key preliminary concepts. Sec. 4 presents both the discriminative
and generative perspectives of the PF model. We then present the deep
variational PF model in Sec. 5. Experimental results related to face
recognition systems can be found in Sec. 6, and our conclusions are drawn in
Sec. 7.

###  1.4 Notations

Throughout this paper, random variables are denoted by capital letters (e.g.,
$X$, $Y$), deterministic values are denoted by small letters (e.g., $x$, $y$),
random vectors are denoted by capital bold letter (e.g., $\mathbf{X}$,
$\mathbf{Y}$), deterministic vectors are denoted by small bold letters (e.g.,
$\mathbf{x}$, $\mathbf{y}$), alphabets (sets) are denoted by calligraphic
fonts (e.g., $\mathcal{X,Y}$), and for specific quantities/values we use sans
serif font (e.g., $\mathsf{x}$, $\mathsf{y}$, $\mathsf{C}$, $\mathsf{D}$).
Also, we use the notation $\left[N\right]$ for the set $\\{1,2,\cdots,N\\}$.
$\mathrm{H}\left(P_{\mathbf{X}}\right)\coloneqq\mathbb{E}_{P_{\mathbf{X}}}\left[-\log
P_{\mathbf{X}}\right]$ denotes the Shannon entropy;
$\mathrm{H}\left(P_{\mathbf{X}}\|Q_{\mathbf{X}}\right)\coloneqq\mathbb{E}_{P_{\mathbf{X}}}\left[-\log
Q_{\mathbf{X}}\right]$ denotes the cross-entropy of the distribution
$P_{\mathbf{X}}$ relative to a distribution $Q_{\mathbf{X}}$; and
$\mathrm{H}\left(P_{\mathbf{Z}\mid\mathbf{X}}\|Q_{\mathbf{Z}\mid\mathbf{X}}\mid
P_{\mathbf{X}}\right)\coloneqq\mathbb{E}_{P_{\mathbf{X}}}\mathbb{E}_{P_{\mathbf{Z}\mid\mathbf{X}}}\left[-\log
Q_{\mathbf{Z}\mid\mathbf{X}}\right]$ denotes the cross-entropy loss for
$Q_{\mathbf{Z}\mid\mathbf{X}}$. The relative entropy is defined as
$\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{X}}\|Q_{\mathbf{X}}\right)\coloneqq\mathbb{E}_{P_{\mathbf{X}}}\big{[}\log\frac{P_{\mathbf{X}}}{Q_{\mathbf{X}}}\big{]}$.
The conditional relative entropy is defined by
$\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{Z}\mid\mathbf{X}}\|Q_{\mathbf{Z}\mid\mathbf{X}}\mid
P_{\mathbf{X}}\right)\coloneqq\mathbb{E}_{P_{\mathbf{X}}}\left[\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{Z}\mid\mathbf{X}=\mathbf{x}}\|Q_{\mathbf{Z}\mid\mathbf{X}=\mathbf{x}}\right)\right]$
and the mutual information is defined by
$\mathrm{I}\left(P_{\mathbf{X}};P_{\mathbf{Z}\mid\mathbf{X}}\right)\coloneqq\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{Z}\mid\mathbf{X}}\|P_{\mathbf{Z}}\mid
P_{\mathbf{X}}\right)$. We abuse notation to write
$\mathrm{H}\left(\mathbf{X}\right)=\mathrm{H}\left(P_{\mathbf{X}}\right)$ and
$\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)=\mathrm{I}\left(P_{\mathbf{X}};P_{\mathbf{Z}\mid\mathbf{X}}\right)$
for random objects222The name random object includes random variables, random
vectors and random processes. $\mathbf{X}\sim P_{\mathbf{X}}$ and
$\mathbf{Z}\sim P_{\mathbf{Z}}$. We use the same notation for the probability
distributions and the associated densities.

##  2 Navigating the Data Privacy Paradigm

Data privacy is an ever-changing domain, propelled by the surge in personal
and sensitive information production and exchange via digital platforms. Its
primary objective is safeguarding the confidentiality, integrity, and
availability of this information, restricting access and usage to authorized
parties only. To realize this aim, data privacy employs various strategies and
technologies such as encryption, access controls, and privacy-enhancing
technologies (PETs). These methods aim to thwart unauthorized access and
misuse of personal and sensitive data while minimizing the volume of
information that is exposed or shared.

One of the key challenges in data privacy is managing the delicate balance
between protecting personal and sensitive information and enabling its access
and use for legitimate purposes. This trade-off becomes particularly
challenging with rapid technological advances and the increasing demand for
data-driven services. Another challenge is the absence of global standards and
regulations for personal and sensitive information protection. Although many
countries have established their own data privacy laws, the significant
variations among these laws complicate the consistent protection of personal
and sensitive data across borders. Despite these challenges, this field of
data privacy is constantly evolving and advancing, with the development of new
technologies and approaches to better protect personal and sensitive
information.

The main challenge in the era of big data lies in balancing the utilization of
data-driven machine learning algorithms with the crucial importance of
protecting individuals’ privacy. The increasing volume of data collected and
used for training machine learning algorithms raises concerns about potential
misuse and privacy invasions. This situation presents several open problems,
such as devising effective methods to anonymize data—ensuring individuals
cannot be identified from training data—and developing robust technologies to
safeguard personal information. Furthermore, there’s an imperative need to
establish ethical guidelines for data use in machine learning, ensuring
adherence to protect individuals’ rights.

###  2.1 Lunch with Turing and Shannon333This section is inspired by the
insightful work of (P. Calmon, 2015; Hsu et al., 2021) and adapted from
(Razeghi, 2023).

Alan Turing visited Bell Labs in 1943, during the peak of World War II, to
examine the X-system, a secret voice scrambler for private telephone
communications between the authorities in London and Washington. While there,
he met Claude Shannon, who was also working on cryptography. In a 28 July 1982
interview with Robert Price in Winchester, MA (Price, 1982), Shannon
reminisced about their regular lunch meetings where they discussed computing
machines and the human brain instead of cryptography (Guizzo, 2003). Shannon
shared with Turing his ideas for what would eventually become known as
information theory, but according to Shannon, Turing did not believe these
ideas were heading in the right direction and provided negative feedback.
Despite this, Shannon’s ideas went on to be influential in the development of
information theory, which has had a significant impact on the fields of
computer science and telecommunications.

Privacy has been a central concern in the fields of information theory and
computer science since its inception. The interaction between Shannon and
Turing highlights the different approaches taken by the two communities to
address the issue of preventing unauthorized access to information contained
in disclosed data. These approaches often involve the use of unique models and
distinct mathematical techniques. It is important to note that these
approaches have evolved over time as technology and threats to privacy have
changed and continue to be an active area of research and development in both
fields.

In the 1970s, two influential papers on privacy were published that
highlighted the differences in approaches between information theory and
computer science. The first paper, authored by Aaron Wyner while he was
working at Bell Labs, introduced the concept of a wire-tap channel (Wyner,
1975), where data is transmitted over a discrete, memoryless channel (DMC)
that is subject to interception by an eavesdropper, who is modeled as a second
DMC observing the output of the first DMC. Wyner showed that it is possible to
achieve perfect secrecy, or the ability to communicate without any information
being disclosed to the interceptor, by designing codes that take advantage of
the noise in the channel observed by the eavesdropper. This approach to
privacy, which does not rely on assumptions about an adversary’s computational
abilities and is often referred to as information-theoretic secrecy, became a
key focus of research within the field of information theory.

In November 1976, Diffie and Hellman published a paper that introduced the
concept of public key cryptography and described how it can be used to achieve
secure communication without the need for a shared secret key (Hellman, 1976).
This approach to cryptography, which is based on the difficulty of discovering
private information without additional knowledge, such as a private key,
ensures security against an adversary who is limited in their computational
abilities. As a result, public key cryptographic systems are easier to
implement and deploy compared to approaches that rely on information-theoretic
secrecy, which do not make assumptions about an adversary’s computational
capabilities. The paper also discussed public key distribution systems and
verifiable digital signatures, which are important tools in ensuring the
security of communication.

Since the publication of the papers on information theory and computer science
approaches to privacy, public key cryptography, which assumes that an
adversary is limited in their computational abilities, has become widely used
in a variety of applications, including banking, healthcare, and public
services. It is used billions of times a day in applications ranging from
digital rights management to cryptocurrency. In contrast, information-
theoretic approaches to secrecy, which do not make assumptions about an
adversary’s computational capabilities, have been less successful in practical
applications. Perfect secrecy against a computationally unbounded adversary
requires strict assumptions, which can lead to elegant mathematical models but
often result in security schemes that are difficult to implement in practice.

The intersection of information theory and computer science approaches to
privacy continues to be relevant in today’s world, where the collection of
individual-level data has increased significantly. This development has
brought both challenges and opportunities for both fields, as the widespread
collection of data has brought significant economic benefits, such as
personalized services and innovative business models, but also poses new
privacy threats. For example, social media posts may be used for undesirable
political targeting (Effing et al., 2011; O’reilly et al., 2018), machine
learning models may reveal sensitive information about the data used for
training (Abadi et al., 2016), and public databases may be deanonymized with
only a few queries (Narayanan and Shmatikov, 2008; Su et al., 2017). Both
fields have faced new challenges and opportunities in addressing these issues.

###  2.2 Identification, Quantification, and Mitigation of Privacy Risks

Addressing privacy risks is paramount across all stages of personal data
handling, including (i) collection, (ii) storage, (iii) processing, and (iv)
sharing (dissemination). This comprehensive perspective ensures robust privacy
protection applicable in various contexts, from traditional data management to
advanced machine learning models. The growing body of research focused on
managing privacy risks addresses three fundamental questions:
‘identification’, ‘quantification’, and ‘mitigation’ of these risks, setting
the foundation for detailed exploration of state-of-the-art practices and
methodologies in the field.

  * (a)

Identification: How can we effectively identify the risk of data leakage and
potential privacy attacks across the entire data lifecycle, from collection
through to processing and sharing?

  * (b)

Quantification: Following the identification of privacy risks, what
metrics444In this document, ‘metric’ is employed not in the traditional
mathematical sense of a distance function but as a quantifier for assessing
privacy risk. can be developed and applied to precisely quantify these risks
and monitor the effectiveness of implemented privacy protection strategies?

  * (c)

Mitigation: With a comprehensive understanding of privacy, what strategies can
be formulated and implemented to mitigate identified risks, ensuring an
optimal balance between operational objectives and privacy, in line with legal
and ethical standards?

The following discussion will provide a brief exploration of these pivotal
questions.

####  2.2.1 Identification of Privacy Risks

Identifying (understanding) privacy risks is a critical first step in
safeguarding privacy across the entire data lifecycle, including collection,
storage, processing, and dissemination (Solove, 2002, 2005). This task becomes
increasingly vital and, at times, complex within the context of both
traditional data management practices and the utilization of machine learning
algorithms (Solove, 2010, 2024). The identification process requires a
detailed understanding of potential vulnerabilities that could lead to data
leakage and privacy attacks, alongside the development of systematic
approaches to detect and assess these risks (Solove, 2010; Smith et al., 2011;
Orekondy et al., 2017; Milne et al., 2017; Beigi and Liu, 2020). We briefly
explore several key methodologies that are essential for the comprehensive
identification of privacy risks in these areas.

##### Data Sensitivity Analysis:

Evaluating the inherent privacy risks associated with specific data types is
fundamental in both conventional data management and machine learning
contexts. This involves a meticulous assessment of datasets to identify data
containing personally identifiable information or sensitive personal
information. Through techniques such as attribute-based risk assessment and
the application of privacy-preserving data mining principles, organizations
can identify data elements that necessitate heightened protection measures.
This analysis is crucial for setting the stage for privacy risk management,
enabling a prioritized focus on the most sensitive data elements.

##### Vulnerability Assessment Across Data Lifecycle:

Conducting comprehensive vulnerability assessments is essential for
identifying potential weaknesses that could lead to privacy breaches or
violations. This process spans the entire data lifecycle, from initial data
collection and storage to its processing and final dissemination. In the
context of machine learning, this includes a detailed examination of model
architectures, training procedures, and data processing pipelines to identify
potential points of data leakage. Utilizing automated tools and frameworks for
privacy audit and analysis plays a pivotal role in facilitating these
comprehensive assessments, ensuring that vulnerabilities can be identified and
addressed proactively.

##### Simulated Privacy Attack Scenarios:

Simulating potential privacy attacks is a proactive strategy for identifying
vulnerabilities within both traditional data handling systems and machine
learning models. Techniques such as adversarial modeling and synthetic data
generation are employed to test the ease with which a model can be manipulated
or data can be re-identified. This approach is particularly pertinent in the
machine learning arena, where algorithms may be susceptible to specific
privacy threats, including model inversion attacks and membership inference
attacks. These simulated attack scenarios are invaluable for assessing the
robustness of privacy protections and for highlighting areas where additional
safeguards are needed.

####  2.2.2 Quantification of Privacy Risks

Following the identification of privacy risks and an in-depth understanding of
the regulatory and ethical standards relevant to data privacy, it is crucial
to precisely define and apply metrics. These metrics are indispensable for
accurately quantifying the identified privacy risks and for diligently
tracking progress towards their effective mitigation. Quantifying privacy
risks requires the development of metrics that can precisely measure the
severity of these risks across various stages of data handling, including
collection, storage, processing, and dissemination. Importantly, the
applicability and specificity of these metrics vary significantly based on the
data lifecycle stage and the specific application context (Duchi et al.,
2013a, 2014; Mendes and Vilela, 2017; Duchi et al., 2018; Wagner and Eckhoff,
2018; Bhowmick et al., 2018; Liao et al., 2019; Hsu et al., 2020; Bloch et
al., 2021; Saeidian et al., 2021). This variability means that different
metrics may have different operational interpretations (Issa et al., 2019;
Kurri et al., 2023), necessitating a nuanced approach to their selection and
implementation. These metrics act as definitive indicators of the privacy
stance of data processing systems, enabling data custodians to make well-
informed decisions about privacy risk management. We review a specific type of
these quantitative metrics in Sec. 3.

####  2.2.3 Mitigation of Privacy Risks

In addressing the spectrum of privacy risks inherent in digital data handling,
it’s imperative to employ a multifaceted strategy that encompasses both the
theoretical frameworks and practical tools available for risk mitigation.
Among these methodologies, PETs emerge as a critical subset, offering targeted
solutions to protect personal data across its lifecycle. PETs are pivotal
category of tools and methodologies designed to directly safeguard personal
privacy by mitigating the risks associated with the collection, storage,
processing, and sharing of personal data. PETs embody the principle of privacy
by design, ensuring that privacy considerations are embedded within the
infrastructure of digital technologies from the ground up. Deploying
foundational techniques in PETs, such as pseudonymization (Chaum, 1981, 1985),
anonymization (Sweeney, 2000, 2002), and encryption (Shannon, 1949; Diffie and
Hellman, 1976; Hellman, 1977), directly addresses privacy risks by protecting
data identifiability and integrity from collection to dissemination. We review
PETs in Sec. 2.3.

###  2.3 Privacy-Enhancing Technologies

PETs play a vital role in safeguarding individual privacy by directly
countering privacy threats. As adversaries constantly refine their methods,
PETs become indispensable in the relentless pursuit to shield sensitive
information from unauthorized access and misuse. Spanning a wide range, these
technologies tackle various facets of privacy and data security.

####  2.3.1 Encryption, Anonymization, Obfuscation, and Information-Theoretic
Technologies

These techniques are at the forefront of PETs, offering robust solutions to
secure data whether it’s stored (at rest), being transmitted (in transit), or
actively used (in use). Encryption technologies, encompassing both symmetric
and asymmetric methods, as well as homomorphic encryption, serve as critical
components by enabling secure data storage and transmission, along with the
capacity for encrypted data processing. Complementing these are data
pseudonymization (Chaum, 1981, 1985) and anonymization (Sweeney, 2000)
techniques, which obscure personal identifiers, effectively anonymizing data
to prevent direct association with individuals. Furthermore, differential
privacy (Dwork et al., 2006b) introduces a probabilistic layer to data
protection, injecting carefully calibrated noise to aggregated data or query
outputs. This ensures that individual data points cannot be discerned, thereby
protecting personal information from inference attacks within statistical
datasets or when deploying machine learning models. Expanding upon these,
information-theoretic privacy techniques offer a fundamental approach to data
security by focusing on the maximum amount of information that can be gained
by an adversary, regardless of the adversary’s computational power. By
assessing the unpredictability and uncertainty in data, these methods ensure a
theoretical limit on the information leakage, making them indispensable in
scenarios where robust privacy guarantees are required, especially in the
absence of assumptions about the adversary’s computational capabilities. In
Sec. 2.4, we review these techniques from the standpoint of the prior
knowledge we have regarding the data distribution.

####  2.3.2 Privacy-Preserving Computation Technologies

Secure computation techniques are essential for maintaining privacy during
data processing (Yao, 1982; Micali and Rogaway, 1992; Mohassel and Rindal,
2018; Juvekar et al., 2018; Keller, 2020; Knott et al., 2021; Neel et al.,
2021). Confidential computing (Mohassel and Rindal, 2018; Mo et al., 2022;
Vaswani et al., 2023), which employs Trusted Execution Environments (TEEs)
(Sabt et al., 2015), is a critical tool, isolating computation to protect data
in use from both internal and external threats. Additionally, Secure Multi-
party Computation (SMPC) (Goldreich, 1998; Du and Atallah, 2001; Cramer et
al., 2015; Knott et al., 2021) facilitates collaborative computation over data
distributed among multiple parties without revealing the data itself, enabling
joint data analysis or model training while preserving the privacy of each
party’s data. Zero-Knowledge Proofs (ZKPs) (Fiege et al., 1987; Kilian, 1992;
Goldreich and Oren, 1994) offer another layer of security, allowing one party
to prove the truth of a statement to another party without revealing any
information beyond the validity of the statement itself, essential for
scenarios requiring validation of data authenticity or integrity without
exposing the data.

####  2.3.3 Decentralized Privacy Technologies

Decentralized privacy-preserving technologies encapsulate methods that enable
collaborative and/or federated data analysis and model training across
dispersed datasets without exposing the underlying data (Shokri and Shmatikov,
2015; McMahan et al., 2017; Dwivedi et al., 2019; Wei et al., 2020; Kaissis et
al., 2020; Kairouz et al., 2021; Shiri et al., 2023). Federated learning
exemplifies this approach by allowing machine learning models to be trained
across various devices or servers. Rather than centralizing raw data, which
could pose significant privacy risks, this technique involves aggregating
model updates derived from local data. Such a decentralized approach not only
maintains the confidentiality of individual contributions but also harnesses
collective insights to improve model accuracy and performance, embodying the
principles of privacy preservation in a distributed computing framework.

###  2.4 Prior-Dependent vs. Prior-Independent Mechanisms in PETs

There are two main types of privacy-enhancing mechanisms: ‘prior-independent’
and ‘prior-dependent’. Prior-independent mechanisms make minimal assumptions
about the data distribution and the information held by an adversary and are
designed to protect privacy regardless of the specific characteristics of the
data being protected or the motivations and capabilities of any potential
adversaries. Prior-dependent mechanisms, on the other hand, make use of
knowledge about the probability distribution of private data and the abilities
of adversaries in order to design privacy-preserving mechanisms. These
mechanisms may be more effective in certain scenarios where the
characteristics of the data and the adversary are known or can be reasonably
estimated but may be less robust in situations where such information is
uncertain or changes over time.

Data anonymization (Sweeney, 2000) techniques, such as $k$-anonymity (Sweeney,
2002), $\ell$-diversity (Machanavajjhala et al., 2006), $t$-closeness (Li et
al., 2007), differential privacy (DP) (Dwork et al., 2006b), and pufferfish
(Kifer and Machanavajjhala, 2012), aim to preserve the privacy of data through
various forms of data perturbation. These techniques focus on queries,
inference algorithms, and probability measures, with DP being the most popular
context-free privacy notion based on the distinguishability of “neighboring”
databases. However, DP does not provide any guarantee on the average or
maximum information leakage (P. Calmon and Fawaz, 2012), and pufferfish, while
able to capture data correlation, does not prioritize preserving data utility.

DP is a privacy metric that measures the impact of small perturbations at the
input of a privacy mechanism on the probability distribution of the output. A
mechanism is said to be $\epsilon$-differentially private if the probability
of any output event does not change by more than a multiplicative factor
$e^{\epsilon}$ for any two neighboring inputs, where the definition of
“neighboring” inputs depends on the chosen metric of the input space. DP is
prior-independent and often used in statistical queries to ensure the result
remains approximately the same regardless of whether an individual’s record is
included in the dataset. The privacy guarantee of DP can typically be achieved
through the use of additive noise mechanisms, such as adding a small
perturbation or random noise from a Gaussian, Laplacian, or exponential
distribution (Dwork et al., 2014).

DP has been modified in various ways since its introduction, with variations
including approximate differential privacy (which allows for a small
additional parameter called delta) (Dwork et al., 2006a), local differential
privacy (which assumes all inputs are neighboring) (Duchi et al., 2013b), and
Rényi differential privacy (which uses Rényi divergence to measure the
difference in output distributions from two neighboring inputs) (Mironov,
2017). DP has two key properties that make it useful for protecting privacy:
(i) it is composable (Dwork et al., 2014; Abadi et al., 2016), meaning that
the aggregate output of multiple observations from a DP mechanism still
satisfies DP requirements; and (ii) it is robust to post-processing (Dwork et
al., 2014), meaning that the output of a DP mechanism remains private even
after further processing. These properties enable the modular construction and
analysis of privacy mechanisms with a specific privacy leakage budget.

Information-theoretic (IT) privacy is the study of designing mechanisms and
metrics that preserve privacy when the statistical properties or probability
distribution of data can be estimated or partially known. IT privacy
approaches (Reed, 1973; Yamamoto, 1983; Evfimievski et al., 2003; Rebollo-
Monedero et al., 2009; P. Calmon and Fawaz, 2012; Sankar et al., 2013; P.
Calmon et al., 2013; Makhdoumi and Fawaz, 2013; Asoodeh et al., 2014; P.
Calmon et al., 2015; Salamatian et al., 2015; Basciftci et al., 2016; Asoodeh
et al., 2016; Kalantari et al., 2017; Rassouli et al., 2018; Asoodeh et al.,
2018; Rassouli and Gündüz, 2018; Liao et al., 2018; Osia et al., 2018;
Tripathy et al., 2019; Hsu et al., 2019; Liao et al., 2019; Sreekumar and
Gündüz, 2019; Xiao and Khisti, 2019; Diaz et al., 2019; Rassouli et al., 2019;
Rassouli and Gündüz, 2019; Razeghi et al., 2020; Zarrabian et al., 2023;
Zamani et al., 2023; Saeidian et al., 2023) model and analyze the trade-off
between privacy and utility using IT metrics, which quantify how much
information an adversary can gain about private features of data from access
to disclosed data. These metrics are often formulated in terms of divergences
between probability distributions, such as f-divergences and Rényi divergence.
IT privacy metrics can be operationalized in terms of an adversary’s ability
to infer sensitive data and can be used to balance the trade-off between
allowing useful information to be drawn from disclosed data and preserving
privacy. By using prior knowledge about the statistical properties of data and
assumptions about the adversary’s inference capabilities, IT privacy can help
to understand the fundamental limits of privacy and how to balance privacy and
utility.

The IT privacy framework is based on the presence of a private variable and a
correlated non-private variable, and the goal is to design a privacy-assuring
mapping that transforms these variables into a new representation that
achieves a specific target utility while minimizing the information inferred
about the private variable. IT privacy approaches provide a context-aware
notion of privacy that can explicitly model the capabilities of data users and
adversaries, but they require statistical knowledge of data, also known as
priors. This framework is inspired by Shannon’s information-theoretic notion
of secrecy (Shannon, 1949), where security is measured through the
equivocation rate at the eavesdropper555A secret listener (wiretapper) to
private conversations., and by Reed (Reed, 1973) and Yamamoto’s (Yamamoto,
1983) treatment of security and privacy from a lossy source coding standpoint.

###  2.5 Challenges in Data-Driven Privacy Preservation Mechanisms

In the era of modern data-driven economies, cryptographic techniques, while
foundational, face limitations in effectively addressing the spectrum of
privacy risks. This limitation arises primarily because adversarial entities
can directly observe disclosed data, complicating the protection of sensitive
information. For instance, in scenarios where statisticians query databases
containing sensitive information, merely encrypting outputs is insufficient.
Such practices fail to prevent potential inference of private information
through multiple queries, a challenge exemplified by the U.S. Census’ efforts
to release population statistics without compromising individual privacy
(Machanavajjhala et al., 2008). Similarly, in machine learning applications
where user data is essential for model training, the dual-edged sword of data
disclosure benefits and privacy risks becomes apparent. Here, the risk emerges
from adversaries’ ability to extract individual-level information by analyzing
the model’s responses.

The overarching objective in these data-centric contexts is not to eliminate
information leakage entirely—a feat that is virtually unattainable against
both computationally bounded and information-theoretic adversaries. Instead,
the goal shifts towards achieving a demonstrable level of privacy that
balances with utility. This nuanced approach stands in contrast to the
absolute zero information leakage ideal of both cryptography and information-
theoretic security models. Here, the privacy threat model envisages
adversaries who scrutinize disclosed data to deduce sensitive information,
such as political preferences or the presence of an individual within a
dataset.

Emerging privacy mechanisms, spearheaded by advances in both computer science
and information theory, eschew assumptions on the adversary’s computational
prowess. These innovative approaches vary in the adversary’s inference
objectives—ranging from probability of correct guesses to minimizing mean-
squared reconstruction errors—and the modeling of private information. A
critical challenge in this domain is the utility-privacy trade-off, which
demands careful consideration of application-specific utility against privacy
needs.

Among the vanguard of data-driven privacy solutions are mechanisms inspired by
Generative Adversarial Networks (GANs). These strategies conceptualize privacy
protection as a strategic game between a defender (or privatizer) and an
adversary, where the privatizer’s goal is to encode datasets to thwart
inference leakage regarding private or sensitive variables (Edwards and
Storkey, 2016; Hamm, 2017; Huang et al., 2017; Tripathy et al., 2019; Huang et
al., 2018). Meanwhile, the adversary endeavors to extract these variables from
the released data. The optimization of privacy-preserving mechanisms through
adversarial training—whether deterministic or stochastic—epitomizes the
dynamic interplay between privacy protection and data utility.

As machine learning technologies evolve and become more pervasive, the
imperative for robust, data-driven privacy mechanisms becomes increasingly
critical. These privacy safeguards are indispensable for maintaining
individual privacy, fostering public trust in organizations and governments,
and mitigating the potential adverse effects of data breaches. Such breaches
can lead to significant repercussions, including reputational damage,
financial loss, and eroded trust in digital institutions. Thus, prioritizing
the development and implementation of potent privacy-preserving strategies is
paramount in safeguarding sensitive information in our increasingly data-
centric world.

###  2.6 Threats to PETs

In this subsection, we briefly explore the challenges confronting PETs,
reviewing the diverse types of attacks that seek to undermine the integrity
and confidentiality of PETs. We discuss adversaries characterized by a broad
range of objectives, as well as the strategies they utilize.

####  2.6.1 Adversary Objectives

Based on our understanding, we can classify the adversary objectives into
three main categories: (i) data reconstruction, (ii) unauthorized access, and
(iii) user re-identification.

##### Data Reconstruction:

The primary aim here is to accurately reconstruct original data from its
encoded or protected form, whether through cryptographic, statistical-
theoretic, information-theoretic, or estimation-theoretic privacy perspectives
(Agrawal and Srikant, 2000; Rebollo-Monedero et al., 2009; Sankar et al.,
2013; Asoodeh et al., 2016; Dwork et al., 2017; Bhowmick et al., 2018;
Ferdowsi et al., 2020; Stock et al., 2022; Razeghi et al., 2023; Shiri et al.,
2024). This encompasses two distinct goals. The first, ‘attribute-level
recovery’ (micro goal), involves extracting specific attributes or features
from protected data, ranging from demographic information to unique behavioral
patterns. Adversaries might utilize these details for actions like
impersonation, targeted phishing attacks, or information profiling. The
second, ‘complete data reconstruction’ (macro goal), goes a step further.
Here, adversaries aim to fully revert privacy-preserving measures to access
entire datasets. This could entail piecing together fragmented personal data
from multiple sources or decrypting encrypted information to reveal sensitive
details, leading to potential misuses such as identity theft, financial fraud,
or corporate espionage.

##### Unauthorized Access:

The objective here is for the adversary to gain access to systems, networks,
or data to which they do not have permission or authorization (Dunne, 1994;
Campbell et al., 2003; Winn, 2007; Mohammed, 2012; Muslukhov et al., 2013;
Sloan and Warner, 2017; Razeghi et al., 2018; Maithili et al., 2018; Prokofiev
et al., 2018; Wang et al., 2019). This can be for various purposes such as
stealing sensitive data, disrupting system operations, planting malware, or
conducting espionage. The underlying goal is to infiltrate a system or
database without being detected and without having legitimate credentials or
authorization.

##### User Re-identification:

The primary objective in user re-identification is the subtle linking of
anonymized data back to identifiable individuals, despite the absence of
direct identifiers (El Emam et al., 2011; Layne et al., 2012; Zheng et al.,
2015; Henriksen-Bulmer and Jeary, 2016; Zheng et al., 2016; Ye et al., 2021).
In this pursuit, adversaries are not just attempting to reveal identities but
are actively aiming to undermine the integrity and purpose of data
anonymization and privacy-preserving measures. Their efforts are directed
toward establishing connections between seemingly unrelated, anonymized data
sets and specific, identifiable individuals. This includes efforts aimed at
correlating disjointed pieces of information to reconstruct identifiable
profiles and, in more extended scenarios, to track individual behaviors and
patterns over time. Such objectives pose significant privacy concerns, as they
directly challenge the effectiveness of data anonymization techniques intended
to protect individual identities and personal information.

####  2.6.2 Adversary Knowledge

##### Technical Model Insights:

The adversary may have a deep understanding of the computational models or
algorithms used in the system. It encompasses comprehensive knowledge of model
architecture, parameters, training methodologies, vulnerabilities, and
potential ways to exploit them (Wang and Gong, 2018; Song et al., 2019; Oseni
et al., 2021; Bober-Irizar et al., 2023; Yang et al., 2023). This includes
detailed knowledge of the model’s architecture such as layers, neurons,
weights, and activation functions, which enables pinpointing and exploiting
inherent weaknesses or backdoors. Additionally, insights into the training
regime, including epochs, learning rates, and loss functions, are crucial in
informing the creation of attacks tailored to the model’s vulnerabilities.

##### Operational System Workflow:

Adversary knowledge in this area encompasses the overall operational workflow
of the system, including system architecture, data flow, user interactions,
security protocols, network infrastructure, third-party integrations, and
external dependencies. A detailed understanding of this area can highlight
areas of vulnerability. This includes tracing the data’s journey from
inception to its ultimate storage or application to unearth vulnerabilities at
various stages and discerning the nuances of the system’s decision-making
process, particularly validation or rejection thresholds, to craft precision-
targeted manipulative inputs.

##### Data and Information Profiling:

In this category, adversary insights into the data used by the system are
covered. It includes understanding the types and sources of data, data
processing methods, inherent biases, security measures for data protection,
and potential data vulnerabilities. Important aspects are understanding the
underlying structure and distribution of features within the dataset,
including anomalies and edge cases, for designing bespoke attack vectors and
analyzing even a limited subset of the true dataset to deduce the full
dataset’s characteristics, particularly useful in reconnaissance attacks.

##### Security Mechanisms and Protocols:

This involves an understanding of the security measures in place, including
authentication processes, encryption techniques, and other security protocols.
Adversaries with this knowledge can plan attacks targeting specific weaknesses
in the security setup.

##### Insider Operational Knowledge:

This includes knowledge acquired from having had legitimate access to the
system or from detailed observations. It encompasses not just technical
aspects but also operational procedures, organizational culture, and internal
policies. This type of knowledge is particularly dangerous as it can lead to
highly targeted and effective attacks.

####  2.6.3 Adversary Strategy

Adversary strategies span a wide array encompassing technical, systemic, data-
centric, sociotechnical, and insider threats, to name a few, which cannot
cover widely in our brief overview. From exploiting algorithmic
vulnerabilities and cryptographic flaws to manipulating system infrastructures
and communication protocols, these strategies reveal the depth and complexity
of modern security challenges. Data-centric approaches further diversify these
tactics through the exploitation of information leakage and the creation of
synthetic identities. Privacy invasions leverage advanced traffic analysis and
misuse of biometric data, while sociotechnical maneuvers utilize
misinformation and target supply chains to undermine security indirectly.
Insider and environmental manipulations, along with the exploitation of
peripheral devices, underscore the necessity for comprehensive and integrated
defense mechanisms. This extensive spectrum of adversary tactics necessitates
advanced, multi-layered security measures that combine technological,
procedural, and organizational strategies to counteract the evolving landscape
of threats. In the context of machine learning and artificial intelligence,
distinct vulnerabilities emerge. Below, we briefly review a couple of these
strategies.

##### Gradient-Oriented Attacks:

In the gradient-oriented attacks, adversaries employ techniques to manipulate
or infer the gradient calculations that are fundamental to machine learning
models. These attacks are categorized into two main approaches: white-box and
black-box attacks (Liu et al., 2016; Papernot et al., 2017; Ilyas et al.,
2018; Bhagoji et al., 2018; Porkodi et al., 2018; Alzantot et al., 2019; Guo
et al., 2019; Sablayrolles et al., 2019; Rahmati et al., 2020; Tashiro et al.,
2020). In white-box attacks, attackers leverage detailed knowledge of the
model’s architecture and parameters to either reverse-engineer the model or
approximate the original training data, thus compromising the model’s
decision-making process. This technique directly facilitates model inference
attacks (Tramèr et al., 2016; Batina et al., 2019; Chandrasekaran et al.,
2020), where the attacker’s goal is to uncover the model’s structure or deduce
characteristics of the training data based on observable outputs or gradients.
Black-box attacks contrast by not requiring direct access to the model’s
internals. Instead, adversaries deploy probing techniques to estimate
gradients and discern underlying data patterns, enabling them to infer
critical insights about the model’s behavior and vulnerabilities. This method
is particularly effective in membership inference attacks (Shokri et al.,
2017), where the attacker aims to determine if specific data points were used
in the model’s training set by observing the model’s predictions in response
to crafted inputs.

##### Temporal Pattern Analysis:

Temporal pattern analysis involves scrutinizing temporal sequences in
verification data to detect vulnerabilities (Kamat et al., 2009; Xiao and
Xiong, 2015; Backes et al., 2016; Grover and Mark, 2017; Leong et al., 2020;
Qi et al., 2020; Zhang et al., 2021; Li et al., 2023). Through sequential
vulnerability detection, adversaries can highlight discernible patterns,
potentially allowing them to predict when the system might introduce new noise
or make updates, making their attacks more timely and effective. Time-based
vulnerability exploitation leverages the analysis of temporal patterns in data
processing, enabling adversaries to identify recurrent vulnerabilities or even
predict system behavior at given times. This strategy could lead to targeted
attacks during moments of predicted vulnerability or broad system disruptions
during peak operation hours.

##### Comprehensive Synergy Attacks:

Comprehensive synergy attacks represent a multifaceted approach by integrating
various data sources and modalities to uncover vulnerabilities and enhance the
efficacy of attacks. This strategy can employ data gathered from
eavesdropping, database interactions, and more. A critical component of this
strategy is data poisoning (Biggio et al., 2012; Guo and Liu, 2020; Tian et
al., 2022; Wang et al., 2022; Ramirez et al., 2022; Carlini et al., 2023),
where adversaries deliberately introduce corrupted, misleading, or
intentionally mislabeled data into the training set. The aim is to compromise
the integrity of the machine learning model, leading to biased outcomes,
incorrect predictions, or complete system failure. By corrupting the
foundation of data on which models are trained, adversaries can significantly
degrade the reliability and fairness of machine learning applications.
Additionally, multi-modal synthesis (Abdullakutty et al., 2021; Liu et al.,
2021; Hu et al., 2022) and noise reduction (Voloshynovskiy et al., 2000, 2001;
Lu et al., 2002; Kloukiniotis et al., 2022; Chen et al., 2023) techniques are
employed to augment data reconstruction efforts or manipulate verification
procedures. For instance, denoising algorithms may be repurposed by
adversaries to retrieve original, unaltered data, aiding in the generation of
synthetic data for identity fraud. Through these comprehensive efforts,
adversaries not only exploit existing vulnerabilities but also proactively
create new ones, making it increasingly challenging to safeguard data and
models against such attacks.

###  2.7 Biometric PETs

Biometric recognition systems automate individual identification through
distinctive behavioral and biological traits. Biometric recognition systems
fundamentally comprise four key subsystems: data capture, signal processing
and feature extraction, comparison, and data storage. These subsystems work in
unison to manage biometric data through its lifecycle, beginning with the
capture of biometric samples, proceeding with the extraction of useful
features, and concluding with the comparison and secure storage of biometric
information. However, the security and privacy vulnerabilities inherent in
face recognition systems, particularly through the potential reconstruction of
face images from stored templates (embeddings), pose significant challenges.

Recent advancements have introduced a suite of Biometric Privacy-Enhancing
Technologies (B-PETs), designed to safeguard sensitive information embedded
within biometric templates. These techniques prioritize the protection of
identity-related data, either by obscuring biometric templates through
template protection schemes or by minimizing the inclusion of privacy-
sensitive attributes such as age, gender, and ethnicity.

The ISO/IEC 24745 standard (ISO/IEC 24745:2022, E) sets forth four primary
requirements for each biometric template protection scheme, encompassing the
principles of cancelability, unlinkability, irreversibility, and the
preservation of recognition performance. These biometric template protection
schemes can be categorized into two main groups: (i) cancelable biometrics,
which encompasses techniques like Bio-Hashing (Jin et al., 2004), MLP-Hash
(Shahreza et al., 2023a), IoM-Hashing (Jin et al., 2017), among others, and
rely on transformation functions dependent on keys to generate protected
templates (Nandakumar and Jain, 2015; Sandhya and Prasad, 2017; Rathgeb et
al., 2022), and (ii) biometric cryptosystems, which include methodologies such
as fuzzy commitment (Juels and Wattenberg, 1999) and fuzzy vault (Juels and
Sudan, 2006), either binding keys to biometric templates or generating keys
from these templates (Uludag et al., 2004; Rathgeb et al., 2022).
Additionally, some researchers have explored the application of Homomorphic
Encryption for template protection in face recognition systems (Boddeti, 2018;
Bassit et al., 2021; Shahreza et al., 2022).

Face recognition systems, as extensively discussed in prior research (Biggio
et al., 2015; Galbally et al., 2010; Marcel et al., 2023), are not only
susceptible to security threats but also face serious privacy vulnerabilities.
These systems rely on facial templates extracted from face images, which
inherently contain sensitive information about the individuals they represent.
Recent studies have even demonstrated an adversary’s capability to reconstruct
face images from templates stored within a face recognition system’s database
(Shahreza and Marcel, 2023b, a).

To address the privacy concerns surrounding face recognition systems, numerous
privacy-preserving methods have emerged in the literature. These privacy-
enhancing techniques predominantly focus on protecting identity-related
information within face templates through the utilization of template
protection schemes (Razeghi et al., 2017; Boddeti, 2018; Rezaeifar et al.,
2019; Mai et al., 2020; Hahn and Marcel, 2022; Shahreza et al., 2022, 2023b;
Abdullahi et al., 2024), or on minimizing the inclusion of privacy-sensitive
attributes, such as age, gender, ethnicity, among others, in these templates
(Morales et al., 2020; Melzi et al., 2023).

###  2.8 Related Works

To address the most closely related works to ours, we consider two categories
of research, which, while seemingly distinct, are indeed related. The first
category encompasses research papers studying and analyzing the privacy funnel
model, and the second comprises works addressing disentangled representation
learning.

Considering the Markov chain
$\mathbf{S}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{X}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{Z}$,
the authors in (Hsu et al., 2020; de Freitas and Geiger, 2022; Huang and
Gamal, 2024) tackle the privacy funnel problem. In (Hsu et al., 2020), the
authors introduce a method to enhance privacy in datasets by identifying and
obfuscating features that leak sensitive information. They propose a framework
for detecting these information-leaking features using information density
estimation, where features with information densities exceeding a predefined
threshold are considered risky and are subsequently obfuscated. This process
is data-driven, utilizing a new estimator known as the trimmed information
density estimator (TIDE) for practical implementation.

In (de Freitas and Geiger, 2022), the authors present the conditional privacy
funnel with side-information (CPFSI) framework. This framework extends the
privacy funnel method by incorporating additional side information to optimize
the trade-off between data compression and maintaining informativeness for a
downstream task. The goal is to learn invariant representations in machine
learning, with a focus on fairness and privacy in both fully and semi-
supervised settings. Through empirical analysis, it is demonstrated that CPFSI
can learn fairer representations with minimal labels and effectively reduce
information leakage about sensitive attributes.

More recently, (Huang and Gamal, 2024) proposes an efficient solver for the
privacy funnel problem by exploiting its difference-of-convex structure,
resulting in a solver with a closed-form update equation. For cases of known
distribution, this solver is proven to converge to local stationary points and
empirically surpasses current state-of-the-art methods in delineating the
privacy-utility trade-off. For unknown distribution cases, where only
empirical samples are accessible, the effectiveness of the proposed solver is
demonstrated through experiments on MNIST and Fashion-MNIST datasets.

In the domain of face recognition, the closest work to ours is (Morales et
al., 2020), where the authors introduce a privacy-preserving feature
representation learning approach designed to eliminate sensitive information,
such as gender or ethnicity, from learned representations while maintaining
data utility. This approach is centered around an adversarial regularizer that
removes sensitive information from the learning objective.

Other fundamental related works include (Tran et al., 2017; Gong et al., 2020;
Park et al., 2021; Li et al., 2022; Suwała et al., 2024), which primarily
focus on disentangled representation learning and algorithmic fairness in face
recognition systems. These works range from introducing methods to mitigate
bias and improve pose-invariant face recognition to developing frameworks for
disentangling data representation into specific types of information to
mitigate discriminatory results in AI systems.

In (Tran et al., 2017), the authors introduce the disentangled representation
learning generative adversarial network (DR-GAN) to address the challenge of
pose variation in face recognition. Unlike conventional methods that either
generate a frontal face from a non-frontal image or learn pose-invariant
features, DR-GAN performs both tasks jointly through an encoder-decoder
generator structure. This enables it to synthesize identity-preserving faces
with arbitrary poses while learning a discriminative representation. The
approach disentangles identity representation from other variations, such as
pose, using a pose code for the decoder and pose estimation in the
discriminator. DR-GAN can process multiple images per subject, fusing them
into a single, robust representation and synthesizing faces in specified
poses.

In (Gong et al., 2020), the authors present an approach to mitigating bias in
automated face recognition and demographic attribute estimation algorithms,
focusing on addressing the observed performance disparities across different
demographic groups. They propose a de-biasing adversarial network, DebFace,
which employs adversarial learning to extract disentangled feature
representations for identity and demographic attributes (gender, age, and
race) in a way that minimizes bias by reducing the correlation among these
feature factors. Their approach combines demographic with identity features to
enhance the robustness and accuracy of face representation across diverse
demographic groups. The network comprises an identity classifier and three
demographic classifiers, trained adversarially to ensure feature
disentanglement and reduce demographic bias in both face recognition and
demographic estimation tasks.

In (Park et al., 2021), the authors introduce a fairness-aware disentangling
variational auto-encoder (FD-VAE) that aims to mitigate discriminatory results
in AI systems related to protected attributes such as gender and age, without
sacrificing beneficial information for target tasks. The FD-VAE model achieves
this by disentangling data representation into three subspaces: target
attribute latent (TAL), protected attribute latent (PAL), and mutual attribute
latent (MAL), each designed to contain specific types of information. A
decorrelation loss is proposed to appropriately align information within these
subspaces, focusing on preserving useful information for the target tasks
while excluding protected attribute information.

In (Li et al., 2022), the authors introduce Debiasing Alternate Networks
(DebiAN) to mitigate biases in deep image classifiers without the need for
labels of protected attributes, aiming to overcome the limitations of previous
methods that require full supervision. DebiAN consists of two networks, a
discoverer and a classifier, trained in an alternating manner to identify and
unlearn multiple unknown biases simultaneously. This approach not only
addresses the challenges of identifying biases without annotations but also
excels in mitigating them effectively. The effectiveness of DebiAN is
demonstrated through experiments on both synthetic datasets, such as the
multi-color MNIST, and real-world datasets, showing its capability to discover
and improve bias mitigation.

Recently, (Suwała et al., 2024) introduces PluGeN4Faces, a plugin for StyleGAN
designed to manipulate facial attributes such as expression, hairstyle, pose,
and age in images while preserving the person’s identity. It employs a
contrastive loss to closely cluster images of the same individual in latent
space, ensuring that changes to attributes do not affect other
characteristics, such as identity.

In comparison to the research mentioned above, our work begins with a purely
information-theoretic formulation of the PF model, which we have named the
discriminative PF framework. We then extend the concept of the discriminative
PF model to develop a generative PF framework. Building upon our objectives
for PF frameworks, as grounded in Shannon’s mutual information, we present a
tractable variational approximation for both our information utility and
information leakage quantities. The variational approximation objectives we
have obtained share some connections with the aforementioned research, thereby
bridging the gap between information-theoretic approaches to privacy and
privacy-preserving machine learning.

##  3 Preliminaries

###  3.1 General Loss Functions for Positive Measures

In data science, the representation of data via positive measures, including
probability distributions, is critical. Positive measures are used extensively
across various scientific fields, from modeling quantum states in physics to
gene expression in biology, as well as representing wealth distribution in
economics (Séjourné et al., 2023). Their role is further magnified in ML
(Bishop and Nasrabadi, 2006; James et al., 2013), where data representation
and manipulation rely on their approximation via discrete (e.g. histograms) or
continuous (e.g. parameterized densities) models.

####  3.1.1 Divergences

Comparative analysis of measures in data science is facilitated by loss
functions, which aim to quantify the similarity or dissimilarity between two
measures. Rooted in distance-based methodologies, a special category of loss
functions, known as divergences, are characteristically non-negative and
definite. This means they are defined such that their value is zero if and
only if the measures being compared are identical. Although the triangle
inequality’s presence is a beneficial feature of these loss functions, imbuing
them with a structure akin to metric spaces, it is not a universal
prerequisite across all applications. Within this context, Csiszár’s concept
of $\mathsf{f}$-divergences (Csiszár, 1967), a family of discrepancies between
positive measures, becomes particularly relevant. They are defined by
integrating pointwise comparisons of two measures and can be formalized as
follows:

######  Definition 1 ($\mathsf{f}$-divergences)

Let $\mathsf{f}:\left(0,\infty\right)\rightarrow\mathbb{R}$ be a convex
function which $\mathsf{f}(1)=0$. The $\mathsf{f}$-divergences between two
probability measures $P$ and $Q$, $P\ll Q$ is defined as (Ali and Silvey,
1966; Csiszár, 1967):

$\mathrm{D}_{\mathsf{f}}\left(P\|Q\right)\coloneqq\mathbb{E}_{Q}\left[\mathsf{f}\left(\frac{\mathrm{d}P}{\mathrm{d}Q}\right)\right].$
(1)

Several specific instances of $\mathsf{f}$-divergences are of particular
interest and have different ‘operational meanings’. Popular instances are
defined as follows (Csiszár et al., 2004; Polyanskiy et al., 2010; Sharma and
Warsi, 2013; Polyanskiy and Wu, 2014; Duchi, 2016):

  1. 1.

Kullback-Leibler (KL) Divergence: The KL-divergence,
$\mathrm{D}_{\text{KL}}(P\|Q)$, is a special case of $\mathsf{f}$-divergence
where the function $\mathsf{f}$ is given by $\mathsf{f}(t)=t\log t$. It is
expressed as
$\mathrm{D}_{\mathsf{KL}}(P\|Q)\coloneqq\mathrm{D}_{\mathsf{f}}(P\|Q)$ for
$\mathsf{f}(t)=t\log t$. It quantifies the amount of information lost when $Q$
is used to approximate $P$. It is widely used in scenarios like statistical
inference.

  2. 2.

Total Variation Distance: The total variation distance, denoted as
$\mathsf{TV}(P,Q)$, is defined by
$\mathsf{TV}(P,Q)\coloneqq\mathrm{D}_{\mathsf{f}}(P\|Q)$ with the function
$\mathsf{f}$ being $\mathsf{f}(t)=|t-1|$. It is widely used in hypothesis
testing and classification tasks in statistics, providing a bound on the
maximum error probability.

  3. 3.

Chi-squared ($\chi^{2}$) Divergence: The $\chi^{2}$-divergence,
$\chi^{2}(P\|Q)$, is another form of $\mathsf{f}$-divergence given by
$\chi^{2}(P\|Q)\coloneqq\mathrm{D}_{\mathsf{f}}(P\|Q)$ for the function
$\mathsf{f}(t)=t^{2}-1$. It is usually used in statistical analysis for
feature selection, particularly in the context of evaluating model fit and
understanding feature importance. It is also used in estimation problems.

  4. 4.

Squared Hellinger Distance: This measure, represented as $H^{2}(P,Q)$, employs
the function $\mathsf{f}(t)=(\sqrt{2}-\sqrt{t})^{2}$ in its definition:
$H^{2}(P,Q)\coloneqq\mathrm{D}_{\mathsf{f}}(P\|Q)$. This distance is
particularly useful in Bayesian statistics. Unlike the KL-divergence, the
Hellinger distance is symmetric and bounded.

  5. 5.

Hockey-Stick Divergence: The hockey-stick divergence, denoted as
$E_{\gamma}(P\|Q)$, is defined for a specific $\gamma$ (where $\gamma\geq 1$)
and employs the function $\mathsf{f}(t)=(t-\gamma)_{+}$ with
$(a)_{+}\coloneqq\max\\{a,0\\}$. Therefore, $E_{\gamma}(P\|Q)\coloneqq
D_{\mathsf{f}}(P\|Q)$ for $\mathsf{f}(t)=(t-\gamma)_{+}$. This divergence can
be particularly useful in decision-making models and risk assessments. The
contraction coefficient of this divergence is also equivalent to the local
Differential Privacy (Asoodeh et al., 2021).

Another important related loss is the Rényi divergence, which is not an
$\mathsf{f}$-divergence but shares a similar purpose in measuring the
discrepancy between probability distributions.

##### Rényi Divergence:

The Rényi divergence (Rényi, 1959, 1961) is denoted as
$D_{\mathsf{R},\alpha}(P\|Q)$ for a parameter $\alpha$, where $\alpha\neq 1$
and $\alpha>0$. It is defined as:

$D_{\mathsf{R},\alpha}(P\|Q)\coloneqq\frac{1}{\alpha-1}\log\left(\mathbb{E}_{Q}\left[\left(\frac{\mathrm{d}P}{\mathrm{d}Q}\right)^{\alpha}\right]\right).$
(2)

This divergence provides a spectrum of metrics between distributions, with the
parameter $\alpha$ controlling the sensitivity to discrepancies. The Kullback-
Leibler divergence is a special case of Rényi divergence as $\alpha\rightarrow
1$. Rényi divergence finds extensive application in fields such as information
theory, data privacy, cryptography, and machine learning, due to its
adaptability and the comprehensive range of distributional differences it can
capture.

####  3.1.2 Optimal Transport Distances

Optimal Transport (OT), a problem introduced by Gaspard Monge in the 18th
century in his work ‘Mémoire sur la théorie des déblais et des remblais’
(Monge, 1781), emerges as a potent tool for probabilistic comparisons. It
provides a uniquely flexible approach to gauge similarities and disparities
between probability distributions, regardless of their supports.

##### Monge’s OT Problem:

Monge’s seminal problem seeks an optimal map
$T:\mathcal{X}\rightarrow\mathcal{X}$ for transferring mass distributed
according to a measure $\mu$ onto another measure $\nu$ on the same space
$\mathcal{X}$. This problem can be metaphorically understood as finding the
most efficient way to move sand to form certain patterns, with $\mu$ and $\nu$
representing the initial and desired distributions of sand, respectively. The
key constraint in Monge’s formulation is represented by the equation
$T_{\\#}\mu=\nu$, where $T_{\\#}$ denotes the push-forward operator. The
integral equation defines the push-forward operator $\int_{\mathcal{X}}f\circ
T\,\mathrm{d}\mu=\int_{\mathcal{X}}f\,\mathrm{d}\nu,\quad\forall
f\in\mathcal{C}(\mathcal{X})$, where $\mathcal{C}(\mathcal{X})$ is the space
of continuous functions on $\mathcal{X}$. This condition ensures that the
measure $\mu$ is effectively transformed onto $\nu$ through the map $T$.
Specifically, it implies that
$T_{\\#}\delta_{\mathbf{x}}=\delta_{T(\mathbf{x})}$ for Dirac measures
$\delta_{\mathbf{x}}$ (Villani, 2008; Peyré et al., 2019; Séjourné et al.,
2023).

In solving Monge’s problem, the objective is to find a measurable map $T$ that
minimizes the total cost of transportation, subject to the aforementioned
constraint. The cost of transporting a unit of mass from location $\mathbf{x}$
to location $\mathbf{y}$ in $\mathcal{X}$ is quantified by a cost function
$\mathsf{c}(\mathbf{x},\mathbf{y})$. A typical choice for
$\mathsf{c}(\mathbf{x},\mathbf{y})$, particularly in Euclidean spaces
$\mathcal{X}=\mathbb{R}^{d}$, is the $p$-th power of the Euclidean distance,
$\mathsf{c}(\mathbf{x},\mathbf{y})=\|\mathbf{x}-\mathbf{y}\|_{p}^{2}$. The
original formulation by Monge is associated with linear transport costs,
corresponding to $p=1$. However, the quadratic case where $p=2$ is often
favored in modern applications due to its advantageous mathematical
properties, including convexity and differentiability.

######  Definition 2 (OT Monge Formulation Between Arbitrary Measures)

Given two arbitrary (probability) measures $\mu$ and $\nu$ supported on
$\mathcal{X}$ and $\mathcal{Y}$, respectively, the optimal transport Monge map
$T^{\ast}$, if it exists, solves the following problem:

$\inf_{T}\,\left\\{\int_{\mathcal{X}}\mathsf{c}\left(\mathbf{x},T(\mathbf{x})\right)\,\mathrm{d}\mu(\mathbf{x}):\quad
T_{\\#}\mu=\nu\right\\},$ (3)

over $\mu$-measurable map $T:\mathcal{X}\rightarrow\mathcal{Y}$.

##### Kantorovich’s OT Problem:

Kantorovich’s formulation of the OT problem addresses the scenario of
arbitrary measure spaces and introduces the concept of ‘mass splitting’
(Villani, 2008; Peyré et al., 2019; Séjourné et al., 2023). This innovative
approach, initially developed by Kantorovich (Kantorovich, 1942) for
applications in economic planning, significantly extends the framework of
Monge’s problem. In Kantorovich’s formulation, the deterministic map $T$ of
Monge’s problem is replaced by a probabilistic measure
$\pi\in\Pi(\mu\times\nu)$, termed as a transport plan. Unlike Monge’s
formulation where mass moves directly from a point $\mathbf{x}$ to
$T(\mathbf{x})$, Kantorovich’s approach allows for the dispersion of mass from
a single point $\mathbf{x}$ to multiple destinations. This flexibility makes
it a generalized, or relaxed, version of Monge’s problem.

######  Definition 3 (Kantorovich’s OT Problem)

Let $\mathcal{X}$ and $\mathcal{Y}$ be two measurable spaces. Let
$\mathcal{P}(\mathcal{X})$ and $\mathcal{P}(\mathcal{Y})$ be the sets of all
positive Radon probability measures on $\mathcal{X}$ and $\mathcal{Y}$,
respectively. For any measurable non-negative cost function
$\mathsf{c}:\mathcal{X}\times\mathcal{Y}\rightarrow\mathbb{R}^{+}$, the
Kantorovich’s OT problem between two positive measures
$\mu\in\mathcal{P}(\mathcal{X})$ and $\nu\in\mathcal{P}(\mathcal{Y})$ is
defined as:

$\mathsf{OT}_{\mathsf{c}}\left(\mu,\nu\right)\coloneqq\mathop{\inf}_{\pi\in\Pi(\mu,\nu)}\int_{\mathcal{X}\times\mathcal{Y}}\mathsf{c}(\mathbf{x},\mathbf{y})\,\mathrm{d}\pi(\mathbf{x},\mathbf{y})=\mathop{\inf}_{\bm{\pi}\in\Pi(\mu,\nu)}\mathbb{E}_{\pi}\left[\,\mathsf{c}(\mathbf{X},\mathbf{Y})\,\right],$
(4)

where $\Pi(\mu,\nu)$ denotes the set of joint distributions (couplings) over
the product space $\mathcal{X}\times\mathcal{Y}$ with marginals $\mu$ and
$\nu$, respectively. That is, for all measurable sets
$\mathcal{A}\subset\mathcal{X}$ and $\mathcal{B}\subset\mathcal{Y}$, we have:

$\Pi(\mu,\nu)\coloneqq\left\\{\pi\in\mathcal{P}(\mathcal{X}\times\mathcal{Y}):\;\pi(\mathcal{A}\times\mathcal{Y})=\mu(\mathcal{A}),\,\pi(\mathcal{X}\times\mathcal{B})=\nu(\mathcal{B})\right\\}.$
(5)

Having established the preliminary concepts of $\mathsf{f}$-divergences and
optimal transport distances as foundational tools in data science, we now
direct our attention to employing these loss functions for the quantification
of privacy leakage and utility performance.

###  3.2 Measuring Privacy Leakage and Utility Performance

We can define a generic privacy risk loss function as a functional tied to the
joint distribution $P_{\mathbf{S},\mathbf{Z}}$, which quantifies the
information leakage about $\mathbf{S}$ when $\mathbf{Z}$ is disclosed. Such a
privacy risk loss function can be represented as
$\mathcal{C}_{\mathsf{S}}:\mathcal{P}\left(\mathcal{S}\times\mathcal{Z}\right)\rightarrow\mathbb{R}^{+}\cup\\{0\\}$.
Analogously, a well-characterized and task-specific generic utility
performance loss function can be formulated as a functional of the joint
distribution $P_{\mathbf{X},\mathbf{Z}}$, capturing the utility retained about
$\mathbf{X}$ through the release of $\mathbf{Z}$. This utility performance
loss function is denoted as
$\mathcal{C}_{\mathsf{U}}:\mathcal{P}\left(\mathcal{U}\times\mathcal{Z}\right)\rightarrow\mathbb{R}^{+}\cup\\{0\\}$.
We can define the $\mathsf{f}$-information between two random objects
$\mathbf{X}$ and $\mathbf{Z}$ as
$\mathrm{I}_{\mathsf{f}}\left(\mathbf{X};\mathbf{Z}\right)=\mathrm{D}_{\mathsf{f}}\left(P_{\mathbf{X,Z}}\|P_{\mathbf{X}}P_{\mathbf{Z}}\right)$,
where $\mathrm{D}_{\mathsf{f}}\left(\cdot\|\cdot\right)$ represents the
$\mathsf{f}$-divergence (Polyanskiy and Wu, 2014), serving as a measure for
both privacy (obfuscation) and utility. Expanding this framework, Arimoto’s
mutual information (Arimoto, 1977) could also be employed to assess
information utility and privacy leakage. In this research, however, we focus
on Shannon mutual information as our primary loss function.

##  4 Privacy Funnel666Metaphorically, the term ‘funnel’ describes processes
that gradually narrow or concentrate, akin to filtering. In the context of the
privacy funnel model in data privacy, ‘funnel’ aptly represents the
progressive obfuscation and narrowing of sensitive personal information
through stages of collection, processing, dissemination, and consumption.
Model: Discriminative and Generative Paradigms

###  4.1 Discriminative Privacy Funnel Method: Optimizing Information
Extraction Under Privacy Constraints

Given two correlated random variables $\mathbf{S}$ and $\mathbf{X}$ with a
joint distribution $P_{\mathbf{S,X}}$, the objective in the (classical)
discriminative PF method (Makhdoumi et al., 2014) is to derive a
representation $\mathbf{Z}$ for useful data $\mathbf{X}$ via a stochastic
mapping $P_{\mathbf{Z}\mid\mathbf{X}}$, satisfying the following constraints:

  * (i)

Formation of a Markov chain
$\mathbf{S}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{X}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{Z}$,

  * (ii)

Maximization of the Shannon mutual information
$\mathrm{I}(\mathbf{X};\mathbf{Z})$ in the representation $\mathbf{Z}$ of
$\mathbf{X}$,

  * (iii)

Minimization of the Shannon mutual information
$\mathrm{I}(\mathbf{S};\mathbf{Z})$ in the representation $\mathbf{Z}$ of
$\mathbf{X}$.

The classical PF method thus addresses the trade-off between information
leakage $\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right)$ and the revealed useful
information $\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)$. This trade-off is
formally represented as:

$\displaystyle\mathsf{DisPF\text{-}MI}\left(R^{\mathrm{s}},P_{\mathbf{S},\mathbf{X}}\right)\coloneqq\mathop{\sup}_{\begin{subarray}{c}P_{\mathbf{Z}\mid\mathbf{X}}:\\\
\mathbf{S}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{X}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{Z}\end{subarray}}\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)\quad\mathrm{subject~{}to}\;\;\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right)\leq
R^{\mathrm{s}}.$ (6)

The $\mathsf{DisPF\text{-}MI}$ curve is defined by the values
$\mathsf{DisPF\text{-}MI}\left(R^{\mathrm{s}},P_{\mathbf{S},\mathbf{X}}\right)$
for different $R^{\mathrm{s}}$. We can use a Lagrange multiplier $\alpha\geq
0$ to represent the $\mathsf{DisPF\text{-}MI}$ problem by the associated
Lagrangian functional:

$\mathcal{L}_{\mathsf{DisPF\text{-}MI}}\left(P_{\mathbf{Z}\mid\mathbf{X}},\alpha\right)\coloneqq\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)-\alpha\;\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right).$
(7)

By creating a unique measure called the $\mathrm{I}$-Measure, we can
geometrically represent the relationship among Shannon’s information measures
(Yeung, 1991; Razeghi et al., 2023). Given that the Markov chain
$\mathbf{S}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{X}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{Z}$
necessitates $\mathrm{I}\left(\mathbf{S};\mathbf{Z}\mid\mathbf{X}\right)=0$,
the corresponding information diagram ($\mathrm{I}$-diagram) vividly
illustrates this and is featured in Figure 3. Note that
$\mathrm{I}\left(\mathbf{S};\mathbf{X};\mathbf{Z}\right)=\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right)-\mathrm{I}\left(\mathbf{S};\mathbf{Z}\mid\mathbf{X}\right)=\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right)\geq
0$.

(a) Discriminative Privacy Funnel $\mathsf{DisPF}$.

(b) Generative Privacy Funnel $\mathsf{GenPF}$.

Figure 2: Comparative overview of generalized privacy funnel (PF) approaches:
(a) the established discriminative (classical) model; (b) the proposed
generative model.

##### Discriminative Privacy Funnel with General Loss Functions:

Consider the extension of the standard discriminative PF objective to
encompass a broader class of loss functions. The aim of this general
discriminative PF approach is to obtain a representation $\mathbf{Z}$ for the
useful data $\mathbf{X}$ via a probabilistic mapping
$P_{\mathbf{Z}\mid\mathbf{X}}$. This objective is subject to the fulfillment
of the following constraints:

  * (i)

Establish a Markov chain
$\mathbf{S}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{X}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{Z}$,

  * (ii)

Minimization of the utility performance loss function
$\mathcal{C}_{\mathsf{U}}\left(P_{\mathbf{X},\mathbf{Z}}\right)$ via
optimizing $\mathbf{Z}$ to preserve the useful information $\mathbf{X}$
pertinent to utility.

  * (iii)

Minimization of the privacy risk loss function
$\mathcal{C}_{\mathsf{S}}\left(P_{\mathbf{S},\mathbf{Z}}\right)$ via
optimizing $\mathbf{Z}$ to limit the leakage about sensitive information
$\mathbf{S}$.

We can formulate this trade-off by imposing a constraint on one of the losses.
Therefore, for a given information leakage constraint $R^{\mathrm{s}}\geq 0$,
the trade-off can be encapsulated within a $\mathsf{DisPF}$ functional:

$\displaystyle\mathsf{DisPF}\left(R^{\mathrm{s}},P_{\mathbf{S},\mathbf{X}}\right)\coloneqq\mathop{\inf}_{\begin{subarray}{c}P_{\mathbf{Z}\mid\mathbf{X}}:\\\
\mathbf{S}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{X}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{Z}\end{subarray}}$
$\displaystyle\mathcal{C}_{\mathsf{U}}\left(P_{\mathbf{X},\mathbf{Z}}\right)$
(8) $\displaystyle\mathrm{subject~{}to}\quad\;$
$\displaystyle\mathcal{C}_{\mathsf{S}}\left(P_{\mathbf{S},\mathbf{Z}}\right)\leq
R^{\mathrm{s}}.$

Any of the previously addressed general loss functions can be utilized in the
above optimization problem.

##### Remark 1:

The discriminative PF model utilizes stochastic mapping
$P_{\mathbf{Z}\mid\mathbf{X}}$ that can undertake either of the following
processes: (i) A domain-preserving transformation, where, for example, in an
image-to-image transition, the output image $\mathbf{Z}$ maintains the same
domain but introduces specific alterations (such as noise or visual
distortions) to obfuscate sensitive information of the original image
$\mathbf{X}$. (ii) A non-domain-preserving transformation, as seen in methods
like image-to-embedding, where the output $\mathbf{Z}$ is a more abstract form
(an embedding) that provides a privacy-preserving representation of the
original data $\mathbf{X}$, but in a different domain. The assessment of the
obfuscated data’s ‘utility’ is performed either through direct analysis of
$\mathbf{Z}$ using
$\mathcal{C}_{\mathsf{U}}\left(P_{\mathbf{X},\mathbf{Z}}\right)$ or, where
applicable, after a decoding phase (indicated in gray in 2(a)) using
$\mathcal{C}_{\mathsf{U}}\big{(}P_{\mathbf{X},\mathbf{\widehat{X}}}\big{)}$.
Information leakage is gauged by using measure
$\mathcal{C}_{\mathsf{S}}\left(P_{\mathbf{S},\mathbf{Z}}\right)$, or in the
case of decoded data, via
$\mathcal{C}_{\mathsf{S}}\big{(}P_{\mathbf{S},\mathbf{\widehat{X}}}\big{)}$.

(a)

(b)

(c)

Figure 3: Information diagrams for
$\mathbf{S}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{X}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{Z}$.
(a) entropy $\mathrm{H}\left(\mathbf{S}\right)$,
$\mathrm{H}\left(\mathbf{X}\right)$, $\mathrm{H}\left(\mathbf{Z}\right)$, and
revealed useful (preserved) information
$\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)$ and information leakage
$\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right)$; (b) residual information
$\mathrm{I}\left(\mathbf{X};\mathbf{Z}\mid\mathbf{S}\right)$ and residual
information $\mathrm{I}\left(\mathbf{S};\mathbf{X}\mid\mathbf{Z}\right)$; (c)
private attribute uncertainty
$\mathrm{H}\left(\mathbf{S}\\!\mid\\!\mathbf{X}\right)$, useful information
decoding uncertainty $\mathrm{H}\left(\mathbf{X}\\!\mid\\!\mathbf{Z}\right)$,
and encoding uncertainty
$\mathrm{H}\left(\mathbf{Z}\\!\mid\\!\mathbf{X}\right)$.

###  4.2 Generative Privacy Funnel Method: Optimizing Data Synthesis Under
Privacy Constraints

The Generative PF model aims to address the fundamental problem of synthetic
data generation with privacy guarantees. Consider the general class of loss
functions discussed in Sec. 3.2. The objective of the Generative Privacy
Funnel model is to generate synthetic data, represented as
$\mathbf{\widetilde{X}}$, from a latent code $\mathbf{\widetilde{Z}}$. This
generation is achieved through a mapping (generator)
$P_{\mathbf{\widetilde{X}}\mid\mathbf{\widetilde{Z}}}$, which can function in
either a probabilistic or deterministic manner. The key goal of this model is
to ensure that the synthetic data $\mathbf{\widetilde{X}}$ preserve useful
information from the real data $\mathbf{X}$, necessary for a specific utility
task while minimizing privacy leakage about sensitive data $\mathbf{S}$. This
objective is subject to the fulfillment of the following constraints:

  * (i)

Establish a Markov chain
$\mathbf{\widetilde{Z}}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{\widetilde{X}}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{S}$,

  * (ii)

Minimization of the utility performance loss function
$\mathcal{C}_{\mathsf{U}}\big{(}P_{\mathbf{X},\mathbf{\widetilde{X}}}\big{)}$
via optimizing $\mathbf{\widetilde{X}}$ to preserve the useful information
$\mathbf{X}$ pertinent to utility.

  * (iii)

Minimization of the privacy risk loss function
$\mathcal{C}_{\mathsf{S}}\big{(}P_{\mathbf{S},\mathbf{\widetilde{X}}}\big{)}$
via optimizing $\mathbf{\widetilde{X}}$ to limit the leakage about sensitive
information $\mathbf{S}$.

We can formulate this trade-off as follows:

$\displaystyle\mathsf{GenPF}\big{(}P_{\mathbf{S,\widetilde{X}}},R^{\mathrm{s}}\big{)}\coloneqq\mathop{\inf}_{\begin{subarray}{c}P_{\mathbf{\widetilde{X}}\mid\mathbf{\widetilde{Z}}}:\\\
\mathbf{\widetilde{Z}}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{\widetilde{X}}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{S}\end{subarray}}$
$\displaystyle\mathcal{C}_{\mathsf{U}}\big{(}P_{\mathbf{X},\mathbf{\widetilde{X}}}\big{)}$
(9) $\displaystyle\mathrm{subject~{}to}\quad\;$
$\displaystyle\mathcal{C}_{\mathsf{S}}\big{(}P_{\mathbf{S},\mathbf{\widetilde{X}}}\big{)}\leq
R^{\mathrm{s}}.$

##### Remark 2:

The objective of the generative PF model is on generating synthetic data that
preserves the utility of the original dataset while ensuring constraints on
sensitive information leakage. This model may optionally include an encoding
step (represented in gray in 2(b)), or it may forego this step, opting to
generate synthetic data directly from a latent noise domain.

##### Generative Privacy Funnel with Self-Information Loss Function:

The objectives of the $\mathsf{GenPF}$ model under self-information loss are
as follows:

  * (i)

Establish a Markov chain
$\mathbf{\widetilde{Z}}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{\widetilde{X}}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{S}$,

  * (ii)

Maximization of the Shannon mutual information
$\mathrm{I}\big{(}\mathbf{X};\mathbf{\widetilde{X}}\big{)}$ in the
representation $\mathbf{\widetilde{X}}$ of $\mathbf{X}$,

  * (iii)

Minimization of the Shannon mutual information
$\mathrm{I}\big{(}\mathbf{S};\mathbf{\widetilde{X}}\big{)}$ in the
representation $\mathbf{\widetilde{X}}$ of $\mathbf{X}$.

$\displaystyle\mathsf{GenPF\text{-}MI}\big{(}P_{\mathbf{S,\widetilde{X}}},R^{\mathrm{s}}\big{)}\coloneqq\mathop{\sup}_{\begin{subarray}{c}P_{\mathbf{\widetilde{X}}\mid\mathbf{\widetilde{Z}}}:\\\
\mathbf{\widetilde{Z}}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{\widetilde{X}}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{S}\end{subarray}}\mathrm{I}\big{(}\mathbf{X};\mathbf{\widetilde{X}}\big{)}\quad\mathrm{subject~{}to}\;\;\mathrm{I}\big{(}\mathbf{S};\mathbf{\widetilde{X}}\big{)}\leq
R^{\mathrm{s}}.$ (10)

where $\mathbf{\widetilde{X}}=G\big{(}\mathbf{\widetilde{Z}}\big{)}$.

##### Remark 3:

The latent code $\mathbf{\widetilde{Z}}$ plays a versatile role in various
generative models. It can represent the latent space in Variational
Autoencoders (VAEs), be sampled from the $\mathcal{W}$ space in StyleGANs,
derived through StyleGAN inversion methods, or constitute the latent space in
diffusion models.

####  4.2.1 Generative Privacy Funnel in Face Recognition Systems

Creating an effective synthetic dataset for facial recognition systems demands
integrating diverse demographic characteristics, encompassing varied ages,
genders, ethnicities, and physical features, to enhance the system’s
capability in recognizing and classifying a broad spectrum of human faces.
Incorporating a range of facial expressions and poses, from happiness to
neutrality, captured in different orientations like frontal and profile views,
is critical for maintaining accuracy across various facial states. The dataset
should also reflect diverse lighting and environmental conditions, including
both indoor and outdoor settings, to ensure robust performance in real-world
scenarios. High-resolution images are crucial for extracting detailed
features, whereas incorporating lower-resolution images equips the system to
handle suboptimal conditions. Including images of people wearing things like
glasses or having part of their face covered is key to making sure we can
recognize faces properly in all sorts of real-life situations. Accurate and
consistent labeling is also a very important aspect to ensure reliable
learning from the dataset. Ethical standards must be considered to prevent
biases in image generation. Lastly, realism in synthetic imagery is imperative
to replicate real-life scenarios accurately, as suboptimal image generation
can significantly impair system performance. This holistic approach in dataset
creation is vital for developing facial recognition systems that are not only
robust and reliable but also versatile for diverse applications.

(a)

(b)

Figure 4: Visualization of the Generative Privacy Funnel in (a) face
recognition systems and (b) face attribute recognition.

Incorporating the principles laid out in the comprehensive approach to
synthetic dataset generation for facial recognition systems, the
$\mathsf{GenPF}$ is aimed to generate synthetic images that not only adhere to
the above-mentioned criteria but also protect the sensitive information from
real dataset samples. This may include protecting personal identities as well
as sensitive attributes such as gender, race, and emotion inherent in facial
images. Moreover, $\mathsf{GenPF}$ has the potential to contribute to the
creation of a balanced dataset, a crucial step in mitigating biases in face
recognition systems. The specifics of this are discussed in Sec. 6.7.1

###  4.3 Threat Model

Our threat model includes the following assumptions:

  * •

We consider an adversary who is interested in a specific attribute
$\mathbf{S}$ related to the data $\mathbf{X}$. This attribute $\mathbf{S}$
could be any function of $\mathbf{X}$, possibly randomized. We limit
$\mathbf{S}$ to a discrete attribute, which accommodates most scenarios of
interest, such as a facial feature or an identity attribute.

  * •

The adversary has access to the released representation $\mathbf{Z}$ and
respects the Markov chain relationship
$\mathbf{S}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{X}\hbox{$\\--$}\kern-1.5pt\hbox{$\circ$}\kern-1.5pt\hbox{$\\--$}\mathbf{Z}$.

  * •

We assume that the adversary knows the mapping $P_{\mathbf{Z}\mid\mathbf{X}}$
designed by the data owner (defender), i.e., the defender’s mechanism is
public knowledge.

##  5 Deep Variational Privacy Funnel

In the following section, we delve into the heart of our methodology: Deep
Variational Privacy Funnel.

###  5.1 Information Leakage Approximation

We provide parameterized variational approximations for information leakage,
including an explicit tight variational bound and an upper bound. This
approximation is designed to be computationally tractable and easily
integrated with deep learning models, which allows for a flexible and
efficient evaluation of privacy guarantees. To better understand the nature of
information leakage, we can express
$\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right)$ as:

$\displaystyle\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right)$
$\displaystyle=\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)-\mathrm{I}\left(\mathbf{X};\mathbf{Z}\mid\mathbf{S}\right)$
(11a)
$\displaystyle=\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)-\mathrm{H}\left(\mathbf{X}\mid\mathbf{S}\right)+\mathrm{H}\left(\mathbf{X}\mid\mathbf{S},\mathbf{Z}\right).$
(11b)

The conditional entropy $\mathrm{H}\left(\mathbf{X}\mid\mathbf{S}\right)$ is
originated from the nature of data since it is out of our control. It can be
interpreted as ‘useful information decoding uncertainty’. Now, we derive the
variational decomposition of $\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)$
and $\mathrm{H}\left(\mathbf{X}\mid\mathbf{S},\mathbf{Z}\right)$. The mutual
information $\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)$ can be interpreted
as ‘information complexity’ or ‘encoder capacity’ (Razeghi et al., 2023). It
can be decomposed as:

$\displaystyle\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)$
$\displaystyle=\mathbb{E}_{P_{\mathbf{X},\mathbf{Z}}}\left[\log\frac{P_{\mathbf{Z}\mid\mathbf{X}}}{Q_{\mathbf{Z}}}\right]-\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{Z}}\|Q_{\mathbf{Z}}\right)$
(12a)
$\displaystyle=\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{Z}\mid\mathbf{X}}\|Q_{\mathbf{Z}}\mid
P_{\mathbf{X}}\right)-\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{Z}}\|Q_{\mathbf{Z}}\right),$
(12b)

where
$Q_{\mathbf{Z}}:\mathcal{Z}\rightarrow\mathcal{P}\left(\mathcal{Z}\right)$ is
variational approximation of the latent space distribution $P_{\mathbf{Z}}$.
The conditional entropy
$\mathrm{H}\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right)$ can be
decomposed as:

$\displaystyle\mathrm{H}\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right)=-\mathbb{E}_{P_{\mathbf{S},\mathbf{X},\mathbf{Z}}}\left[\log
P_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}}\right]$
$\displaystyle=-\mathbb{E}_{P_{\mathbf{S},\mathbf{X}}}\left[\mathbb{E}_{P_{\mathbf{Z}\mid\mathbf{X}}}\left[\log
Q_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}}\right]\right]-\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}}\|Q_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}}\right)$
(13a)
$\displaystyle\leq-\mathbb{E}_{P_{\mathbf{S},\mathbf{X}}}\left[\mathbb{E}_{P_{\mathbf{Z}\mid\mathbf{X}}}\left[\log
Q_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}}\right]\right]$ (13b)
$\displaystyle=\mathrm{H}\left(P_{\mathbf{X}\mid\mathbf{S,Z}}\|Q_{\mathbf{X}\mid\mathbf{S,Z}}\mid
P_{\mathbf{S,Z}}\right)\eqqcolon\mathrm{H}^{\mathrm{U}}\\!\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right),$
(13c)

where
$Q_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}}\\!:\\!\mathcal{S}\\!\times\\!\mathcal{Z}\\!\rightarrow\\!\mathcal{P}\left(\mathcal{X}\right)$
is variational approximation of the optimal uncertainty decoder distribution
$P_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}}$, and the inequality in (13c) follows
by noticing that
$\mathrm{D}_{\mathrm{KL}}(P_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}}\|Q_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}})$
$\geq 0$. Using (11), (12) and (13), the variational upper bound of
information leakage is given as:

$\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right)\leq\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{Z}\mid\mathbf{X}}\|Q_{\mathbf{Z}}\mid
P_{\mathbf{X}}\right)-\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{Z}}\|Q_{\mathbf{Z}}\right)+\mathrm{H}^{\mathrm{U}}\\!\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right).$
(14)

Having the variational upper bound of information leakage, we now approximate
the parameterized variational bound using neural networks. Let
$P_{\bm{\phi}}(\mathbf{Z}\\!\mid\\!\mathbf{X})$ represent the family of
encoding probability distributions $P_{\mathbf{Z}\mid\mathbf{X}}$ over
$\mathcal{Z}$ for each element of space $\mathcal{X}$, parameterized by the
output of a deep neural network $f_{\bm{\phi}}$ with parameters $\bm{\phi}$.
Analogously, let
$P_{\bm{\varphi}}\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right)$
denote the corresponding family of decoding probability distributions
$Q_{\mathbf{X}\mid\mathbf{S},\mathbf{Z}}$, driven by $g_{\bm{\varphi}}$.
Lastly, $Q_{\bm{\psi}}(\mathbf{Z})$ denotes the parameterized prior
distribution, either explicit or implicit, that is associated with
$Q_{\mathbf{Z}}$.

Using (12), the parameterized variational approximation of
$\mathrm{I}\left(\mathbf{X};\mathbf{Z}\right)$ can be defined as:

$\displaystyle\mathrm{I}_{\bm{\phi},\bm{\psi}}\left(\mathbf{X};\mathbf{Z}\right)\coloneqq\mathrm{D}_{\mathrm{KL}}\\!\left(P_{\bm{\phi}}(\mathbf{Z}\\!\mid\\!\mathbf{X})\,\|\,Q_{\bm{\psi}}(\mathbf{Z})\mid
P_{\mathsf{D}}(\mathbf{X})\right)-\mathrm{D}_{\mathrm{KL}}\\!\left(P_{\bm{\phi}}(\mathbf{Z})\,\|\,Q_{\bm{\psi}}(\mathbf{Z})\right).$
(15)

The parameterized variational approximation of conditional entropy
$\mathrm{H}^{\mathrm{U}}\left(\mathbf{X}\mid\mathbf{S},\mathbf{Z}\right)$ in
(13c) can be defined as:

$\displaystyle\mathrm{H}_{\bm{\phi},\bm{\varphi}}^{\mathrm{U}}\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right)\coloneqq-\mathbb{E}_{P_{\mathbf{S},\mathbf{X}}}\left[\mathbb{E}_{P_{\bm{\phi}}(\mathbf{Z}\mid\mathbf{X})}\left[\log
P_{\bm{\varphi}}(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z})\right]\right].$
(16)

Let $\mathrm{I}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S};\mathbf{Z}\right)$ denote
the parameterized variational approximation of information leakage
$\mathrm{I}\left(\mathbf{S};\mathbf{Z}\right)$. Using (14), an upper bound of
$\mathrm{I}_{\bm{\phi},\bm{\xi}}\\!\left(\mathbf{S};\mathbf{Z}\right)$ can be
given as:

$\displaystyle\mathrm{I}_{\bm{\phi},\bm{\xi}}(\mathbf{S};\mathbf{Z})\;\leq\\!\\!\\!\\!\underbrace{\mathrm{I}_{\bm{\phi},\bm{\psi}}\left(\mathbf{X};\mathbf{Z}\right)}_{\mathrm{Information~{}Complexity}}\\!\\!+\\!\underbrace{\mathrm{H}_{\bm{\phi},\bm{\varphi}}^{\mathrm{U}}\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right)}_{\mathrm{Information~{}Uncertainty}}\\!\\!\\!\\!+\,\mathrm{c}\eqqcolon\;\mathrm{I}_{\bm{\phi},\bm{\psi},\bm{\varphi}}^{\mathrm{U}}\left(\mathbf{S};\mathbf{Z}\right)+\mathrm{c},$
(17)

where $\mathrm{c}$ is a constant term, independent of the neural network
parameters.

This upper bound encourages the model to reduce both the information
complexity, represented by
$\mathrm{I}_{\bm{\phi},\bm{\psi}}\left(\mathbf{X};\mathbf{Z}\right)$, and the
information uncertainty, denoted by
$\mathrm{H}_{\bm{\phi},\bm{\varphi}}^{\mathrm{U}}\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right)$.
Consequently, this leads the model to ‘forget’ or de-emphasize the sensitive
attribute $\mathbf{S}$, which subsequently reduces the uncertainty about the
useful data $\mathbf{X}$. In essence, this nudges the model towards an
accurate reconstruction of the data $\mathbf{X}$.

(a)

(b)

Figure 5: Core architectural components of the (a) deep discriminative privacy
funnel and (b) deep generative privacy funnel models.

Now, let us derive another parameterized variational bound of information
leakage $\mathrm{I}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S};\mathbf{Z}\right)$.
We can decompose
$\mathrm{I}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S};\mathbf{Z}\right)$ as
follows:

$\displaystyle\mathrm{I}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S};\mathbf{Z}\right)$
$\displaystyle=\mathbb{E}_{P_{\mathbf{S},\mathbf{X}}}\left[\mathbb{E}_{P_{\bm{\phi}}\left(\mathbf{Z}\mid\mathbf{X}\right)}\left[\log
P_{\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)\right]\right]+\mathbb{E}_{P_{\mathbf{S}}}\left[\log
P_{\bm{\xi}}(\mathbf{S})\right]-\mathbb{E}_{P_{\mathbf{S}}}\Big{[}\log\frac{P_{\mathbf{S}}}{P_{\bm{\xi}}(\mathbf{S})}\Big{]}$
(18a)
$\displaystyle=\underbrace{-\mathrm{H}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)+\mathrm{H}\left(P_{\mathbf{S}}\,\|\,P_{\bm{\xi}}(\mathbf{S})\right)}_{\mathrm{Prediction~{}Fidelity}}\,\,\,-\\!\\!\\!\\!\underbrace{\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{S}}\,\|\,P_{\bm{\xi}}(\mathbf{S})\right)}_{\mathrm{Distribution~{}Discrepancy}}$
(18b)
$\displaystyle\geq-\mathrm{H}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)\,\,\,-\,\,\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{S}}\,\|\,P_{\bm{\xi}}(\mathbf{S})\right)\eqqcolon\,\,\mathrm{I}_{\bm{\phi},\bm{\xi}}^{\mathrm{L}}\left(\mathbf{S};\mathbf{Z}\right),$
(18c)

where $P_{\bm{\xi}}(\mathbf{S}\\!\mid\\!\mathbf{Z})$ denotes the corresponding
family of decoding probability distribution $Q_{\mathbf{S}\mid\mathbf{Z}}$,
where
$Q_{\mathbf{S}\mid\mathbf{Z}}:\mathcal{Z}\rightarrow\mathcal{P}(\mathcal{S})$
is a variational approximation of optimal decoder distribution
$P_{\mathbf{S}\mid\mathbf{Z}}$. Let us interpret the MI decomposition in Eq
(18b):

  * •

Negative Conditional Cross-Entropy
$-\mathrm{H}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)$:
This term aims to maximize the uncertainty in predicting $\mathbf{S}$ given
$\mathbf{Z}$.
$\mathrm{H}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)$
can be as low as $0$ when $\mathbf{S}$ is deterministically predictable given
$\mathbf{Z}$. This means that knowing $\mathbf{Z}$ gives us full information
about $\mathbf{S}$. A negative sign encourages the model (encoder) to increase
the entropy of $\mathbf{S}$ given $\mathbf{Z}$, which means making
$\mathbf{S}$ less predictable when you know $\mathbf{Z}$. In the case of a
discrete sensitive attribute $\mathbf{S}$, the conditional entropy is
maximized when all the conditional distributions
$P_{\mathbf{S\mid\mathbf{Z}=\mathbf{z}}}$ are uniform. The maximum entropy is
$\log_{2}|\mathcal{S}|$, where $|\mathcal{S}|$ is the number of possible
states (or values, or classes) for $\mathbf{S}$. This means the adversary,
lacking any additional information, can do no better than ‘random guessing’.
This scenario equates to a potential lower boundary for
$-\mathrm{H}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)$
at $-\log_{2}|\mathcal{S}|$.

  * •

Cross-Entropy
$\mathrm{H}\left(P_{\mathbf{S}}\,\|\,P_{\bm{\xi}}(\mathbf{S})\right)$: This
term encourages the classifier to produce correct predictions for
$\mathbf{S}$. The minimum value is equal to the entropy of $P_{\mathbf{S}}$,
i.e., $\mathrm{H}(P_{\mathbf{S}})$, which is achieved when
$P_{\bm{\xi}}(\mathbf{S})=P_{\mathbf{S}}$. Given that $\mathbf{S}$ is
discrete, the maximum value is $\log_{2}|\mathcal{S}|$.

  * •

This term ensures the model’s inferred distribution,
$P_{\bm{\xi}}(\mathbf{S})$, aligns tightly with the actual distribution
$P_{\mathbf{S}}$. Ideally, the divergence measure,
$\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{S}}\|P_{\bm{\xi}}(\mathbf{S})\right)$,
is minimized to zero when $P_{\bm{\xi}}(\mathbf{S})$ aligns perfectly with
$P_{\mathbf{S}}$.

By pushing both
$\mathrm{H}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)$
and $\mathrm{H}\left(P_{\mathbf{S}}\,\|\,P_{\bm{\xi}}(\mathbf{S})\right)$ to
their maximum values of $\log_{2}N$, and simultaneously minimizing
distributional gap
$\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{S}}\,\|\,P_{\bm{\xi}}(\mathbf{S})\right)$,
the mutual information
$\mathrm{I}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S};\mathbf{Z}\right)$ will
approach zero, indicating that $\mathbf{Z}$ has minimal information about
$\mathbf{S}$.

Using (17) and (18) we have:

$-\mathrm{H}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)-\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{S}}\,\|\,P_{\bm{\xi}}(\mathbf{S})\right)\;\leq\;\mathrm{I}_{\bm{\phi},\bm{\xi}}(\mathbf{S};\mathbf{Z})\;\leq\;\mathrm{I}_{\bm{\phi},\bm{\psi}}\left(\mathbf{X};\mathbf{Z}\right)+\mathrm{H}_{\bm{\phi},\bm{\varphi}}^{\mathrm{U}}\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right)+\mathrm{c}.$
(19)

The above lower and upper bounds lead us towards two alternative models. The
upper bound in (19) encourages the model to directly minimizes the information
complexity
$\mathrm{I}_{\bm{\phi},\bm{\psi}}\left(\mathbf{X};\mathbf{Z}\right)$ as well
as the information uncertainty
$\mathrm{H}_{\bm{\phi},\bm{\varphi}}^{\mathrm{U}}\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right)$.
By minimizing the information uncertainty
$\mathrm{H}_{\bm{\phi},\bm{\varphi}}^{\mathrm{U}}\left(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z}\right)$
the model forces to forget the sensitive attribute $\mathbf{S}$ at the expense
of reducing the uncertainty about the original data $\mathbf{X}$, i.e.,
encourages the model to reconstruct the original data $\mathbf{X}$. In
contrast, the lower bound in (19) encourages the model to maximizes (i)
uncertainty about the sensitive attribute $\mathbf{S}$ given the released
representation $\mathbf{Z}$, i.e.,
$\mathrm{H}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)$,
as well as (ii) the distribution discrepancy measure
$\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{S}}\,\|\,P_{\bm{\xi}}(\mathbf{S})\right)$.
Note that minimizing the lower bound of information leakage may not
necessarily minimize the average maximal possible leakage. Furthermore note
that although the lower bound in (19) does not explicitly depend on the
information complexity
$\mathrm{I}_{\bm{\phi},\bm{\psi}}\left(\mathbf{X};\mathbf{Z}\right)$, it
depends implicitly through the encoder $f_{\bm{\phi}}$.

###  5.2 Information Utility Approximation

In this subsection, we turn our focus on quantifying the utility of
information. As with information leakage, we provide a careful decomposition
of the mutual information $\mathrm{I}(\mathbf{X};\mathbf{Z})$ and derive a
parameterized variational approximation for information utility. These
measures form the foundation of the Deep Variational PF framework and pave the
way for practical and scalable privacy preservation in deep learning
applications. The end-to-end parameterized variational approximation
associated to the information utility $\mathrm{I}(\mathbf{X};\mathbf{Z})$ can
be defined as:

$\displaystyle\mathrm{I}_{\bm{\phi},\bm{\theta}}\left(\mathbf{X};\mathbf{Z}\right)$
$\displaystyle\coloneqq\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\mathbb{E}_{P_{\bm{\phi}}\left(\mathbf{Z}\mid\mathbf{X}\right)}\Big{[}\log\frac{P_{\bm{\theta}}\left(\mathbf{X}\\!\mid\\!\mathbf{Z}\right)}{P_{\mathsf{D}}(\mathbf{X})}\Big{]}\right]$
(20a)
$\displaystyle=\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\mathbb{E}_{P_{\bm{\phi}}\left(\mathbf{Z}\mid\mathbf{X}\right)}\left[\log
P_{\bm{\theta}}\left(\mathbf{X}\\!\mid\\!\mathbf{Z}\right)\right]\right]-\mathrm{D}_{\mathrm{KL}}\left(P_{\mathsf{D}}(\mathbf{X})\,\|\,P_{\bm{\theta}}(\mathbf{X})\right)+\mathrm{H}\left(P_{\mathsf{D}}(\mathbf{X})\,\|\,P_{\bm{\theta}}(\mathbf{X})\right)$
(20b)
$\displaystyle\geq\underbrace{-\mathrm{H}_{\bm{\phi},\bm{\theta}}\left(\mathbf{X}\\!\mid\\!\mathbf{Z}\right)}_{\mathrm{Reconstruction~{}Fidelity}}\,\,-\\!\\!\underbrace{\mathrm{D}_{\mathrm{KL}}\left(P_{\mathsf{D}}(\mathbf{X})\,\|\,P_{\bm{\theta}}(\mathbf{X})\right)}_{\mathrm{Distribution~{}Discrepancy~{}Loss}}\\!\\!\\!\eqqcolon\,\,\,\mathrm{I}_{\bm{\phi},\bm{\theta}}^{\mathrm{L}}\left(\mathbf{X};\mathbf{Z}\right),$
(20c)

where
$\mathrm{H}_{\bm{\phi},\bm{\theta}}\left(\mathbf{X}\\!\mid\\!\mathbf{Z}\right)\coloneqq\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\mathbb{E}_{P_{\bm{\phi}}\left(\mathbf{Z}\mid\mathbf{X}\right)}\left[\log
P_{\bm{\theta}}\left(\mathbf{X}\\!\mid\\!\mathbf{Z}\right)\right]\right]$.
Note that in our VAE-based framework, the end-to-end parameterized information
utility $\mathrm{I}_{\bm{\phi},\bm{\theta}}\left(\mathbf{X};\mathbf{Z}\right)$
can equivalently be expressed as
$\mathrm{I}_{\bm{\phi},\bm{\theta}}\big{(}\mathbf{X};\mathbf{\widehat{X}}\big{)}$.
This means it serves as a parameterized approximation of
$\mathrm{I}\big{(}\mathbf{X};\mathbf{\widetilde{X}}\big{)}$ in
$\mathsf{Gen\text{-}MI}$ problem (10).

###  5.3 Deep Variational Privacy Funnel Objectives

Considering (7) and using the addressed parameterized approximations, one can
obtain the $\mathsf{DisPF}$ and $\mathsf{GenPF}$ Lagrangian functionals. We
recast the following maximization objectives:

$(\textsf{P1})\\!:\;\mathcal{L}_{\mathsf{DisPF\text{-}MI}}\left(\bm{\phi},\bm{\theta},\bm{\xi},\alpha\right)\coloneqq\overbrace{-\mathrm{H}_{\bm{\phi},\bm{\theta}}\left(\mathbf{X}\\!\mid\\!\mathbf{Z}\right)-\mathrm{D}_{\mathrm{KL}}\left(P_{\mathsf{D}}(\mathbf{X})\,\|\,P_{\bm{\theta}}(\mathbf{X})\right)}^{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\mathrm{Information~{}Utility:}~{}\mathrm{I}_{\bm{\phi},\bm{\theta}}^{\mathrm{L}}\left(\mathbf{X};\mathbf{Z}\right)}}\\\
\;\;\;\;\;-\,\alpha\,\underbrace{\Big{(}-\mathrm{H}_{\bm{\phi},\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)-\mathrm{D}_{\mathrm{KL}}\left(P_{\mathbf{S}}\,\|\,P_{\bm{\xi}}(\mathbf{S})\right)\Big{)}}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\mathrm{Information~{}Leakage:}~{}\mathrm{I}_{\bm{\phi},\bm{\xi}}^{\mathrm{L}}\left(\mathbf{S};\mathbf{Z}\right)}}.$
(21)

$(\textsf{P2})\\!:\;\mathcal{L}_{\mathsf{GenPF\text{-}MI}}\left(\bm{\phi},\bm{\theta},\bm{\psi},\bm{\varphi},\alpha\right)\coloneqq\overbrace{-\mathrm{H}_{\bm{\phi},\bm{\theta}}\left(\mathbf{X}\\!\mid\\!\mathbf{Z}\right)-\mathrm{D}_{\mathrm{KL}}\left(P_{\mathsf{D}}(\mathbf{X})\,\|\,P_{\bm{\theta}}(\mathbf{X})\right)}^{{\color[rgb]{.5,0,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,0,.5}\mathrm{Information~{}Utility:}~{}\mathrm{I}_{\bm{\phi},\bm{\theta}}^{\mathrm{L}}\left(\mathbf{X};\mathbf{Z}\right)}}\\\
-\alpha\underbrace{\Big{(}\mathrm{I}_{\bm{\phi},\bm{\psi}}\left(\mathbf{X};\mathbf{Z}\right)+\mathrm{H}_{\bm{\phi},\bm{\varphi}}^{\mathrm{U}}\left(\mathbf{X}\mid\mathbf{S},\mathbf{Z}\right)\Big{)}}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\mathrm{Information~{}Leakage}:~{}\mathrm{I}_{\bm{\phi},\bm{\psi},\bm{\varphi}}^{\mathrm{U}}\left(\mathbf{S};\mathbf{Z}\right)}}.$
(22)

###  5.4 Learning Framework

System Designer: Consider a set of independent and identically distributed
(i.i.d.) training samples
$\\{\left(\mathbf{u}_{n},\mathbf{x}_{n}\right)\\}_{n=1}^{N}$
$\subseteq\mathcal{U}\times\mathcal{X}$. We train deep neural networks (DNNs)
$f_{\bm{\phi}}$, $g_{\bm{\theta}}$, $g_{\bm{\xi}}$ (or $g_{\bm{\varphi}}$),
$D_{\bm{\eta}}$, $D_{\bm{\tau}}$, and $D_{\bm{\omega}}$ jointly using a
stochastic gradient descent (SGD)-type approach. The goal is to maximize a
Monte Carlo approximation of the deep variational PF functional over the
parameters $\bm{\phi}$, $\bm{\theta}$, $\bm{\xi}$ (or $\bm{\varphi}$),
$\bm{\eta}$, $\bm{\tau}$, and $\bm{\omega}$ as illustrated in Figure 6. Our
framework requires backpropagation through random samples from the posterior
distribution $P_{\bm{\phi}}(\mathbf{Z}\\!\\!\mid\\!\\!\mathbf{X})$, which
presents a challenge since backpropagation cannot flow via random nodes. To
address this, we employ the reparameterization trick (Kingma and Welling,
2014).

We typically infer the posterior distribution to be a multivariate Gaussian
with a diagonal covariance matrix, represented as
$P_{\bm{\phi}}(\mathbf{Z}\\!\mid\\!\mathbf{x})=\mathcal{N}\big{(}\bm{\mu}_{\bm{\phi}}(\mathbf{x}),$
$\mathsf{diag}(\bm{\sigma}_{\bm{\phi}}(\mathbf{x}))\big{)}$. Let’s assume
$\mathcal{Z}=\mathbb{R}^{d}$. We start by sampling a random variable
$\bm{\mathcal{E}}$ i.i.d. from
$\mathcal{N}\\!\left(\bm{0},\mathbf{I}_{d}\right)$. Then, given a data sample
$\mathbf{x}\in\mathcal{X}$, we generate the sample
$\mathbf{z}=\bm{\mu}_{\bm{\phi}}(\mathbf{x})+\bm{\sigma}_{\bm{\phi}}(\mathbf{x})\odot\bm{\varepsilon}$,
where $\odot$ denotes the element-wise (Hadamard) product. The prior
distribution in the latent space is generally assumed to be a fixed, standard
isotropic multivariate Gaussian in $d$ dimensions, denoted as
$Q_{\mathbf{Z}}=\mathcal{N}\left(\bm{0},\mathbf{I}_{d}\right)$. With this
assumption, the upper bound of the information complexity can be expressed in
a closed form. It is formulated as
$\mathbb{E}_{P_{\bm{\phi}}(\mathbf{X},\mathbf{Z})}\big{[}\log\frac{P_{\bm{\phi}}\left(\mathbf{Z}\mid\mathbf{X}\right)}{Q_{\mathbf{Z}}}\big{]}=\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\mathrm{D}_{\mathrm{KL}}\left(P_{\bm{\phi}}\\!\left(\mathbf{Z}\\!\mid\\!\mathbf{X}\right)\|Q_{\mathbf{Z}}\right)\right]$.
The explicit form of this upper bound is given as:

$2\,\mathrm{D}_{\mathrm{KL}}\left(P_{\bm{\phi}}\,\left(\mathbf{Z}\,\mid\,\mathbf{X}=\mathbf{x}\right)\|\right.\left.Q_{\mathbf{Z}}\right)={\|\bm{\mu}_{\bm{\phi}}(\mathbf{x})\|}_{2}^{2}+d+\sum_{i=1}^{d}(\bm{\sigma}_{\bm{\phi}}(\mathbf{x})_{i}-\log\bm{\sigma}_{\bm{\phi}}(\mathbf{x})_{i}).$
(23)

The KL divergences in the equations (15), (18), and (20) can be estimated
utilizing the density-ratio trick, as mentioned in (Nguyen et al., 2010;
Sugiyama et al., 2012). This trick is also employed in the GAN framework to
align the distributions of the generated model and the data directly. The
essence of the trick lies in expressing the two distributions as conditional
distributions, based on a label $C\in\\{0,1\\}$, thereby transforming the task
into a binary classification problem. The central idea here is that it allows
us to estimate the KL divergence by gauging the ratio of two distributions
without the need to model each distribution explicitly.

(a)

(b)

Figure 6: The training architectures associated with: (a)
$\mathsf{DisPF\text{-}MI}$ (P1); (b) $\mathsf{GenPF\text{-}MI}$ (P2).

Learning Procedure: The deep variational PF models $(\textsf{P1})$ (21) and
$(\textsf{P2})$ (22) are trained via a six-step alternating block coordinate
descent process. In this process, steps 1, 5, and 6 are specific for each
model, while steps 2, 3, and 4 are identical for both $(\textsf{P1})$ and
$(\textsf{P2})$. Figure 6 illustrates the training architectures for
$(\textsf{P1})$ (21) and $(\textsf{P2})$ (22). The training of the DVPF model
$(\textsf{P1})$ follows this six-step alternating block coordinate descent
process:

(1) Train the Encoder $\bm{\phi}$, Utility Decoder $\bm{\theta}$ and
Uncertainty Decoder $\bm{\xi}$.

$\mathop{\max}_{\bm{\phi},\bm{\theta},\bm{\xi}}\;\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\mathbb{E}_{P_{\bm{\phi}}(\mathbf{Z}\mid\mathbf{X})}\left[\log
P_{\bm{\theta}}(\mathbf{X}\\!\mid\\!\mathbf{Z})\right]\right]-\alpha\;\mathbb{E}_{P_{\mathbf{S},\mathbf{X}}}\left[\mathbb{E}_{P_{\bm{\phi}}\left(\mathbf{Z}\mid\mathbf{X}\right)}\left[\log
P_{\bm{\xi}}\left(\mathbf{S}\\!\mid\\!\mathbf{Z}\right)\right]\right].$ (24)

(2) Train the Latent Space Discriminator $\bm{\eta}$.

$\displaystyle\mathop{\min}_{\bm{\eta}}\quad\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\,\mathbb{E}_{P_{\bm{\phi}}(\mathbf{Z}\mid\mathbf{X})}\left[-\log
D_{\bm{\eta}}(\mathbf{Z})\right]\,\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log(1-D_{\bm{\eta}}(\mathbf{Z}))\,\right].$
(25)

(3) Train the Encoder $\bm{\phi}$ and Prior Distribution Generator $\bm{\psi}$
Adversarially.

$\displaystyle\mathop{\max}_{\bm{\phi},\bm{\psi}}\quad\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\,\mathbb{E}_{P_{\bm{\phi}}(\mathbf{Z}\mid\mathbf{X})}\left[-\log
D_{\bm{\eta}}(\mathbf{Z})\right]\,\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log(1-D_{\bm{\eta}}(\mathbf{Z}))\,\right].$
(26)

(4) Train the Utility Output Space Discriminator $\bm{\omega}$.

$\displaystyle\mathop{\min}_{\bm{\omega}}\quad\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\,-\log
D_{\bm{\omega}}(\mathbf{X})\,\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log\left(1-D_{\bm{\omega}}(\,g_{\bm{\theta}}(\mathbf{Z})\,)\right)\,\right].$
(27)

(5) Train the Prior Distribution Generator $\bm{\psi}$, Utility Decoder
$\bm{\theta}$, and Uncertainty Decoder $\bm{\xi}$ Adversarially.

$\displaystyle\mathop{\max}_{\bm{\psi},\bm{\theta},\bm{\xi}}\quad\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log\left(1-D_{\bm{\omega}}(\,g_{\bm{\theta}}(\mathbf{Z})\,)\right)\,\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log\left(1-D_{\bm{\tau}}(\,g_{\bm{\xi}}(\mathbf{Z})\,)\right)\,\right].$
(28)

(6) Train Uncertainty Output Space (Sensitive Attribute Class) Discriminator
$\bm{\tau}$.

$\displaystyle\mathop{\min}_{\bm{\tau}}\,\mathbb{E}_{P_{\mathbf{S}}}\left[-\log
D_{\bm{\tau}}(\mathbf{S})\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[-\log\left(1-D_{\bm{\tau}}(g_{\bm{\xi}}(\mathbf{Z}))\right)\right].$

The DVPF model $(\textsf{P2})$ is trained using alternating block coordinate
descent across six steps as follows:

(1) Train the Encoder $\bm{\phi}$, Utility Decoder $\bm{\theta}$ and
Uncertainty Decoder $\bm{\varphi}$.

$\mathop{\max}_{\bm{\phi},\bm{\theta},\bm{\varphi}}\quad\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\mathbb{E}_{P_{\bm{\phi}}(\mathbf{Z}\mid\mathbf{X})}\left[\log
P_{\bm{\theta}}(\mathbf{X}\\!\mid\\!\mathbf{Z})\right]\right]-\alpha\;\;\mathrm{D}_{\mathrm{KL}}\left(P_{\bm{\phi}}(\mathbf{Z}\mid\mathbf{X})\|Q_{\bm{\psi}}(\mathbf{Z})\\!\mid\\!P_{\mathsf{D}}(\mathbf{X})\right)\\\
-\alpha\;\;\mathbb{E}_{P_{\mathbf{S},\mathbf{X}}}\left[\mathbb{E}_{P_{\bm{\phi}}(\mathbf{Z}\mid\mathbf{X})}\left[-\log
P_{\bm{\varphi}}(\mathbf{X}\\!\mid\\!\mathbf{S},\mathbf{Z})\right]\right].$
(29)

(2) Train the Latent Space Discriminator $\bm{\eta}$.

$\displaystyle\mathop{\min}_{\bm{\eta}}\quad\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\,\mathbb{E}_{P_{\bm{\phi}}(\mathbf{Z}\mid\mathbf{X})}\left[-\log
D_{\bm{\eta}}(\mathbf{Z})\right]\,\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log(1-D_{\bm{\eta}}(\mathbf{Z}))\,\right].$
(30)

(3) Train the Encoder $\bm{\phi}$ and Prior Distribution Generator $\bm{\psi}$
Adversarially.

$\displaystyle\mathop{\max}_{\bm{\phi},\bm{\psi}}\quad\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\,\mathbb{E}_{P_{\bm{\phi}}(\mathbf{Z}\mid\mathbf{X})}\left[-\log
D_{\bm{\eta}}(\mathbf{Z})\right]\,\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log(1-D_{\bm{\eta}}(\mathbf{Z}))\,\right].$
(31)

(4) Train the Utility Output Space Discriminator $\bm{\omega}$.

$\displaystyle\mathop{\min}_{\bm{\omega}}\quad\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\,-\log
D_{\bm{\omega}}(\mathbf{X})\,\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log\left(1-D_{\bm{\omega}}(\,g_{\bm{\theta}}(\mathbf{Z})\,)\right)\,\right].$
(32)

(5) Train the Prior Distribution Generator $\bm{\psi}$, Utility Decoder
$\bm{\theta}$, and Uncertainty Decoder $\bm{\varphi}$ Adversarially.

$\displaystyle\mathop{\max}_{\bm{\psi},\bm{\theta},\bm{\varphi}}\quad\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log\left(1-D_{\bm{\omega}}(\,g_{\bm{\theta}}(\mathbf{Z})\,)\right)\,\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log\left(1-D_{\bm{\omega}}(\,g_{\bm{\varphi}}(\mathbf{S},\mathbf{Z})\,)\right)\,\right].$
(33)

(6) Train the Uncertainty Output Space Discriminator $\bm{\omega}$.

$\displaystyle\mathop{\min}_{\bm{\omega}}\quad\mathbb{E}_{P_{\mathsf{D}}(\mathbf{X})}\left[\,-\log
D_{\bm{\omega}}(\mathbf{X})\,\right]+\mathbb{E}_{Q_{\bm{\psi}}(\mathbf{Z})}\left[\,-\log\left(1-D_{\bm{\omega}}(\,g_{\bm{\varphi}}(\mathbf{S},\mathbf{Z})\,)\right)\,\right].$
(34)

The detailed training algorithms of these models are shown in Algorithm 1 and
Algorithm 2.

1:Input: Training Dataset:
$\\{\left(\mathbf{s}_{n},\mathbf{x}_{n}\right)\\}_{n=1}^{N}$; Hyper-Parameter:
$\alpha$

2:$\bm{\phi},\bm{\theta},\bm{\psi},\bm{\varphi},\bm{\eta},\bm{\omega}\;\leftarrow$
Initialize Network Parameters

3:repeat(1) Train the Encoder $\bm{\phi}$, Utility Decoder $\bm{\theta}$,
Uncertainty Decoder $\bm{\xi}$

4: Sample a mini-batch $\\{\mathbf{x}_{m},\mathbf{s}_{m}\\}_{m=1}^{M}\sim
P_{\mathsf{D}}(\mathbf{X})P_{\mathbf{S}\mid\mathbf{X}}$

5: Compute encoder outputs
$\bm{\mu}_{m}^{\mathsf{enc}},\bm{\sigma}_{m}^{\mathsf{enc}}=f_{\bm{\phi}}(\mathbf{x}_{m}),\forall
m\in[M]$

6: Apply reparametrization trick
$\mathbf{z}_{m}^{\mathsf{enc}}=\bm{\mu}_{m}^{\mathsf{enc}}+\bm{\epsilon}_{m}\odot\bm{\sigma}_{m}^{\mathsf{enc}},\;\bm{\epsilon}_{m}\sim\mathcal{N}(0,\mathbf{I}),\;\forall
m\in[M]$

7: Sample $\\{\mathbf{n}_{m}\\}_{m=1}^{M}\sim\mathcal{N}(\bm{0},\mathbf{I})$

8: Compute
$\bm{\mu}_{m}^{\mathsf{prior}},\bm{\sigma}_{m}^{\mathsf{prior}}=g_{\bm{\psi}}(\mathbf{n}_{m}),\forall
m\in[M]$

9: Compute
$\mathbf{z}_{m}^{\mathsf{prior}}\\!\\!=\\!\bm{\mu}_{m}^{\mathsf{prior}}\\!+\bm{\epsilon}_{m}^{\prime}\odot\bm{\sigma}_{m}^{\mathsf{prior}},\bm{\epsilon}_{m}^{\prime}\\!\sim\\!\mathcal{N}(0,\mathbf{I}),\forall
m\\!\in\\![M]\\!$

10: Compute
$\mathbf{\widehat{x}}_{m}=g_{\bm{\theta}}(\mathbf{z}_{m}^{\mathsf{enc}}),\forall
m\in[M]$

11: Compute
$\mathbf{\widehat{s}}_{m}=g_{\bm{\xi}}(\mathbf{z}_{m}^{\mathsf{enc}}),\forall
m\in[M]$

12: Back-propagate loss:
$\quad\mathcal{L}\left(\bm{\phi},\bm{\theta},\bm{\xi}\right)=\\!-\frac{1}{M}\sum_{m=1}^{M}\\!\Big{(}\mathsf{dis}(\mathbf{x}_{m},\mathbf{\widehat{x}}_{m})-\,\alpha\;\log
P_{\bm{\xi}}(\mathbf{s}_{m}\\!\mid\\!\mathbf{z}_{m}^{\mathsf{enc}})\Big{)}$
(2) Train the Latent Space Discriminator $\bm{\eta}$

13: Sample $\\{\mathbf{x}_{m}\\}_{m=1}^{M}\sim P_{\mathsf{D}}(\mathbf{X})$

14: Sample $\\{\mathbf{n}_{m}\\}_{m=1}^{M}\sim\mathcal{N}(\bm{0},\mathbf{I})$

15: Compute $\mathbf{z}_{m}^{\mathsf{enc}}\\!$ from
$\\!f_{\\!\bm{\phi}}(\mathbf{x}_{m})\\!$ with reparametrization, $\forall
m\\!\in\\![M]\\!$

16: Compute $\mathbf{z}_{m}^{\mathsf{prior}}\\!\\!$ from
$\\!g_{\bm{\psi}}(\mathbf{n}_{m}\\!)\\!$ with reparametrization, $\\!\forall
m\\!\in\\![M]\\!$

17: Back-propagate loss:
$\;\;\;\mathcal{L}\left(\bm{\eta}\right)=-\frac{\alpha}{M}\;\sum_{m=1}^{M}\log
D_{\bm{\eta}}(\mathbf{z}_{m}^{\mathsf{enc}})+\log\big{(}1-D_{\bm{\eta}}(\,\mathbf{z}_{m}^{\mathsf{prior}}\,)\big{)}$
(3) Train the Encoder $\bm{\phi}$ and Prior Distribution Generator $\bm{\psi}$
Adversarially

18: Sample $\\{\mathbf{x}_{m}\\}_{m=1}^{M}\sim P_{\mathsf{D}}(\mathbf{X})$

19: Compute $\mathbf{z}_{m}^{\mathsf{enc}}\\!$ from
$\\!f_{\\!\bm{\phi}}(\mathbf{x}_{m})\\!$ with reparametrization, $\forall
m\\!\in\\![M]$

20: Sample $\\{\mathbf{n}_{m}\\}_{m=1}^{M}\sim\mathcal{N}(\bm{0},\mathbf{I})$

21: Compute $\mathbf{z}_{m}^{\mathsf{prior}}\\!\\!$ from
$\\!g_{\bm{\psi}}(\mathbf{n}_{m}\\!)\\!$ with reparametrization, $\\!\forall
m\\!\in\\![M]\\!$

22: Back-propagate loss:
$\;\;\;\mathcal{L}\left(\bm{\phi},\bm{\psi}\right)=\frac{\alpha}{M}\;\sum_{m=1}^{M}\log
D_{\bm{\eta}}(\mathbf{z}_{m}^{\mathsf{enc}})+\log\big{(}1-D_{\bm{\eta}}(\,\mathbf{z}_{m}^{\mathsf{prior}}\,)\big{)}$
(4) Train the Utility Output Space Discriminator $\bm{\omega}$

23: Sample $\\{\mathbf{x}_{m}\\}_{m=1}^{M}\sim P_{\mathsf{D}}(\mathbf{X})$

24: Sample
$\\{\mathbf{n}_{m}\\}_{m=1}^{M}\sim\mathcal{N}\\!\left(\bm{0},\mathbf{I}\right)$

25: Compute $\mathbf{z}_{m}^{\mathsf{prior}}\\!\\!$ from
$\\!g_{\bm{\psi}}(\mathbf{n}_{m}\\!)\\!$ with reparametrization, $\\!\forall
m\\!\in\\![M]\\!$

26: Compute
$\mathbf{\widehat{x}}_{m}=g_{\bm{\theta}}(\mathbf{z}_{m}^{\mathsf{prior}}),\forall
m\in[M]$

27: Back-propagate loss:
$\mathcal{L}\left(\bm{\omega}\right)=-\frac{1}{M}\sum_{m=1}^{M}\log
D_{\bm{\omega}}(\mathbf{x}_{m})+\log\left(1-D_{\bm{\omega}}(\,\mathbf{\widehat{x}}_{m}\,)\right)$
(5) Train the Prior Distribution Generator $\bm{\psi}$, Utility Decoder
$\bm{\theta}$, and Uncertainty Decoder $\bm{\xi}$ Adversarially

28: Sample
$\\{\mathbf{n}_{m}\\}_{m=1}^{M}\sim\mathcal{N}\\!\left(\bm{0},\mathbf{I}\right)$

29: Compute $\mathbf{z}_{m}^{\mathsf{prior}}\\!\\!$ from
$\\!g_{\bm{\psi}}(\mathbf{n}_{m}\\!)\\!$ with reparametrization, $\\!\forall
m\\!\in\\![M]\\!$

30: Compute
$\mathbf{\widehat{x}}_{m}=g_{\bm{\theta}}\left(\mathbf{z}_{m}^{\mathsf{prior}}\right),\forall
m\in[M]$

31: Compute
$\mathbf{\widehat{s}}_{m}=g_{\bm{\xi}}\left(\mathbf{z}_{m}^{\mathsf{prior}}\right),\forall
m\in[M]$

32: Back-propagate loss:
$\;\;\;\;\mathcal{L}\left(\bm{\psi},\bm{\theta},\bm{\xi}\right)\\!=\\!\frac{1}{M}\\!\\!\sum_{m=1}^{M}\\!\\!\log\left(1\\!-\\!D_{\bm{\omega}}(\,\mathbf{\widehat{x}}_{m}\,)\right)+\log\left(1\\!-\\!D_{\bm{\tau}}(\,\mathbf{\widehat{s}}_{m}\,)\right)\\!\\!\\!\\!$
(6) Train Uncertainty Output Space Discriminator $\bm{\omega}$

33: Sample a mini-batch $\\{\mathbf{s}_{m},\mathbf{x}_{m}\\}_{m=1}^{M}\sim
P_{\mathsf{D}}(\mathbf{X})P_{\mathbf{S}\mid\mathbf{X}}$

34: Sample $\\{\mathbf{n}_{m}\\}_{m=1}^{M}\sim\mathcal{N}(\bm{0},\mathbf{I})$

35: Compute $\mathbf{z}_{m}^{\mathsf{prior}}\\!\\!$ from
$\\!g_{\bm{\psi}}(\mathbf{n}_{m}\\!)\\!$ with reparametrization, $\\!\forall
m\\!\in\\![M]\\!$

36: Compute $\mathbf{\widehat{s}}_{m}\sim
g_{\bm{\xi}}\left(\mathbf{z}_{m}^{\mathsf{prior}}\right),\forall m\in[M]$

37: Back-propagate loss:
$\mathcal{L}\left(\bm{\tau}\right)=\frac{1}{M}\sum_{m=1}^{M}\log
D_{\bm{\tau}}(\,\mathbf{s}_{m}\,)+\log\left(1-D_{\bm{\tau}}(\,\mathbf{\widehat{s}}_{m}\,)\right)$

38:until Convergence

39:return $\bm{\phi},\bm{\theta},\bm{\psi},\bm{\varphi},\bm{\eta},\bm{\omega}$

Algorithm 1 Deep Variational $\mathsf{DisPF}$ training algorithm associated
with $\textsf{DisPF\text{-}MI}\;(\textsf{P1})$

1:Input: Training Dataset:
$\\{\left(\mathbf{s}_{n},\mathbf{x}_{n}\right)\\}_{n=1}^{N}$; Hyper-Parameter:
$\alpha$

2:$\bm{\phi},\bm{\theta},\bm{\psi},\bm{\varphi},\bm{\eta},\bm{\omega}\;\leftarrow$
Initialize Network Parameters

3:repeat(1) Train the Encoder $\bm{\phi}$, Utility Decoder $\bm{\theta}$,
Uncertainty Decoder $\bm{\varphi}$

4: Sample a mini-batch $\\{\mathbf{x}_{m},\mathbf{s}_{m}\\}_{m=1}^{M}\sim
P_{\mathsf{D}}(\mathbf{X})P_{\mathbf{S}\mid\mathbf{X}}$

5: Compute encoder outputs
$\bm{\mu}_{m}^{\mathsf{enc}},\bm{\sigma}_{m}^{\mathsf{enc}}=f_{\bm{\phi}}(\mathbf{x}_{m}),\forall
m\in[M]$

6: Apply reparametrization trick
$\mathbf{z}_{m}^{\mathsf{enc}}=\bm{\mu}_{m}^{\mathsf{enc}}+\bm{\epsilon}_{m}\odot\bm{\sigma}_{m}^{\mathsf{enc}},\;\bm{\epsilon}_{m}\sim\mathcal{N}(0,\mathbf{I}),\;\forall
m\in[M]$

7: Sample $\\{\mathbf{n}_{m}\\}_{m=1}^{M}\sim\mathcal{N}(\bm{0},\mathbf{I})$

8: Compute
$\bm{\mu}_{m}^{\mathsf{prior}},\bm{\sigma}_{m}^{\mathsf{prior}}=g_{\bm{\psi}}(\mathbf{n}_{m}),\forall
m\in[M]$

9: Compute
$\mathbf{z}_{m}^{\mathsf{prior}}\\!\\!=\\!\bm{\mu}_{m}^{\mathsf{prior}}\\!+\bm{\epsilon}_{m}^{\prime}\odot\bm{\sigma}_{m}^{\mathsf{prior}},\bm{\epsilon}_{m}^{\prime}\\!\sim\\!\mathcal{N}(0,\mathbf{I}),\forall
m\\!\in\\![M]\\!$

10: Compute