#  $\mathbb{S}$ci$\mathbb{F}$ix: Outperforming GPT3 on Scientific Factual
Error Correction

David S. Hippocampus
Department of Computer Science
Cranberry-Lemon University
Pittsburgh, PA 15213
<EMAIL_ADDRESS>
Use footnote for providing further information about author (webpage,
alternative address)— _not_ for acknowledging funding agencies.

##  1 Introduction

##  2 Related Work

##  3 Method

##  4 Implementation Details

##  5 Experiment 1: Correction Dataset

##  6 Limitations and Future Work

##  7 Conclusion

## Appendix

##  Appendix A Experiment 2: Verification Dataset

##  Appendix B Ablation and Analysis


# Note on a theorem of Professor X.

Levent Alpöge

###### Abstract.

Between his arrival in Frankfurt in $1922$ and and his proof of his famous
finiteness theorem for integral points in $1929$, Siegel had no publications.
He did, however, write a letter to Mordell in $1926$ in which he explained a
proof of the finiteness of integral points on hyperelliptic curves.
Recognizing the importance of this argument (and Siegel’s views on
publication), Mordell sent the relevant extract to be published under the
pseudonym "X".

The purpose of this note is to explain how to optimize Siegel’s $1926$
technique to obtain the following bound. Let $K$ be a number field, $S$ a
finite set of places of $K$, and $f\in\mathfrak{o}_{K,S}[t]$ monic of degree
$d\geq 5$ with discriminant $\Delta_{f}\in\mathfrak{o}_{K,S}^{\times}$. Then:

$\\#|\\{(x,y):x,y\in\mathfrak{o}_{K,S},y^{2}=f(x)\\}|\leq
2^{\mathop{\mathrm{rank}}{\mathrm{Jac}(C_{f})(K)}}\cdot
O(1)^{d^{3}\cdot([K:\mathbb{Q}]+\\#|S|)}.$

This improves bounds of Evertse-Silverman and Bombieri-Gubler from $1986$ and
$2006$, respectively.

The main point underlying our improvement is that, informally speaking, we
insist on "executing the descents in the presence of only one root (and not
three) until the last possible moment".

##  1 Introduction.

The technique introduced in Siegel’s $1926$ letter to Mordell [8] to prove the
finiteness of integral points on hyperelliptic curves111By this common and
incorrect abbreviation we really mean integral solutions of $y^{2}=f(x)$.
Siegel’s $1926$ proof does not control integral solutions of e.g.
$y^{3}+x\cdot y=x^{4}$, or, said another way, integral points with respect to
an effective divisor containing no nonzero divisor symmetric under the
hyperelliptic involution. Baker effectivized Siegel’s $1926$ argument and thus
gave an effective finiteness proof in the $y^{2}=f(x)$ cases, but he did not
effectivize Siegel’s $1929$ finiteness theorem in the case of hyperelliptic
curves. can be summarized as: $2$-descent on the curve, then $3$-descent on
$\mathbb{G}_{m}$. From this summary the intuitive guess for the bound that the
argument "should" produce is of course something of shape
$2^{\mathop{\mathrm{rank}}}\cdot 3^{\text{$\\#$ of prime factors of the
discriminant}}$. However a consultation of the literature yields worse
estimates.222To our knowledge the best bounds in the literature are the $2006$
estimate of Bombieri-Gubler [4, Theorem $5.3.5$] and the $1986$ estimate of
Evertse-Silverman [6]. In both bounds the ”expected”
$2^{\mathop{\mathrm{rank}}}$ factor is replaced by a power of the size of the
$2$-part of the class group of an extension gotten by adjoining three
Weierstrass points.

In this note we rectify the situation.

######  Theorem 1.1.

Let $K/\mathbb{Q}$ be a number field. Let $S$ be a finite set of places of
$K$. Let $f\in\mathfrak{o}_{K,S}[t]$ be monic of degree $d\geq 5$ with
discriminant $\Delta_{f}\in\mathfrak{o}_{K,S}^{\times}$. Let
$\mathcal{C}_{f}^{\mathrm{aff.}}:y^{2}=f(x)$ be the Weierstrass model of the
hyperelliptic curve $C_{f}$ corresponding to $f$. Then:

$\\#|\mathcal{C}_{f}^{\mathrm{aff.}}(\mathfrak{o}_{K,S})|\leq
2^{\mathop{\mathrm{rank}}{\mathrm{Jac}(C_{f})(K)}}\cdot
O(1)^{d^{3}\cdot([K:\mathbb{Q}]+\\#|S|)}.$

Also, for all $K$-irreducible $g_{i}\in\mathfrak{o}_{K,S}[t]$ with
$\sum_{i}\deg{g_{i}}\geq 3$ and $\prod_{i}g_{i}\big{|}f$,

$\\#|\mathcal{C}_{f}^{\mathrm{aff.}}(\mathfrak{o}_{K,S})|\leq\left(\prod_{i}\\#|\mathrm{Cl}(\mathfrak{o}_{K_{g_{i}},S})[2]|\right)\cdot
O(1)^{d^{3}\cdot([K:\mathbb{Q}]+\\#|S|)},$

where $K_{g_{i}}:=K[t]/(g_{i})$ and
$\mathfrak{o}_{K_{g_{i}},S}:=\mathfrak{o}_{K_{g_{i}}}\otimes_{\mathfrak{o}_{K}}\mathfrak{o}_{K,S}$.

We include the second statement in case e.g. the product of the linear and
quadratic irreducible factors of $f$ is of degree at least $3$.

In the elliptic curve case an even stronger bound is available because one can
execute the whole descent over $K$ (following Mordell) [1]333See also Chapter
$2$ of [2], which is based on the same work.. In the superelliptic
($y^{m}=f(x),m>2$) case as usual one does not need to execute a $3$-descent on
$\mathbb{G}_{m}$ so the bound also improves.

Let us explain how the argument goes. First, the $2$-descent. À la Fermat,
given $y^{2}=f(x)$, adjoin a root $\rho$ and conclude that $x-\rho$ is almost
a square, say $x-\rho=\alpha\cdot\beta^{2}$ with $\alpha,\beta\in
K[\rho]:=K[t]/(f)$.

Now adjoin three roots. Let $L/K$ be an extension containing three roots, say
$\rho_{1},\rho_{2},\rho_{3}$. Then, for all $K$-maps $\sigma:K[\rho]\to L$, we
find $x-\sigma(\rho)=\sigma(\alpha)\cdot\sigma(\beta)^{2}$. Thus (obvious
notation)
$\rho_{j}-\rho_{i}=\alpha_{i}\cdot\beta_{i}^{2}-\alpha_{j}\cdot\beta_{j}^{2}$.

Now pass to
$L^{\prime}:=L(\sqrt{\alpha_{1}},\sqrt{\alpha_{2}},\sqrt{\alpha_{3}})$. We
obtain the six elements
$\gamma_{ij,\pm}:=\beta_{i}\sqrt{\alpha_{i}}\pm\beta_{j}\sqrt{\alpha_{j}}$,
all divisors of $\rho_{j}-\rho_{i}$. By hypothesis they are therefore all
$S^{\prime}$-units, with $S^{\prime}$ the set of places of $L^{\prime}$ above
a place in $S$.

Now for the $3$-descent. We may then write
$\gamma_{ij,\pm}=\delta_{ij,\pm}\cdot\varepsilon_{ij,\pm}^{3}$ with
$\delta_{ij,\pm}$ representatives for
$\mathfrak{o}_{L^{\prime},S^{\prime}}^{\times}/3$. The relation
$\gamma_{12,+}-\gamma_{23,+}=\gamma_{13,-}$ becomes the cubic Thue equation
$\frac{\delta_{12,+}}{\delta_{13,-}}\cdot\left(\frac{\varepsilon_{12,+}}{\varepsilon_{13,-}}\right)^{3}-\frac{\delta_{23,+}}{\delta_{13,-}}\cdot\left(\frac{\varepsilon_{23,+}}{\varepsilon_{13,-}}\right)^{3}=1$.

We conclude with a bound of Evertse.

As a last remark: note that
$2^{\mathop{\mathrm{rank}}{\mathrm{Jac}\,{(C_{f})}(K)}}$ is $\ll 1$ on average
(ordering as usual by height) [3, 7]. This statement is why we tried for the
stronger bound [1] in the elliptic curve case, after which we questioned why a
similar bound did not exist in the hyperelliptic curve case.

##  2 Acknowledgments.

This note is based on Chapter $4$ of the author’s Ph.D. thesis at Princeton
University. I would like to thank both my advisor Manjul Bhargava and Peter
Sarnak for their patience and encouragement. I would also like to thank Jacob
Tsimerman and Nina Zubrilina for informative discussions. Finally I thank the
National Science Foundation (via their grant DMS-$2002109$), Columbia
University, and the Society of Fellows for their support during the pandemic.

##  3 Proof of Theorem 1.1.

###### Proof of Theorem 1.1.

Let $g\in\mathfrak{o}_{K,S}[t]$ with $g|f$ be a monic divisor of $f$ with
$\deg{g}\geq 3$ (for the first part of the theorem we will take $g=f$). Write
$f=:gh$ with $h\in\mathfrak{o}_{K,S}[t]$. Let $L/K$ be an extension of minimal
degree containing at least three roots of $g$.

Factorize $g=:\prod_{i}g_{i}$ into $K$-irreducible factors
$g_{i}\in\mathfrak{o}_{K,S}[t]$. Note that $(g_{i},g_{j})=(1)$ for each $i\neq
j$ because $\Delta_{f}\in\mathfrak{o}_{K,S}^{\times}$. Thus
$K_{g}\simeq\bigoplus_{i}K_{g_{i}}$, with $K_{g_{i}}:=K[t]/(g_{i})$ and
$K_{g}:=K[t]/(g)$. We will repeatedly write $\mathfrak{o}_{K_{g_{i}},S}$ etc.
for the evident localizations, e.g.
$\mathfrak{o}_{K_{g_{i}},S}:=\mathfrak{o}_{K_{g_{i}}}\otimes_{\mathfrak{o}_{K}}\mathfrak{o}_{K,S}$.
Write $\rho^{(i)}$ for the image of $t$ in $K_{g_{i}}=K[t]/(g_{i})$.

By the Chebotarev density theorem (and its explicit error term) applied to the
Hilbert class field of $K_{g_{i}}$, each ideal class of $K_{g_{i}}$ contains a
prime of norm $\ll|\Delta_{K_{g_{i}}}|^{O(1)}$. (We will only use existence.)
Let then $P^{(i)}$ be a minimal set of prime representatives of
$\mathrm{Cl}(\mathfrak{o}_{K_{g_{i}},S})[2]$ in $K_{g_{i}}$ of norm
$\ll|\Delta_{K_{g_{i}}}|^{O(1)}$.

Now let us begin the argument. For
$(x,y)\in\mathcal{C}_{f}^{\mathrm{aff.}}(\mathfrak{o}_{K,S})$,
$(y)^{2}=(x-\rho^{(i)})\cdot\left(\frac{g_{i}(x)}{x-\rho^{(i)}}\right)\cdot\prod_{j\neq
i}(g_{j}(x))\cdot(h(x))$ as ideals in $\mathfrak{o}_{K_{g_{i}},S}$. Without
loss of generality $f(x)\neq 0$. Since
$\Delta_{f}\in\mathfrak{o}_{K,S}^{\times}$, it follows that there is an ideal
$\mathfrak{a}_{i}\subseteq\mathfrak{o}_{K_{g_{i}},S}$ with
$(x-\rho^{(i)})=\mathfrak{a}_{i}^{2}$. Thus there is a $\mathfrak{p}_{i}\in
P^{(i)}$ with $\mathfrak{p}_{i}\equiv\mathfrak{a}_{i}$ modulo principal ideals
of $K_{g_{i}}$. Hence $\mathfrak{a}_{i}$ is principal in
$\mathfrak{o}_{K_{g_{i}},S\cup\\{\mathfrak{p}_{i}\\}}$ (obvious meaning).

Let $\alpha_{i}\in\mathfrak{o}_{K_{g_{i}},S\cup\\{\mathfrak{p}_{i}\\}}$ be
such that $\mathfrak{a}_{i}=(\alpha_{i})$ as ideals of
$\mathfrak{o}_{K_{g_{i}},S\cup\\{\mathfrak{p}_{i}\\}}$. Thus
$x-\rho^{(i)}=\alpha_{i}^{2}\cdot(\in\mathfrak{o}_{K_{g_{i}},S\cup\\{\mathfrak{p}_{i}\\}}^{\times})$,
where $(\in\mathfrak{o}_{K_{g_{i}},S\cup\\{\mathfrak{p}_{i}\\}}^{\times})$
denotes an element of the $(S\cup\\{\mathfrak{p}_{i}\\})$-units of
$K_{g_{i}}$.

Let $U^{(i)}$ be a minimal set of representatives of
$\mathfrak{o}_{K_{g_{i}},S\cup\\{\mathfrak{p}_{i}\\}}^{\times}/2$ (that is,
modulo squares). It follows that there are $\gamma_{i}\in U^{(i)}$ and
$u^{(i)}\in\mathfrak{o}_{K_{g_{i}},S\cup\\{\mathfrak{p}_{i}\\}}^{\times}$ such
that $x-\rho^{(i)}=\gamma_{i}\cdot(\alpha_{i}\cdot u^{(i)})^{2}$. Let
$\eta_{i}:=\alpha_{i}\cdot u^{(i)}$. Thus
$x-\rho^{(i)}=\gamma_{i}\cdot\eta_{i}^{2}$.

Therefore we find that, for each $K$-embedding
$\sigma:K_{g_{i}}\hookrightarrow L$, we have
$x-\sigma(\rho^{(i)})=\sigma(\gamma_{i})\cdot\sigma(\eta_{i})^{2}$.

By definition of $L$, for each $1\leq k\leq 3$ there is an $i_{k}$ and a
$K$-embedding $\tau_{k}:K_{g_{i_{k}}}\hookrightarrow L$ such that the
corresponding roots $\tau_{k}(\rho^{(i_{k})})$ are pairwise distinct. Let
then:

$\displaystyle\kappa_{k}$ $\displaystyle:=\tau_{k}(\rho^{(i_{k})}),$
$\displaystyle\lambda_{k}$ $\displaystyle:=\tau_{k}(\gamma_{i_{k}}),$
$\displaystyle\mu_{k}$ $\displaystyle:=\tau_{k}(\eta_{i_{k}}).$

Hence $x-\kappa_{k}=\lambda_{k}\cdot\mu_{k}^{2}$ as elements of $L$. Hence for
$k\neq\ell$ we find that
$\kappa_{\ell}-\kappa_{k}=\lambda_{k}\cdot\mu_{k}^{2}-\lambda_{\ell}\cdot\mu_{\ell}^{2}$.

We are done with the $2$-descent on $C_{f}$. Now it is time for the
$3$-descent on $\mathbb{G}_{m}$. Let
$L_{k\ell}:=L\left(\sqrt{\lambda_{k}},\sqrt{\lambda_{\ell}}\right)$. Thus
$(1)=(\kappa_{\ell}-\kappa_{k})=\left(\sqrt{\lambda_{k}}\mu_{k}-\sqrt{\lambda_{\ell}}\mu_{\ell}\right)\cdot\left(\sqrt{\lambda_{k}}\mu_{k}+\sqrt{\lambda_{\ell}}\mu_{\ell}\right)$
as ideals of
$\mathfrak{o}_{L_{k\ell},S\cup\\{\mathfrak{p}_{k},\mathfrak{p}_{\ell}\\}}$
(obvious meaning), the first equality following from
$\Delta_{f}\in\mathfrak{o}_{K,S}^{\times}$.

Let $V_{k\ell}$ be a minimal set of representatives of
$\mathfrak{o}_{L_{k\ell},S\cup\\{\mathfrak{p}_{k},\mathfrak{p}_{\ell}\\}}^{\times}/3$
(that is, modulo cubes). It follows that there are $v_{k\ell,\pm}\in
V_{k\ell},\zeta_{k\ell,\pm}\in\mathfrak{o}_{L_{k\ell},S\cup\\{\mathfrak{p}_{k},\mathfrak{p}_{\ell}\\}}^{\times}$
for which
$\sqrt{\lambda_{k}}\mu_{k}\pm\sqrt{\lambda_{\ell}}\mu_{\ell}=v_{k\ell,\pm}\cdot\zeta_{k\ell,\pm}^{3}$.
Hence:

$\displaystyle 0$
$\displaystyle=(\sqrt{\lambda_{1}}\mu_{1}\pm\sqrt{\lambda_{2}}\mu_{2})\mp(\sqrt{\lambda_{2}}\mu_{2}\pm\sqrt{\lambda_{3}}\mu_{3})-(\sqrt{\lambda_{1}}\mu_{1}-\sqrt{\lambda_{3}}\mu_{3})$
$\displaystyle=v_{12,\pm}\cdot\zeta_{12,\pm}^{3}\mp
v_{23,\pm}\cdot\zeta_{23,\pm}^{3}-v_{13,-}\cdot\zeta_{13,-}^{3}.$

We may of course rearrange this as
$\frac{v_{12,\pm}}{v_{13,-}}\cdot\left(\frac{\zeta_{12,\pm}}{\zeta_{13,-}}\right)^{3}\mp\frac{v_{23,\pm}}{v_{13,-}}\cdot\left(\frac{\zeta_{23,\pm}}{\zeta_{13,-}}\right)^{3}=1$.

Now for Evertse’s bound. Let $F_{\pm}(X,Y):=\frac{v_{12,\pm}}{v_{13,-}}\cdot
X^{3}\mp\frac{v_{23,\pm}}{v_{13,-}}\cdot Y^{3}\in\mathfrak{o}_{L_{123}}[X,Y]$,
with $L_{123}:=L(\sqrt{\lambda_{1}},\sqrt{\lambda_{2}},\sqrt{\lambda_{3}})$.
Then $F_{\pm}$ is a cubic form with nonzero discriminant. Moreover if
$F_{\pm}(X,Y)=F_{\pm}(tX,tY)=1$ then $t^{3}=1$. Therefore by [5] the number of
solutions of $F_{\pm}(X,Y)=1$ with
$(X,Y)\in\mathfrak{o}_{L_{123},S\cup\\{\mathfrak{p}_{1},\mathfrak{p}_{2},\mathfrak{p}_{3}\\}}$
is $\ll O(1)^{[L:\mathbb{Q}]+[L:K]\cdot\\#|S|}$.

We are done with the argument. Now let us examine the tally. Let us first show
that our original point $(x,y)$ can be recovered from the pairs
$\left(\frac{\zeta_{12,\pm}}{\zeta_{13,-}},\frac{\zeta_{23,\pm}}{\zeta_{13,-}}\right)$
up to $O(1)$ many possibilities. To see this first multiply the two
$X$-coordinates together to form
$\frac{\zeta_{12,+}\cdot\zeta_{12,-}}{\zeta_{13,-}^{2}}$. Note that:

$\displaystyle v_{12,+}\cdot
v_{12,-}\cdot\zeta_{12,+}^{3}\cdot\zeta_{12,-}^{3}$
$\displaystyle=(\sqrt{\lambda_{1}}\mu_{1}+\sqrt{\lambda_{2}}\mu_{2})\cdot(\sqrt{\lambda_{1}}\mu_{1}-\sqrt{\lambda_{2}}\mu_{2})$
$\displaystyle=\lambda_{1}\mu_{1}^{2}-\lambda_{2}\mu_{2}^{2}$
$\displaystyle=\kappa_{2}-\kappa_{1}.$

Thus it follows that
$\zeta_{12,+}^{3}\cdot\zeta_{12,-}^{3}=\frac{\kappa_{2}-\kappa_{1}}{v_{12,+}\cdot
v_{12,-}}$. Hence the cube of the product of the two $X$-coordinates is
$\zeta_{13,-}^{-6}\cdot\left(\frac{\kappa_{2}-\kappa_{1}}{v_{12,+}\cdot
v_{12,-}}\right)$. We note that the term in parentheses is fixed (in terms of
our choices up til now). Thus we may recover $\zeta_{13,-}^{6}$, and hence
$\zeta_{13,-}$ up to at most six choices. Having done so we return to the
$X$-coordinates of both solutions and recover $\zeta_{12,+}$ and
$\zeta_{12,-}$. Then the equality

$\displaystyle 2\sqrt{\lambda_{1}}\mu_{1}$
$\displaystyle=(\sqrt{\lambda_{1}}\mu_{1}+\sqrt{\lambda_{2}}\mu_{2})+(\sqrt{\lambda_{1}}\mu_{1}-\sqrt{\lambda_{2}}\mu_{2})$
$\displaystyle=(v_{12,+}\cdot\zeta_{12,+}^{3})+(v_{12,-}\cdot\zeta_{12,-}^{3})$

implies that we can recover $2\sqrt{\lambda_{1}}\mu_{1}$. Squaring this we
find that we can recover $4\lambda_{1}\mu_{1}^{2}=4(x-\kappa_{1})$. Since
$\kappa_{1}$ is fixed we can recover $x$, and then there are at most two
choices for $y$ given $x$, so we can indeed recover the point up to $O(1)$
many choices.

So we see that a point
$(x,y)\in\mathcal{C}_{f}^{\mathrm{aff.}}(\mathfrak{o}_{K,S})$ is determined up
to $O(d^{3})$ (arising from the choice of three roots in $L$) many choices by
the data

$(\mathfrak{p}_{1},\mathfrak{p}_{2},\mathfrak{p}_{3},\gamma_{1},\gamma_{2},\gamma_{3},v_{12,+},v_{12,-},v_{23,+},v_{23,-},v_{31,-}).$

The number of choices for each $\mathfrak{p}_{i}$ is
$\\#|P^{(i)}|=\\#|\mathrm{Cl}(\mathfrak{o}_{K_{g_{i}},S})[2]|$. The number of
choices for each $\gamma_{i}\in U^{(i)}$ is $\\#|U^{(i)}|\ll
2^{\deg{g_{i}}\cdot(\\#|S|+[K:\mathbb{Q}])}$ by Dirichlet. Similarly the
number of choices for each $v_{k\ell,\pm}\in V_{k\ell}$ is $\\#|V_{k\ell}|\ll
3^{4\cdot[L:\mathbb{Q}]+4\cdot[L:K]\cdot\\#|S|}$. Of course $[L:K]\leq d^{3}$.

Therefore the total number of tuples
$(\mathfrak{p}_{1},\mathfrak{p}_{2},\mathfrak{p}_{3},\gamma_{1},\gamma_{2},\gamma_{3},v_{12,+},v_{12,-},v_{23,+},v_{23,-},v_{31,-})$
is

$\ll\left(\prod_{i}\\#|\mathrm{Cl}(\mathfrak{o}_{K_{g_{i}},S})[2]|\right)\cdot
O(1)^{d^{3}\cdot([K:\mathbb{Q}]+\\#|S|)}.$

We are done with the second part of the theorem.

So take $g=f$, and let us count the ideal classes of
$\bigoplus_{i}\mathfrak{o}_{K_{g_{i}},S}\simeq\mathfrak{o}_{K_{f},S}$ that
could possibly arise in the $2$-descent step (we have bounded said count by
$\leq\prod_{i}\\#|\mathrm{Cl}(\mathfrak{o}_{K_{g_{i}},S})[2]|$ and we claim it
is also $\ll 2^{\mathop{\mathrm{rank}}{\mathrm{Jac}\,{(C_{f})}(K)}}\cdot
O(1)^{d}$).

Let $J_{f}:=\mathrm{Jac}\,{C_{f}}$. Let $W\subseteq
C_{f}(\overline{\mathbb{Q}})$ be the set of Weierstrass points of $C_{f}$. Let
$\infty\in W$ be a point at infinity. We embed $C_{f}\hookrightarrow
J_{f}=\mathrm{Pic}^{0}(C_{f})$ via $P\mapsto P-\infty$. Thus as
$\mathop{\mathrm{Gal}}(\overline{\mathbb{Q}}/K)$-modules
$J_{f}[2]\simeq\mathbb{F}_{2}[W-\\{\infty\\}]/\mathbb{F}_{2}\cdot\left(\sum_{P\in
W-\\{\infty\\}}P\right)$. In other words,
$\mathrm{Ind}_{K}^{K_{f}}\mathbb{F}_{2}\simeq\mathbb{F}_{2}\oplus J_{f}[2]$,
where we have written
$\mathrm{Ind}_{K}^{K_{f}}(\bullet):=\bigoplus_{i}\mathrm{Ind}_{K}^{K_{g_{i}}}(\bullet)$.
Thus $H^{1}(K_{f},\mathbb{F}_{2})\simeq
H^{1}(K,\mathrm{Ind}_{K}^{K_{f}}\mathbb{F}_{2})\simeq
H^{1}(K,\mathbb{F}_{2})\oplus H^{1}(K,J_{f}[2])$, where the first isomorphism
follows by Shapiro’s lemma (and
$H^{1}(K_{f},\bullet):=\bigoplus_{i}H^{1}(K_{g_{i}},\bullet)$). By Kummer it
follows that
$H^{1}(K,J_{f}[2])\simeq(K_{f}^{\times}/2)_{\mathrm{Nm}=\square}$.

Thus by taking invariants of $0\to J_{f}[2]\to J_{f}\to J_{f}\to 0$ we obtain
$J_{f}(K)/2\hookrightarrow
H^{1}(K,J_{f}[2])\simeq(K_{f}^{\times}/2)_{\mathrm{Nm}=\square}$. Write
$G\subseteq H^{1}(K,J_{f}[2])$ for the image of this map. Note that the
restriction of this map to $C_{f}(K)-W$ is simply $(x,y)\mapsto x-\rho$, so
similarly write $G^{\prime}\subseteq G$ for the image of
$\mathcal{C}^{\mathrm{aff.}}_{f}(\mathfrak{o}_{K,S})-W$.

We have already seen that each $g\in G^{\prime}\subseteq
G\hookrightarrow(K_{f}^{\times}/2)_{\mathrm{Nm}=\square}$ gives rise to a
class represented by an $\alpha\in K_{f}^{\times}$ for which
$v_{\mathfrak{p}}(\alpha)$ is even for all primes
$\mathfrak{p}\subseteq\mathfrak{o}_{K_{f},S}$ (because
$\Delta_{f}\in\mathfrak{o}_{K,S}^{\times}$). From such a class we may produce
an element of $\mathrm{Cl}(\mathfrak{o}_{K_{f},S})[2]$ via
$\alpha\mapsto\mathfrak{a}$ such that $(\alpha)=\mathfrak{a}^{2}$ as ideals of
$\mathfrak{o}_{K_{f},S}$.

The corresponding map
$\mathcal{C}^{\mathrm{aff.}}_{f}(\mathfrak{o}_{K,S})-W\to\mathrm{Cl}(\mathfrak{o}_{K_{f},S})[2]$
is precisely (up to our choice of representatives) our map
$(x,y)\mapsto(\mathfrak{p}_{i})_{i}$. It therefore suffices to show that
$\\#|G^{\prime}|\ll 2^{\mathop{\mathrm{rank}}{J_{f}(K)}}\cdot O(1)^{d}$.
However $\\#|G|=\\#|(J_{f}[2])(K)|\cdot 2^{\mathop{\mathrm{rank}}{J_{f}(K)}}$
and $G^{\prime}\subseteq G$. ∎

## References.

  * [1] Levent Alpöge and Wei Ho, _The second moment of the number of integral points on elliptic curves is bounded_ , (2018), arXiv:1807.03761.
  * [2] Levent Hasan Ali Alpöge, _Points on Curves_ , ProQuest LLC, Ann Arbor, MI, 2020, Thesis (Ph.D.)–Princeton University. MR 4209988
  * [3] Manjul Bhargava and Benedict H. Gross, _The average size of the 2-Selmer group of Jacobians of hyperelliptic curves having a rational Weierstrass point_ , Automorphic representations and $L$-functions, Tata Inst. Fundam. Res. Stud. Math., vol. 22, Tata Inst. Fund. Res., Mumbai, 2013, pp. 23–91. MR 3156850
  * [4] Enrico Bombieri and Walter Gubler, _Heights in Diophantine geometry_ , New Mathematical Monographs, vol. 4, Cambridge University Press, Cambridge, 2006\. MR 2216774
  * [5] J.-H. Evertse, _On equations in $S$-units and the Thue-Mahler equation_, Invent. Math. 75 (1984), no. 3, 561–584. MR 735341
  * [6] J.-H. Evertse and J. H. Silverman, _Uniform bounds for the number of solutions to $Y^{n}=f(X)$_, Math. Proc. Cambridge Philos. Soc. 100 (1986), no. 2, 237–248. MR 848850
  * [7] Arul Shankar and Xiaoheng Wang, _Rational points on hyperelliptic curves having a marked non-Weierstrass point_ , Compos. Math. 154 (2018), no. 1, 188–222. MR 3719247
  * [8] X, _The integer solutions of the equation $y^{2}=ax^{n}+bx^{n-1}+\cdots+k$_, Journal of the London Mathematical Society s1-1 (1926), no. 2, 66–68, (Extract from a letter from C. L. Siegel to L. J. Mordell.).


11institutetext: NTT Social Informatics Laboratories, Japan, 11email:
<EMAIL_ADDRESS>22institutetext: Gunma University, Faculty of Informatics,
Japan

# On Rényi Differential Privacy
in Statistics-Based Synthetic Data Generation

Takayuki Miura 11 Toshiki Shibahara 11 Masanobu Kii 11 Atsunori Ichikawa 11
Juko Yamamoto 11 Koji Chida 22

###### Abstract

Privacy protection with synthetic data generation often uses differentially
private statistics and model parameters to quantitatively express theoretical
security. However, these methods do not take into account privacy protection
due to the randomness of data generation. In this paper, we theoretically
evaluate Rényi differential privacy of the randomness in data generation of a
synthetic data generation method that uses the mean vector and the covariance
matrix of an original dataset. Specifically, for a fixed $\alpha>1$, we show
the condition of $\varepsilon$ such that the synthetic data generation
satisfies $(\alpha,\varepsilon)$-Rényi differential privacy under a bounded
neighboring condition and an unbounded neighboring condition, respectively. In
particular, under the unbounded condition, when the size of the original
dataset and synthetic dataset is 10 million, the mechanism satisfies
$(4,0.576)$-Rényi differential privacy. We also show that when we translate it
into the traditional $(\varepsilon,\delta)$-differential privacy, the
mechanism satisfies $(4.00,10^{-10})$-differential privacy.

###### keywords:

synthetic data generation Rényi differential privacy privacy protection

##  1 Introduction

Personal data is expected to be utilized in various fields such as finance,
healthcare, and medicine, but sharing personal data collected by one
organization with another organization requires attention to individual
privacy. Traditional anonymization techniques such as $k$-anonymization [40]
and randomized response [43] have struggled to find a good trade-off between
utility and privacy for high-dimensional data [2]. In contrast, a synthetic
data generation technique has emerged as a privacy protection method that
preserves data utility even for high-dimensional data such as images and
tabular data with multi-attributes [6]. In synthetic data generation, values,
which we call generative parameters, are extracted from the original raw
dataset, and then synthetic data are generated randomly as shown in Fig. 1(a).
The synthetic data are the same format as the original data and statistically
similar to them. Typical generative parameters are statistics of original data
and trained parameters of deep neural networks [38, 26, 5, 15, 45, 46, 30, 17,
44, 24, 36]. After the synthetic data are generated, they are shared with
other organizations, but the generative parameters are typically discarded
without being disclosed.

To guarantee privacy protection theoretically, differential privacy [11] is
used as a standard framework. By adding randomness in generative parameter
calculation, the generative parameters become differentially private [29, 45,
1]. The post-processing property of differential privacy guarantees that
synthetic data generated with differentially private generative parameters
also satisfy differential privacy as shown in Fig. 1(b). Although the
synthetic data generated with non-differentially private generative parameters
have high utility, those with differentially private parameters are known to
have lower utility [41].

Figure 1: (a) Output = Only synthetic data: The generative parameters are
discarded after data are generated. We evaluate privacy protection by the
randomness in generation.
(b) Output = Generative parameters: By computing or training generative
parameters with intentional randomness, we obtain differentially private
generative parameters that also generate differentially private synthetic
data.

We address this problem by evaluating differential privacy of randomness in
data generation when using non-differentially private generative parameters.
As mentioned above, in the context of anonymization, the generative parameters
are often discarded without disclosing them to the public. When the output is
not generative parameters but only synthetic data, we can consider that it has
already been protected by the randomness even if the generative parameters are
not protected with differential privacy as shown in Fig. 1(a). If privacy
protection in data generation is quantitatively evaluated, theoretically
guaranteed synthetic data can be obtained without degrading the utility.
Moreover, by incorporating this result into traditional methods, we expect to
keep the same level of security with smaller additional randomness; that is,
we can obtain higher utility synthetic data.

In this paper, we regard a record as a $d$-dimensional vector and focus on a
synthetic data generation mechanism with the mean vector and the covariance
matrix of the original dataset shown in Fig. 2. We theoretically evaluate
Rényi differential privacy [34], which is a relaxed concept of differential
privacy, by randomness in generation for the method. We explicitly derive the
condition of $\varepsilon$ such that the synthetic data generation mechanism
satisfies $(\alpha,\varepsilon)$-Rényi differential privacy for a fixed
$\alpha>1$ under the unbounded neighboring condition (Theorem 3.1) and the
bounded neighboring condition (Corollary 3.2). Furthermore, we conduct a
numerical evaluation with reference to the Adult dataset [9] and compute
$\varepsilon$ concretely. We demonstrate that when the size of original
dataset is 10 million and the mechanism outputs data the same size as the
input dataset, it satisfies $(4,0.576)$-Rényi differential privacy under the
unbounded condition and $(4,2.307)$-Rényi differential privacy under the
bounded condition (Table 1). If they are translated into the traditional
$(\varepsilon,\delta)$-differential privacy, the mechanism satisfies
$(4.00,10^{-10})$ and $(7.88,10^{-10})$ differential privacy under the
unbounded and bounded condition, respectively (Table 2). These values are
mostly similar to ones used by Apple [4] and US Census [42].

##  2 Preliminaries

In this section, we introduce basic notations and concepts for later
discussion.

###  2.1 Notations

In this paper, we denote the determinant of a square matrix
$A\in\mathbb{R}^{d\times d}$ by $|A|:=\det A$. The transposes of a vector
$x\in\mathbb{R}^{d}$ and a matrix $A\in\mathbb{R}^{d_{1}\times d_{2}}$ are
denoted by ${}^{t}x$ and ${}^{t}A$. We assume that datasets are tabular but
all discussions can be applied to other datasets such as images since we
consider records as vectors. In a tabular dataset, a record is expressed as a
combination of several attribution values. Each attribution value is a
numerical value and normalized into a range $[-1,1]$. Thus, a record is
regarded as a vector $x\in[-1,1]^{d}$, and a dataset with $n$ records is
regarded as $D=\\{x_{i}\\}_{i=1,\ldots,n}\in[-1,1]^{d\times n}=:\mathcal{D}$.

###  2.2 Differential Privacy

In this subsection, we introduce $(\varepsilon,\delta)$-differential privacy
and $(\alpha,\varepsilon)$-Rényi differential privacy. First, we define
neighboring datasets.

######  Definition 2.1 (Neighboring datasets)

Datasets $D,D^{\prime}\in\mathcal{D}$ are neighboring datasets if $D$ and
$D^{\prime}$ are different only in one record. When datasets have a fixed size
$n$, we call the neighboring condition a bounded condition [23]. In this case,
neighboring means changing the value of exactly one record. When datasets have
no such restriction, we call the neighboring condition an unbounded condition
[23]. In this case, neighboring means either adding or removing one
record.111This difference is important for the sensitivity of queries. For
example, the sensitivity of the mean value query under the bounded condition
is twice as large as that under the unbounded condition.

$(\varepsilon,\delta)$-differential privacy [11] is defined as follows.

######  Definition 2.2 (differential privacy [11])

A randomized function $\mathcal{M}:\mathcal{D}\to\mathcal{Y}$ satisfies
$(\varepsilon,\delta)$-differential privacy ($(\varepsilon,\delta)$-DP) if for
any neighboring $D,D^{\prime}\in\mathcal{D}$ and $S\subset\mathcal{Y}$

$\Pr[\mathcal{M}(D)\in S]\leq e^{\varepsilon}\Pr[\mathcal{M}(D^{\prime})\in
S]+\delta.$

In particular, $\mathcal{M}$ satisfies $\varepsilon$-DP if it satisfies
$(\varepsilon,0)$-DP.

Next, we define Rényi divergence, which is necessary to define Rényi
differential privacy.

######  Definition 2.3 (Rényi Divergence)

Let $P,Q$ be probability distributions on $\mathbb{R}^{d}$. For $\alpha>1$,
the Rényi Divergence of order $\alpha$ is

$D_{\alpha}(P||Q):=\frac{1}{\alpha-1}\log\left(\int_{\mathbb{R}^{d}}P(x)^{\alpha}Q(x)^{1-\alpha}dx\right).$

######  Definition 2.4 (Rényi differential privacy [34])

For $\alpha>1$ and $\varepsilon>0$, a randomized function
$\mathcal{M}:\mathcal{D}\to\mathbb{R}^{d}$ satisfies
$(\alpha,\varepsilon)$-Rényi differential privacy ($(\alpha,\varepsilon)$-RDP)
if for neighboring datasets $D,D^{\prime}\in\mathcal{D}$,

$D_{\alpha}(\mathcal{M}(D)||\mathcal{M}(D^{\prime}))\leq\varepsilon.$

The smaller $\varepsilon$ is, the stronger the protection, and the larger
$\alpha$ is, the stronger the protection. To satisfy
$(\alpha,\varepsilon)$-RDP for any $\alpha$ is equivalent to $\varepsilon$-DP.

The composition theorem [12, 22] holds for Rényi differential privacy as well
as $(\varepsilon,\delta)$-DP. Furthermore, Rényi differential privacy can be
translated into $(\varepsilon,\delta)$-DP.

######  Proposition 2.5 (Composition of Rényi differential privacy [34])

Let $\mathcal{M}_{1}:\mathcal{D}\to\mathbb{R}^{d_{1}}$ be
$(\alpha,\varepsilon_{1})$-RDP and
$\mathcal{M}_{2}:\mathcal{D}\times\mathbb{R}^{d_{1}}\to\mathbb{R}^{d_{2}}$
$(\alpha,\varepsilon_{2})$-RDP. Then the mechanism
$\mathcal{M}:\mathcal{D}\to\mathbb{R}^{d_{1}}\times\mathbb{R}^{d_{2}}$ defined
as $\mathcal{M}(D)=(\mathcal{M}_{1}(D),\mathcal{M}_{2}(D,\mathcal{M}_{1}(D)))$
satisfies $(\alpha,\varepsilon_{1}+\varepsilon_{2})$-RDP.

######  Proposition 2.6 (Translation from $(\alpha,\varepsilon)$-RDP to
$(\varepsilon,\delta)$-DP [34])

If $\mathcal{M}$ is an $(\alpha,\varepsilon)$-RDP mechanism, it also satisfies
$(\varepsilon+\frac{\log\frac{1}{\delta}}{\alpha-1},\delta)$-DP for any
$0<\delta<1$.

By the following lemma, the result with the unbounded condition can be reduced
to the bounded condition.

######  Lemma 2.7 (Weak triangle inequality [34])

Let $P,Q,R$ be probability distributions on $\mathbb{R}^{d}$. For $\alpha>1$
and $\frac{1}{p}+\frac{1}{q}=1$, it holds

$D_{\alpha}(P||Q)\leq\frac{\alpha-\frac{1}{p}}{\alpha-1}D_{p\alpha}(P||R)+D_{q(\alpha-\frac{1}{p})}(R||Q).$

###  2.3 Synthetic Data Generation with Mean Vector and Covariance Matrix

In this paper, we focus on a simple synthetic data generation with the mean
vector and the covariance matrix of the original dataset
$\mathcal{M}_{G}:\mathcal{D}\to[-1,1]^{d}$ as shown in Fig. 2. This method is
identical to the Gaussian copula [38] with the assumption that the marginal
distributions are all normal distributions.

The mechanism $\mathcal{M}_{G}$ generates synthetic data as follows. First,
for dataset $D=\\{x_{i}\\}_{i=1,\ldots,n}\in\mathcal{D}$, the mean vector
$\mu\in\mathbb{R}^{d}$ and the covariance matrix $\Sigma\in\mathbb{R}^{d\times
d}$ are computed:

$\mu:=\frac{1}{n}\sum_{i=1}^{n}x_{i},\ \
\Sigma:=\frac{1}{n}\sum_{i=1}^{n}x{}^{t}x-\mu{}^{t}\mu.$

Next, a sample is drawn from a multivariate normal distribution
$\mathcal{N}(\mu,\Sigma)$, and its values are cut into the range $[-1,1]^{d}$.

We denote by $\mathcal{M}_{G}^{n}:\mathcal{D}\to[-1,1]^{d\times n}$ the
mechanism that simultaneously outputs $n$ records by $\mathcal{M}_{G}$. By
Proposition 2.5, we see that if $\mathcal{M}_{G}$ satisfies
$(\alpha,\varepsilon)$-RDP, then $\mathcal{M}_{G}^{n}$ also satisfies
$(\alpha,n\varepsilon)$-RDP.

Figure 2: Synthetic data generation algorithms $\mathcal{M}_{G}$ and
$\mathcal{M}_{G}^{n}$

###  2.4 Properties of Symmetric Matrices

We explain properties of symmetric matrices for the proof of the main theorem.

######  Definition 2.8 (symmetric matrix)

A square matrix $A$ is called symmetric if $A={}^{t}A$ holds.

######  Definition 2.9 (positive-definite, semi-positive definite)

For a $d$-dimensional symmetric matrix $A$, the following two conditions are
equivalent:
(1) For all $x\in\mathbb{R}^{d}\backslash\\{0\\}$, it holds ${}^{t}xAx>0$
$(\geq 0)$;
(2) All eigenvalues of $A$ are positive $($non-negative$)$.
If $A$ satisfies these conditions, then $A$ is called positive-definite
(positive semi-definite).

The following two lemmas are well-known facts [18].

###### Lemma 2.10

Let $A,B$ be positive-definite symmetric matrices. If $AB$ is symmetric, then
$AB$ is also positive-definite.

###### Lemma 2.11

Let $A$ be a positive-definite symmetric matrix. For an invertible matrix $S$
that is the same size as $A$, ${}^{t}SAS$ is also positive-definite.

###### Proposition 2.12

Let $A,B,C$ be positive-definite symmetric real matrices. If $ABC$ is
symmetric, then $ABC$ is also positive-definite.

######  Proof 2.1.

Set $D:=ABC=CBA$. Since $C$ is positive-definite, we can obtain the spectral
decomposition $C:=\sum_{i=1}^{d}\lambda_{i}\theta_{i}{}^{t}\theta_{i}$, where
$\lambda_{i}>0$ for all $i=1,\ldots,d$. Then we set
$S:=\sum_{i=1}^{d}\sqrt{\lambda_{i}}\theta_{i}{}^{t}\theta_{i}$. We see that
$S$ is symmetric and $C=S^{2}$ holds. We have

$S^{-1}DS^{-1}=S^{-1}AS^{-1}SBS=SBSS^{-1}AS^{-1}.$

By applying $S^{-1}AS^{-1}$ and $SBS$ to Lemma 2.10 and Lemma 2.11, we see
that $S^{-1}DS^{-1}$ is positive-definite. Thus, $D$ is also positive-
definite.

##  3 Main Theorem

In this paper, we prove the upper bound of $\varepsilon$ such that the
mechanism $\mathcal{M}_{G}$ satisfies $(\alpha,\varepsilon)$-Rényi
differential privacy for a fixed $\alpha$. We assume that all datasets have a
limitation for the minimum eigenvalue of their covariance matrices.
Specifically, for a fixed $\sigma>0$, we define the set of datasets as

$\mathcal{D}_{\sigma}:=\\{D\in[-1,1]^{n\times d}\mid z\in
S^{d-1},{}^{t}z\Sigma_{D}z\geq\sigma\\}.$

We also set $\tau:=\frac{4d}{\sigma}$.

First, the result under the unbounded condition is the following theorem. We
assume that the number of records in an original dataset is $n$ and that in
its neighboring dataset is $n+1$.

###### Theorem 3.1

Under the unbounded condition, let $\alpha>1$. We assume that

$\frac{n}{n+1}<\tau,\
\alpha<\min\left\\{n+1,\frac{n^{2}}{\tau(n+1)-n}\right\\}.$ (1)

Then, the synthetic data generation mechanism $\mathcal{M}_{G}$ satisfies
$(\alpha,\varepsilon_{\alpha})$-RDP for
$\varepsilon_{\alpha}:=\max\\{\varepsilon_{\alpha 1},\varepsilon_{\alpha
2}\\}$. Here,

$\displaystyle\varepsilon_{\alpha 1}$ $\displaystyle=$
$\displaystyle\frac{\alpha}{2}\cdot\frac{\tau}{(n+1)(n+1-\alpha)}+\frac{\alpha
d}{2(\alpha-1)}\log\frac{n}{n+1}-\frac{d}{2(\alpha-1)}\log\left(1-\frac{\alpha}{n+1}\right)$
$\displaystyle-\frac{1}{2(\alpha-1)}\log\min\left\\{1,\frac{1+\alpha\frac{n\tau}{(n+1)(n+1-\alpha)}}{(1+\frac{\tau}{n+1})^{\alpha}}\right\\}$

and

$\displaystyle\varepsilon_{\alpha 2}$ $\displaystyle=$
$\displaystyle\frac{\alpha}{2}\cdot\frac{\tau}{n(n+\alpha)-\alpha(n+1)\tau}+\frac{\alpha
d}{2(\alpha-1)}\log\frac{n+1}{n}-\frac{d}{2(\alpha-1)}\log\left(1+\frac{\alpha}{n}\right)$
$\displaystyle-\frac{1}{2(\alpha-1)}\log\min\Biggl{\\{}1,\frac{1-\frac{\alpha(n+1)\tau}{(n+\alpha)n}}{(1-\frac{\tau}{n})^{\alpha}}\Biggr{\\}}.$

Next, under the bounded condition, we obtain the following statement as a
corollary of Theorem 3.1.

###### Corollary 3.2

Under the bounded condition, let $\alpha>1$. We set

$c:=\min\left\\{n+1,\frac{n^{2}}{\tau(n+1)-n}\right\\}$

and assume that

$\alpha<\frac{c^{2}}{2c-1}.$ (2)

Then, the synthetic data generation mechanism $\mathcal{M}_{G}$ satisfies
$(\alpha,\varepsilon_{\alpha})$-RDP for the following $\varepsilon$:

$\varepsilon_{\alpha}=\inf_{\frac{c-1}{c-\alpha}<p<\frac{c}{\alpha}}\frac{\alpha-\frac{1}{p}}{\alpha-1}\varepsilon\left(p\alpha,n\right)+\varepsilon\left(\frac{p\alpha-1}{p-1},n+1\right),$
(3)

where $\varepsilon(\alpha,n)$ is the $\varepsilon$ in Theorem 3.1.

######  Proof 3.1.

For any neighboring datasets $D_{1},D_{2}$ under the bounded condition, there
exists a dataset $D_{3}$ such that $D_{1}$ and $D_{3}$ are neighboring and
$D_{2}$ and $D_{3}$ are neighboring under the unbounded condition. Then, to
obtain Equation (3), we use Lemma 2.7. Here, the weak triangle inequality
holds for all $p>1$, and the following condition is necessary:

$\max\left\\{p\alpha,\frac{p\alpha-1}{p-1}\right\\}<c.$

This is equivalent to

$\frac{c-1}{c-\alpha}<p<\frac{c}{\alpha}.$

The existence of $p$ is equivalent to Equation (2).

##  4 Proof of Theorem 3.1

In this section, we prove Theorem 3.1. The following proposition is essential.

######  Proposition 4.1 (Gil et al. [16])

Let $\alpha>1$ and $\mathcal{N}(\mu_{1},\Sigma_{1})$,
$\mathcal{N}(\mu_{2},\Sigma_{2})$ be multivariate normal distributions. If a
matrix

$T_{\alpha}:=\alpha\Sigma_{1}^{-1}+(1-\alpha)\Sigma_{2}^{-1}$

is positive-definite, then it holds

$\displaystyle
D_{\alpha}(\mathcal{N}(\mu_{1},\Sigma_{1})||\mathcal{N}(\mu_{2},\Sigma_{2}))$
$\displaystyle=\frac{\alpha}{2}{}^{t}(\mu_{1}-\mu_{2})\Sigma_{\alpha}^{-1}(\mu_{1}-\mu_{2})-\frac{1}{2(\alpha-1)}\log\frac{|\Sigma_{\alpha}|}{|{\Sigma_{1}}|^{1-\alpha}|{\Sigma_{2}}|^{\alpha}},$

where $\Sigma_{\alpha}:=(1-\alpha)\Sigma_{1}+\alpha\Sigma_{2}$.

For neighboring datasets $D_{1},D_{2}\in\mathcal{D}_{\sigma}$, we set the mean
vectors as $\mu_{1},\mu_{2}$ and the covariance matrices as
$\Sigma_{1},\Sigma_{2}$. If
$D_{\alpha}(\mathcal{N}(\mu_{1},\Sigma_{1})||\mathcal{N}(\mu_{2},\Sigma_{2}))\leq\varepsilon$,
the mechanism $\mathcal{M}_{G}$ satisfies $(\alpha,\varepsilon)$-RDP. Here we
set

$L_{1}:={}^{t}(\mu_{1}-\mu_{2})\Sigma_{\alpha}^{-1}(\mu_{1}-\mu_{2}),\ \
L_{2}:=\frac{|\Sigma_{\alpha}|}{|{\Sigma_{1}}|^{1-\alpha}|{\Sigma_{2}}|^{\alpha}}.$

Then we see

$D_{\alpha}(\mathcal{N}(\mu_{1},\Sigma_{1})||\mathcal{N}(\mu_{2},\Sigma_{2}))=\frac{\alpha}{2}L_{1}-\frac{1}{2(\alpha-1)}\log
L_{2}.$

Thus, an upper bound $\varepsilon$ is described by the maximum of $L_{1}$ and
the minimum of $L_{2}$. The outline of proof is as follows. First, by using
the different record, we represent the difference between mean vectors and the
difference between covariance matrices (Lemma 4.2). Next, we determine the
positive-definiteness of $T_{\alpha}$ (Lemma 4.3). Finally, we compute the
upper bound of $L_{1}$ (Lemma 4.4) and the lower bound of $L_{2}$ (Lemma 4.5).

Set $\\#D_{1}=n$ and $\\#D_{2}=n+s$, where $s=1$ when we ”add” a record and
$s=-1$ when we ”remove” a record. The common records are denoted by
$x_{1},\ldots,x_{n}\in[-1,1]^{d}$ and the different record by
$x\in[-1,1]^{d}$. We set each mean vector as $\mu_{1},\mu_{2}$ and covariance
matrix as $\Sigma_{1},\Sigma_{2}$. We also denote by $\sigma_{min}$ the
minimum eigenvalue of $\Sigma_{1}$. Note that $\sigma_{min}\geq\sigma$ by the
assumption.

######  Lemma 4.2 (Representations of difference)

The following equations hold:

$\mu_{d}:=\mu_{2}-\mu_{1}=\frac{s}{n+s}x-\frac{s}{n(n+s)}\sum_{i=1}^{n}x_{i},$
$X:=\Sigma_{2}-\frac{n}{n+s}\Sigma_{1}=\frac{ns}{(n+s)^{2}}(x-\mu_{1}){}^{t}(x-\mu_{1}).$

######  Proof 4.1.

It is easily shown by calculation.

The rank of $X$ is one. $X$ is semi-positive definite when $s=1$ and semi-
negative definite when $s=-1$.

######  Lemma 4.3 (Positive-definiteness of $T_{\alpha}$)

If the following two inequalities hold, $T_{\alpha}$ is positive-definite:

$\frac{n-1}{n}<\tau,\ \ \alpha<\min\left\\{n+1,\ \frac{(n-1)^{2}}{\tau
n-(n-1)}\right\\}.$ (4)

######  Proof 4.2.

Since
$T_{\alpha}=\Sigma_{1}\Sigma_{\alpha}\Sigma_{2}=\Sigma_{2}\Sigma_{\alpha}\Sigma_{1}$,
by Lemma 2.12, the positive-definiteness of $T_{\alpha}$ is reduced to the
positive-definiteness of $\Sigma_{\alpha}$. By Lemma 4.2, we have

$\Sigma_{\alpha}=(1-\alpha)\Sigma_{1}+\alpha\left(\frac{n}{n+s}\Sigma_{1}+X\right)=\left(1-\frac{s\alpha}{n+s}\right)\Sigma_{1}+\alpha
X.$

When $s=1$, since $\Sigma_{1}$ is positive-definite and $X$ is semi-positive
definite, it is enough to be $\alpha<n+1$. We consider the case when $s=-1$.
For an arbitrary vector $z\in\mathbb{R}^{d}$ whose norm is one, we seek a
condition where the minimum of ${}^{t}z\Sigma_{\alpha}z$ is positive. Here we
can consider that the vector $x-\mu_{1}$ is contained in a ball with a radius
$2\sqrt{d}$. Thus, we obtain the minimum when the following two conditions
hold:

  * •

$z$ is parallel to the eigenvector of the minimum eigenvalue $\sigma_{min}$ of
$\Sigma_{1}$;

  * •

$x-\mu_{1}$ is parallel to $z$.

Hence we see that $\Sigma_{\alpha}$ is positive-definite if

$\displaystyle{}^{t}z\Sigma_{\alpha}z$ $\displaystyle=$
$\displaystyle\left(1+\frac{\alpha}{n-1}\right)\sigma_{min}-\alpha\frac{n}{(n-1)^{2}}4d$
$\displaystyle=$
$\displaystyle\sigma_{min}-\alpha\cdot\frac{4dn-(n-1)\sigma_{min}}{(n-1)^{2}}$
$\displaystyle\geq$
$\displaystyle\sigma-\alpha\cdot\frac{4dn-(n-1)\sigma}{(n-1)^{2}}>0.$

When the inequalities in Equation (4) hold, this inequality also holds.

######  Lemma 4.4 (Upper bound of $L_{1}$)

If $s=1$, then we have

$L_{1}\leq\frac{\tau}{(n+1)(n+1-\alpha)},$

and if $s=-1$, then we have

$L_{1}\leq\frac{\tau}{(n-1)(n-1+\alpha)-\alpha n\tau}.$

######  Proof 4.3.

Now $\mu_{d}$ is contained in a ball with a radius $\frac{2\sqrt{d}}{n+s}$ by
Lemma 4.2 and $\Sigma_{\alpha}$ is positive-definite by Lemma 4.3. By
multiplying the reciprocal of the minimum of ${}^{t}z\Sigma_{\alpha}z$ for a
unit vector $z\in\mathbb{R}^{d}$ by $\frac{4d}{(n+s)^{2}}$, we can obtain the
maximum of ${}^{t}\mu_{d}\Sigma_{\alpha}^{-1}\mu_{d}$. Here, we see

${}^{t}z\Sigma_{\alpha}z={}^{t}z\left(1-\frac{s\alpha}{n+s}\right)\Sigma_{1}z+\frac{s\alpha
n}{(n+s)^{2}}({}^{t}z(x-\mu_{1}))^{2}.$

Hence when $s=1$, the minimum is

$\left(1-\frac{\alpha}{n+1}\right)\sigma_{min}.$

When $s=-1$, since $x-\mu_{1}$ is contained in a ball with a radius
$2\sqrt{d}$, the minimum is

$\left(1+\frac{\alpha}{n-1}\right)\sigma_{min}-\frac{\alpha n}{(n-1)^{2}}\cdot
4d.$

Thus, we obtain the inequality.

######  Lemma 4.5 (Lower bound of $L_{2}$)

It holds

$L_{2}\geq\frac{(1-\frac{s\alpha}{n+s})^{d}}{(\frac{n}{n+s})^{\alpha
d}}\cdot\min\left\\{1,\frac{1+\frac{\alpha
ns\tau}{(n+s-s\alpha)(n+s)}}{(1+\frac{s\tau}{n+s})^{\alpha}}\right\\}.$

######  Proof 4.4.

We see that

$L_{2}:=\frac{|(1-\frac{s\alpha}{n+s})\Sigma_{1}+\alpha
X|}{|{\Sigma_{1}}|^{1-\alpha}|{\frac{n}{n+s}\Sigma_{1}+X}|^{\alpha}}=\frac{(1-\frac{s\alpha}{n+s})^{d}|I+\frac{n+s}{n+s-s\alpha}\alpha\Sigma_{1}^{-1}X|}{(\frac{n}{n+s})^{\alpha
d}|{I+\frac{n+s}{n}\Sigma_{1}^{-1}X}|^{\alpha}}.$

Since the rank of $X$ is one and $\Sigma_{1}^{-1}$ is invertible, the rank of
$\Sigma_{1}^{-1}X$ is also one. Thus, there is only one non-zero eigenvalue,
and it is set as $\lambda$. We also set
$A:=(1-\frac{s\alpha}{n+s})^{d}/(\frac{n}{n+s})^{\alpha d}$. Since the other
eigenvalues are all zero, we see

$L_{2}=\frac{1+\frac{n+s}{n+s-s\alpha}\alpha\lambda}{(1+\frac{n+s}{n}\lambda)^{\alpha}}\cdot
A.$

By differentiating this equation with respect to $\lambda$, we obtain

$\frac{\partial
L_{2}}{\partial\lambda}=\alpha(\alpha-1)\frac{n+s}{n(n+s-s\alpha)}\cdot\frac{s-(n+s)\lambda}{(1+\frac{n+s}{n}\lambda)^{\alpha+1}}\cdot
A.$

We see that $\frac{\partial L_{2}}{\partial\lambda}>0$ when
$\frac{s}{n+s}<\lambda$ and $\frac{\partial L_{2}}{\partial\lambda}<0$ when
$\frac{s}{n+s}>\lambda$. Hence the minimum of $L_{2}$ is obtained at the edges
of the range of $\lambda$.

Next, we will find the range of $\lambda$, which is the only one non-zero
eigenvalue of $\Sigma_{1}^{-1}X$. Since $\Sigma_{1}$ is positive-definite, we
can obtain the spectral decomposition of $\Sigma_{1}$:

$\Sigma_{1}=\sum_{i=1}^{d}\sigma_{i}p_{i}{}^{t}p_{i},$

where $\sigma_{1},\ldots,\sigma_{d}$ are the eigenvalues of $\Sigma_{1}$ and
$p_{1},\ldots,p_{d}$ are their eigenvectors whose norms are one. Since
$p_{1},\ldots,p_{d}$ is a basis of $\mathbb{R}^{d}$, there exist
$r_{1},\ldots,r_{d}\in\mathbb{R}$ such that

$x-\mu_{1}=\sum_{i=1}^{d}r_{i}p_{i}.$

Squaring both sides, we obtain a condition $4d\geq\sum_{i=1}^{d}r_{i}^{2}>0$.
Set $e_{1}:=\sum_{i=1}^{d}\frac{r_{i}}{\sigma_{i}}p_{i}$. Then we have

$\displaystyle\Sigma_{1}^{-1}Xe_{1}$ $\displaystyle=$
$\displaystyle\Sigma_{1}^{-1}\frac{ns}{(n+s)^{2}}\sum_{i=1}^{d}r_{i}p_{i}((x-\mu_{1})\cdot
e_{1})$ $\displaystyle=$ $\displaystyle\frac{ns}{(n+s)^{2}}((x-\mu_{1})\cdot
e_{1})e_{1}$ $\displaystyle=$
$\displaystyle\frac{ns}{(n+s)^{2}}(\sum_{i=1}^{d}\frac{r_{i}^{2}}{\sigma_{i}})e_{1}.$

Thus, we have
$\lambda=\frac{ns}{(n+s)^{2}}\sum_{i=1}^{d}\frac{r_{i}^{2}}{\sigma_{i}}$.
Therefore, we have
$0<\lambda\leq\frac{4dn}{(n+1)^{2}\sigma_{min}}\leq\frac{4dn}{(n+1)^{2}\sigma}$
when $s=1$, and
$-\frac{4dn}{(n-1)^{2}\sigma}\leq-\frac{4dn}{(n-1)^{2}\sigma_{min}}\leq\lambda<0$
when $s=-1$.

##  5 Numerical Evaluations

In Theorem 3.1 and Corollary 3.2, we obtain the concrete upper bounds. Thus,
in this section, we compute the value $\varepsilon$ concretely and observe the
results.

###  5.1 Setting of Numerical Parameters

We set $d=6$, $\sigma=0.01$ since the number of numerical attributions in
Adult Dataset [9] is six and the minimum eigenvalue for the data normalized
into $[-1,1]$ is $\sigma_{min}=0.01$.

###  5.2 Relation between $\alpha$ and $\varepsilon$

Figure 3: $\alpha$-$\varepsilon$ curve ($d=6$, $\sigma=0.01$) : Vertical axis
is logarithmic scale. The curves are drawn for each of the four sample sizes
$n$.

The relations between $\alpha$ and $\varepsilon$ are shown in Fig. 3
($\alpha$-$\varepsilon$ curves). For all curves, $\varepsilon$ is
monotonically increasing with respect to $\alpha$. We also see that as $n$
increases exponentially, $\varepsilon$ becomes smaller at equal intervals on a
logarithmic scale. In particular, if $n=10^{4}$, the condition in Equation (1)
is

$\alpha<c:=\min\left\\{n+1,\frac{n^{2}}{\tau(n+1)-n}\right\\}\fallingdotseq
4.1679$

and the condition in Equation (2) is

$\alpha<\frac{c^{2}}{2c-1}\fallingdotseq 2.3680.$

Thus, the curves stop at these values.

###  5.3 The Case Input and Output are the Same Sizes

For $\alpha=4$, the values of $\varepsilon$ for which the mechanism
$\mathcal{M}_{G}^{n}$ satisfies $(\alpha,\varepsilon)$-RDP are shown in Table
1. By the composition theorem in Proposition 2.5, the values of $\varepsilon$
are ones in Theorem 3.1 and Corollary 3.2 multiplied by $n$. We can show that
values of $\varepsilon$ are within a practical range when $n\geq 10^{6}$ under
both conditions. In particular, under the unbounded condition,
$\varepsilon=0.5764$ when $n=10^{7}$, which is very small. We also see that
$\varepsilon$’s under the unbounded condition are four times larger than those
under the bounded condition.

Table 1: Values of $\varepsilon$ in the case that input and output are the
same size $n$. ($\alpha=4,d=6,\sigma=0.01$)

$n$ $10^{4}$ $10^{5}$ $10^{6}$ $10^{7}$ Unbounded $\varepsilon$ 3535.17
62.5859 5.8064 0.5764 Bounded $\varepsilon$ - 266.7349 23.3577 2.3071

###  5.4 Translation into $(\varepsilon,\delta)$-DP

Table 2: Values of $\varepsilon$ in $(\varepsilon,\delta)$-DP with under
unbounded condition and bounded condition ($d=6,\sigma=0.01$)

Case : $n=10^{6}$ Unbounded condition Bounded condition $\delta$ $10^{-2}$
$10^{-5}$ $10^{-10}$ $10^{-15}$ $10^{-20}$ $10^{-2}$ $10^{-5}$ $10^{-10}$
$10^{-15}$ $10^{-20}$ $\alpha=2$ 7.499 14.407 25.920 37.433 48.946 16.209
23.116 34.629 46.142 57.655 $\alpha=4$ 7.341 9.644 13.482 17.319 21.157 24.893
27.195 31.033 34.871 38.708 $\alpha=7$ 10.978 12.130 14.048 15.967 17.886
42.046 43.198 45.116 47.035 48.954 $\alpha=10$ 15.170 15.937 17.217 18.496
19.775 60.070 60.838 62.117 63.396 64.675 $\alpha=20$ 30.046 30.410 31.016
31.622 32.228 123.482 123.846 124.452 125.058 125.663 $\alpha=30$ 45.624
45.863 46.260 46.657 47.054 191.710 191.948 192.345 192.742 193.139 Case :
$n=10^{7}$ Unbounded condition Bounded condition $\delta$ $10^{-2}$ $10^{-5}$
$10^{-10}$ $10^{-15}$ $10^{-20}$ $10^{-2}$ $10^{-5}$ $10^{-10}$ $10^{-15}$
$10^{-20}$ $\alpha=2$ 4.893 11.801 23.314 34.827 46.340 5.758 12.666 24.179
35.692 47.205 $\alpha=4$ 2.112 4.414 8.252 12.089 15.927 3.842 6.145 9.982
13.820 17.658 $\alpha=7$ 1.777 2.928 4.847 6.766 8.685 4.809 5.960 7.879 9.798
11.717 $\alpha=10$ 1.954 2.722 4.001 5.280 6.559 6.291 7.058 8.337 9.617
10.896 $\alpha=20$ 3.132 3.496 4.102 4.708 5.313 11.838 12.201 12.807 13.413
14.019 $\alpha=30$ 4.500 4.739 5.136 5.533 5.930 17.608 17.846 18.243 18.640
19.037

By Proposition 2.6, we see that $(\alpha,\varepsilon)$-RDP can be translated
into $(\varepsilon,\delta)$-DP.

The values translated into $(\varepsilon,\delta)$-DP under the unbounded
condition are shown in Table 2. When $\delta=0.01$, we see that
$\varepsilon=7.341$ for $n=10^{6}$ and $\varepsilon=1.777$ for $n=10^{7}$.
When $\delta=10^{-10}$, we also see that $\varepsilon=13.482$ for $n=10^{6}$
and $\varepsilon=4.001$ for $n=10^{7}$. These values are reasonable [4, 42].

The results under the bounded condition are shown in Table 2. When
$\delta=0.01$, we see that $\varepsilon=16.209$ for $n=10^{6}$ and
$\varepsilon=3.842$ for $n=10^{7}$. When $\delta=10^{-10}$, we also see that
$\varepsilon=31.033$ for $n=10^{6}$ and $\varepsilon=7.879$ for $n=10^{7}$.

The values of $\varepsilon$ under the bounded condition are about twice as
large as those under the unbounded condition.

###  5.5 Summary of Results

To sum up the results of numerical evaluations, we see the following:

  * •

We see that $\varepsilon$ is monotonically increasing with respect to
$\alpha$. This result is intuitive.

  * •

If $n$ increases exponentially, the curve becomes smaller at equal intervals
on a logarithmic scale.

  * •

When $n=10^{4}$, a range where $\alpha$ satisfies the assumption of being very
narrow. When $n=10^{7}$, the value of $\varepsilon$ is practical.

##  6 Related Work

In this section, we describe the related work and mention the difference from
our result.

###  6.1 Differentially Private Synthetic Data Generation

In synthetic data generation, the post-processing property of differential
privacy guarantees that synthetic data generated from differentially private
generative parameters also satisfy differential privacy as shown in Fig. 1(b).
Methods to generage differentially private synthetic data for tabular data are
classified to two types.

The first type is also called a “select-measure-generate” scheme [29].
Statistics and (conditional) probability distributions are used as the
generative parameters. Typical statistics are mean vectors and covariance
matrices of original datasets. In particular, synthetic data generation with
copulas has been researched actively [38, 26, 5, 15]. To learn conditional
distributions, graphical models such as Bayesian networks have been applied to
synthetic data generation [45, 46, 30, 31].

In the second type, generative models with deep neural networks are used to
generate synthetic data. The model parameters trained with the original data
are regarded as the generative parameters. By training deep neural networks
with differentially private stochastic gradient descent (DP-SGD) [1], we
obtain differentially private model parameters. Methods based on generative
adversarial networks (GAN) such as CTGAN [44], DPCTGAN [13], CTAB-GAN [47],
and CTAB-GAN+ [48], are widely used. A method based on diffusion model such as
TabDDPM [25] has also attracted attention recently.

In both types of approaches, generative parameters are computed by various
differentially private mechanisms [1, 32] (Fig. 1(b)). In contrast, we
evaluate differential privacy of randomness in data generation when using non-
differentially private generative parameters.

###  6.2 Privacy Attacks against Synthetic Data Generation

Many methods empirically evaluate the privacy protection of synthetic data
generations from attack success rates of membership inference attacks [37] and
attribute inference attacks [14]. Most of them assume that an adversary has
access to the target trained model such as GAN [8, 19, 20] and diffusion
models [7, 21, 10, 28].

On the other hands, there are several methods where an adversary only has
access to output synthetic data. Stadler et al. [39] discussed membership
inference attacks and attribute inference attacks for tabular data in such
setting, and Oprisanu et al. [35] applied such attacks to genomic data.
Annamalai et al. [3] conducted attribute inference with linear reconstruction
in this setting.

Although these studies and ours share a common perspective in that they focus
on the privacy protection of generated synthetic data alone, these studies
differ from ours in that they experimentally evaluate synthetic data
generation from an attack perspective. In contrast, our perspective is to
prove Rényi differential privacy theoretically.

###  6.3 Differential Privacy of Randomness in Synthetic Data Generation

To the best of our knowledge, only Lin et al. [27] have evaluated the privacy
protection by the randomness in outputs of synthetic data generations. They
theoretically evaluated probabilistic differential privacy [33] of GAN-sampled
data. However, the concretely evaluated bound is hard to compute since it
needs a GAN’s generalization error. In addition, they assume that training
datasets are far larger than the number of model parameters. Thus, their main
contribution is to give the theoretical bound, but we cannot compute the bound
as a concrete numerical value.

In contrast, although we focus on only a simple synthetic data generation, we
give the concretely computable bound.

##  7 Conclusion

In this paper, we evaluated the privacy protection due to the randomness of
synthetic data generation without adding intentional randomness. We proved
Rényi differential privacy of a synthetic data generation with a mean vector
and covariance matrix (Theorem 3.1, Corollary 3.2). We also conducted
numerical evaluations using the Adult dataset as a model case. Concretely, we
demonstrated that the mechanism $\mathcal{M}_{G}^{n}$ satisfies
$(4,0.576)$-RDP under the unbounded condition and $(4,2.307)$-RDP under the
bounded condition (Table 1). If they are translated into
$(\varepsilon,\delta)$-DP, $\mathcal{M}_{G}^{n}$ satisfies
$(\varepsilon,\delta)$-DP for a practical $\varepsilon$ (Table 2). In future
work, we will apply our evaluation method to more advanced synthetic data
generation algorithms.

## References

  * [1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang.  Deep learning with differential privacy.  In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308–318, 2016.
  * [2] Charu C Aggarwal.  On k-anonymity and the curse of dimensionality.  In VLDB, volume 5, pages 901–909, 2005.
  * [3] Meenatchi Sundaram Muthu Selva Annamalai, Andrea Gadotti, and Luc Rocher.  A linear reconstruction approach for attribute inference attacks against synthetic data.  arXiv preprint arXiv:2301.10053, 2023.
  * [4] Apple.  Apple differential privacy technical overview.  https://www.apple.com/privacy/docs/Differential_Privacy_Overview.pdf.  Accessed: 2023-02-22.
  * [5] Hassan Jameel Asghar, Ming Ding, Thierry Rakotoarivelo, Sirine Mrabet, and Dali Kaafar.  Differentially private release of datasets using gaussian copula.  Journal of Privacy and Confidentiality, 10(2), 2020.
  * [6] Sam Bond-Taylor, Adam Leach, Yang Long, and Chris G Willcocks.  Deep generative modelling: A comparative review of vaes, gans, normalizing flows, energy-based and autoregressive models.  IEEE transactions on pattern analysis and machine intelligence, 2021\.
  * [7] Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace.  Extracting training data from diffusion models.  arXiv preprint arXiv:2301.13188, 2023.
  * [8] Dingfan Chen, Ning Yu, Yang Zhang, and Mario Fritz.  Gan-leaks: A taxonomy of membership inference attacks against generative models.  In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, pages 343–362, 2020.
  * [9] Dheeru Dua and Casey Graff.  UCI machine learning repository, 2017.
  * [10] Jinhao Duan, Fei Kong, Shiqi Wang, Xiaoshuang Shi, and Kaidi Xu.  Are diffusion models vulnerable to membership inference attacks?  arXiv preprint arXiv:2302.01316, 2023.
  * [11] Cynthia Dwork.  Differential privacy.  In International Colloquium on Automata, Languages, and Programming, pages 1–12. Springer, 2006.
  * [12] Cynthia Dwork, Aaron Roth, et al.  The algorithmic foundations of differential privacy.  Found. Trends Theor. Comput. Sci., 9(3-4):211–407, 2014.
  * [13] Mei Ling Fang, Devendra Singh Dhami, and Kristian Kersting.  Dp-ctgan: Differentially private medical data generation using ctgans.  In Artificial Intelligence in Medicine: 20th International Conference on Artificial Intelligence in Medicine, AIME 2022, Halifax, NS, Canada, June 14–17, 2022, Proceedings, pages 178–188. Springer, 2022.
  * [14] Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart.  Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing.  In 23rd $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 14), pages 17–32, 2014.
  * [15] Sébastien Gambs, Frédéric Ladouceur, Antoine Laurent, and Alexandre Roy-Gaumond.  Growing synthetic data through differentially-private vine copulas.  Proceedings on Privacy Enhancing Technologies, 2021(3):122–141, 2021.
  * [16] Manuel Gil, Fady Alajaji, and Tamas Linder.  Rényi divergence measures for commonly used univariate continuous distributions.  Information Sciences, 249:124–131, 2013.
  * [17] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  Advances in neural information processing systems, 27, 2014.
  * [18] David A Harville.  Matrix algebra from a statistician’s perspective, 1998.
  * [19] Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro.  LOGAN: membership inference attacks against generative models.  Proceedings of Privacy Enhancing Technologies, 2019(1):133–152, 2019.
  * [20] Aoting Hu, Renjie Xie, Zhigang Lu, Aiqun Hu, and Minhui Xue.  Tablegan-mca: Evaluating membership collisions of gan-synthesized tabular data releasing.  In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 2096–2112, 2021.
  * [21] Hailong Hu and Jun Pang.  Membership inference of diffusion models.  arXiv preprint arXiv:2301.09956, 2023.
  * [22] Peter Kairouz, Sewoong Oh, and Pramod Viswanath.  The composition theorem for differential privacy.  In International conference on machine learning, pages 1376–1385. PMLR, 2015.
  * [23] Daniel Kifer and Ashwin Machanavajjhala.  No free lunch in data privacy.  In Proceedings of the 2011 ACM SIGMOD International Conference on Management of data, pages 193–204, 2011.
  * [24] Diederik P. Kingma and Max Welling.  Auto-encoding variational bayes.  In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
  * [25] Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko.  Tabddpm: Modelling tabular data with diffusion models.  arXiv preprint arXiv:2209.15421, 2022.
  * [26] Haoran Li, Li Xiong, Lifan Zhang, and Xiaoqian Jiang.  Dpsynthesizer: Differentially private data synthesizer for privacy preserving data sharing.  In Proceedings of the VLDB Endowment International Conference on Very Large Data Bases, volume 7, page 1677. NIH Public Access, 2014.
  * [27] Zinan Lin, Vyas Sekar, and Giulia Fanti.  On the privacy properties of gan-generated samples.  In International Conference on Artificial Intelligence and Statistics, pages 1522–1530. PMLR, 2021.
  * [28] Tomoya Matsumoto, Takayuki Miura, and Naoto Yanai.  Membership inference attacks against diffusion models.  arXiv preprint arXiv:2302.03262, 2023.
  * [29] Ryan McKenna, Gerome Miklau, and Daniel Sheldon.  Winning the nist contest: A scalable and general approach to differentially private synthetic data.  arXiv preprint arXiv:2108.04978, 2021.
  * [30] Ryan McKenna, Brett Mullins, Daniel Sheldon, and Gerome Miklau.  Aim: An adaptive and iterative mechanism for differentially private synthetic data.  arXiv preprint arXiv:2201.12677, 2022.
  * [31] Ryan McKenna, Daniel Sheldon, and Gerome Miklau.  Graphical-model based estimation and inference for differential privacy.  In International Conference on Machine Learning, pages 4435–4444. PMLR, 2019.
  * [32] Frank McSherry and Kunal Talwar.  Mechanism design via differential privacy.  In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07), pages 94–103. IEEE, 2007.
  * [33] Sebastian Meiser.  Approximate and probabilistic differential privacy definitions.  Cryptology ePrint Archive, 2018.
  * [34] Ilya Mironov.  Rényi differential privacy.  In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pages 263–275. IEEE, 2017.
  * [35] Bristena Oprisanu, Georgi Ganev, and Emiliano De Cristofaro.  On utility and privacy in synthetic genomic data.  arXiv preprint arXiv:2102.03314, 2021.
  * [36] Danilo Rezende and Shakir Mohamed.  Variational inference with normalizing flows.  In International conference on machine learning, pages 1530–1538. PMLR, 2015.
  * [37] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov.  Membership inference attacks against machine learning models.  In 2017 IEEE Symposium on Security and Privacy (SP), pages 3–18. IEEE, 2017.
  * [38] M Sklar.  Fonctions de repartition an dimensions et leurs marges.  Publ. inst. statist. univ. Paris, 8:229–231, 1959.
  * [39] Theresa Stadler, Bristena Oprisanu, and Carmela Troncoso.  Synthetic data–anonymisation groundhog day.  In 31st USENIX Security Symposium (USENIX Security 22), pages 1451–1468, 2022.
  * [40] Latanya Sweeney.  k-anonymity: A model for protecting privacy.  International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(05):557–570, 2002.
  * [41] Yuchao Tao, Ryan McKenna, Michael Hay, Ashwin Machanavajjhala, and Gerome Miklau.  Benchmarking differentially private synthetic data generation algorithms.  arXiv preprint arXiv:2112.09238, 2021.
  * [42] United States Census Bureau.  Census bureau sets key parameters to protect privacy in 2020 census results.  https://www.census.gov/newsroom/press-releases/2021/2020-census-key-parameters.html.  Accessed: 2023-02-22.
  * [43] Stanley L Warner.  Randomized response: A survey technique for eliminating evasive answer bias.  Journal of the American Statistical Association, 60(309):63–69, 1965.
  * [44] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni.  Modeling tabular data using conditional gan.  In Advances in Neural Information Processing Systems, 2019.
  * [45] Jun Zhang, Graham Cormode, Cecilia M. Procopiuc, Divesh Srivastava, and Xiaokui Xiao.  Privbayes: Private data release via bayesian networks.  ACM Trans. Database Syst., 42(4), October 2017.
  * [46] Zhikun Zhang, Tianhao Wang, Jean Honorio, Ninghui Li, Michael Backes, Shibo He, Jiming Chen, and Yang Zhang.  Privsyn: Differentially private data synthesis.  2021\.
  * [47] Zilong Zhao, Aditya Kunar, Robert Birke, and Lydia Y Chen.  Ctab-gan: Effective table data synthesizing.  In Asian Conference on Machine Learning, pages 97–112. PMLR, 2021\.
  * [48] Zilong Zhao, Aditya Kunar, Robert Birke, and Lydia Y Chen.  Ctab-gan+: Enhancing tabular data synthesis.  arXiv preprint arXiv:2204.00401, 2022.