<s>[INST] Distance between the product of marginal distributions and the joint distribution Given a joint distribution $P(A,B,C)$, we can compute various marginal distributions. Now suppose:
\begin{align}
P1(A,B,C) &= P(A) P(B) P(C)  \\
P2(A,B,C) &= P(A,B) P(C)  \\
P3(A,B,C) &= P(A,B,C)
\end{align}
Is it true that $d(P1,P3) \geq d(P2,P3)$ where $d$ is the total variation distance?
In other words, is it provable that $P(A,B) P(C)$ is a better approximation of $P(A,B,C)$ than $P(A) P(B) P(C)$ in terms of the total variation distance? Intuitively I think it's true but could not find out a proof.
 [/INST] I just find the following counter-example. Suppose $A,B,C$ are discrete variables. $A,B$ can each take two values while $C$ can take three values. 
The joint distribution $P(A,B,C)$ is:
\begin{array}{cccc}
A & B & C & P(A,B,C) \\
1 & 1 & 1 & 0.1/3 \\
1 & 1 & 2 & 0.25/3 \\
1 & 1 & 3 & 0.25/3 \\
1 & 2 & 1 & 0.4/3 \\
1 & 2 & 2 & 0.25/3 \\
1 & 2 & 3 & 0.25/3 \\
2 & 1 & 1 & 0.4/3 \\
2 & 1 & 2 & 0.25/3 \\
2 & 1 & 3 & 0.25/3 \\
2 & 2 & 1 & 0.1/3 \\
2 & 2 & 2 & 0.25/3 \\
2 & 2 & 3 & 0.25/3 \\
\end{array}
So the marginal distribution $P(A,B)$ is:
\begin{array}{ccc}
A & B & P(A,B) \\
1 & 1 & 0.2 \\
1 & 2 & 0.3 \\
2 & 1 & 0.3 \\
2 & 2 & 0.2 \\
\end{array}
The marginal distributions $P(A), P(B)$ and $P(C)$ are uniform.
So we can compute that:
\begin{align}
d(P1,P3) &= 0.1 \\
d(P2,P3) &= 0.4/3
\end{align}
</s>