The application of FERMI is not obviously a large improvement over its introduction in (Lowy+ 2021): we want to optimize a weird fairness-constrained loss, so we instead optimize an upper bound on it, which admits a stochastic convergence analysis, and also handles non-binary classification. The added contribution here is, in the paper's phrasing, "a careful analysis of how the Gaussian noises [necessary for DP-SGD] propagate through the optimization trajectories". I don't have much feel for what constitutes an "interesting" convergence analysis, but the conceptual novelty here is unclear, and the introduction is a bit slippery about what is novel and what is borrowed from the FERMI paper.
The paper also struggles to explain its technical contributions in terms between a very high level summary and a long, opaque theorem statement. I suggest changing the focus of the paper to 1) reduce, relegate to the appendix, or eliminate the discussion of demographic parity (an extremely coarse notion of fairness that, IMO, the fairness literature needs to move past, and has only been discussed this long because it's very simple), which takes up over a page of the main body without meaningfully adding to the story told by the equalized odds results alone, 2) extending the discussion of how Theorem 3.1 works and what it accomplishes (the current statement is a blizzard of notation with little explanation -- I still don't know what W is doing), along with Theorem 3.2, and 3) extending the equalized odds results to more datasets (why are Parkinsons and Retired Adult results only reported for demographic parity? it seems like equalized odds should also apply here, and an empirical story built on 2 datasets seems thin). I think 2) would help provide a clearer explanation of the paper's improvement over (Lowy+ 2021) and 3) would make a stronger empirical case separate from the convergence analysis.
Other questions/comments:
I'd appreciate a table in the appendix attempting to concisely explain all of the relevant variables -- by my count, Theorem B.1 has well over a dozen.
Why is Tran 2021b a single point where the other methods have curves? More generally, perhaps I missed the explanation of this in the text, but what is varied to generate the curves?
As far as I can tell, the paper does not discuss the tightness of the upper bound offered by ERMI, nor does it explicitly write out the expression for equalized odds. This makes it hard to contextualize the convergence guarantee in terms of the underlying loss we actually want to optimize.
Figure 4 "priavte"
3) extending the equalized odds results to more datasets (why are Parkinsons and Retired Adult results only reported for demographic parity? it seems like equalized odds should also apply here, and an empirical story built on 2 datasets seems thin). I think