1. Writing should be seriously improved. It is really tedious to get through the paper. Often terms are not defined properly and the reader really has to rely on the context to understand used terms. This is not possible always. See a list of writing issues below:
* I find parts of Fig. 2 unclear. The caption should describe the key features of the model. Why is there a connection between Decoder and External classifier but no arrow. What is it supposed to mean?
* line 224 Should describe $d, \kappa$ clearly and what they denote. Is the feature encoding coming from an external network, the classifier $f$ itself etc. What is the distance metric? Euclidean distance?
* Implicit/Explicit classifier should be described clearly when describing CoLa-DCE in Sec. 4. What is their input, output domains, what are their roles etc. I guess one of them is $f$. I assume they originate from prior literature but they also seem central to your method so need clear concise description.
* line 249: Should define $\mathcal{N}$ or state its the set of natural numbers (can be confused with normal distribution as you use it in Eq. 1).
* line 259 Why is the external classifier modelled as p(x|y) (which should be for diffusion model). Wouldn't the classifier be modelled as p(y|x)?
* The notations $h, g$ almost appear out of the blue. What are their input, output domains. It is the same issue with $\delta$ but it at least has some brief description.
* line 260. You provide absolutely no explanation of notations about concepts, how they are represented, what the binary constraints denote exactly. It is difficult to understand $\lambda_1, ..., \lambda_k$ and $\theta_1, ..., \theta_k$. Are the lambda's just subset of natural numbers from 1 to K? Your notation for $\theta$ also does not seem consistent. For starters, line 236 has it going from 0 to k while at other places it is 1 to k. More importantly, Eq. 7 makes it seem like $\theta$'s denote subset of indices but they are supposed to be binary masks. I am not sure what are these representations exactly, beyond the basic idea that they control which concepts to condition on.
* There are two terms for datasets (line 219, 226) $X', \hat{X}$. What is the difference between the two? Is the reference data classification training/validation dataset and the other test data?
* line 222 "As the model perception of the data shall be represented, the class predictions of the model are used to determine class affiliation." Really odd phrasing, please make it more clear.
* line 319-320 "Using the intermediate ... high flip ratios" I am not sure how you are drawing this conclusion from Tab. 1. Please elaborate on this how you come to this conclusion.
* line 469 What is $attr$ supposed to denote exactly. The attribution map for a particular feature/concept? If yes, why are you computing absolute magnitude for relative alignment? Is it the Frobenius norm of the difference of attributions? It should be described more clearly.
* Please explain more clearly what the "confidence" metric is? Is it the difference between classifier's probability for the initially predicted class before and after the modifications?
* line 295 mentions l2 norm between original and counterfactual image. Was it supposed to be a metric in Tab. 1?
2. The quantitative metrics of CoLa-DCE seem weak. LDCE seems to clearly outperform CoLA-DCE on "Flip-ratio" and "Confidence" metrics while being close on FID.
* The notations $h, g$ almost appear out of the blue. What are their input, output domains. It is the same issue with $\delta$ but it at least has some brief description.