1 The paper poses mostly subjective points which are themselves largely not novel or are flawed in themselves. It starts with the emphasized sentence on page 1 attributed to Jacovi et al which I will call "Opinion A": "As an explanation method, the evaluation criteria of attribution methods should be how accurately it reflects the true reasoning process of the model (faithfulness), not how convincing it is to humans (plausibility)" This point is debatable. Whether attributions should be purely faithful to model behaviour or offer human-interpretability is a decision to be made in the definition of an attribution method or its motivation. Neither option is wrong as long as the method is applied to its intended use case. Further, the noted choice of faithfulness has implications on some of the "logic traps" described subsequently. I elaborate on each in the comments section below.
Significant positioning fixes need to be implemented in this work starting with Opinion A. Either arguments for faithfulness need to be made or the paper needs to restrict itself to attributions whose primary concern is faithfulness. The latter option would have to be written from the perspective that faithfulness is an apriori goal.
The novelty of arguments being made needs to be addressed as well. I find that the cores of the "logic traps" are all known points and are often well considered in attribution, evaluation, and robustness work. Presently the paper does not demonstrate that (logic traps) "in existing evaluation methods have been ignored for a long time". If there is a gap in some segment of the community, a survey or systemization paper where applicable would be more appropriate.
2 Arguments and experiments with regard to "model reasoning process" are vague in key definitions and thus non-convincing. Specifically, no definition of "model reasoning process" is given.
Experiment outlined in Figure 7 argues that extractor's bits are equivalent to "model reasoning process" but this is arguable. The extractor can derive the same bits using different means or different bits using very similar means. While I do not disagree with the main points regarding model reasoning process and robustness, I do not think the experiments demonstrate them.
Detailed comments regarding Weakness 1): * Logic Trap 1: The point being made is obvious: "The decision-making process of neural networks is not equal to the decision-making process of humans."
The example Experiment 1 incorporates not a single attribution method. Is it presumed that all attributions will be wrong as there is no human explanation possible? Can we also say that any human explanation would be wrong for the same reason? If there is no ground truth, how can it be wrong for an attribution method to say anything?
Regardless, no-one *expects* models to fully replicate the reasoning of a human. Replicating human behaviour may be useful to gauge human-interpretability of explanations but this is precluded by Opinion A. Finally, it is well understood that what is correct to a human may not be correct in a model as per faithfulness vs. explainability discussion which is had alongside attribution evaluations in literature and in Opinion A (some examples from cited works below).
* Logic Trap 2: The second point made is a roundabout way of saying that ablation orders are varied: "Using an attribution method as the ground truth to evaluate the target attribution method."
The authors are rightly pointing out that numerous forms of attribution evaluation based on ablation or reconstruction inputs have been used to motivate attribution methods. Here the opposite conclusion to Opinion A is helpful. Expecting humans to interpret an attribution one way may lead to one ablation order whereas another form of interpretation may lead to another. There is no single correct metric because there is no single interpretation. This point has been made at least in [a].
* Logic Trap 3: The third point is known: "The change in attribution scores maybe because the model reasoning process is really changed rather than the attribution method is unreliable."
Ghorbani et al. (2019) note in their concluding discussion that fragility of attribution is reflective of fragility of the model (and that their attacks have not "broken" an attribution).
Likewise Alvarez-Melis et al. (2018) point out that the non-robust artifacts in attributions they discover are actually reasonable if faithfulness to model is the only goal of an attribution.
Thus Logic Trap 3 does not seem to add anything beyond Opinion A. * 3.1 -- "Attacking attribution methods by replacing the target model."
This section seems to be pointing out that attribution methods have a problem of domain faithfulness in that they often internally apply a model to inputs that they have not been trained on or don't expect to operate reliably on.
* 3.2 -- "Should We Use Attributions Methods in a Black-Box Way?"
The paper argues that black-box is not a worthwhile goal. This is again highly subjective as there are several reasons, despite presented arguments, to prefer black-box methods, like 1) having explanations of the semantics of what a model is modeling instead of an explanation tainted by how the model is implemented, 2) black-box means model agnostic, hence can apply to any model structure, 3) some scenarios just do not have access to the model internals.
Other comments: - The term "logic traps" is not define or explained. Dictionaries and reference materials equate them to logical fallacies which I'm unsure is the intended meaning here.
- The term "erasure-based" is used in the intro but later the term "ablation" is used.
- Typo/word choice near "with none word overlap".
- Typo/grammar near "there are works (...) disprove" - Suggestion of 3.3, point 2 is unclear. Adversarial examples can be highly confident and I suspect means of incorporating confidence can themselves be subverted adversarially.
- Figure 6 is not useful. I presume the paths are supposed to be indicative of model behaviour but as mentioned in earlier comments, defining it is a crucial problem in explanation work. The Figure is suggestive of it being a triviality.
- Several points in the paper use phrases like "A lot of works ..." or "most existing methods ...".
I think it would be more appropriate to list the works or methods instead of noting the relative abundance.
- There are some inconsistencies in the notation for AOPC and the k parameter with potential related typos in its definition.
- Potential grammar issue near "can be seen as an attribution".
- Grammar issue near "results are mainly depend" - Grammar issue near "achieve 86.4% accuracy" - Where is footnote 1?
- In Figure 4, the resulting attributions for the target word "good" in LOO, Marg are not presented or indicated by color.
References from comments: [a] - Wang et al. "Interpreting Interpretations: Organizing Attribution Methods by Criteria".
* 3.1 -- "Attacking attribution methods by replacing the target model." This section seems to be pointing out that attribution methods have a problem of domain faithfulness in that they often internally apply a model to inputs that they have not been trained on or don't expect to operate reliably on.