W1) The authors should compare their approach (methodologically as well as experimentally) to other concept-based explanations for high-dimensional data such as (Kim et al., 2018), (Ghorbani et al., 2019) and (Goyal et al., 2019). The related work claims that (Kim et al., 2018) requires large sets of annotated data. I disagree. (Kim et al., 2018) only requires a few images describing the concept you want to measure the importance of. This is significantly less than the number of annotations required in the image-to-image translation experiment in the paper where the complete dataset needs to be annotated. In addition, (Kim et al., 2018) allows the flexibility to consider any given semantic concept for explanation while the proposed approach is limited either to semantic concepts captured by frequency information, or to semantic concepts automatically discovered by representation learning, or to concepts annotated in the complete dataset. (Ghorbani et al., 2019) also overcomes the issue of needing annotations by discovering useful concepts from the data itself. What advantages does the proposed approach offer over these existing methods?
W2) Faithfulness of the explanations with the pretrained classifier. The methods of disentangled representation and image-to-image translation require training another network to learn a lower-dimensional representation. This runs the risk of encoding some biases of its own. If we find some concerns with the explanations, we cannot infer if the concerns are with the trained classifier or the newly trained network, potentially making the explanations useless.
W3) In the 2-module approach proposed in the paper, the second module can theoretically be any explainability approach for low-dimensional data. What is the reason that the authors decide to use Shapely instead of other works such as (Breiman, 2001) or (Ribeiro et al., 2016)?
W4) Among the three ways of transforming the high-dimensional data to low-dimensional latent space, what criteria should be used by a user to decide which method to use? Or, in other words, what are the advantages and disadvantages of each of these methods which might make them more or less suitable for certain tasks/datasets/applications?
W5) The paper uses the phrase “human-interpretable explainability”. What other type of explainability could be possible if it’s not human-interpretable? I think the paper might benefit with more precise definitions of these terms in the paper.
References mentioned above which are not present in the main paper:
(Ghorbani et al., 2019) Amirata Ghorbani, James Wexler, James Zou, Been Kim. Towards Automatic Concept-based Explanations. NeurIPS 2019.
(Goyal et al., 2019) Yash Goyal, Amir Feder, Uri Shalit, Been Kim. Explaining Classifiers with Causal Concept Effect (CaCE). ArXiv 2019.
—————————————————————————————————————————————————————————————— ——————————————————————————————————————————————————————————————
Update after rebuttal: I thank the authors for their responses to all my questions. However, I believe that these answers need to be justified experimentally in order for the paper’s contributions to be significant for acceptance. In particular, I still have two major concerns. 1) the faithfulness of the proposed approach. I think that the authors’ answer that their method is less at risk to biases than other methods needs to be demonstrated with at least a simple experiment. 2) Shapely values over other methods. I think the authors need to back up their argument for using Shapely value explanations over other methods by comparing experimentally with other methods such as CaCE or even raw gradients. In addition, I think the paper would benefit a lot by including a significant discussion on the advantages and disadvantages of each of the three ways of transforming the high-dimensional data to low-dimensional latent space which might make them more or less suitable for certain tasks/datasets/applications. Because of these concerns, I am keeping my original rating.
2) Shapely values over other methods. I think the authors need to back up their argument for using Shapely value explanations over other methods by comparing experimentally with other methods such as CaCE or even raw gradients. In addition, I think the paper would benefit a lot by including a significant discussion on the advantages and disadvantages of each of the three ways of transforming the high-dimensional data to low-dimensional latent space which might make them more or less suitable for certain tasks/datasets/applications. Because of these concerns, I am keeping my original rating.