of this approach (e.g., it does not take into account language compositionally). I appreciate that the authors used different methods to extract influential objects: Human attention (in line with previous works), text explanation (to rely on another modality), and question parsing (to remove the need of extra annotation). As a complementary analysis, I would have compared object sets (Jaccard Distance) which are extracted with visual cues and text description. Indeed, the VQA-X dataset contains both information for each question/answer pairs. The method is more or less correctly explained. The training details seems complete and allow for reproducibility. The authors do not provide code source although they mentioned it in the reproducibility checklist. The empirical results are quite convincing and the necessary baselines and ablation studies are correctly provided. The formatting is simple and clear! It would have been perfect to provide the error bar as the number of experiments remains low (and over a small number of epochs) The cherry on the cake would be to run similar experiments on VQAv1 / VQA-CP1? To increase the impact of the paper, I would recommend extending the setting to either dense image captioning, or question answering (if possible). I feel that the discussion section raise some excellent points: - I really like table 4, that clearly show that the method perform as expected (I would have add HINT for being exhaustive) - the ablation study is convincing But, a lot of open-questions are still left open and could have been discussed. For instance, I would have appreciated a more in-depth analysis of model errors. What about the model complexity? Why only reweighting L_{crit}. How does evolve L_crit and L_infl at training time? On a more general note, I think the overall writing and paper architecture can be greatly improved. For instance, - the introduction and related work can be partially merged and summarized. - 4.2 starts by providing high-level intuition while 4.1 does not. - Training details incorporate some result discussion Generic questions (sorted by impact): - What is the impact of |I|, do you have the performance ration according to the number of top |I| influential objects - Eq1 is a modified version of GardCAM, however, the modifications are not highlighted (neither explained). For instance, why did the authors remove the ReLU - Even if the weight sensitivity in equation 5 is well motivated, it is not supported by previous works. Thus, did you perform an ablation study? It would be very have been nice in the discussion section. - What is the actual computation cost of the two losses? What is the relative additional time required? +5%, +20%, +200%? - As you used heuristics to retrieve influential objects, did you try to estimate the impact of false negatives in the loss. - How did you pick 0.6 for glove embedding similarity? Did you perform k-cross-validation? What is the potential impact - Have you tried other influential loss (Eq3)? For instance, replacing the min with a mean or NDCG? Remarks: - I would use a different notation for SV(.,.,.) as it is not symmetric. For instance SV_{a}(v_i || v_j) would avoid confusion (I am using KL notation here) - Non-formal expression should be avoided: Ex: "l32 What's worse" - The references section is full of format inconsistencies. Besides, some papers are published with proceeding but are referred to arxiv papers. - 3.1 introduces non-important notation, e.g., function h(.) or f(.) that are never used in the paper. - Several subsections could be gathered together, or define as a paragraph: 2.1/2.2/2.3 ; 5.1/5.2/5.3, etc. It would have save space for more experiments Conclusion: The paper introduces two losses to better tie influential objects and potential answers. The method is convincing, and the experimental results are diverse and good. However, I still think that the paper requires further polishing to improve the readability. I would also advocate providing more element to support the proposed method and to analyze the strengths and weaknesses. Although the current experiences are quite convincing, I would advocate adding more analysis to definitely conclude the efficiency of the method. ---------------------------------- The rebuttal was clearly written and insightful. it answered most of my questions, and the authors demonstrate their ability to update the paper accordingly. Therefore, I am happy to increase my score, and accept the paper
- What is the actual computation cost of the two losses? What is the relative additional time required? +5%, +20%, +200%?