weakness of this paper, in my opinion, is that it lacks strong empirical results. In particular, from what I can tell, none of the human-optimized models actually beat models that are optimized by some easily-computable proxy for interpretability (e.g., mean path length in a decision tree). Indeed, the bulk of the paper talks about optimizing for interpretability proxies, though the title and rhetoric focus on the human-in-the-loop component. What, then, do we get out of human-optimization? The authors argue in line 249 that âHuman response times suggest preferences between proxies that rank models differently.â This statement comes from Figure 3a, but the variance in the results shown in that figure seems so large that itâs hard to make any conclusive claims. Do you have some measure of statistical significance? Looking at Fig 4a, it seems like models that optimize for path length do as well as models that optimize for the number of non-zero features. Despite the above shortcomings, I think that this paper contains interesting ideas and is a solid contribution to human-in-the-loop ML research. It is hard to criticize the paper too much for not being able to show state-of-the-art results on such a challenging task, and this paper certainly stands out in terms of trying to involve humans in model building in a meaningful way. Other comments: 1) Is the likelihood formulation necessary? It complicates notation, and itâs not obvious to me that the SILF likelihood or a prior based on mean-response-time is particularly probabilistically meaningful. I think it might simplify the discourse to just say that we want accuracy to be above some threshold, and then we also want mean response time to be high, without dealing with all of the probabilistic notions. (e.g., as a reader, I didnât understand the SILF measure or its motivation). 2) Does mean response time take into account correctness? A model that users can simulate quickly but incorrectly seems like it should be counted as uninterpretable. 3) Also, why is HIS proportional to the *negative* inverse of mean response time? 4) The local explanations part of the paper is the least convincing. It seems like a model that is locally explainable can be globally opaque; and none of the human results seem to have been run on the CoverType dataset anyway? Given that the paper is bursting at the seams in terms of content (and overflowing into the supplement :), perhaps leaving that part out might help the overall organization. 5) On local explanations, line 154 says: âWe note that these local models will only be nontrivial if the data point x is in the vicinity of a decision boundary; if not, we will not succeed in fitting a local model.â Why is this true? Why canât we measure differences in probabilities, e.g., âif feature 1 were higher, the model would be less confident in class yâ?
4) The local explanations part of the paper is the least convincing. It seems like a model that is locally explainable can be globally opaque; and none of the human results seem to have been run on the CoverType dataset anyway? Given that the paper is bursting at the seams in terms of content (and overflowing into the supplement :), perhaps leaving that part out might help the overall organization.