/Questions:
I only have a few minor points:
1.) For equation (7), does treating | u − l |
as the length require the bins to be equally spaced? I don't think this is stated.
2.) It may be good to briefly mention the negligible computational cost of CHR (which is in the appendix) in the main paper to help motivate the method. A rough example of some run-times in the experiments may also be useful for readers looking to apply the method.
3.) Just a few typographical/communication points:
I found Section 2.2 slightly difficult to read, as the notation gets a little heavy. This may not be necessary, but the authors could consider presenting the nested intervals without randomization (e.g. after Line 119), with the randomization in the Appendix, as it is not needed in Theorem 2. This would give more room for intuitive discussions, related to my next point.
It may be helpful to introduce some intuition on the conformity score in equation (12) and why we need the sets to be nested for readers unfamiliar with previous work, perhaps at the start of Section 2.3.
Line 113: ϵ
is mentioned here before it is defined
Line 188: 'increased' instead of 'increase' ##################################################################### Overall:
This paper is an interesting extension of previous work, and the provided asymptotic justifications of attaining oracle width and conditional coverage is useful. The method is also general and can empirically provide better average widths and conditional coverage than other methods, particularly under skewed data, making it useful in practice. ##################################################################### References:
Romano, Y., Sesia, M., & Candes, E. (2020). Classification with Valid and Adaptive Coverage. Advances in Neural Information Processing Systems, 33, 3581-3591.
Gupta, C., Kuchibhotla, A. K., & Ramdas, A. K. (2019). Nested conformal prediction and quantile out-of-bag ensemble methods. arXiv preprint arXiv:1910.10562.
The authors have described the limitations of their method - in particular their method does not control for upper and lower miscoverage, and they provide alternative recommendations.
2.) It may be good to briefly mention the negligible computational cost of CHR (which is in the appendix) in the main paper to help motivate the method. A rough example of some run-times in the experiments may also be useful for readers looking to apply the method.