of their presented approach, showing promising results on MNIST variants and drawbacks on more realistic tasks like Cifar10. Clarity
The paper is well-organized, but some details are confusing or unclear.
- Discuss difference to original capsule work.
- Why are 1-3 refinement iterations chosen, what happens after more iterations?
- How many iterations were necessary for Cifar10?
- Compare the computational cost of baseline and capsules, as well as the cost of the refinement steps.
- What happens when the invariant sum over the coupled prediction vectors in equation (2) and the associated non-linearity are replaced by a simple linear layer and standard non-linearity?
- line 135: "We test using a single model with no ... data augmentation". A couple of lines before, the authors mention they do moderately augment data with shifts. Why do shifts improve performance, given that the authors claim capsules are designed to be robust to such variations? Originality
The presented work is original as it introduces a new routing principle for capsule networks.
However, novelty with respect to classical capsules should be discussed more clearly.
Relevant related work, either dealing with separating filter response into magnitude and orientation, estimating keypoints or doing locally adaptive filtering, as opposed to global normalization of STNs: https://arxiv.org/abs/1701.01833 https://arxiv.org/abs/1703.06211 https://arxiv.org/abs/1612.04642 https://arxiv.org/abs/1706.00598 https://arxiv.org/abs/1605.01224 https://arxiv.org/abs/1605.09673 Significance
The presented results are not very strong and it is hard to say how significant the findings are, as the authors do not thoroughly investigate more interesting domains than digits.
Performance on MNIST is a very limited metric, given that:
i) Saturated at this point
ii) Scattering has shown that local invariance wrt deformation, translation and rotation is enough to achieve very good performance
iii) It lacks ambiguity and many other properties that make natural images so challenging, especially the assumption of only one entity per location becomes questionable under clutter
The robustness results on affine and overlapping MNIST are promising, but should be validated on more realistic tasks with more challenging local statistics.
It would be great if the authors would provide the reader with insight into strengths and weaknesses on more realistic problems.
Some suggestions:
i) More thorough evaluation + visualisations on Cifar10. The results seem weak for now, but might shed some light on failure modes and work to be accomplished by follow-up papers
ii) Check if affine robustness holds for Cifar10 as well to similar degree, this would change my vote on the paper
iii) The iterative Tagger (Graeff et al.) might give some inspiration for additional experiments with more natural ambiguity and should be discussed in related work as well
A strong analysis on the drawbacks of the presented method and open problems would make this a very useful paper, but in its current form, it is hard to tell what the suggested method achieves beyond being potentially more representationally efficient and robust on variants of MNIST.
- What happens when the invariant sum over the coupled prediction vectors in equation (2) and the associated non-linearity are replaced by a simple linear layer and standard non-linearity?