<p>Yup, that is the current problem with Capsule Networks. It works well with MNIST because of the dataset's simplicity. All you require is some edges and blobs to be detected in order to classify each data. For more complex datasets, naively stacking the capsules and hoping it to perform well does not work. However, works are being performed currently to tweak the current CapsNet architecture to make it perform better than now. When CNN was developed during the days, it also had this same problem. It took many years for CNN to be what is it now. </p>

<p>Refer to this paper if you want to know the performance of CapsNet on different datasets : <a href="https://arxiv.org/abs/1712.03480" rel="nofollow noreferrer">https://arxiv.org/abs/1712.03480</a></p>

<p>Earlier I mentioned there are works being done to improve CapsNet. However, there are already some works that has been done so far. You can refer to these:</p>

<p><a href="http://openaccess.thecvf.com/content_CVPR_2019/papers/Rajasegaran_DeepCaps_Going_Deeper_With_Capsule_Networks_CVPR_2019_paper.pdf" rel="nofollow noreferrer">http://openaccess.thecvf.com/content_CVPR_2019/papers/Rajasegaran_DeepCaps_Going_Deeper_With_Capsule_Networks_CVPR_2019_paper.pdf</a></p>

<p><a href="http://proceedings.mlr.press/v97/jeong19b/jeong19b.pdf" rel="nofollow noreferrer">http://proceedings.mlr.press/v97/jeong19b/jeong19b.pdf</a></p>

<p>Bear in mind that the time it takes to train CapsNet is very much higher than CNN. Therefore, it is not easy to test out these architectures.</p>
Is there any Capsnet Implementation on CIFAR10 that achieve better accuracy than common CNN?
<p>I implement Capsule Network by EM-Routing based-on Sara Sabour &amp; Hinton's article, It works great on MNIST dataset and some other grayscale dataset as same as MNIST such as Hoda (Persian/Arabic Digits) but When I tried on CIFAR10 the accuracy was unbelievable low.</p>