In search for more accurate predictive models, we customize capsule networks for the learning to diagnose problem. We also propose Spectral Capsule Networks, a novel variation of capsule networks, that converge faster than capsule network with EM routing. Spectral capsule networks  consist of spatial coincidence filters that detect entities based on the alignment of extracted features on a one-dimensional linear subspace. Experiments on a public benchmark learning to diagnose dataset not only shows the success of capsule networks on this task, but also confirm the faster convergence of the spectral capsule networks. The potential for improvement of the quality of care via artificial intelligence has led to significant advances in predictive modeling for healthcare BID11 BID1 BID16 BID0 BID12 BID19 BID15 . For accurate prediction, the models in healthcare need to not only identify risk factors, but also distill the complex and hierarchal temporal interactions among symptoms, conditions, and medications.It has been argued that traditional deep neural networks might not be efficient in capturing the hierarchical structure of the entities in the images BID13 BID2 BID4 BID23 . They argue that networks that preserve variations in the input perform superior to those that drop variations (equivariant vs. invariant architectures), as the upper layer can have access to the spatial relationship of the entities detected by the lower layers. In particular, in capsule networks BID7 BID17 BID8 ) the capsules are designed to have both activation and pose components, where the latter is responsible for preserving the variations in the detected entity.In this work, we first develop a version of capsule networks with EM routing (EM-Capsules) and show that it can accurately predict diagnoses. We observe that EM-Capsules converge slowly in our dataset and are sensitive to the selection of hyperparameters such as learning rate. To address these issues, we propose Spectral Capsule Networks (S-Capsules) that are also spatial coincidence filters, similar to EM-Capsules. In contrast to EM-Capsules, S-Capsules measure the coincidence as the degree of alignment of the votes from below capsules in a one-dimensional linear subspace, rather than centralized clusters. In S-Capsules, the variation (pose) component is the normal vector of a linear subspace that preserves most of the variance in the votes coming from below capsules and the activation component is computed based on the ratio of preserved variance.Our experiments on a benchmark learning to diagnose task BID5 ) defined on the publicly available MIMIC-III dataset BID9 highlight the success of capsule networks. Moreover, we confirm that the proposed S-Capsules converge faster than EM-Capsules. Finally, we show that the elements of the S-Capsules' variation (pose) vector are significantly correlated with the commonly used hand-engineered features. In this work, we customized capsule networks with EM routing BID8 ) for learning to diagnose task. We also proposed spectral capsule networks to improve stability and convergence speed of the capsule networks. Similar to EM-Capsules, S-Capsules are also spatial coincidence filters and look for agreement of the below capsules. However, spectral capsules measure the agreement by the amount of alignment in a linear subspace, rather than a centralized cluster. Setting aside the attention mechanism in EM-Capsules, the connection between S-Capsules and EM-Capsules is analogous to the connection between Gaussian Mixture Models and Principal Component Analysis. This analogy suggests why S-Capsules are more robust during the training. Our preliminary results confirm the superior convergence speed of the proposed S-Capsule network and preservation of variations in the data in its pose vectors.
A new capsule network that converges faster on our healthcare benchmark experiments. Presents a variant of capsule networks that instead of using EM routing employs a linear subspace spanned by the dominant eigenvector on the weighted votes matrix from the previous capsule. The paper proposes an improved routing method, which employs tools of eigendecomposition to find capsule activation and pose.