Update README.md

README.md

@@ -1,3 +1,21 @@
 ---
 license: apache-2.0
 ---
+# Skimformer
+
+## Model description
+Skimformer is a two-stage Transformer that replaces self-attention with Skim-Attention, a self-attention module that computes attention solely based on the 2D positions of tokens in the page. The model adopts a two-step approach: first, the skim-attention scores are computed once and only once using layout information alone; then, these attentions are used in every layer of a text-based Transformer encoder. For more details, please refer to our paper: 
+
+[Skim-Attention: Learning to Focus via Document Layout](https://arxiv.org/abs/2109.01078)
+Laura Nguyen, Thomas Scialom, Jacopo Staiano, Benjamin Piwowarski, [EMNLP 2021](https://2021.emnlp.org/papers)
+
+## Citation
+
+``` latex
+@article{nguyen2021skimattention,
+    title={Skim-Attention: Learning to Focus via Document Layout}, 
+    author={Laura Nguyen and Thomas Scialom and Jacopo Staiano and Benjamin Piwowarski},
+    journal={arXiv preprint arXiv:2109.01078}
+    year={2021},
+}
+```