Create README.md

README.md

@@ -0,0 +1,15 @@
+## Pretrained models for the paper *Scaling up Masked Diffusion Models on Text*
+
+**Scaling law experiments**: We provided all pre-trained models in the *ar_safetensors* and *mdm_safetensors* folders. 
+For instance, the checkpoint `mdm-1028M-1600e18.safetensors` represents an MDM model with 1,028 million non-embedding 
+parameters and 1,600e18 training FLOPs. Similarly, the checkpoint `mdm-170M-100e18-rsl-0.01.safetensors` indicates 
+an MDM model with 170 million non-embedding parameters, 100e18 training FLOPs, and 1% of the dataset subjected 
+to random sequence lengths during pretraining.
+
+**Conditional generation**: please see the *sharegpt_safetensors* folder.
+
+**Reverse curse**: please see the *reverse_safetensors* folder
+
+For all models, we provide models in `.pth` and `.safetensors` formats.
+
+