Update README.md

README.md

@@ -1,3 +1,29 @@
 ---
 license: gpl-3.0
+datasets:
+- yahma/alpaca-cleaned
+language:
+- en
+pipeline_tag: text2text-generation
+tags:
+- alpaca
+- llama
+- chat
 ---
+
+This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The checkpoint is the output of instruction following fine-tuning process with the following settings on 8xA100(40G) DGX system.
+- Dataset: [cleaned-up Alpaca dataset](https://github.com/gururise/AlpacaDataCleaned) up to 04/06/23
+- Training script: borrowed from the official [Alpaca-LoRA](https://github.com/tloen/alpaca-lora) implementation
+- Training script:
+```shell
+python finetune.py \
+    --base_model='decapoda-research/llama-13b-hf' \
+    --num_epochs=10 \
+    --cutoff_len=512 \
+    --group_by_length \
+    --output_dir='./lora-alpaca' \
+    --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
+    --lora_r=16 \
+    --batch_size=... \
+    --micro_batch_size=...
+```