Update README.md

README.md

@@ -7,4 +7,110 @@ language:
 - ko
 base_model:
 - Qwen/Qwen2-7B-Instruct
----
+---
+
+## Quickstart
+
+Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+device = "cuda" # the device to load the model onto
+
+model = AutoModelForCausalLM.from_pretrained(
+    "Qwen/Qwen2-7B-Instruct",
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("CarrotAI/Rabbit-Ko-15B-Instruct")
+
+prompt = "Give me a short introduction to large language model."
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(device)
+
+generated_ids = model.generate(
+    model_inputs.input_ids,
+    max_new_tokens=512
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+```
+
+### Processing Long Texts
+1. **Install vLLM**: You can install vLLM by running the following command.
+
+```bash
+pip install "vllm>=0.4.3"
+```
+
+Or you can install vLLM from [source](https://github.com/vllm-project/vllm/).
+
+2. **Configure Model Settings**: After downloading the model weights, modify the `config.json` file by including the below snippet:
+    ```json
+        {
+            "architectures": [
+                "Qwen2ForCausalLM"
+            ],
+            // ...
+            "vocab_size": 152064,
+
+            // adding the following snippets
+            "rope_scaling": {
+                "factor": 4.0,
+                "original_max_position_embeddings": 32768,
+                "type": "yarn"
+            }
+        }
+    ```
+    This snippet enable YARN to support longer contexts.
+
+3. **Model Deployment**: Utilize vLLM to deploy your model. For instance, you can set up an openAI-like server using the command:
+
+    ```bash
+    python -m vllm.entrypoints.openai.api_server --served-model-name Qwen2-7B-Instruct --model path/to/weights
+    ```
+
+    Then you can access the Chat API by:
+
+    ```bash
+    curl http://localhost:8000/v1/chat/completions \
+        -H "Content-Type: application/json" \
+        -d '{
+        "model": "CarrotAI/Rabbit-Ko-15B-Instruct",
+        "messages": [
+            {"role": "system", "content": "You are a helpful assistant."},
+            {"role": "user", "content": "Your Long Input Here."}
+        ]
+        }'
+    ```
+
+    For further usage instructions of vLLM, please refer to our [Github](https://github.com/QwenLM/Qwen2).
+
+
+### Applications
+This fine-tuned model is particularly suited for [mention applications, e.g., chatbots, question-answering systems, etc.]. Its enhanced capabilities ensure more accurate and contextually appropriate responses in these domains.
+
+### Limitations and Considerations
+While our fine-tuning process has optimized the model for specific tasks, it's important to acknowledge potential limitations. The model's performance can still vary based on the complexity of the task and the specificities of the input data. Users are encouraged to evaluate the model thoroughly in their specific context to ensure it meets their requirements.
+
+If you liked this model, please use the card below
+
+```
+@article{RabbitKo15BInstruct,
+  title={CarrotAI/Rabbit-Ko-15B-Instruct Card},
+  author={CarrotAI (L, GEUN)},
+  year={2024},
+  url = {https://huggingface.co/CarrotAI/Rabbit-Ko-15B-Instruct}
+}
+```