Create Readme

README.md

@@ -0,0 +1,111 @@
+---
+license: llama3
+language:
+- en
+library_name: transformers
+tags:
+- llama-3
+- llama-3.2
+- bitcoin
+- finance
+- instruction-following
+- fine-tuning
+- merged
+base_model: meta-llama/Llama-3.2-3B-Instruct
+datasets:
+- tahamajs/bitcoin-llm-finetuning-dataset
+---
+
+# Llama-3.2-3B Instruct - Advanced Bitcoin Analyst
+
+This repository contains a highly specialized version of `meta-llama/Llama-3.2-3B-Instruct`, expertly fine-tuned to function as a **Bitcoin and cryptocurrency market analyst**. This model is the result of a "continuation training" process, where an already specialized model was further refined on a targeted dataset.
+
+This model excels at understanding and responding to complex instructions related to blockchain technology, financial markets, and technical/fundamental analysis of cryptocurrencies.
+
+## 🧠 Training Procedure
+
+The final model was created through a sophisticated multi-stage process designed to build upon and deepen existing knowledge.
+
+### Stage 1: Initial Specialization (Adapter Merge)
+The process began with the base `meta-llama/Llama-3.2-3B-Instruct` model. This base was then merged with a previously fine-tuned, high-performing LoRA adapter to create an initial specialized model.
+- **Initial Adapter:** `tahamajs/llama-3.2-3b-instruct-bitcoin-analyst-perfect`
+
+### Stage 2: Continued Fine-Tuning (New LoRA)
+A new LoRA adapter was then trained on top of the already-merged model from Stage 1. This continuation training allowed the model to further refine its expertise using a specific dataset, improving its nuance and instruction-following on relevant topics.
+- **Dataset:** `tahamajs/bitcoin-llm-finetuning-dataset`
+
+### Stage 3: Final Merge
+The final step was to merge the newly trained adapter from Stage 2 into the model. This repository hosts this **fully merged, standalone model**, which contains the cumulative knowledge of the base model, the first specialized adapter, and the second round of continuation training.
+
+---
+
+## 📊 Training Details
+
+### Hyperparameters
+The second stage of LoRA fine-tuning was performed with the following key hyperparameters:
+
+| Parameter | Value |
+| :--- | :--- |
+| `learning_rate` | 1e-4 |
+| `num_train_epochs` | 1 |
+| `lora_r` | 16 |
+| `lora_alpha` | 32 |
+| `optimizer` | paged_adamw_32bit |
+| `precision` | bf16 |
+
+### Training Loss
+The training loss shows a clear downward trend, indicating that the model was successfully learning from the dataset. The process started with a loss of ~2.18 and converged to a loss in the ~1.4-1.6 range, demonstrating effective knowledge acquisition. The fluctuations are normal during training and reflect the varying difficulty of the data in each batch.
+
+![Training Loss](https://googleusercontent.com/file_content/0)
+
+---
+
+## 🚀 How to Use
+
+This is a fully merged model and can be used directly with the `transformers` library. For best results, use the Llama 3 chat template to format your prompts.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Use the ID of the repository where this model is hosted
+model_id = "tahamajs/llama-3.2-3b-instruct-bitcoin-analyst-perfect_v2"
+
+# Load the tokenizer and model
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+# Use the Llama 3 chat template for instruction-following
+messages = [
+    {"role": "user", "content": "Analyze the current sentiment around Bitcoin based on the concept of the Fear & Greed Index. What does a high 'Greed' value typically imply for the short-term market?"},
+]
+
+# Apply the chat template and tokenize
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+# Generate a response
+outputs = model.generate(
+    input_ids,
+    max_new_tokens=512,
+    do_sample=True,
+    temperature=0.6,
+    top_p=0.9,
+)
+
+# Decode and print the output
+response = outputs[0][input_ids.shape[-1]:]
+print(tokenizer.decode(response, skip_special_tokens=True))
+````
+
+## Disclaimer
+
+This model is provided for informational and educational purposes only. It is **not financial advice**. The outputs are generated by an AI and may contain errors or inaccuracies. Always perform your own due diligence and consult with a qualified financial professional before making any investment decisions.
+