Upload 13 files

README.md

@@ -0,0 +1,27 @@
+Tiny Llama Project Guide
+This repository provides a comprehensive guide for students and researchers to experiment with the TinyLlama-1.1B-Chat-v1.0 model, an open-source language model developed by the TinyLlama organization. The goal is to enable accessible AI experimentation without any fees or personal information requirements.
+Model Details
+
+Model: TinyLlama-1.1B-Chat-v1.0
+Source: Hugging Face - TinyLlama/TinyLlama-1.1B-Chat-v1.0
+Organization: TinyLlama
+Description: A lightweight, efficient 1.1B parameter model optimized for chat and text generation tasks, suitable for low-resource environments like laptops with 16GB RAM.
+License: Refer to the model's official Hugging Face page for licensing details (typically Apache 2.0).
+
+Resources
+
+Code: Includes scripts for downloading the model, fine-tuning, and running a Flask-based chat UI.
+Dataset: A small JSON dataset for fine-tuning tests.
+Loss Plot: Training loss plot from fine-tuning (loss_plot.png).
+
+
+Usage
+This repository provides:
+
+A Flask app for local inference with a user-friendly chat interface.
+Fine-tuning scripts using LoRA for efficient training.
+Detailed setup instructions in document.txt.
+
+Note: Model weights are not included in this repository. Users must download them from the official Hugging Face repository using their access token.
+Attribution
+This project uses the TinyLlama-1.1B-Chat-v1.0 model by the TinyLlama organization. All credits for the model go to the original authors. For more details, visit the TinyLlama Hugging Face page.

app.py

@@ -0,0 +1,61 @@
+from flask import Flask, request, render_template
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+import re
+
+app = Flask(__name__)
+
+# Load model and tokenizer
+model_path = "./tinyllama_model"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForCausalLM.from_pretrained(model_path)
+device = torch.device("cpu")
+model.to(device)
+
+# Load simple knowledge base
+knowledge_base = {}
+try:
+    with open("knowledge_base.txt", "r") as f:
+        for line in f:
+            if ":" in line:
+                key, value = line.strip().split(":", 1)
+                knowledge_base[key.lower()] = value.strip()
+except FileNotFoundError:
+    knowledge_base = {
+        "ceo of meta": "Mark Zuckerberg",
+        "founder of meta": "Mark Zuckerberg",
+        "meta founding year": "2004"
+    }
+
+@app.route('/')
+def home():
+    return render_template('index.html')
+
+@app.route('/generate', methods=['POST'])
+def generate():
+    user_input = request.form['prompt'].strip().lower()
+    
+    # Check knowledge base for factual answers
+    response = None
+    for key in knowledge_base:
+        if key in user_input:
+            response = knowledge_base[key]
+            break
+    
+    if not response:
+        # Create chat prompt template
+        prompt = f"<|SYSTEM|> You are a helpful assistant providing accurate and concise answers. Avoid fabricating details or repeating user input unnecessarily. If unsure, say so. <|USER|> {user_input} <|ASSISTANT|> "
+        inputs = tokenizer(prompt, return_tensors="pt").to(device)
+        outputs = model.generate(**inputs, max_length=100, num_return_sequences=1, pad_token_id=tokenizer.eos_token_id)
+        generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+        # Clean response
+        response = generated_text.split("<|ASSISTANT|> ")[-1] if "<|ASSISTANT|> " in generated_text else generated_text
+        response = re.sub(r"\s+", " ", response).strip()
+        # Avoid repetitive or irrelevant output
+        if user_input in response.lower() and len(response.split()) < 10:
+            response = "I'm not sure how to respond to that. Could you clarify or ask something else?"
+
+    return response
+
+if __name__ == '__main__':
+    app.run(debug=True)

document.txt

@@ -0,0 +1,95 @@
+Tiny Llama Project Guide: Running TinyLlama-1.1B-Chat-v1.0 Locally
+
+This document provides a step-by-step guide to run the TinyLlama-1.1B-Chat-v1.0 model locally on a laptop with 16GB RAM, i5 processor, and Windows OS. The guide includes setting up the environment, downloading the model, fine-tuning, and running a Flask-based chat UI.
+
+---
+
+System Requirements
+Operating System: Windows
+RAM: 16GB
+Processor: Intel i5 or equivalent
+Python Version: 3.10.9
+- IDE: Visual Studio Code (VS Code)
+- Internet: Required for downloading model and libraries
+
+---
+
+Step-by-Step Setup
+
+1. Install Python 3.10.9
+   - Download and install Python 3.10.9 from https://www.python.org/downloads/release/python-3109/.
+   - Ensure Python and pip are added to your system PATH.
+
+2. Set Up a Virtual Environment
+   - Open VS Code terminal in your project directory (e.g., C:\path\to\TinyLlama-1.1B).
+   - Run:
+     ```
+     python -m venv venv
+     .\venv\Scripts\activate
+     ```
+
+3. Install Required Libraries
+   - In the activated virtual environment, run:
+     ```
+     pip install transformers torch huggingface_hub datasets peft trl accelerate flask matplotlib
+     ```
+   - This installs libraries for model handling, fine-tuning, Flask app, and plotting.
+
+4. Download the TinyLlama Model
+   - Create a file `download_model.py` with the following code:
+     ```python
+     from huggingface_hub import login, snapshot_download
+     login(token="YOUR_ACCESS_TOKEN_HERE")
+     snapshot_download(repo_id="TinyLlama/TinyLlama-1.1B-Chat-v1.0", local_dir="./tinyllama_model")
+     ```
+   - Replace `YOUR_ACCESS_TOKEN_HERE` with your Hugging Face access token (get it from https://huggingface.co/settings/tokens).
+   - Run: `python download_model.py`
+   - Model weights will be saved in the `tinyllama_model` folder.
+
+5. Run Inference with Flask UI
+   - Create a `finetune` folder in your project directory.
+   - Copy `app.py` and `templates/index.html` from the repository to the `finetune` folder.
+   - Run: `python app.py`
+   - Open http://127.0.0.1:5000 in your browser to access the chat UI.
+   - Enter prompts to interact with the model.
+
+6. Fine-Tune the Model (Optional)
+   - In the `finetune` folder, ensure `dataset.json` and `finetune.py` are present.
+   - Run: `python finetune.py`
+   - Fine-tuned weights will be saved in `finetune/finetuned_weights`.
+   - Update `app.py` to point to `./finetuned_weights` for inference with the fine-tuned model.
+   - Check `loss_plot.png` for training loss visualization.
+
+7. View Training Metrics
+   - After fine-tuning, check the console for final train loss and learning rate.
+   - Open `loss_plot.png` in the `finetune` folder for a graphical view of training loss.
+
+---
+
+Project Structure
+- `tinyllama_model/`: Model weights downloaded from Hugging Face.
+- `finetune/`: Contains fine-tuning scripts and fine-tuned weights.
+  - `dataset.json`: Small dataset for fine-tuning.
+  - `finetune.py`: Fine-tuning script with LoRA.
+  - `app.py`: Flask app for inference.
+  - `templates/index.html`: Chat UI.
+  - `loss_plot.png`: Training loss plot.
+- `requirements.txt`: List of required libraries.
+- `document.txt`: This guide.
+- `README.md`: Project overview.
+
+---
+
+Attribution
+- **Model**: TinyLlama-1.1B-Chat-v1.0
+- **Source**: https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0
+- **Organization**: TinyLlama
+- **License**: Check the model's Hugging Face page for licensing details.
+
+---
+
+Notes
+- Model weights are not included in this repository to respect licensing terms.
+- Download the model directly from Hugging Face using your access token.
+- Ensure sufficient disk space (~2-3GB) for model weights and fine-tuned weights.
+- For support, refer to the TinyLlama Hugging Face page or community forums.

download_model.py

@@ -0,0 +1,7 @@
+from huggingface_hub import login, snapshot_download
+
+# Log in to Hugging Face
+login(token="hf_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")
+
+# Download TinyLlama-1.1B model to local folder
+snapshot_download(repo_id="TinyLlama/TinyLlama-1.1B-Chat-v1.0", local_dir="./tinyllama_model")

finetune/app.py

@@ -0,0 +1,29 @@
+from flask import Flask, request, render_template
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+app = Flask(__name__)
+
+# Load fine-tuned model and tokenizer
+model_path = "./finetuned_weights"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForCausalLM.from_pretrained(model_path)
+device = torch.device("cpu")
+model.to(device)
+
+@app.route('/')
+def home():
+    return render_template('index.html')
+
+@app.route('/generate', methods=['POST'])
+def generate():
+    user_input = request.form['prompt'].strip()
+    prompt = f"<|USER|> {user_input} <|ASSISTANT|> "
+    inputs = tokenizer(prompt, return_tensors="pt").to(device)
+    outputs = model.generate(**inputs, max_length=100, num_return_sequences=1, pad_token_id=tokenizer.eos_token_id)
+    generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+    response = generated_text.split("<|ASSISTANT|> ")[-1] if "<|ASSISTANT|> " in generated_text else generated_text
+    return response
+
+if __name__ == '__main__':
+    app.run(debug=True)

finetune/dataset.json

@@ -0,0 +1,7 @@
+[
+    {"prompt": "What is the capital of France?", "response": "The capital of France is Paris."},
+    {"prompt": "Who is the CEO of Tesla?", "response": "The CEO of Tesla is Elon Musk."},
+    {"prompt": "What is 2 + 2?", "response": "2 + 2 equals 4."},
+    {"prompt": "What is Python?", "response": "Python is a high-level programming language."},
+    {"prompt": "Who founded Microsoft?", "response": "Microsoft was founded by Bill Gates and Paul Allen."}
+]

finetune/finetune.py

@@ -0,0 +1,86 @@
+from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
+from datasets import Dataset
+from peft import LoraConfig, get_peft_model
+import torch
+import json
+import matplotlib.pyplot as plt
+
+# Load model and tokenizer
+model_path = "../tinyllama_model"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForCausalLM.from_pretrained(model_path)
+
+# Set pad token
+tokenizer.pad_token = tokenizer.eos_token
+model.config.pad_token_id = tokenizer.eos_token_id
+
+# Load dataset from JSON
+with open("dataset.json", "r") as f:
+    data = json.load(f)
+dataset = Dataset.from_list(data)
+
+# Tokenize dataset and include labels
+def tokenize_function(examples):
+    inputs = [f"<|USER|> {p} <|ASSISTANT|> {r}" for p, r in zip(examples["prompt"], examples["response"])]
+    tokenized = tokenizer(inputs, padding="max_length", truncation=True, max_length=128, return_tensors="pt")
+    tokenized["labels"] = tokenized["input_ids"].clone()
+    return tokenized
+
+tokenized_dataset = dataset.map(tokenize_function, batched=True, remove_columns=["prompt", "response"])
+
+# Configure LoRA for efficient fine-tuning
+lora_config = LoraConfig(
+    r=8,
+    lora_alpha=32,
+    target_modules=["q_proj", "v_proj"],
+    lora_dropout=0.1,
+    bias="none",
+    task_type="CAUSAL_LM"
+)
+model = get_peft_model(model, lora_config)
+
+# Training arguments
+training_args = TrainingArguments(
+    output_dir="./finetuned_weights",
+    num_train_epochs=3,
+    per_device_train_batch_size=1,
+    save_strategy="epoch",
+    logging_steps=1,
+    learning_rate=2e-4,
+    fp16=False,
+    report_to="none"
+)
+
+# Trainer (no validation dataset due to small size)
+trainer = Trainer(
+    model=model,
+    args=training_args,
+    train_dataset=tokenized_dataset,
+)
+
+# Fine-tune model
+train_result = trainer.train()
+
+# Save fine-tuned weights
+model.save_pretrained("./finetuned_weights")
+tokenizer.save_pretrained("./finetuned_weights")
+
+# Extract metrics
+train_loss = [log["loss"] for log in trainer.state.log_history if "loss" in log]
+learning_rate = [log["learning_rate"] for log in trainer.state.log_history if "learning_rate" in log]
+
+# Print final metrics
+print(f"Final Train Loss: {train_loss[-1] if train_loss else 'N/A'}")
+print(f"Final Learning Rate: {learning_rate[-1] if learning_rate else 'N/A'}")
+
+# Plot train loss
+plt.figure(figsize=(10, 6))
+if train_loss:
+    plt.plot(range(len(train_loss)), train_loss, label="Train Loss", color="#2563eb")
+plt.xlabel("Steps")
+plt.ylabel("Loss")
+plt.title("Training Loss")
+plt.legend()
+plt.grid()
+plt.savefig("loss_plot.png")
+plt.show()

finetune/templates/index.html

@@ -0,0 +1,156 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>TinyLlama Chat</title>
+    <link href="https://cdn.jsdelivr.net/npm/[email protected]/dist/tailwind.min.css" rel="stylesheet">
+    <style>
+        body {
+            background: linear-gradient(135deg, #1e3a8a, #3b82f6);
+            min-height: 100vh;
+            font-family: 'Arial', sans-serif;
+        }
+        .chat-container {
+            max-width: 700px;
+            margin: 2rem auto;
+            background: white;
+            border-radius: 1rem;
+            box-shadow: 0 10px 20px rgba(0, 0, 0, 0.2);
+            overflow: hidden;
+        }
+        .chat-header {
+            background: #2563eb;
+            color: white;
+            padding: 1rem;
+            text-align: center;
+            font-size: 1.5rem;
+            font-weight: bold;
+        }
+        .chat-body {
+            max-height: 500px;
+            overflow-y: auto;
+            padding: 1rem;
+        }
+        .message {
+            margin: 0.5rem 0;
+            padding: 0.75rem;
+            border-radius: 0.5rem;
+            max-width: 80%;
+        }
+        .user-message {
+            background: #dbeafe;
+            margin-left: auto;
+            text-align: right;
+        }
+        .ai-message {
+            background: #f3f4f6;
+            margin-right: auto;
+        }
+        .input-container {
+            display: flex;
+            padding: 1rem;
+            background: #f9fafb;
+            border-top: 1px solid #e5e7eb;
+        }
+        .input-container input {
+            flex: 1;
+            padding: 0.75rem;
+            border: 1px solid #d1d5db;
+            border-radius: 0.5rem 0 0 0.5rem;
+            outline: none;
+        }
+        .input-container button {
+            padding: 0.75rem 1.5rem;
+            background: #2563eb;
+            color: white;
+            border: none;
+            border-radius: 0 0.5rem 0.5rem 0;
+            cursor: pointer;
+            transition: background 0.3s;
+        }
+        .input-container button:hover {
+            background: #1e40af;
+        }
+        .loading {
+            display: none;
+            margin: 1rem auto;
+            width: 40px;
+            height: 40px;
+            border: 4px solid #f3f3f3;
+            border-top: 4px solid #2563eb;
+            border-radius: 50%;
+            animation: spin 1s linear infinite;
+        }
+        @keyframes spin {
+            0% { transform: rotate(0deg); }
+            100% { transform: rotate(360deg); }
+        }
+        .attribution {
+            text-align: center;
+            margin: 1rem;
+            font-size: 0.9rem;
+            color: #f3f4f6;
+        }
+    </style>
+</head>
+<body>
+    <div class="chat-container">
+        <div class="chat-header">TinyLlama Chat</div>
+        <div class="chat-body" id="chat-body">
+            <div class="message ai-message">Hello! How can I assist you today?</div>
+        </div>
+        <div class="input-container">
+            <input type="text" id="prompt" placeholder="Type your message...">
+            <button onclick="sendMessage()">Send</button>
+        </div>
+        <div class="loading" id="loading"></div>
+    </div>
+    <div class="attribution">
+        Powered by </a href="https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0" target="_blank">TinyLlama-1.1B-Chat-v1.0</a> by TinyLlama
+    </div>
+    <script>
+        async function sendMessage() {
+            const promptInput = document.getElementById('prompt');
+            const chatBody = document.getElementById('chat-body');
+            const loading = document.getElementById('loading');
+            const prompt = promptInput.value.trim();
+            
+            if (!prompt) return;
+
+            // Add user message
+            const userMessage = document.createElement('div');
+            userMessage.className = 'message user-message';
+            userMessage.textContent = prompt;
+            chatBody.appendChild(userMessage);
+
+            // Show loading animation
+            loading.style.display = 'block';
+            promptInput.value = '';
+            chatBody.scrollTop = chatBody.scrollHeight;
+
+            // Send request to Flask
+            const response = await fetch('/generate', {
+                method: 'POST',
+                headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
+                body: `prompt=${encodeURIComponent(prompt)}`
+            });
+            const aiResponse = await response.text();
+
+            // Hide loading animation
+            loading.style.display = 'none';
+
+            // Add AI response
+            const aiMessage = document.createElement('div');
+            aiMessage.className = 'message ai-message';
+            aiMessage.textContent = aiResponse;
+            chatBody.appendChild(aiMessage);
+            chatBody.scrollTop = chatBody.scrollHeight;
+        }
+
+        document.getElementById('prompt').addEventListener('keypress', (e) => {
+            if (e.key === 'Enter') sendMessage();
+        });
+    </script>
+</body>
+</html>

inference.py

@@ -0,0 +1,24 @@
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+# Load the model and tokenizer
+model_path = "./tinyllama_model"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForCausalLM.from_pretrained(model_path)
+
+# Set device to CPU
+device = torch.device("cpu")
+model.to(device)
+
+# Input prompt
+prompt = "Hello, how can I assist you today?"
+
+# Tokenize input
+inputs = tokenizer(prompt, return_tensors="pt").to(device)
+
+# Generate text
+outputs = model.generate(**inputs, max_length=100, num_return_sequences=1)
+
+# Decode and print output
+generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(generated_text)

knowledge_base.txt

@@ -0,0 +1,3 @@
+ceo of meta:Mark Zuckerberg
+founder of meta:Mark Zuckerberg
+meta founding year:2004

requirements.txt

@@ -0,0 +1,9 @@
+transformers==4.44.2
+torch==2.4.1
+huggingface_hub==0.25.1
+datasets==3.0.0
+peft==0.12.0
+trl==0.11.1
+accelerate==0.33.0
+flask==3.0.3
+matplotlib==3.9.2

templates/index.html

@@ -0,0 +1,156 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>TinyLlama Chat</title>
+    <link href="https://cdn.jsdelivr.net/npm/[email protected]/dist/tailwind.min.css" rel="stylesheet">
+    <style>
+        body {
+            background: linear-gradient(135deg, #1e3a8a, #3b82f6);
+            min-height: 100vh;
+            font-family: 'Arial', sans-serif;
+        }
+        .chat-container {
+            max-width: 700px;
+            margin: 2rem auto;
+            background: white;
+            border-radius: 1rem;
+            box-shadow: 0 10px 20px rgba(0, 0, 0, 0.2);
+            overflow: hidden;
+        }
+        .chat-header {
+            background: #2563eb;
+            color: white;
+            padding: 1rem;
+            text-align: center;
+            font-size: 1.5rem;
+            font-weight: bold;
+        }
+        .chat-body {
+            max-height: 500px;
+            overflow-y: auto;
+            padding: 1rem;
+        }
+        .message {
+            margin: 0.5rem 0;
+            padding: 0.75rem;
+            border-radius: 0.5rem;
+            max-width: 80%;
+        }
+        .user-message {
+            background: #dbeafe;
+            margin-left: auto;
+            text-align: right;
+        }
+        .ai-message {
+            background: #f3f4f6;
+            margin-right: auto;
+        }
+        .input-container {
+            display: flex;
+            padding: 1rem;
+            background: #f9fafb;
+            border-top: 1px solid #e5e7eb;
+        }
+        .input-container input {
+            flex: 1;
+            padding: 0.75rem;
+            border: 1px solid #d1d5db;
+            border-radius: 0.5rem 0 0 0.5rem;
+            outline: none;
+        }
+        .input-container button {
+            padding: 0.75rem 1.5rem;
+            background: #2563eb;
+            color: white;
+            border: none;
+            border-radius: 0 0.5rem 0.5rem 0;
+            cursor: pointer;
+            transition: background 0.3s;
+        }
+        .input-container button:hover {
+            background: #1e40af;
+        }
+        .loading {
+            display: none;
+            margin: 1rem auto;
+            width: 40px;
+            height: 40px;
+            border: 4px solid #f3f3f3;
+            border-top: 4px solid #2563eb;
+            border-radius: 50%;
+            animation: spin 1s linear infinite;
+        }
+        @keyframes spin {
+            0% { transform: rotate(0deg); }
+            100% { transform: rotate(360deg); }
+        }
+        .attribution {
+            text-align: center;
+            margin: 1rem;
+            font-size: 0.9rem;
+            color: #f3f4f6;
+        }
+    </style>
+</head>
+<body>
+    <div class="chat-container">
+        <div class="chat-header">TinyLlama Chat</div>
+        <div class="chat-body" id="chat-body">
+            <div class="message ai-message">Hello! How can I assist you today?</div>
+        </div>
+        <div class="input-container">
+            <input type="text" id="prompt" placeholder="Type your message...">
+            <button onclick="sendMessage()">Send</button>
+        </div>
+        <div class="loading" id="loading"></div>
+    </div>
+    <div class="attribution">
+        Powered by </a href="https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0" target="_blank">TinyLlama-1.1B-Chat-v1.0</a> by TinyLlama
+    </div>
+    <script>
+        async function sendMessage() {
+            const promptInput = document.getElementById('prompt');
+            const chatBody = document.getElementById('chat-body');
+            const loading = document.getElementById('loading');
+            const prompt = promptInput.value.trim();
+            
+            if (!prompt) return;
+
+            // Add user message
+            const userMessage = document.createElement('div');
+            userMessage.className = 'message user-message';
+            userMessage.textContent = prompt;
+            chatBody.appendChild(userMessage);
+
+            // Show loading animation
+            loading.style.display = 'block';
+            promptInput.value = '';
+            chatBody.scrollTop = chatBody.scrollHeight;
+
+            // Send request to Flask
+            const response = await fetch('/generate', {
+                method: 'POST',
+                headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
+                body: `prompt=${encodeURIComponent(prompt)}`
+            });
+            const aiResponse = await response.text();
+
+            // Hide loading animation
+            loading.style.display = 'none';
+
+            // Add AI response
+            const aiMessage = document.createElement('div');
+            aiMessage.className = 'message ai-message';
+            aiMessage.textContent = aiResponse;
+            chatBody.appendChild(aiMessage);
+            chatBody.scrollTop = chatBody.scrollHeight;
+        }
+
+        document.getElementById('prompt').addEventListener('keypress', (e) => {
+            if (e.key === 'Enter') sendMessage();
+        });
+    </script>
+</body>
+</html>