README.md

metadata

tags:
  - computer-engineering
  - llama-3
  - 1b
  - lora
  - 8bit
license: llama3.2
license_link: https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE
base_model:
  - meta-llama/Llama-3.2-1B
datasets:
  - wikitext-2-raw-v1
  - computer-engineering-corpus

Specialized 1B Parameter Model for Computer Engineering
Fine-tuned with LoRA on 8-bit quantized Llama-3-1B

---

🛠️ Technical Specifications

Architecture

Component	Specification
Base Model	Meta-Llama-3-1B
Hidden Size	2048
Layers	16
Attention Heads	32
Quantization	8-bit via BitsAndBytes
Fine-Tuning Method	LoRA (Low-Rank Adaptation)
Tokenizer Vocabulary	128,256 tokens

Training Data

• Wikitext-2-raw-v1 (General knowledge)
• Custom computer engineering corpus:
  • Hardware design principles
  • Processor architectures
  • Embedded systems documentation

---

Installation and usage

Option 1: From Hugging Face Hub (Recommended)

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Irfanuruchi/Llama-3.2-1B-Computer-Engineering-LLM"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto", 
    trust_remote_code=True
)

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    use_fast=False  # Required for proper Llama tokenization
)

prompt = "Explain the von Neumann architecture:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)  

outputs = model.generate(
    **inputs,
    max_new_tokens=200,  
    temperature=0.7,     
    top_p=0.9,          
    do_sample=True,   
    repetition_penalty=1.1  
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Option 2: Local Installation (Git LFS Required)

from transformers import AutoModelForCausalLM, AutoTokenizer

# Replace with your local path
model_path = "./Llama-3.2-1B-ComputerEngineeringLLM"  

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="auto",
    local_files_only=True
)
tokenizer = AutoTokenizer.from_pretrained(
    model_path,
    use_fast=False,  # Required for Llama tokenizer
    local_files_only=True
)

Recomended Config

outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.7, 
    top_p=0.9,     
    do_sample=True,
    repetition_penalty=1.1  
)

---

Licence complience

This model is governed by the Llama 3.2 Community License. Key requirements:

Non-commercial use only Attribution to Meta required Cannot be used to train other LLMs Attribution Notice: "Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc."

---

Limitations

Specialized for computer engineering (general performance may vary) Occasional repetition in outputs Requires prompt engineering for optimal results Knowledge cutoff: January 2025

---

Citation

If using for academic research, please cite:

@misc{llama3.2-1b-eng-2025,
  title = {Llama-3.2-1B-Computer-Engineering-LLM},
  author = {Irfanuruchi},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Irfanuruchi/Llama-3.2-1B-Computer-Engineering-LLM},
}