File size: 4,295 Bytes
e151d79 ccde10f e151d79 ccde10f e151d79 ccde10f e151d79 ccde10f e151d79 ccde10f e151d79 ccde10f e151d79 6f18a97 e151d79 6f18a97 e151d79 6f18a97 e151d79 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 |
---
tags:
- computer-engineering
- llama-3
- 1b
- lora
- 8bit
license: llama3.2
license_link: https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE
base_model:
- meta-llama/Llama-3.2-1B
datasets:
- wikitext-2-raw-v1
- computer-engineering-corpus
---
**Specialized 1B Parameter Model for Computer Engineering**
*Fine-tuned with LoRA on 8-bit quantized Llama-3-1B*
<div align="center">
<a href="https://github.com/IrfanUruchi/Llama-3.2-1B-ComputerEngineeringLLM">
<img src="https://img.shields.io/badge/🔗_GitHub-Repo-181717?style=for-the-badge&logo=github" alt="GitHub">
</a>
<a href="https://huggingface.co/Irfanuruchi/Llama-3.2-1B-Computer-Engineering-LLM">
<img src="https://img.shields.io/badge/🤗_HuggingFace-Model_Repo-FFD21F?style=for-the-badge" alt="HuggingFace">
</a>
<br>
<img src="https://img.shields.io/badge/Model_Size-1B_parameters-blue" alt="Model Size">
<img src="https://img.shields.io/badge/Quantization-8bit-green" alt="Quantization">
<img src="https://img.shields.io/badge/Adapter-LoRA-orange" alt="Adapter">
<img src="https://img.shields.io/badge/Context-8k-lightgrey" alt="Context">
</div>
---
## 🛠️ Technical Specifications
### Architecture
| Component | Specification |
|------------------------|---------------------------------|
| Base Model | Meta-Llama-3-1B |
| Hidden Size | 2048 |
| Layers | 16 |
| Attention Heads | 32 |
| Quantization | 8-bit via BitsAndBytes |
| Fine-Tuning Method | LoRA (Low-Rank Adaptation) |
| Tokenizer Vocabulary | 128,256 tokens |
### Training Data
- Wikitext-2-raw-v1 (General knowledge)
- Custom computer engineering corpus:
- Hardware design principles
- Processor architectures
- Embedded systems documentation
---
## Installation and usage
### Option 1: From Hugging Face Hub (Recommended)
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Irfanuruchi/Llama-3.2-1B-Computer-Engineering-LLM"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype="auto",
trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
model_id,
use_fast=False # Required for proper Llama tokenization
)
prompt = "Explain the von Neumann architecture:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=200,
temperature=0.7,
top_p=0.9,
do_sample=True,
repetition_penalty=1.1
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
### Option 2: Local Installation (Git LFS Required)
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
# Replace with your local path
model_path = "./Llama-3.2-1B-ComputerEngineeringLLM"
model = AutoModelForCausalLM.from_pretrained(
model_path,
device_map="auto",
local_files_only=True
)
tokenizer = AutoTokenizer.from_pretrained(
model_path,
use_fast=False, # Required for Llama tokenizer
local_files_only=True
)
```
*Recomended Config*
```python
outputs = model.generate(
**inputs,
max_new_tokens=200,
temperature=0.7,
top_p=0.9,
do_sample=True,
repetition_penalty=1.1
)
```
---
## Licence complience
This model is governed by the Llama 3.2 Community License. Key requirements:
Non-commercial use only
Attribution to Meta required
Cannot be used to train other LLMs
Attribution Notice:
"Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc."
---
## Limitations
Specialized for computer engineering (general performance may vary)
Occasional repetition in outputs
Requires prompt engineering for optimal results
Knowledge cutoff: January 2025
---
## Citation
If using for academic research, please cite:
```bibtex
@misc{llama3.2-1b-eng-2025,
title = {Llama-3.2-1B-Computer-Engineering-LLM},
author = {Irfanuruchi},
year = {2025},
publisher = {Hugging Face},
url = {https://huggingface.co/Irfanuruchi/Llama-3.2-1B-Computer-Engineering-LLM},
}
```
|