ziyingchen1106/Llama-3.2-3B-Instruct-fp16-lora-gptqmodel-4bit

Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "ziyingchen1106/Llama-3.2-3B-Instruct-fp16-lora-gptqmodel-4bit"
model = AutoModelForCausalLM.from_pretrained(
   model_name,
   torch_dtype=torch.float16,
   device_map="cuda:0"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

Attribution

• Built with Llama
• Llama 3.2 Community License © Meta Platforms, Inc.