Azzedde/llama3.2-3b-sql-expert-1-epoch

Model Card for Llama3.2-3B-SQL-Expert-1Epoch

Model Details

Model Description

Llama3.2-3B-SQL-Expert-1Epoch is a fine-tuned version of Meta’s Llama-3.1-3B, specifically optimized for generating SQL queries from natural language input. The model has been trained using Unsloth for efficient fine-tuning and inference.

Developed by: Azzedine (GitHub: Azzedde)
Funded by [optional]: N/A
Shared by [optional]: Azzedde
Model Type: Large Language Model (LLM) optimized for SQL query generation
Language(s) (NLP): English
License: Apache 2.0
Finetuned from model [optional]: Meta-Llama-3.1-3B-Instruct

Model Sources

Repository: Hugging Face
Paper [optional]: N/A
Demo [optional]: N/A

Uses

Direct Use

This model is designed for generating SQL queries based on natural language inputs and is useful for:

Database management and administration
Automated query generation
Data analytics pipelines
SQL education and training
Business intelligence applications

Downstream Use [optional]

Embedding into LLM-based database assistants
Automating SQL-based analytics
Assisting developers in writing optimized queries

Out-of-Scope Use

General NLP tasks unrelated to SQL query generation
Applications requiring strong factual accuracy outside SQL

Bias, Risks, and Limitations

Incorrect or suboptimal queries: The model may generate queries that are syntactically correct but do not yield the intended results.
Lack of query optimization: The generated queries are not always optimized for performance; users should validate execution plans.
English-only support: The model primarily supports English-language inputs.
Limited schema understanding: The model does not validate database structures and may assume incorrect relationships between tables.

Recommendations

Users should:

Always validate generated SQL queries before executing them.
Use the model as an assistant, not a replacement for SQL expertise.
Fine-tune the model further for domain-specific databases.

How to Get Started with the Model

Use the following code to load and use the model:

from unsloth import FastLanguageModel
from transformers import AutoTokenizer

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("Azzedde/llama3.2-3b-sql-expert-1-epoch")
model = FastLanguageModel.from_pretrained("Azzedde/llama3.2-3b-sql-expert-1-epoch")

# Example inference
sql_prompt = """Below is a SQL database schema and a question. Generate an SQL query to answer the question.

### Schema:
{schema}

### Question:
{question}

### SQL Query:
"""
input_text = sql_prompt.format(
    schema="CREATE TABLE employees (id INT PRIMARY KEY, name VARCHAR, salary DECIMAL, department_id INT);",
    question="Find the average salary per department."
)

# Tokenize and generate query
inputs = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)
print(tokenizer.decode(outputs[0]))

Training Details

Training Data

The model was fine-tuned on a structured SQL dataset, including a mix of publicly available SQL benchmarks and synthetically generated SQL queries.

Training Procedure

Preprocessing: Tokenized using standard SQL syntax formatting
Training Hyperparameters:
- batch_size = 4
- gradient_accumulation_steps = 8
- num_train_epochs = 1
- learning_rate = 2e-4
- fp16 = True

Evaluation

Testing Data

The model was evaluated on a separate test set of SQL queries derived from real-world database schemas.

Evaluation Metrics

Exact Match Accuracy: Percentage of queries that exactly match ground-truth SQL
Execution Success Rate: Percentage of generated queries that execute without errors

Results

High accuracy for common SQL queries
Some errors in complex multi-table joins and nested queries

Environmental Impact

Hardware Type: Tesla T4 (Google Colab)
Training Duration: ~1.5 hours
Compute Region: N/A
Estimated Carbon Emissions: Minimal

Technical Specifications

Model Architecture and Objective

Based on Llama-3.1 3B, fine-tuned with LoRA for SQL generation.

Compute Infrastructure

Fine-tuned using Unsloth for efficient training and inference.

Hardware

GPU: Tesla T4
Max Reserved Memory: ~6.5 GB

Software

Libraries Used: unsloth, transformers, TRL, datasets

Citation [optional]

BibTeX:

@article{llama3.2-3B-SQL-Expert,
  author    = {Azzedde},
  title     = {Llama3.2-3B-SQL-Expert: An SQL Query Generation Model},
  year      = {2025},
  url       = {https://huggingface.co/Azzedde/llama3.2-3b-sql-expert-1-epoch}
}

APA:

Azzedde. (2025). Llama3.2-3B-SQL-Expert: An SQL Query Generation Model. Retrieved from Hugging Face

More Information

For questions, reach out via Hugging Face discussions or GitHub issues.

Model Card Authors

Azzedde (GitHub: Azzedde)

Model Card Contact

Contact: Hugging Face Profile

Azzedde
/

llama3.2-3b-sql-expert-1-epoch