benisonjac/finetune-of-stable-diffuson-on-Zappos-shoe-dataset

Finetuned Stable Diffusion for Shoe Image Generation

This repository hosts a finetuned Stable Diffusion model specialized in generating high-quality shoe images. The model is trained on the UT Zappos50K dataset, which consists of over 50,000 catalog-style shoe images with clean backgrounds.

Model Overview

• Base model: Stable Diffusion (v1.4/v2, specify exact version)
• Finetuned on the UT Zappos50K dataset of shoe images
• Generates product-style shoe images with consistent framing and high detail
• Ideal for concepting shoe designs, data augmentation, and e-commerce visual content

Usage

Install dependencies:

pip install diffusers transformers accelerate torch

Example Python code for inference:

import torch
from diffusers import StableDiffusionPipeline

model_id = "benisonjac/finetune-of-stable-diffuson-on-Zappos-shoe-dataset"

pipe = StableDiffusionPipeline.from_pretrained(
model_id,
torch_dtype=torch.float16,
safety_checker=None,
use_safetensors=True
).to("cuda")

prompt = "studio product photo of a black leather ankle boot, side view, isolated on white background, high detail"
negative_prompt = "blurry, low resolution, watermark, deformed"

image = pipe(
prompt,
negative_prompt=negative_prompt,
guidance_scale=7.5,
num_inference_steps=30,
generator=torch.Generator("cuda").manual_seed(42)
).images

image.save("generated_shoe.png")

Prompting Tips

• Use phrases like "studio product photo," "isolated on white background," and "high detail" to emphasize the product catalog style.
• Specify shoe type, color, material, and view angle to guide generation.
• Use negative prompts such as "blurry," "watermark," and "deformed" to improve image clarity.

Dataset

The model is finetuned on the UT Zappos50K dataset, which contains over 50,000 images of shoes including boots, sandals, slippers, and more, all with consistent white backgrounds and catalog styling.

Limitations

• Best suited for single shoe objects with clean backgrounds.
• May produce artifacts or less realistic results for complex scenes or multiple objects.
• Not intended for non-shoe or human image generation.
• Safety checker disabled by default; use caution for downstream deployment.

License and Citation

Please adhere to the licensing terms of the base Stable Diffusion model and the UT Zappos50K dataset when using or redistributing this finetuned model.

If you use this model in your work, please cite the UT Zappos50K dataset and the Stable Diffusion original model.

Acknowledgements

• Stable Diffusion and Hugging Face Diffusers
• UT Austin Zappos50K dataset creators
• Hugging Face Hub for model hosting infrastructure

---

Feel free to open issues for questions or improvement suggestions.