metadata

title: BrushNet
emoji: ⚡
colorFrom: yellow
colorTo: indigo
sdk: gradio
sdk_version: 3.50.2
python_version: 3.9
app_file: app.py
pinned: false
license: apache-2.0

BrushNetApi

This repo implements an API that allows other applications to call it with http request.

Instructions to Run the Application

Create a virtual environment and install dependencies.

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Run Honcho to Start all the Services

honcho start

Or Run the Redis, Celery and Web Server in seperate tabs

Start Redis Server

redis-server

Start the Celery Worker

# In a new terminal window:
./start_celery_worker.sh

Run the FastAPI Application

# In a new terminal window:
uvicorn app_api:app --host 0.0.0.0 --port 8000

How to Use the API

a. Initiate Image Generation

Send a POST request to /inpaint with the required parameters.

Request Body

{
  "input_image": "<base64-encoded input image>",
  "input_mask": "<base64-encoded input mask>",
  "prompt": "A beautiful cake on the table",
  "negative_prompt": "ugly, low quality",
  "control_strength": 1.0,
  "guidance_scale": 12.0,
  "num_inference_steps": 50,
  "seed": 551793204,
  "randomize_seed": false,
  "blended": false,
  "invert_mask": true,
  "count": 1,
  "webhook_url": "http://your-webhook-url.com/notify"  # Optional
}

Response

{
  "job_id": "<task_id>"
}

b. Check Task Status

Send a GET request to /status/{job_id}.

Response

{
  "status": "PENDING"  # Or "STARTED", "SUCCESS", "FAILURE"
}

c. Retrieve Result

Once the status is “SUCCESS”, send a GET request to /result/{job_id}.

Response:

{
  "images": ["<base64-encoded image>", "..."]
}

d. Webhook Notification

If you provided a webhook_url in your request, the server will send a POST request to that URL with the images once the task is complete.

{
  "images": ["<base64-encoded image>", "..."]
}

The webhook functionality is included in the generate_image_task function. If a webhook_url is provided in the request, the server will send a POST request to that URL with the generated images.

BrushNet(original README below )

This repository contains the gradio demo of the paper "BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion"

Keywords: Image Inpainting, Diffusion Models, Image Generation

Xuan Ju¹², Xian Liu¹², Xintao Wang^1*, Yuxuan Bian², Ying Shan¹, Qiang Xu^2*
¹ARC Lab, Tencent PCG ²The Chinese University of Hong Kong ^*Corresponding Author

🤝🏼 Cite Us

@misc{ju2024brushnet,
  title={BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion},
  author={Xuan Ju and Xian Liu and Xintao Wang and Yuxuan Bian and Ying Shan and Qiang Xu},
  year={2024},
  eprint={2403.06976},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

💖 Acknowledgement

Our code is modified based on diffusers, thanks to all the contributors!