An elastic deployment of Stable Diffusion with Discord on AWS
AWS Architecture BlogAn elastic deployment of Stable Diffusion with Discord on AWSby Steven Warren | on23 DEC 2022| inAmazon CloudWatch,Amazon EC2,Amazon Elastic Container Service,Architecture|Permalink|ShareStable Diffusion is a state-of-the-art text-to-image model that generates images from text. Deploying text-to-image models such as Stable Diffusion can be difficult. Currently, Stable Diffusion requires specific computer hardware known as graphical processing units (GPUs). You can lower the bar to entry by offloading the text-to-image generation onto Amazon Web Services (AWS).Discord is a popular voice, video, and text communication service. It provides a user interface that people can use to make text-to-image requests. When deployed, all members of a Discord server can create images by using Discord Slash Commands.In this post, we discuss how to deploy a highly available solution on AWS. This solution will perform text-to-image generation with Stable Diffusion and use Discord as the user interface.Solution architectureMany of the services selected areserverless, which will offer many benefits. At the time of writing, Stable Diffusion requires a GPU for inference. Amazon Elastic Compute Cloud (Amazon EC2) was selected because it provides GPUs. The solution architecture is shown in the Figure 1.Figure 1. Solution architecture diagramLet us walk through the architecture of this solution.Auto scaling with custom metricsTo properly scale the system, a custom Amazon CloudWatch metric is created. This custom CloudWatch metric calculates the number of Amazon Elastic Container Service (Amazon ECS) tasks required to adequately handle the amount of Amazon Simple Queue Service (Amazon SQS) messages. You should have a high-resolution CloudWatch metric to scale up quickly. For this use case, a high-resolution CloudWatch metric of every 10 seconds was implemented.Next, let’s create the custom CW metric. Amazon EventBridge rules provide a serverless solution for starting actions on a schedule. Here we use an Amazon EventBridge rule, which initiates an AWS Step Function Express Workflow every minute. With the Express Workflow, we can create serverless workflows that take less than five minutes, which helps us avoid long running AWS Lambda functions. The Express Workflow runs a Lambda function every 10 seconds over a one-minute period, which generates the custom CloudWatch metric.Two high-resolution CloudWatch alarms scale the system up and down, and are initiated by the custom CloudWatch metric. One CloudWatch alarm increases the ECS tasks and EC2 machines. The other alarm decreases the ECS tasks and EC2 machines.Handling Discord requestsSomeone on Discord sends a request. The Amazon API Gateway HTTP API receives the request and passes the information to an AWS Lambda function. The HTTP API provides a cost-effective option compared to REST APIs and provides tools for authentication and authorization. The HTTP API uses cross-origin resource sharing (CORS), which provides security because it only allows discord.com as an origin.The AWS Lambda function provides a serverless solution for responding to the HTTP API requests. It transforms the HTTP API request and sends a message to the SQS First-In-First-Out (FIFO) queue. SQS seamelessly decouples the architecture between user requests and backend processing. A FIFO queue ensures that user requests are processed in the order they were requested. The AWS Lambda function sends a response back to the HTTP API within three seconds, which is a requirement of Discord Slash Commands.When scaling up, an EC2 instance is registered with the ECS cluster. The EC2 instance type was selected because it provides GPU instances. ECS provides a repeatable solution to deploy the container across a variety of instance types. This solution currently only uses the g4dn.xlarge instance type. The ECS service will then place an ECS task onto the eligible EC2 instance. The ECS task will use the Amazon Elastic Container Registry (Amazon ECR) private registry to pull the image,  perform text-to-image processing, and respond to the Discord request. The ECR private registry is a managed container registry that manages the image.Once there is an ECS task running on an Amazon EC2 instance, the ECS task will consume messages from the queue using long polling. This reduces the amount of ReceiveMessage requests the ECS task needs to send. When the ECS task receives a message from the queue, it will then processes the request.Estimated monthly costThe example assumes 1,000 requests per month and each request takes 16 seconds to complete. Extra EC2 time was added for the time to begin processing messages (seven minutes) and auto scaling cooldown time (30 minutes). You can adjust the pricing calculations with theAWS Pricing Calculatorto reflect your usage and estimated cost.PrerequisitesThis blog assumes familiarity withTerraform,Docker,Discord,Amazon EC2,Amazon Elastic Block Store (Amazon EBS),Amazon Virtual Private Cloud (Amazon VPC),AWS Identity and Access Management (IAM),Amazon API Gateway,AWS Lambda,Amazon SQS,Amazon Elastic Container Registry (Amazon ECR),Amazon ECS,Amazon EventBridge,AWS Step Functions, andAmazon CloudWatch.For this walkthrough, you should have the following prerequisites:Access to an AWS account, with permissions to create the resources described in the installation steps sectionA virtual private cloud (VPC) with public subnets that is associated with an internet gateway in the region you are deploying intoWe suggest using the default VPC. The subnets will need the tag of key: Tier and value: Public and be attached to the VPC. If you decided to create your own VPC with subnets, make sure thatauto-assign IP settingsis enabled.AnIAM userwith the required permissions to deploy the infrastructureA new Discord application that is registered to a Discord server you own with the scope applications.command. Usethis tutorialif you need a starting point on creating a Discord application.Discord Bot tokenDiscord Application IDDiscord Public KeyAHugging Face accountA computer with the following packages installed:Terraform >=1.3.6gitAWS CLIdockerWalkthroughComplete the following steps to deploy this solution on AWS.Increase EC2 limitsThis solution uses the g4dn.xlarge instance type, which might require you torequest an EC2 limit increase. Check your current limit ofRunning On-Demand All G and VT instances. Make sure you have more than 4 vCPU; a single g4dn.xlarge requires 4 vCPU. We suggest requesting 8 vCPU so that you can access 2 g4dn.xlarge instances.Deploy the infrastructureEnsure you have at least 60 GB of storage available and you’re running on a 64-bit x86 architecture system.Open a command line on the machine you will be deploying from.Log in as your AWS userthrough the AWS CLI with the commandaws configure. If you are using an EC2 instance, create and use an instance profile rather than using the AWS CLI.The region you select will be the one you will deploy into.Clone the Terraform repository:git clone https://github.com/aws-samples/amazon-scalable-infra-discord-diffusion.gitNavigate into the Terraform repository:cd amazon-scalable-infra-discord-diffusionCustomize the variables interraform.tfvarsto match your deployment.Export the following secrets to the command line:export TF_VAR_discord_bot_secret='DISCORD_BOT_SECRET_HERE'export TF_VAR_huggingface_password='HUGGINGFACE_PASSWORD_HERE'Initialize the repository:terraform initApply the infrastructure (this takes about 2 minutes):terraform applySave the outputs for future use.Set up DiscordThis setup adds the Discord interactions URL to your Discord application. Afterterraform applycomes back successfully, move onto these steps.OpenDiscord Application Page->General Information.Copy and paste the value fromdiscord_interactions_endpoint_urlinto the Interactions Endpoint URL, and then save the changes.If successful, there should be a green box withAll your edits have been carefully recorded.Docker image and Amazon Elastic Container RegistryIn this section, you will create a docker image with the Stable Diffusion model.Exit the terraform repository:cd ..Clone the Docker build repository:git clone https://github.com/aws-samples/amazon-scalable-discord-diffusion.gitNavigate to the Docker repository:cd amazon-scalable-discord-diffusionBuild and pushthe docker image to ECR. This requires docker to be installed on the machine and actively running.You can find the commands for your deployment from the Amazon ECR repository.Figure 2. View push commands for Amazon ECRThis is a large image (10GB) and can take over 20 minutes to push depending on your machine’s internet connection.Request an image with Discord Slash CommandsThis section will describe how to request a text to image response with Discord.Log in to Discord and navigate to the server with your Discord application deployed.Navigate to a text channel.Type the command/sparkle.A box withCOMMANDS MATCHING /sparklewill appear. Select the/sparklecommand box.Depending on how you customized your Discord Application, the avatar image shown in Figure 3 might be different from what you have.Figure 3. Writing a Discord Slash CommandType in a prompt such asa corgi, style of monet.A response fromYourBotNameshould appear with the responseSubmitted to Sparkle: YourPromptHere, as shown in Figure 4.Figure 4. First response from AWS Lambda functionIt will take 10 minutes for an EC2 instance to start with an ECS Task running on the instance. Once an ECS Task is running on the instance, inference times should reduce to under 30 seconds, depending on the request.When an ECS Task is running your request, you will see aProcessing your Sparklemessage, as shown in Figure 5.Figure 5. Amazon ECS task processing a requestThe message is complete when it saysCompleted your Sparkle!as shown in Figure 6.Figure 6. Amazon ECS task returning the final responseCleaning upTo avoid incurring future charges, delete the resources created by the Terraform script.Return to the directory where you deployed your terraform script.To destroy the infrastructure in AWS, run the commandterraform destroy.When prompted to confirm that you want to destroy the infrastructure, typeyesand pressEnter.ConclusionIn summary, we created a solution that allows members of a Discord server to create images from text with a Stable Diffusion model. With this implementation, the deployment can scale to many Discord Servers and handle over one hundred requests per second.Create projects on AWS that lower the bar to entry for people wanting to try text to image models.TAGS:Architecture,aws,discord,highly available,scale,solution,stable diffusion,text to imageSteven WarrenSteven is a Technical Account Manager at AWS. He is passionate about building fun and relatable projects. He enjoys riding electric longboards during the summer and cross-country skiing / snowboarding during the winter with his family.
Stable Diffusion is a state-of-the-art text-to-image model that generates images from text. Deploying text-to-image models such as Stable Diffusion can be difficult. Currently, Stable Diffusion requires specific computer hardware known as graphical processing units (GPUs). You can lower the bar to entry by offloading the text-to-image generation onto Amazon Web Services (AWS). Discord is a […]
Steven Warren
Amazon CloudWatch Amazon EC2 Amazon Elastic Container Service Architecture