When the first version of Stable Diffusion appeared back in August 2022, there was an explosion of innovation around it. In a matter of days after its release, the open-source community created lots of compelling derivatives, like the material diffusion models which generated tiling images, the inpainting models that gave you an AI paintbrush, and animation models that could interpolate between prompts.

Fast forward nearly two years, and Stable Diffusion 3 (SD3) just came out this week. People are hyped about the capabilities of this new model, and we expect to see a lot of the same open-source innovation we saw in Stable Diffusion’s early days.

In this post, we’ll show you how to get your own custom version of Stable Diffusion 3 running on Replicate.

Prerequisites

To follow this guide, you’ll need:

• Git You’ll need Git to clone the cog-stable-diffusion-3 repository.
• Docker. You’ll be using the Cog command-line tool to build and push your model. Cog uses Docker to create a container for your model. You’ll need to install and start Docker before you can run Cog. You can confirm Docker is running by typing docker info in your terminal.
• An account on Replicate.

Step 0: Create your model on Replicate

Start off by creating a page for your model on Replicate. Visit replicate.com/create and you’ll see a form to create the model.

Create the model under your personal account, or in an organization if you want to share access to the model with your team. Choose the username or organization from top left corner of the screen.

Choose a name for your model, set the hardware to “GPU A40 Small”, and click Create.

Your model will be private by default, but you can make it public at any time, even after you create it.

Step 1: Install Cog

Next, you’ll use Cog to package SD3 and push it to Replicate. Cog is an open-source tool that makes it easy to put a machine learning model in a Docker container.

Run the following command to install Cog:

Step 2: Clone the SD3 repository

Next, use Git to clone the cog-stable-diffusion-3 repository:

This repo contains an implementation of SD3 that uses the popular Diffusers Python library.

Step 3: Push to Replicate

Now it’s time to push your model to Replicate:

The owner name and model name must match the values you set when you created the model above.

Note: You can also set the image property in your cog.yaml file. This allows you to run cog push without specifying the image, and also makes your model page on Replicate more discoverable for folks reading your model’s source code.

Step 4: Iterate

You should now have your own custom version of SD3 running on Replicate! You can run in on the Replicate website or via the API.

Now it’s time to start experimenting with the model to customize it to your needs. Here are some ideas:

• Add support for inpainting.
• Add an upscaler to resize your generated images to a higher resolution.
• Update the prediction interface to allow more inputs to the underlying model.

Happy hacking!