Controlnet Models SDXL

ControlNet is a technique for adding conditional control to text-to-image diffusion models. It allows users to specify conditions or constraints on the generated images, such as the presence of specific objects, the style of the artwork, or the overall composition of the image.

Tip

Controlnets can sometimes introduce artifacts into the generated images. Ensure you use negative prompts to remove these.

Python
JavaScript
PHP

import requests
import json
url = "https://api.imagepipeline.io/sdxl/controlnet/v1"
headers = {
    "API-Key": "Your API Key"
}
data = {
    "prompt": "Snow White's face, best quality, extremely detailed",
    "negative_prompt": "monochrome, lowres, bad anatomy, worst quality, low quality",
    "model_id": "sdxl", //Add any sdxl checkpoint model_id from Models page
    "controlnets": ["canny"], //Single controlnet supported
    "controlnet_weights": [0.5],
    "init_images": ["https://cdn.pariscityvision.com/library/image/5449.jpg"], //Init img needs to be publicly available links
    "num_inference_steps": 20,
    "samples": 1,
    "guidance_scale": 7.5,
    "height": 1024,
    "width": 1024,
    "lora_models": [], //Add any sdxl LoRA model_ids from Models page
    "lora_weights": [0.5],
    "embeddings": [], //Add any sdxl embeddings model_ids from Models page
    "scheduler": "UniPCMultistepScheduler"
    //"server_id": "" //For enterprise users only
}
response = requests.post(url, headers=headers, data=json.dumps(data))
print(response.json())

const url = "https://api.imagepipeline.io/sdxl/controlnet/v1";
const headers = {
  "API-Key": "Your API Key",
  "Content-Type": "application/json"
};
const data = {
    "prompt": "Snow White's face, best quality, extremely detailed",
    "negative_prompt": "monochrome, lowres, bad anatomy, worst quality, low quality",
    "model_id": "sdxl", //Add any sdxl checkpoint model_id from Models page
    "controlnets": ["canny"], //Single controlnet supported
    "controlnet_weights": [0.5],
    "init_images": ["https://cdn.pariscityvision.com/library/image/5449.jpg"], //Init imgs needs to be publicly available links
    "num_inference_steps": 20,
    "samples": 1,
    "guidance_scale": 7.5,
    "height": 1024,
    "width": 1024,
    "lora_models": [], //Add any sdxl LoRA model_ids from Models page
    "lora_weights": [0.5],
    "embeddings": [], //Add any sdxl embeddings model_ids from Models page
    "scheduler": "UniPCMultistepScheduler"
    //"server_id": "" //For enterprise users only
};
fetch(url, {
  method: "POST",
  headers: headers,
  body: JSON.stringify(data)
})
.then(response => response.json())
.then(data => console.log(data))
.catch((error) => console.error('Error:', error));

<?php
$url = "https://api.imagepipeline.io/sdxl/controlnet/v1";
$headers = array(
  "API-Key: Your-API-Key",
  "Content-Type: application/json"
);
$data = array(
    "prompt"=> "Snow White's face, best quality, extremely detailed",
    "negative_prompt"=> "monochrome, lowres, bad anatomy, worst quality, low quality",
    "model_id"=> "sdxl", //Add any sdxl checkpoint model_id from Models page
    "controlnets"=> ["canny"], //Single controlnet supported
    "controlnet_weights"=> [0.5],
    "init_images"=> ["https://cdn.pariscityvision.com/library/image/5449.jpg"], //Init img needs to be publicly available links
    "num_inference_steps"=> 20,
    "samples"=> 1,
    "guidance_scale"=> 7.5,
    "height"=> 1024,
    "width"=> 1024,
    "lora_models"=> [], //Add any sdxl LoRA model_ids from Models page
    "lora_weights"=> [0.5],
    "embeddings"=> [], //Add any sdxl embeddings model_ids from Models page
    "scheduler"=> "UniPCMultistepScheduler"
    //"server_id"=> "" //For enterprise users only
);
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
$response = curl_exec($ch);
if($response === FALSE){
  die(curl_error($ch));
}
$responseData = json_decode($response, TRUE);
curl_close($ch);
var_dump($responseData);
?>

JSON Parameters

Parameter	Permissible values	Notes
model_id	`str`	model_id can be found in models page. Filter by foundation SDXL models
prompt	`str, 75 tokens`	Check our Prompt Guide for tips. Please add embeddings prompts to your prompt
negative_prompt	`str, 75 tokens`	Check our Prompt Guide for tips. Please add embeddings prompts to your prompt
num_inference_steps	`int, [1-100]`	Noise is removed with each step, resulting in a higher-quality image over time. Ideal value 20-30
samples	`int, [1-4]`	Generates a maximum of 4 samples per API call
guidance_scale	`float, [1-20]`	Higher guidance scale prioritizes text prompt relevance but sacrifices image quality. Ideal value 7.5-12.5
width	`int`	Width in pixels. Lower than or equal to 1024 for best results
height	`int`	Height in pixels. Lower than or equal to 1024 for best results
seed	`int`	Controlling the seed can help you generate reproducible images
controlnets	`str, array`	Pass the controlnet model name/id. Default values: canny, depth, openpose, softedge.
controlnet_weights	`float, array`	Strength of the Controlnet effect
lora_models	`str, array`	Pass the model_id(s) of LoRA models that can be found in models page
lora_weights	`float, array`	Strength of the LoRA effect
safety_checker	`boolean`	Checks for NSFW images and filters explicit images

Status

Your response will include a status.

If the status = SUCCESS, you will also have download_urls that will have the links to your generated image based on the number of samples you have entered. The maximum number of samples that can be generated is 4.
If the status = PENDING, you will receive a id. You can use the status endpoint to fetch your image using the id.
If the status = FAILURE, you will receive only an error message.

Use this endpoint to fetch the status:

https://api.imagepipeline.io/sdxl/controlnet/v1/status/{{id}}

Pass the API-Key as the authorization in the header.

](ControlNets-SD1.5.md)