Skip to main content

Prompt Guide

Stable Diffusion 1.5​

Prompt engineering is a crucial aspect of utilizing Stable Diffusion Models 1.5 (SD 1.5) and the more advanced SDXL. This process involves designing prompts to achieve desired results in image generation. Whether you're working with single models or multimodels, effective prompt engineering significantly influences the quality of the generated images.

Positive Prompt Engineering Tips:

  • Provide detailed prompts to enhance the model's understanding. For example, instead of a generic a cat use a black and white tabby cat sitting in a window.

  • Employ keywords that directly relate to the image you want. For a cat image, include keywords like cat, feline, and furry.

  • Exclude undesired elements using negative prompts. For instance, use no clothes to ensure a cat image without clothing accessories.

  • Try various prompts to find the most effective approach. There's no one-size-fits-all method for prompt engineering.

Negative Prompt Engineering Tips:

  • To use the negative prompt parameter, simply use the negative_prompt parameter followed by the things you don't want in your image, separated by commas.

  • You can also use the negative prompt parameter to exclude entire categories of things. For example, if you don't want any humans in your image, you would use the following prompt:

  • prompt: A black cat sitting on a red couch

  • negative prompt: humans

Here are some more examples of negative prompts using the negative prompt parameter:

Example 1

  • prompt: A photorealistic image of a black cat sitting on a red couch
  • negative prompt: text, watermarks, blurry, low quality, cartoon

Example 2

  • prompt: A digital painting of a tabby cat playing with a ball of yarn in a field of flowers
  • negative prompt: cartoons, illustrations, blurry, logos

Example 3

  • prompt: A cartoonish image of a person holding a beer while standing on an iceberg
  • negative prompt: low-quality, blurry, disfigured, poorly drawn hands, multiple bodies, multiple heads, too many fingers

Example 4

  • prompt: A realistic landscape painting with mountains and a lake, in the style of Bob Ross
  • negative prompt: abstract images, surreal images, low quality

Multimodel Generation Prompt​

  • Use LoRAs and Controlnets to control the style and content of the generated images. LoRAs and Controlnets can be used to make the generated images more realistic, artistic, or abstract.
  • Use embeddings to represent the different concepts in your prompt. Embeddings can help the model to better understand your prompt and generate the desired images.

Here is an example of a prompt that you could use for a multimodel generation:

  • prompt: a fluffy white cat sitting on a red couch
  • models: stable_diffusion, lora_diffusion, controlnet

Stable Diffusion XL​

Here are some of the key differences between SDXL and Stable Diffusion 1.5:

  • SDXL is a pipeline model, while Stable Diffusion 1.5 is a single-stage model. This means that SDXL is made up of multiple stages, each of which performs a specific task. This allows SDXL to generate more complex and detailed images.
  • SDXL is trained on a larger dataset of images and text than Stable Diffusion 1.5. This gives SDXL a better understanding of the world and allows it to generate more realistic and believable images.
  • SDXL is able to generate images in a wider variety of styles than Stable Diffusion 1.5. This includes realistic, cartoonish, artistic, and abstract styles.
  • SDXL is better at understanding and following complex prompts than Stable Diffusion 1.5. This means that you can give SDXL detailed instructions about what you want in the image, and it is more likely to be able to generate the image correctly.
Tip

Remember to use clear and concise language when prompting SDXL. Do not use long prompts.

  • Provide detailed prompts for SDXL, considering its sensitivity to prompt nuances.

  • Use negative prompts to avoid common issues like distorted faces or unrealistic proportions.

  • Exercise caution with keyword weights, ensuring a balanced approach for optimal results.

  • Understand that SDXL is still under development; regularly test and revise prompts for desired outcomes.

Examples of Effective Prompts for SDXL:

  1. Prompt Engineering:

    • A photorealistic portrait of a young woman with curly brown hair and green eyes, smiling at the camera
    • A painting of a forest in a vibrant autumn style, with a river running through it
    • A digital art illustration of a futuristic city, with towering skyscrapers and flying cars

  2. Negative Prompt:

    • Distorted faces, Unrealistic body proportions, Cartoons, Anime, Low quality, Blurry

Emphasis and Weights​

Introduction: In the realm of Stable Diffusion Models 1.5 and the advanced SDXL, mastering prompt engineering involves not only crafting precise prompts but also utilizing emphasis and weights. This guide explores how parentheses and weights can be employed to enhance the model's understanding and control the generated images.

Example with Emphasis and Weights:

  • prompt: A black cat sitting on a red couch
  • negative prompts: No clothes, no humans, no cartoons
  • weights: cat+++ couch++

Explanation: In this example, the parentheses are used to influence the weight of keywords. The prompt cat+++ couch++ emphasizes a strong focus on the cat and a lesser emphasis on the couch.

If a higher emphasis on the cat is desired, additional parentheses can be employed:

Enhanced Cat Emphasis:

  • prompt: A black (((cat))) sitting on a red couch.

The triple parentheses around cat signal the model to assign the highest possible weight to this keyword.

Combining Emphasis with Negative Prompts:

  • prompt: A black (((cat))) sitting on a red couch, (((no humans)))

Here, the prompt instructs the model to give a strong emphasis to the cat while also emphasizing the absence of humans. Caution is advised to avoid overusing parentheses, as it may lead to unintended effects.

Increasing Weight for Photorealism:

  • prompt: A ((((photorealistic))) image of a black (((cat+++))) sitting on a red couch

In this instance, the model is directed to generate a photorealistic image with a strong emphasis on both the cat and the photorealism.

Warning

Using parentheses sparingly is crucial to prevent the model from disregarding the keywords within them.