Basic Tutorial for Stable Diffusion ComfyUI (Part 2): Text-to-Image Basic Workflow

Introduction

Basic Tutorial on Stable Diffusion ComfyUI (Part 1)

In this lesson, we will begin learning about ComfyUI, starting with the basics of text-to-image generation. To make things easier (especially for those of us who aren't fluent in English), most of the nodes in my article are translated into Chinese. If you also struggle with English, please refer to my previous content on how to install the translation plugin.

Before we dive in, let's recall the features we used for text-to-image generation using the Web UI.

These include the main model, positive prompts, negative prompts, sampler, steps, width and height, VAE, and displaying images. Now that we know the nodes involved, let's connect them in a workflow.

Creating the Workflow in ComfyUI



ComfyUI Main Model Loader

  1. Node Creation: Right-click and select "New Node -> Loader". You'll see many loaders with familiar names like "Checkpoint Loader (Main Model Loader)", "VAE Loader", and "Lora Loader". Start by adding the main model loader (you can also use the Checkpoint Loader from the ComfyUI-Custom-Scripts plugin we installed in the last lesson, which allows thumbnail previews).
  2. Model Selection: Click the "Checkpoint Name" field to choose the model you need.

ComfyUI Text Input Nodes (CLIP Text Encoder)

Next, add the keyword input nodes. Note that there's no distinction between positive and negative keywords; they're both entered using the "CLIP Text Encoder" node.

  1. Node Creation: Right-click and select "New Node -> Condition -> CLIP Text Encoder". Add two CLIP Text Encoder nodes for positive and negative keywords.
  2. Naming and Coloring: To differentiate, right-click above the node, select "Title", and enter a name. To change the node color, right-click, select "Color", and choose a color.



You'll notice both the Checkpoint Loader and the CLIP Text Encoder have a yellow "CLIP" connection point. Connect these points by dragging with the left mouse button.

Note: The Checkpoint Loader has only one CLIP connection point, but it can output to multiple nodes, while each input can connect to only one node.

ComfyUI Sampler

  1. Node Creation: Right-click and select "Sample -> Sampler" (choose the regular sampler).

The sampler has seven adjustable options:

  • Random Seed: Each image has a unique random identifier.
  • Post-Run Operation: Choose to fix, increase, decrease, or randomize the seed.
  • Steps: Set the number of denoising steps.
  • CFG: Prompt guidance coefficient; higher values align more closely with the prompt, typically around 8.
  • Sampler: Choose a sampling method (e.g., Euler Ancestral, DPM++ 2m).
  • Scheduler: Controls noise amount per iteration step, usually set to normal or karras.
  • Denoising: Related to steps; 1 means 100% denoising as per the steps, 0.1 means 10%.

Connect "Model" to the Checkpoint Loader's model, "Positive Prompt" to the positive prompt condition, and "Negative Prompt" to the negative prompt condition. The "Latent" output connects to the node controlling image width and height.



ComfyUI Width and Height (Empty Latent)

Click and drag from "Latent" to create an "Empty Latent" node.

Here, you can set the width, height, and batch size (number of images generated at once).

ComfyUI VAE Decode

Note: Choose "VAE Decode" instead of "VAE Loader".

  1. Node Creation: Right-click and select "New Node -> Latent -> VAE Decode". You can also drag to create this node.
  2. Connections: Connect the "VAE Decode Latent" to the sampler's latent. Connect the remaining VAE point on the left to the Checkpoint Loader's VAE.

Note: Most main models include a VAE model. You can connect directly or add a "VAE Loader" to load a separate VAE model.



ComfyUI Save Image

  1. Node Creation: Right-click and select "New Node -> Image -> Save Image". You can also drag to create this node.

Saving images will store them in the "output" folder under ComfyUI, while still providing a preview.



ComfyUI Generating the Image

Once all nodes are connected, enter your keywords and adjust settings like the model, steps, and dimensions. Click the right panel's "Prompt Queue" or press "ctrl+enter" to generate the image.

If the image is successfully generated, it means the nodes are correctly connected. Save your workflow as taught in the previous lesson for future use. During the generation process, the current step will be highlighted in green, helping you understand the workflow better.



ComfyUI Additional Tips

  • Copy: Ctrl+C
  • Paste: Ctrl+V
  • Paste with Connections: Ctrl+Shift+V
  • Multi-Select: Shift + Left-click
  • Box Select: Ctrl + Left-click drag
  • Move Multiple Nodes: Shift + Left-click drag
  • Resize Nodes: Drag the bottom-right corner
  • Group Nodes: Right-click on an empty space, select "New Group", and customize the title and color.

By following these steps, you'll have a basic understanding of using ComfyUI for text-to-image generation with Stable Diffusion.


Comments

Post a Comment

Popular posts from this blog

Guide to Using the Image to Vector Node in ComfyUI

Image
Converting images to vectors is a very useful technique that transforms pixel-based images into vector graphics. Unlike pixel images, vector graphics are composed of mathematical equations rather than pixels, allowing them to be scaled infinitely without losing quality. In the field of AI-generated art, converting images to vectors can be used for post-processing generated results, creating editable vector line art. the source website Recently, a developer has packaged the vtracer image-to-vector algorithm as a ComfyUI node, making it convenient for us to use within ComfyUI workflows. This guide will provide detailed instructions on how to install and use this node. Example Results Before diving into the specific steps, let's take a look at a few examples to see the node's effects: Works best for simple graphics: Average results for realistic images and complex illustrations: ![Complex images to vector effect][] As we can see, the conversion works very well for images with simp...
Read more

Basic Tutorial on Stable Diffusion ComfyUI (Part 1)

Image
Installing ComfyUI and Common Plugins Introduction  ComfyUI : Most of you probably use the web UI interface to work with Stable Diffusion (SD). But have you heard about ComfyUI? ComfyUI is a node-based interface for Stable Diffusion, enabling more precise workflow customization and excellent reproducibility. Each module in ComfyUI has a specific function, and by adjusting the connections between modules, you can achieve various image generation effects. However, the node-based workflow can make it slightly more challenging to use. On the bright side, thanks to internal optimizations, ComfyUI generates images faster than the web UI and has lower VRAM requirements (it runs well on just 4GB). ComfyUI  Pros and Cons: Pros: Lower VRAM requirements:  ComfyUI starts up quickly and generates images faster. Greater generation freedom:  Offers more flexibility in creating images. Shared environment and models:  You can share the same environment and models with the web UI...
Read more