Basic Tutorial on Stable Diffusion ComfyUI (Part 1)

Installing ComfyUI and Common Plugins

Introduction ComfyUI:

Most of you probably use the web UI interface to work with Stable Diffusion (SD). But have you heard about ComfyUI?

ComfyUI is a node-based interface for Stable Diffusion, enabling more precise workflow customization and excellent reproducibility. Each module in ComfyUI has a specific function, and by adjusting the connections between modules, you can achieve various image generation effects. However, the node-based workflow can make it slightly more challenging to use. On the bright side, thanks to internal optimizations, ComfyUI generates images faster than the web UI and has lower VRAM requirements (it runs well on just 4GB).

ComfyUI  Pros and Cons:



Pros:

  • Lower VRAM requirements: ComfyUI starts up quickly and generates images faster.
  • Greater generation freedom: Offers more flexibility in creating images.
  • Shared environment and models: You can share the same environment and models with the web UI.
  • Custom workflows: You can build and export your workflows and share them with others, easily identifying errors in the process.
  • Workflow restoration: Dragging a generated image back into ComfyUI restores the entire workflow and selects the appropriate model.

Cons:

  • Higher operation threshold: Requires a clear logical understanding to operate.
  • Smaller ecosystem: While it supports many common plugins, the ecosystem isn’t as extensive as the web UI. However, there are some interesting plugins developed specifically for ComfyUI.

By following this guide, you'll get a good grasp of how to install ComfyUI, use its features, and make the most out of its plugins to enhance your Stable Diffusion experience. The ComfyUI interface not only improves your workflow efficiency but also provides a more customizable and flexible way to manage your image generation tasks.

Installation ComfyUI  Guide:

First, let's download the integrated package from GitHub that the author has prepared, which includes the environment and dependencies (no need for us to set it up, perfect for programming novices).

ComfyUI GitHub Repository

In the middle of the webpage, find the "Installing" section and click the "Direct link to download" button to start the download.

After downloading, extract the package to a suitable location. Open the folder, and you will see a file named run_nvidia_gpu. Double-click this file to launch ComfyUI. However, note that we don't have any models inside the folder yet.


ComfyUI  Configuring Models:

Model Locations:

Although ComfyUI comes with the environment and dependencies set up, it doesn't include any models. We need to place the models in the appropriate directories, for example:

  • Main Models: Place in ComfyUI_windows_portable\ComfyUI\models\checkpoints
  • VAE Models: Place in ComfyUI_windows_portable\ComfyUI\models\vae
  • Lora Models: Place in ComfyUI_windows_portable\ComfyUI\models\loras

Sharing Models with Web UI:

If you already have Web UI installed, don't worry. We can make ComfyUI share the same models to avoid duplicating large files and wasting space.

  1. In the ComfyUI directory, find the file named extra_model_paths.yaml.example.
  2. Rename this file to extra_model_paths.yaml (remove the .example).
  3. Open the file with a text editor (Notepad will do).

Edit the base_path: to the path where your Web UI is installed. For example, if your Web UI is installed at G:\sd-webui, you would modify it accordingly.

For ControlNet, whether you need to modify the path depends on where your ControlNet models are installed. If they are under the ControlNet plugin directory like mine, set it to extensions\sd-webui-controlnet\models. If you follow the older directory structure where ControlNet models are in model\controlnet\, then no changes are necessary.

ComfyUI  Updating:

In the ComfyUI_windows_portable\update directory, you will find two files: update_comfyui and update_comfyui_and_python_dependencies. These are used to update ComfyUI and configure the environment, respectively.

  1. Click on update_comfyui to update ComfyUI. Wait until you see "Done," indicating the update was successful.
  2. As for environment configuration, if everything is running smoothly, there's no need to touch it, and updates here are relatively infrequent.

ComfyUI  Adding Plugins:

Now that we have ComfyUI installed, let's add a few plugins to enhance our experience. Some plugins are needed for workflow processes, while others adjust the UI. I'll integrate the workflow-related plugins into the upcoming tutorial steps.

ComfyUI Plugin Installation:

ComfyUI Prompt Styler:

GitHub - twri/sdxl_prompt_styler

ComfyUI Custom Scripts:

GitHub - pythongosssss/ComfyUI-Custom-Scripts

ComfyUI Manager:

GitHub - ltdrdata/ComfyUI-Manager

ComfyUI Installation Method 1:

  1. Go to the required plugin links above.
  2. Click on "Code - Download ZIP" to download.
  3. After downloading, extract the files and place them in the ComfyUI_windows_portable\ComfyUI\custom_nodes directory.

If you have trouble downloading, I have uploaded the files to a cloud drive (link at the end).

ComfyUI Installation Method 2:

  1. Ensure git is installed.

  2. Navigate to the ComfyUI_windows_portable\ComfyUI\custom_nodes directory.

  3. Right-click to open the terminal in this directory.

  4. Copy and paste the following commands into the terminal:

    git clone https://github.com/twri/sdxl_prompt_styler.git
git clone https://github.com/pythongosssss/ComfyUI-Custom-Scripts.git
git clone https://github.com/ltdrdata/ComfyUI-Manager.git

  5. Press Enter and wait for the installation to complete.

ComfyUI Installation Method 3:

Prerequisite: Install the "ComfyUI Manager" plugin first. You can use the git method or the ZIP method mentioned above, or:

  1. Place the install-manager-for-portable-version.bat file into the ComfyUI_windows_portable directory.
  2. Double-click the install-manager-for-portable-version.bat file to install.

After installation, a "Manager" button will appear in the right-side menu. Click it to open the operation window.

Through this plugin, you can install, delete, and disable other plugins, download models, and update ComfyUI. The key features are "Install Nodes" and "Install Missing Nodes."

  1. Click "Install Nodes," search for the node you need, and click "Install." Wait for the installation to complete.
  2. If you find missing plugins (nodes turn red) while using shared workflows, click "Install Missing Nodes" to install them.

Using the "Manager" is simpler than the git method and also handles the manual configuration requirements for some plugins (though not all; specific installation requirements depend on the plugin).

ComfyUI-Custom-Scripts: The Swiss Army Knife of Enhancements

The ComfyUI-Custom-Scripts plugin is a collection of many handy tools, each marked with a small green snake icon for easy identification. Let's go over a few commonly used features that can significantly boost your efficiency.

1. Adding Lora and Checkpoint Loader Nodes:

These nodes can display preview images. Simply place a PNG or JPG file next to your model, and it will show up in the list when you hover over it.

2. Image Feed Display:

This feature records the images generated during the current session. By clicking on the settings panel, you'll find two options at the bottom: "Image Feed Location" and "Image Feed Direction." These settings allow you to choose the display location and sort order of new and old images. Once set, close the popup, and you'll see the image feed panel on the page. You can adjust the panel size and image display size using the "Resize Feed" option.

3. Node File Storage:

You can save your connected workflows as JSON files for easy sharing and loading. Here's how:

  • In the settings panel, click the triangle on the right side of "Save" and select "Save to Workflow." Enter a name and save it.
  • After saving, you'll notice a triangle next to the "Load" button. Click it to load your saved workflow.

These features make using ComfyUI more efficient and convenient, allowing for a smoother and more productive workflow. By leveraging ComfyUI-Custom-Scripts, you can enhance your ComfyUI experience and streamline your processes.

Comments

Post a Comment

Popular posts from this blog

Guide to Using the Image to Vector Node in ComfyUI

Image
Converting images to vectors is a very useful technique that transforms pixel-based images into vector graphics. Unlike pixel images, vector graphics are composed of mathematical equations rather than pixels, allowing them to be scaled infinitely without losing quality. In the field of AI-generated art, converting images to vectors can be used for post-processing generated results, creating editable vector line art. the source website Recently, a developer has packaged the vtracer image-to-vector algorithm as a ComfyUI node, making it convenient for us to use within ComfyUI workflows. This guide will provide detailed instructions on how to install and use this node. Example Results Before diving into the specific steps, let's take a look at a few examples to see the node's effects: Works best for simple graphics: Average results for realistic images and complex illustrations: ![Complex images to vector effect][] As we can see, the conversion works very well for images with simp...
Read more

Basic Tutorial for Stable Diffusion ComfyUI (Part 2): Text-to-Image Basic Workflow

Image
Introduction Basic Tutorial on Stable Diffusion ComfyUI (Part 1) In this lesson, we will begin learning about  ComfyUI , starting with the basics of text-to-image generation. To make things easier (especially for those of us who aren't fluent in English), most of the nodes in my article are translated into Chinese. If you also struggle with English, please refer to my previous content on how to install the translation plugin. Before we dive in, let's recall the features we used for text-to-image generation using the Web UI. These include the main model, positive prompts, negative prompts, sampler, steps, width and height, VAE, and displaying images. Now that we know the nodes involved, let's connect them in a workflow. Creating the Workflow in ComfyUI ComfyUI Main Model Loader Node Creation : Right-click and select "New Node -> Loader". You'll see many loaders with familiar names like "Checkpoint Loader (Main Model Loader)", "VAE Loader"...
Read more