Sign In

💪 Flexi-Workflow :: Flux · SDXL [ Illustrious · Pony ] · SD 3.5 · SD 1.5 · Kolors · Hunyuan-DiT

41
1k
24
Type
Workflows
Stats
139
0
Reviews
Published
Dec 27, 2024
Base Model
Flux.1 D
Hash
AutoV2
6F82C1D6AB
The FLUX.1 [dev] Model is licensed by Black Forest Labs. Inc. under the FLUX.1 [dev] Non-Commercial License. Copyright Black Forest Labs. Inc.
IN NO EVENT SHALL BLACK FOREST LABS, INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.

Introduction

💪 This is my attempt at a flexible and extensible workflow framework in variants of Flux (including compatibility with Chroma), SDXL (including compatibility with Illustrious, NoobAI, and Pony), SD 3.5, SD 1.5, Kolors, and Hunyuan-DiT; with others in development: PixArt Sigma, HiDream, and Wan (as a separate FlexiVid-Workflow). Many customizable pathways are possible to create particular recipes 🥣 from the available components, without unnecessary obfuscation (e.g., noodle convolution, stacking nodes over others, etc.) and arguably capable of rendering results of similar quality to more complicated specialized workflows.

The workflow was developed and tested on the following system:

  • Operating system: Linux Mint 21.3 Cinnamon with 62 GB RAM

  • Processor: 11th Gen Intel© Core™ i9-11900 @ 2.50GHz × 8

  • Graphics card: NVIDIA GeForce RTX 3060 with 12 GB VRAM

  • Browser: Google Chrome

Please report bugs 🪲 or errors 🚫, as well as successes 🤞 and requests/suggestions 📝. Post and share your SFW creations!!! I spent a lot of time working on this project (((for no 💰))), so I hope others make good use of it and find it helpful.

Flux [v5]

The Flux version 5 💪 flexi-workflow is highlighted by Flux Tools 🛠️: Fill (optionally with ACE++), Canny & Depth, and Redux. Basic ControlNets are also available, including regional control, along with two face swap tools (PuLID and ZenID), detailing, relighting, EasyControl 🚧, InstantCharacter 🚧, facial expression editor, and RF inversion. Additional accessories include four captioners (Florence2, JoyCaption2, WD14 Tagger, and SuperPrompt), three background removers (RemBG, RMBG, and Florence2 + SAM2), XY plots, five upscalers (basic, Thera, InvSR, Ultimate SD, and SUPIR; with tiled diffusion nodes also available), three video generators (DepthFlow, Sonic for talking avatars, and Wan), and stereoscope. A simple interface for Gemini AI is included. Core 🦴 and lite 🪶 editions are also available in the package.

Flux :: Compatibility

Flux :: What's new?

  • [v4] Added Gemini AI, facial expression editor, and Thera upscaler.

  • [v4] Replaced Hunyuan video with Wan 2.1 video:

    • employs (mostly) native nodes

    • text-to-video, image-to-video (default), and video-to-video (using ControlNet) options

    • ControlNet Fun and LoRA models implemented; VACE not yet available

    • simple upscaling and interpolation

  • [v5] Greatly simplified the Wan video group, now limited to image-to-video and first-last-frame-to-video options; a more robust separate Wan workflow is in development.

  • [v4] Replaced OmniGen with EasyControl 🚧, but the current implementation is experimental and exceeding my available VRAM to even run.

  • [v5] Added InstantCharacter 🚧, but the current implementation is experimental and exceeding my available VRAM to even run.

  • [v4] MutiGPU loaders now default, except for Wan where they seemed to be a source of instabilities.

  • [v4] Overhauled ControlNets + groups:

    • simplified Redux

    • restructured basic ControlNets to allow three different models concurrently

    • regional control that respects different LoRAs

  • [v4] Cleaned up and improved workflow:

    • more color-coding of nodes

    • better organization and sorting of bookmarks

    • added global seed node

    • added simple latent operations (between samplers)

    • fixed default masking bug

    • upgraded inpainting crop-and-stitch

    • added model switch, for easier implementation of specialized recipes

  • [v5] Reconfigured second sampling, which should offer more flexibility.

  • [v5] Replaced image output comparison node, which works better and offers more functions.

  • [v5] Redux now uses ReduxFineTune node, for more intuitive controls.

  • [v5] Stereoscope now offers rendering of videos.

  • [v5.1] Removed Golden Noise node.

  • [v5.2] Added latent combo option and cleaned up latent group.

  • [v5.3] Added tiled diffusion node for an additional upscaling option.

  • [v5.3] Added SD3 model sampling node, which may improve rendering even in Flux.

  • [v5.3] Removed requirement for Crystools nodes.

Flux :: Known Issues

  • [v5] Running the EasyControl 🚧 and InstantCharacter 🚧 groups currently exceed my available VRAM and I have been unable to run or test these locally.

  • [v5] The text-to-speech audio nodes (PC-ding-dong) don't seem to work most of the time.

Flux :: Quick start

  1. Install or update ComfyUI to the very latest version. Follow your favorite YouTube installation video, if needed.

  2. Install ComfyUI Manager.

  3. Download the following models or equivalents. Follow the Quickstart Guide to Flux.1, if needed.

  4. Download the FLUX.1-Turbo-Alpha LoRA.

  5. Open the Flux :: Flexi-Workflow in ComfyUI. You may want to start with the lite 🪶 edition, especially if you are new to ComfyUI.

  6. Use the Manager to Install Missing Custom Nodes. It is recommended to install just a few custom node packages at a time until you get through all of them. You may need to set security_level = normal- (notice the dash/minus!) in the config.ini file to download some custom nodes.

  7. Restart ComfyUI.

  8. Load models (01a) and LoRAs (03c) according to your folder structure.

  9. Run the default text-to-image recipe 🥣.

  10. Enjoy your generated image creations! 😎

Flux :: Navigation

The workflow is structured for flexibility. With just a few adjustments, it can flip from text-to-image to image-to-image to inpainting or application of Flux Tools 🛠️. Additional unlinked nodes have been included to provide options and ideas for even more adjustments, such as linking in nodes for increasing details. (The workflow does not employ Anything Everywhere, so if a node connection looks empty, it really is empty.)

In the Switchboard, flip the yes|no 🔵 toggles to activate or deactivate groups and the jump arrows ➡️ to quickly move to particular groups for checking and making adjustments to the settings/switches.

🛑 DO NOT RUN THE WORKFLOW WITH ALL SWITCHES FLIPPED TO "YES"! 🛑

There are also bookmarks 🔖 to help you navigate quickly.

In the rgthree settings, it is also recommended to show fast toggles in group headers for muting.

In the Lite Graph section of the settings, enable the fast-zoom shortcut and set the zoom speed to around 1.5–1.75. The workflow was built with a snap to grid size of 20.

Most of the workflow is unpinned 📌, so grab any empty space with your mouse (while pressing the control key) to navigate around. You are welcome to pin 📌 anything to prevent accidentally moving groups or nodes.

Flux :: Recipes

This is the default text-to-image recipe 🥣 and should be run first to make sure you have the basics configured correctly.

💪 :: Toggle to "yes" 01a; 02b; 03 all; 04; and 05
03a :: Latent switch = 1 (empty)
03b :: Conditioning switch = 1 (no ControlNets +)
03e :: Denoise = 1; Guidance = 1.2–5; Steps 20–30, or 8–12 w/ Turbo LoRA
05 :: Image switch = 2 (save generated image)

Reference the Start Here group to find additional workflow recipes 🥣.

SDXL [v5]

The SDXL version 5 💪 flexi-workflow includes all of the components of the Flux variant, except for Flux Tools, RF inversion, EasyControl, and InstantCharacter. Core 🦴 and lite 🪶 editions are also available in the package.

SDXL :: Compatibility

SDXL :: Quick start

  1. Install or update ComfyUI to the very latest version. Follow your favorite YouTube installation video, if needed.

  2. Install ComfyUI Manager.

  3. Download one the following models or equivalent. Follow the SDXL 1.0 Overview (possibly slightly outdated), if needed.

  4. Optionally, download the SDXL+Pony Lowstep LoRA 3-4 CFG 9-16 Steps LoRA.

  5. Open the SDXL :: Flexi-Workflow in ComfyUI. You may want to start with the lite 🪶 edition, especially if you are new to ComfyUI.

  6. Use the Manager to Install Missing Custom Nodes. It is recommended to install just a few custom node packages at a time until you get through all of them. You may need to set security_level = normal- (notice the dash/minus!) in the config.ini file to download some custom nodes.

  7. Restart ComfyUI.

  8. Load models (01a) and LoRAs (03c) according to your folder structure.

  9. Run the default text-to-image recipe 🥣.

  10. Enjoy your generated image creations! 😎

SD 3.5 [v5]

The SD 3.5 version 5 💪 flexi-workflow includes all of the components of the SDXL core 🦴 and lite 🪶 editions.

SD 3.5 :: Compatibility

SD 3.5 :: Quick start

  1. Install or update ComfyUI to the very latest version. Follow your favorite YouTube installation video, if needed.

  2. Install ComfyUI Manager.

  3. Download the following models or equivalents. Follow Getting Started with Stable Diffusion 3.5, if needed.

  4. Optionally, download the TensorArt TurboX LoRA 8 Steps [SD3.5 Large] LoRA.

  5. Open the SD 3.5 :: Flexi-Workflow core 🦴 or lite 🪶 edition in ComfyUI.

  6. Use the Manager to Install Missing Custom Nodes. It is recommended to install just a few custom node packages at a time until you get through all of them. You may need to set security_level = normal- (notice the dash/minus!) in the config.ini file to download some custom nodes.

  7. Restart ComfyUI.

  8. Load models (01a) and LoRAs (03c) according to your folder structure.

  9. Run the default text-to-image recipe 🥣.

  10. Enjoy your generated image creations! 😎

SD 1.5 [v5]

The SD 1.5 version 5 💪 flexi-workflow includes all of the components of the SDXL core 🦴 and lite 🪶 editions.

SD 1.5 :: Compatibility

SD 1.5 :: Quick start

  1. Install or update ComfyUI to the very latest version. Follow your favorite YouTube installation video, if needed.

  2. Install ComfyUI Manager.

  3. Download the following model or equivalent.

  4. Open the SD 1.5 :: Flexi-Workflow core 🦴 or lite 🪶 edition in ComfyUI.

  5. Use the Manager to Install Missing Custom Nodes. It is recommended to install just a few custom node packages at a time until you get through all of them. You may need to set security_level = normal- (notice the dash/minus!) in the config.ini file to download some custom nodes.

  6. Restart ComfyUI.

  7. Load models (01a) and LoRAs (03c) according to your folder structure.

  8. Run the default text-to-image recipe 🥣.

  9. Enjoy your generated image creations! 😎

Kolors [v5]

The Kolors version 5 💪 flexi-workflow includes all of the components of the SDXL core 🦴 and lite 🪶 editions. The ControlNets group was reworked to be compatible, but with limited testing.

Kolors :: Compatibility

  • Kolors UNET variants

Kolors :: Quick start

  1. Install or update ComfyUI to the very latest version. Follow your favorite YouTube installation video, if needed.

  2. Install ComfyUI Manager.

  3. Download the following models or equivalents.

  4. Open the Kolors :: Flexi-Workflow core 🦴 or lite 🪶 edition in ComfyUI.

  5. Use the Manager to Install Missing Custom Nodes. It is recommended to install just a few custom node packages at a time until you get through all of them. You may need to set security_level = normal- (notice the dash/minus!) in the config.ini file to download some custom nodes.

  6. Restart ComfyUI.

  7. Load models (01a) and LoRAs (03c) according to your folder structure.

  8. Run the default text-to-image recipe 🥣.

  9. Enjoy your generated image creations! 😎

Hunyuan DiT [v5]

The Hunyuan-DiT version 5 💪 flexi-workflow includes all of the components of the SDXL core 🦴 and lite 🪶 editions. No ControlNets have been tested at all.

Hunyuan DiT :: Compatibility

Hunyuan DiT :: Quick start

  1. Install or update ComfyUI to the very latest version. Follow your favorite YouTube installation video, if needed.

  2. Install ComfyUI Manager.

  3. Download the following models or equivalents.

  4. Open the Hunyuan DiT :: Flexi-Workflow core 🦴 or lite 🪶 edition in ComfyUI.

  5. Use the Manager to Install Missing Custom Nodes. It is recommended to install just a few custom node packages at a time until you get through all of them. You may need to set security_level = normal- (notice the dash/minus!) in the config.ini file to download some custom nodes.

  6. Restart ComfyUI.

  7. Load models (01a) and LoRAs (03c) according to your folder structure.

  8. Run the default text-to-image recipe 🥣.

  9. Enjoy your generated image creations! 😎


PixArt Sigma [v5]

The PixArt Sigma version 5 💪 flexi workflow is in development.

HiDream [v5]

The HiDream version 5 💪 flexi workflow is in development.

Wan [v5]

The Wan version 5 💪 flexi workflow is in development, but will likely be uploaded as a separate FlexiVid-Workflow.