A ComfyUI custom node plugin for the open-source framework DiffSynth-Studio. It integrates DiffSynth-Studio's model configuration, quantization, LoRA, and inference pipeline capabilities into ComfyUI, enabling node-based image, video, and audio generation and editing.
ComfyUI Desktop / ComfyUI-Manager:
- Open the sidebar Manage Extensions or Manager
- Search for DiffSynth-ComfyUI
- Click Install, then restart ComfyUI
- Install ComfyUI
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
pip install -r requirements.txt- Install DiffSynth-Studio
pip install git+https://github.com/modelscope/DiffSynth-Studio.git- Install this plugin
cd custom_nodes
git clone https://github.com/modelscope/DiffSynth-ComfyUI.git
cd ..
python main.pyIf you need quantization features, install the quantization extras:
pip install "diffsynth[quant] @ git+https://github.com/modelscope/DiffSynth-Studio.git"
Once started, open the ComfyUI address shown in the terminal (typically http://127.0.0.1:8188) in your browser.
Open the ComfyUI interface and load a preset using either method:
- Template: Click the Template tab in the left sidebar and select a DiffSynth template.
- JSON file: Choose a workflow from the Workflow Index. Use the
.jsonfile from your localexample_workflows/directory, or download it from GitHub using Download raw file on the file page, then drag it onto the ComfyUI canvas.
Modify parameters such as prompt and seed in the Inference node, provide any required input images, video, or audio, then click Run.
Right-click on a blank canvas to add nodes, then connect them in the following order:
VRAMConfig → ModelConfig → MergeModelConfigs ───┐
├──→ Loader → Inference → SaveImage
VRAMLimit ────────┘
- Add a VRAM Config node to configure VRAM strategy for each stage (optional)
- Add 3 ModelConfig nodes with the following settings:
model_id:Qwen/Qwen-Image,origin_file_pattern:transformer/diffusion_pytorch_model*.safetensorsmodel_id:Qwen/Qwen-Image,origin_file_pattern:text_encoder/model*.safetensorsmodel_id:Qwen/Qwen-Image,origin_file_pattern:vae/diffusion_pytorch_model.safetensors
- Connect the VRAM Config output to the
vram_configinput of all 3 ModelConfig nodes - Add a Merge ModelConfigs node and connect the outputs of the 3 ModelConfig nodes
- Add a VRAM Limit node (optional)
- Add a Qwen Image Loader node and connect
model_configsandvram_limit - Add a Qwen Image Inference node and connect the
pipeoutput from the Loader - Add a built-in SaveImage node and connect the
imageoutput from the Inference node - Fill in the
promptin the Inference node and click Run
All workflow JSON files are in example_workflows/. Both general_* and featured_* files can be imported using the steps above.
| Model | Workflows |
|---|---|
| Anima | Preview |
| Boogu Image | Base · Turbo · Edit |
| ERNIE Image | Standard · Turbo |
| Fibo 2.0 | Dev |
| FLUX.1 | Dev · FLEX.2 Preview |
| FLUX.2 | Dev · Klein 4B · Klein 9B |
| FLUX.3 | Klein · EntroPack |
| HiDream O1 | Image |
| Ideogram 4 | FP8 |
| JoyAI Image | Edit |
| Krea 2 | Raw · Turbo |
| MiniMax H3 Image | Text-to-Image · Edit · EntroPack |
| MOSS Image | Base · SFT |
| Qwen Image | Standard · 2512 · Edit · Edit 2511 Lightning |
| Qwen Image 2.1 | Featured · EntroPack |
| SenseNova U1.5 | MoT |
| Stable Diffusion | v1.5 · XL Base 1.0 |
| Z-Image | Standard · Turbo · Turbo Featured · Turbo EntroPack |
| Model | Workflows |
|---|---|
| LingBot Video | Text-to-Video |
| LTX-2 | Two-Stage · 2.3 Two-Stage |
| MiniMax H3 | FL2VA · FL2VA EntroPack · Ref2VA · Ref2VA EntroPack · Pruned NF4 |
| MOVA | 360P Image-to-Audio-Video |
| Qwen Video Edit | Video Editing |
| Wan Video | Text-to-Video 1.3B · Text-to-Video 14B · Fun A14B Control |
| Model | Workflows |
|---|---|
| ACE-Step 1.5 | Standard · XL SFT |
| MiniMax Music 3 | Music Generation |
| YuE2 | Music Generation |
All nodes are located under the DiffSynth category in the ComfyUI node menu, organized into 4 subcategories:
| Node | Function |
|---|---|
| VRAM Config | Configures DiffSynth-Studio's four-level VRAM management strategy (offload / onload / preparing / computation), specifying device and data dtype for each stage. Connect the output to the vram_config input of ModelConfig nodes. |
| VRAM Limit | Limits the available VRAM during inference, reserving buffer_size GB for the system. Connect the output to the vram_limit input of the Loader. |
| ModelConfig | Declares a model file's download source (model_id) and file path (origin_file_pattern). |
| Merge ModelConfigs | Merges multiple ModelConfig nodes into a single list for the Loader's model_configs input. |
| Quantization Config | Configures the model quantization scheme. method selects the quantization method (supports bitsandbytes NF4/FP4, etc.), mode selects the quantization mode. Optionally use target_modules and exclude_modules to control the quantization scope. Connect the output to the quant_config input of ModelConfig. |
| Mixed Quantize Config | Combines multiple Quantization Configs to apply different quantization strategies to different modules. Outputs a merged quant_config after connecting multiple Quantization Config inputs. |
Each pipeline has a corresponding Loader node that loads model weights, applies VRAM management strategy, and returns a pipe object.
Inputs:
model_configs(required): Merged configuration from Merge ModelConfigstorch_dtype: Model precisiondevice: Inference devicevram_limit(optional): From a VRAM Limit node- Other optional parameters: Vary by pipeline
Each pipeline has a corresponding Inference node that receives a pipe object and performs inference.
Parameters fall into two categories:
- Widgets: Parameters such as prompt, seed, num_inference_steps, height, and width that can be edited directly on the node
- Connection inputs: Parameters such as input_image and edit_image that are passed in via connections
| Node | Function |
|---|---|
| LoRA Clear | Clears all loaded LoRA weights on the pipeline. Usage: connect the pipe output from the Loader, then chain to LoRA Load after clearing old weights |
| LoRA Load | Loads a LoRA weight to a specified module. lora_config connects to a ModelConfig (declaring the LoRA file source), module specifies the target module, and alpha controls the strength. Multiple LoRA Load nodes can be chained to stack multiple LoRAs: Loader → LoRA Clear → LoRA Load → LoRA Load → Inference |
