ComfyUI Step-by-step tutorial - Wan Animate 2.2 workflow build from scratch. Learn ComfyUI video

Introduction

This tutorial guides you through building the Wan Animate 2.2 workflow from scratch using ComfyUI. You'll learn to create a complex workflow involving image processing, character animation, and background isolation. The skills gained here will enhance your ability to use ComfyUI for generating artificial images and videos.

Step 1: Open the Wan Animate Template and Download Models

• Start by opening the Wan Animate template in ComfyUI.
• Download the six necessary models required for the workflow.
• Ensure you have a stable internet connection for downloading.

Step 2: Install Missing Custom Nodes and Video Helper Suite

• Navigate to the ComfyUI interface.
• Install any missing custom nodes and the Video Helper Suite to ensure all functionalities are available.
• Restart ComfyUI if prompted after installation.

Step 3: Understand the Workflow Process

• Familiarize yourself with the basic concepts of the Wan Animate workflow.
• Review the intended outcomes: character animation and character replacement, using the Wan Animate model.

Step 4: Set Up the Empty Wan Animate Canvas

• Create a new canvas in ComfyUI.
• Begin with adding a K Sampler node to your canvas.

Step 5: Decode Latents and Connect to Video Combine Node

• Add a decoding node to process the latents.
• Connect this node to the Video Combine node to prepare for video processing.

Step 6: Add Models to the Workflow

• Include the following models in your workflow:
  • Wan Animate diffusion model
  • Loras
  • SD3 shift
• Group these models for easier management.

Step 7: Integrate Wan Animate to Video Node

• Add the Wan Animate to Video node to the workflow.
• This node will help in linking the animation outputs to the video processing.

Step 8: Add Text Encode and Clip Vision Model Load Nodes

• Incorporate the text encode node.
• Add the Clip Vision model load nodes for processing images.

Step 9: Connect and Encode Reference Image

• Connect the reference image using the VAE and Clip Vision nodes.
• Ensure proper encoding for accurate output.

Step 10: Create and Connect Face and Poses

• Use the DWPose 'controlnet' preprocessors to create face and pose nodes.
• Connect these nodes to the animation video reference.

Step 11: Complete Workflow Settings and Run Animation

• Review all settings in your workflow for consistency.
• Run the animation workflow to start processing.

Step 12: Correct Reference Image in Video Output

• Use the trim latent node to adjust the reference image.
• This step ensures the output maintains the desired quality.

Step 13: Use Subgraphs for Video Concatenation

• Create subgraphs to concatenate sets of sampled videos.
• This allows for the extension of the video output seamlessly.

Step 14: Correct Second Animation Subgraph

• Focus on the second animation subgraph to concatenate the videos.
• Ensure batch images are handled correctly for smooth transitions.

Step 15: Correct Repeated Frames

• Use the Image From Batch node to manage repeated frames.
• Apply the trim image function to refine the output.

Step 16: Create Mask for Character Replacement

• Design a mask over the character to isolate it.
• Use tools within ComfyUI to ensure the background is effectively removed.

Step 17: Utilize BF16 Wan Animate Model

• Implement the BF16 Wan Animate model in your workflow for enhanced performance.

Step 18: Install ComfyUI-GGUF for Quantized Models

• Install ComfyUI-GGUF to load GGUF (quantized) Wan Animate models.
• This enables efficient processing of larger models.

Conclusion

You now have a comprehensive understanding of how to build the Wan Animate 2.2 workflow using ComfyUI. Practice these steps to refine your skills in creating complex workflows for animation and video generation. Explore additional resources and communities for further learning and support as you continue your journey in artificial image and video creation.