Lately, I've been experimenting with AI-based synthesis tools that are revolutionizing video production. What are your experiences with them? Especially in terms of realism, fluidity of movement, and lip-sync, which methods seem to work best? What are the best practices to follow to get optimized outputs in my own projects? Are there any end-to-end workflows, render settings, or post-production tips that you pay attention to? I'm also interested in performance comparisons of different approaches.
How to improve synthesis with video AI tools?
👁️ 2 views💬 1 replies❤️ 0 likes
1 Replies
Yes, as someone who's messed around a lot with video synthesis tools recently, I can say the most critical part is **the quality of the key frames used as reference for motion**. Especially with tools like Stable Diffusion Video or Runway ML, the details and pose of the first frame given determine the realism of the entire video. In my projects, I got the most realistic results using high-resolution (4K) and well-lit animated reference videos (not motion capture, but manually crafted puppeteering). Even the expression on the face in the starting image needs to be optimized; otherwise, the whole render comes out rough.
As for the method, there's a three-step workflow worth trying:
- In the first render, **keep the denoising level low** (most tools work best between 0.2-0.4; anything higher causes blurring).
- **Fix the iteration count (iterations) around 30-50**—increasing it further doesn’t improve performance linearly.
- After that, I boost smoothness by **upscaling with Topaz Video AI** or using **frame interpolation** (like RIFE). For audio sync, the most reliable method is auto-syncing with Adobe Audition and then making small manual adjustments.