Latest model achieves state-of-the-art performance when adding voice to any video

SAN FRANCISCO: #AIContentTools--Tavus, a leading AI video research company backed by Sequoia, today announced the release of Hummingbird-0 into research preview, a zero-shot lip sync model created from components of its flagship Phoenix-3 replica model. Now, with just one video and any voice track, developers can bring faces to life—instantly—without model training or manual tweaking. This step up in quality opens the door to high-quality user-generated content, foreign language dubbing for localization, and personalized videos created at scale, in minutes.

“Lip sync technology has been around for years, but until now, it’s never really been great — open source or otherwise,” said Effie Goenawan, Head of Product at Tavus. “With Hummingbird-0, we're giving developers access to a state-of-the-art lip sync model that unlocks an entirely new level of creative potential. It actually emerged as a happy accident while we were developing our full-face replica rendering model, Phoenix-3, and it’s a testament to the brilliance and curiosity of our research team.”

Helping Content Creation Take Flight

The Hummingbird model is designed to modify the lip movements in a given video to match the content of a driving audio signal. The guiding principle is to preserve the original identity, expressions, and visual quality of the person in the video while synchronizing their lip movements with the new audio.

Notably, with Hummingbird-0, ​​users can create content much faster because they don't have to train a model. All that’s needed is a video of a person speaking–one already in existence or one created using a video generator like Veo or Kling. From making memes talk to instantly localizing thousands of B2B videos, Hummingbird-0 puts high-quality lip sync just an API call away.

“Text-to-video generation models have become enormously popular for content creation, but there is a problem in that the video is muted; there’s no voice,” said Hassaan Raza, CEO of Tavus. “We are adding that voice that can go on top of any video where there is a human. This serves as an enabler not just for more, different, or better content, but for new types of products and experiences altogether. Once developers get a taste of Hummingbird-0, they want to know more about what our entire family of models can do. Hummingbird-0 barely scratches the surface of our capabilities as we continue developing the human layer of AI.”

Hummingbird-0 specifically gives developers the tools to overcome challenges associated with video content creation. For example, it offers:

• Scalable Personalization: Transform a single source video into thousands of personalized versions with different audio tracks, dramatically reducing production costs for marketing, educational, and localized content.
• Editing Video Dialogue in Post: Build editing workflows into any video app. Users can update or adapt existing footage of dialogue using text or audio—no reshoots, no heavy post-production.
• Integrate with Video Generation: Build an AI film studio. Add dialogue, the missing puzzle piece to videos generated by Sora, Veo, Runway, Kling, and more.
• Efficient Content Repurposing: Leverage existing footage to generate new videos with updated messaging or corrections without costly reshoots or complex post-production workflows.

Unparalleled Performance

Hummingbird-0 is already demonstrating best-in-class performance in visual quality, lip sync accuracy, and identity preservation— outperforming all other lip sync models on the market. Because it was built using Phoenix-3 components, Hummingbird-0 yields state-of-the-art results. Tavus tested Hummingbird-0 against industry-leading zero-shot lip sync solutions, displaying:

• Superior Visual Quality: FID score of 63.92 (37% better than closest competitor)
• Strong Lip Synchronization: LSE-D score of 6.74 (7% better than closest competitor)
• Exceptional Identity Preservation: Arcface score of 0.84 (7% better than closest competitor).

“The Tavus team was able to take an existing product and transform it into a complementary solution. By being nimble, knowledgeable, and committed to pushing the envelope through research, Tavus now enables developers to quickly and easily add native-quality voice to any video, unlocking limitless video editing possibilities,” added Goenawan.

Learn more about how Hummingbird-0 works here and find Hummingbird-0 on Tavus or FAL today.

About Tavus

Tavus is a market-leading generative AI video research company building foundational models and operating systems for human-AI interaction. Inspired by the human brain, Tavus’ cognitive architecture enables developers to build hyper-realistic AI video agents that see, listen, and respond, bringing the human touch to digital experiences at scale. Its AI models and APIs power virtual humans for real-time conversations and lifelike video generation, transforming industries like education, healthcare, recruiting, marketing, sales, financial services, and more. Tavus’ technology is used by Fortune 500 companies and innovative startups alike to create AI-driven experiences that feel truly engaging and interactive. Headquartered in San Francisco, Tavus is backed by Sequoia Capital, Scale Venture Partners, Y Combinator, HubSpot, and other leading investors.

Fonte: Business Wire