| Rank | Model | Score |
|---|---|---|
| 1 | lfm3-omni-2b | 46.79 |
1 phaseActive
A 3,212-question multiple-choice benchmark for speaker-centric audiovisual reasoning in real-world videos, with fusion-grounded question design that embeds audiovisual dependencies into question semantics.
Quick answer: AV-SpeakerBench ("See, Hear, and Understand: Benchmarking Audiovisual Human Speech Understanding in Multimodal Large Language Models") is a benchmark introduced by Le Thien Phuc Nguyen, Zhuoran Yu, Samuel Low Yu Hang, Subin An, Jeongik Lee, Yohan Ban, SeungEun Chung, Thanh-Huy Nguyen, JuWan Maeng, Soochahn Lee, and Yong Jae Lee (submitted December 2025, accepted as Findings of CVPR 2026), featuring 3,212 multiple-choice questions focused on speaker-centric audiovisual reasoning in real-world videos.
Most video benchmarks remain visually solvable or only coarsely evaluate speech, offering limited insight into whether models can align who speaks, what is said, and when it occurs. AV-SpeakerBench treats speakers — not scenes — as the core reasoning unit, with fusion-grounded question design embedding audiovisual dependencies directly into question semantics, and expert-curated annotations ensuring temporal precision and cross-modal validity. The paper's own evaluations found the Gemini family consistently outperforming open-source systems, with Gemini 2.5 Pro achieving the best results, while among open models Qwen3-Omni-30B approaches Gemini 2.0 Flash but remains behind Gemini 2.5 Pro — primarily due to weaker audiovisual fusion rather than visual perception.
| Model | Score | Source | Date |
|---|---|---|---|
| LFM3-Omni-2B | 46.79% | Liquid AI model card | 2026-10-27 |