Evaluating full-duplex vision-speech capabilities in conversational agents.
Best and second-best non-human entries are in bold and underlined. Timing reports TOR-Alignment percentage above median latency below. Full per-dynamic breakdowns are in the paper appendix.
| Model | Fluency ↑ | Conv. Flow ↑ | Vis. Ground. ↑ | Overall ↑ | Timing ↑ |
|---|---|---|---|---|---|
| Human reference | 4.16 | 4.20 | 4.24 | 4.20 | 90% / 1400 ms |
| Closed-Source Full-Duplex AV2AV models | |||||
| Tavus Griffin Lite | 3.60 | 3.67 | 3.92 | 3.73 | 73.8% / 2232 ms |
| Closed-source full-duplex speech-vision (AV2A) | |||||
| Gemini 2.5 Flash Native | 3.33 | 2.81 | 3.37 | 3.17 | 72% / 3160 ms |
| Gemini 3.1 Flash Live | 3.15 | 2.20 | 3.16 | 2.84 | 66% / 1720 ms |
| OpenAI gpt-realtime-mini | 2.91 | 2.37 | 2.90 | 2.73 | 66% / 5320 ms |
| OpenAI gpt-realtime | 2.72 | 2.50 | 3.02 | 2.75 | 72% / 5400 ms |
| Open-source full-duplex speech-vision (AV2A) | |||||
| MiniCPM-o 4.5 | 3.03 | 3.54 | 3.63 | 3.40 | 73% / 720 ms |
| MiniOmni2 | 0.65 | 1.37 | 1.54 | 1.19 | 64% / 3080 ms |
| VITA-1.5 | 1.19 | 1.57 | 2.53 | 1.76 | 58% / 400 ms |
| Audio-only baselines (A2A; same agents, video withheld) | |||||
| Gemini 2.5 Flash Native | 3.35 | 2.98 | 3.17 | 3.17 | 73% / 2760 ms |
| Gemini 3.1 Flash Live | 3.40 | 2.64 | 3.03 | 3.03 | 69% / 1240 ms |
| OpenAI gpt-realtime-mini | 3.05 | 2.48 | 3.12 | 2.88 | 69% / 5000 ms |
| OpenAI gpt-realtime | 2.93 | 2.37 | 3.59 | 2.97 | 67% / 4440 ms |
| MiniCPM-o 4.5 | 3.45 | 3.76 | 3.10 | 3.44 | 72% / 920 ms |
| MiniOmni2 | 1.48 | 1.70 | 2.15 | 1.72 | 69% / 2760 ms |
| VITA-1.5 | 1.62 | 1.37 | 3.02 | 2.00 | 61% / 800 ms |
Upload your model's per-sample outputs to the VideoFDB evaluation Space to score them and produce a leaderboard row.