Research · Oct 30, 2026, 00:00 UTC

Research finds MLLM retrieval heads are largely shared across audio-visual modalities

Ablating the shared retrieval heads, which are under 2% of all heads, removes nearly all audio-visual retrieval and also degrades document understanding. Training them with a ranking objective raises Qwen3-VL-8B page Recall@1 on MMDocIR by 6.03 points.

• LabAlibaba
• ModelQwen3-VL-8B
• Topicresearch