VIKI-R

VIKI-R is a two-stage fine-tuning framework designed to equip vision–language models (VLMs) with robust visual reasoning capabilities for multi-agent collaboration. The first stage is a warmup phase based on supervised fine-tuning (SFT), where the model learns from high-quality Chain-of-Thought (CoT) annotations, enabling it to capture domain-specific reasoning patterns by optimizing both intermediate steps and final decisions. In the second stage, reinforcement fine-tuning further refines the model using the Grouped Relative Proximal Optimization (GRPO) algorithm. Here, the model explores multiple answer candidates and is guided by a reward function that evaluates both correctness and format, with policy updates performed under a KL-regularized objective for stable learning. Together, these two phases enable VIKI-R to achieve advanced perception-driven planning and compositional reasoning in complex, visual multi-agent tasks.

Results

Our experiments reveal three key findings. First, closed-source models outperform open-source ones in zero-shot settings, with GPT-4o excelling at trajectory perception and Gemini-2.5-Flash-preview achieving the best agent activation accuracy. Second, model scale is crucial for open-source VLMs: Qwen2.5-VL (72B) matches or exceeds some closed models, while downsizing to 32B significantly harms planning and trajectory performance. Third, our two-stage fine-tuning framework VIKI-R surpasses supervised baselines like Ans-SFT and VIKI-R-zero, especially in out-of-domain generalization, highlighting the value of reinforcement learning for multi-agent visual reasoning.

Additional Analysis

Observation 1

We conduct an ablation study on reasoning tasks from the VIKI-L2 level to evaluate the impact of incorporating a step penalty during training. This penalty is applied when the predicted plan deviates in length from the ground-truth plan—i.e., when the agent takes too many steps compared to the optimal plan, it receives no reward. The goal is to encourage precise and efficient planning behavior.

1. Improved Generalization: With the step penalty, VIKI-R achieves 46.8% accuracy on out-of-domain (OOD-H) tasks, compared to only 7.1% without it. This shows that penalizing suboptimal plans significantly improves the model's ability to generalize to unseen scenarios.

2. Better Planning Efficiency: The average difference in plan length (Δ Steps) drops from 1.97 to 0.05, indicating that the model generates action sequences that are nearly identical to the ground truth. Meanwhile, in-domain accuracy improves from 8.0% to 96.0%, demonstrating that the step penalty also enhances planning precision in familiar environments.

Observation 2

The curve in the right illustrates a dip during early format optimization, followed by a rise as task reasoning emerges. In the early stages of training, output length decreases as the model prioritizes format compliance to secure the format reward. At this point, the model learns to produce syntactically correct responses with minimal reasoning content. Once format accuracy saturates, the policy gradually shifts focus toward maximizing task correctness. This leads to a steady increase in output length, as the model begins to include more detailed reasoning steps necessary to solve the tasks accurately.

Figure: Output length dynamics across training.

Citation

If you find this project useful, welcome to cite us.

@article{kang2025viki,
    title={VIKI‑R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement Learning},
    author={},
    journal={},
    year={2025}
}