Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification

Yuting Yang^†   Haichao Jiang^†   Tianming Liang   Quan Zhang   Jian-Fang Hu*  

Sun Yat-sen University  

🌟 Highlights

When the user query lacks complete details to uniquely distinguish the intended target, existing methods tend to arbitrarily guess user preferences, while our IC-Seg proactively interacts with the user to clarify their real intention.

🎯 Framework

IC-Seg resolves ambiguities via multi-turn dialogues with an MLLM-based User Simulator. Our Hi-GRPO algorithm empowers the agent through a hierarchical reward chain: supervising final localization accuracy at the trajectory level, inquiry quality at the turn level, and fine-grained reasoning steps using expert-diagnosed signals.

🙏 Acknowledgements

Our work is built upon Seg-ReSearch and SDPO. We sincerely appreciate these excellent works.

📝 Citation

If you find our work helpful for your research, please consider citing our paper.

@misc{yang2026dontguessjustask,
      title={Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification}, 
      author={Yuting Yang and Haichao Jiang and Tianming Liang and Quan Zhang and Jian-Fang Hu},
      year={2026},
      eprint={2605.17531},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2605.17531}, 
}