---
license: cc-by-4.0
datasets:
- jihyoung/M3C
language:
- en
base_model:
- Qwen/Qwen2-VL-2B-Instruct
---

# Enabling Chatbots with Eyes and Ears: An Immersive Multimodal Conversation System for Dynamic Interactions
[\[📜 Paper\]](https://arxiv.org/abs/2506.00421)   [\[🖥️ Project Page\]](https://m3c-dataset.github.io/)  [\[📖 Dataset\]](https://huggingface.co/datasets/jihyoung/M3C)   [\[🤗 Model Weights\]](https://huggingface.co/jihyoung/M3C-dialogue)

<div align="center">
  <img width="500" alt="image" src="https://github.com/user-attachments/assets/76309007-498a-45ed-a4c2-dac43ee39bfc">
  <br>
  <sub>Image Generated by DALL·E</sub>
</div>

## ✅ TODO List

- [ ] Write documentation (README)
- [ ] Release M³C dataset
- [ ] Release dialogue module weight
- [ ] Release retrieval module weight
- [ ] Release training code
- [ ] Release inference code
- [ ] Release model self-chat code
- [ ] Launch Gradio demo for live chat

## 📚 Citation

```bibtex
@article{jang2025enabling,
  title={Enabling Chatbots with Eyes and Ears: An Immersive Multimodal Conversation System for Dynamic Interactions},
  author={Jang, Jihyoung and Bae, Minwook and Kim, Minji and Hakkani-Tur, Dilek and Kim, Hyounghun},
  journal={arXiv preprint arXiv:2506.00421},
  year={2025}
}