---
license: mit
datasets:
- mjjung/Charades-VTune
language:
- en
---

---
license: mit
datasets:
- ShuhuaiRen/TimeIT
language:
- en
---

# TimeChat-7B-Charades-VTune Model

## Model details

We trained [TimeChat](https://arxiv.org/abs/2312.02051) using VTune, a developed instruction-tuning method specifically designed to account for consistency. 

For the tuning, we utilized 5K training videos from Charades-STA with 99K automatically generated annotations.

## Evaluation
We evaluated the model on Charades-CON and Charades-STA.

- Charades-CON
| Metric          | Value       |
|-----------------|-------------|
| Ground          | 76.2        |
| R-Ground        | 69.2 (90.8) |
| S-Ground        | 36.2 (47.5) |
| H-Verify        | 44.8 (58.8) |
| C-Verify        | 42.4 (55.7) |

- Charades-STA
| Metric          | Value   |
|-----------------|---------|
| R@1 IoU=0.3     | 72.74   |
| R@1 IoU=0.5     | 58.47   |
| R@1 IoU=0.7     | 34.70   |
| mIoU            | 50.65   |

**Paper and Code for more information:**
[Paper](https://arxiv.org/abs/2411.12951), [Code](https://github.com/minjoong507/consistency-of-video-llm)

## Citation
If you find our research and codes useful, please consider starring our repository and citing our paper:

```
@article{jung2024consistency,
  title={On the Consistency of Video Large Language Models in Temporal Comprehension},
  author={Jung, Minjoon and Xiao, Junbin and Zhang, Byoung-Tak and Yao, Angela},
  journal={arXiv preprint arXiv:2411.12951},
  year={2024}
}
```