update for chat template

README.md

@@ -115,6 +115,27 @@ output = model.generate(**inputs_video, max_new_tokens=100, do_sample=False)
 print(processor.decode(output[0][2:], skip_special_tokens=True))
 ```
 
+-----------
+From transformers>=v4.48, you can also pass image/video url or local path to the conversation history, and let the chat template handle the rest.
+For video you also need to indicate how many `num_frames` to sample from video, otherwise the whole video will be loaded.
+Chat template will load the image/video for you and return inputs in `torch.Tensor` which you can pass directly to `model.generate()`.
+
+```python
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {"type": "image", "url": "https://www.ilankelman.org/stopsigns/australia.jpg"}
+            {"type": "video", "path": "my_video.mp4"},
+            {"type": "text", "text": "What is shown in this image and video?"},
+        ],
+    },
+]
+
+inputs = processor.apply_chat_template(messages, num_frames=8, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors"pt")
+output = model.generate(**inputs, max_new_tokens=50)
+```
+
 ### Inference with images as inputs
 
 To generate from images use the below code after loading the model as shown above: