README.md

---
license: apache-2.0
pipeline_tag: image-text-to-text
library_name: transformers
---

# WebDancer: Towards Autonomous Information Seeking Agency

<div align="center">

[📄Paper](https://huggingface.co/papers/2505.22648) | [🏠Homepage](https://osatlas.github.io/) | [💻Code](https://github.com/Alibaba-NLP/WebAgent)

</div>

WebDancer is a native agentic search reasoning model using the ReAct framework, aiming towards autonomous information seeking agency and _Deep Research_-like capabilities. This model was presented in the paper [WebDancer: Towards Autonomous Information Seeking Agency](https://huggingface.co/papers/2505.22648).

Addressing intricate real-world problems necessitates in-depth information seeking and multi-step reasoning. WebDancer presents a cohesive paradigm for building end-to-end agentic information seeking agents from a data-centric and training-stage perspective. The approach consists of four key stages: (1) browsing data construction, (2) trajectories sampling, (3) supervised fine-tuning for effective cold start, and (4) reinforcement learning for enhanced generalisation. Empirical evaluations on the challenging information seeking benchmarks, GAIA and WebWalkerQA, demonstrate the strong performance of WebDancer, achieving considerable results and highlighting the efficacy of this training paradigm.

<div align="center">
<p align="center">
  <img src="https://github.com/user-attachments/assets/cf2ee020-5e15-4087-9a7e-75cc43662494" width="800px" />
</p>
</div>

## Key Features

*   **Native agentic search reasoning model**: Utilizes the ReAct framework for autonomous information seeking, inspired by "Deep Research"-like models.
*   **Four-stage training paradigm**: Comprises browsing data construction, trajectory sampling, supervised fine-tuning for effective cold start, and reinforcement learning for improved generalization, enabling the agent to autonomously acquire search and reasoning skills.
*   **Data-centric approach**: Integrates trajectory-level supervision fine-tuning and reinforcement learning (DAPO) to develop a scalable pipeline for training agentic systems via SFT or RL.
*   **Strong Performance**: WebDancer achieves a Pass@3 score of 64.1% on GAIA and 62.0% on WebWalkerQA.

## Quick Start

You need to enter the `WebDancer` folder in the [WebAgent repository](https://github.com/Alibaba-NLP/WebAgent) for the following commands.

### Step 0: Set Up the Environment

```bash
conda create -n webdancer python=3.12
pip install -r requirements.txt
```

For `WebDancer-QwQ-32B` (which is based on Qwen2-VL), ensure that the necessary dependencies are installed:
```bash
pip install transformers
pip install qwen-vl-utils
```

### Step 1: Deploy the Model

Download the WebDancer model from [🤗 HuggingFace](https://huggingface.co/Alibaba-NLP/WebDancer-QwQ-32B) and deploy it using the provided scripts with [sglang](https://github.com/sgl-project/sglang).

```bash
cd scripts
bash deploy_model.sh WebDancer_PATH
```

> **Note:** Replace `WebDancer_PATH` with the actual path to the downloaded model.

### Step 2: Run the Demo

Edit the following keys in [`WebDancer/scripts/run_demo.sh`](https://github.com/Alibaba-NLP/WebAgent/blob/main/WebDancer/scripts/run_demo.sh):

-   `GOOGLE_SEARCH_KEY`, you can get it from [serper](https://serper.dev/).
-   `JINA_API_KEY`, you can get it from [jina](https://jina.ai/api-dashboard/).
-   `DASHSCOPE_API_KEY`, you can get it from [dashscope](https://dashscope.aliyun.com/).

Then, launch the demo with Gradio to interact with the WebDancer model:

```bash
cd scripts
bash run_demo.sh
```

### Inference Example

WebDancer models accept images of any size as input. The model outputs are normalized to relative coordinates within a 0-1000 range (either a center point or a bounding box defined by top-left and bottom-right coordinates). For visualization, remember to convert these relative coordinates back to the original image dimensions.

Here's a minimal Python inference example for `WebDancer-QwQ-32B` (based on Qwen2-VL):

```python
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
from PIL import Image
import torch

# Load the model and processor
# Default: Load the model on the available device(s)
model_name = "Alibaba-NLP/WebDancer-QwQ-32B"
model = Qwen2VLForConditionalGeneration.from_pretrained(
    model_name, torch_dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_name)

# Define the task with an image and text instruction
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                # Replace with a local image path or a URL
                "image": "./examples/images/web_6f93090a-81f6-489e-bb35-1a2838b18c01.png",
            },
            {"type": "text", "text": "In this UI screenshot, what is the position of the element corresponding to the command \"switch language of current page\" (with bbox)?"},
        ],
    }
]

# Preparation for inference
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
image_inputs, video_inputs = process_vision_info(messages) # video_inputs will be empty for this task
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
)
inputs = inputs.to("cuda") # Ensure inputs are on the correct device

# Inference: Generation of the output
generated_ids = model.generate(**inputs, max_new_tokens=128)

generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]

output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=False, clean_up_tokenization_spaces=False
)
print(output_text[0])
# Expected output: <|object_ref_start|>language switch<|object_ref_end|><|box_start|>(576,12),(592,42)<|box_end|><|im_end|>
```

## Demos

WebDancer can execute long-horizon tasks with multiple steps and complex reasoning, such as web traversal, information seeking, and question answering.

<div align="center">
    <h3>WebWalkerQA</h3>
    <video src="https://github.com/user-attachments/assets/0bbaf55b-897e-4c57-967d-a6e8bbd2167e" />
</div>

<div align="center">
    <h3>GAIA</h3>
    <video src="https://github.com/user-attachments/assets/935c668e-6169-4712-9c04-ac80f0531872" />
</div>

<div align="center">
    <h3>Daily Use</h3>
    <video src="https://github.com/user-attachments/assets/d1d5b533-4009-478b-bd87-96b86389327d" />
</div>

## Citation

If this work is helpful, please kindly cite as:

```bibtex
@misc{wu2025webdancer,
      title={WebDancer: Towards Autonomous Information Seeking Agency},
      author={Jialong Wu and Baixuan Li and Runnan Fang and Wenbiao Yin and Liwen Zhang and Zhengwei Tao and Dingchu Zhang and Zekun Xi and Yong Jiang and Pengjun Xie and Fei Huang and Jingren Zhou},
      year={2025},
      eprint={2505.22648},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.22648},
}
```