Given the task definition and input, reply with output. In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

In all our experiments, we used the out-of-the-box BERT models without any task-specific fine-tuning. 
 Question: How does their model differ from BERT?

Their model does not differ from BERT.