Given the task definition and input, reply with output. In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

The comparative evaluator is trained with maximum likelihood estimation (MLE) objective, as described in eq DISPLAY_FORM6

where $\mathcal {X}$ is the set of pairwise training examples contructed as described above, $Q(x_1, x_2) \in \lbrace >,<,\approx \rbrace $ is the true label for the pair ($x_1$, $x_2$), $D_\phi ^q(x_1, x_2)$ is the probability of the comparative discriminator's prediction being $q$ ($q \in \lbrace >,<,\approx \rbrace $) for the pair ($x_1$, $x_2$). 
 Question: How they add human prefference annotation to fine-tuning process?

human preference annotation is available $Q(x_1, x_2) \in \lbrace >,<,\approx \rbrace $ is the true label for the pair