model for [<span style="font-variant:small-caps;">Bustle</span>]{}, but instead of predicting whether an expression is a sub-expression of a solution, we predict which operations will be used in a solution. Then, for each benchmark attempt, we simply don’t use the 4 operations that the model predicts are least likely to appear (where 4 was chosen using a hyper-parameter search over the values in {1, 2, 3, 4, 5}).

The results as a function of number of candidate expressions considered are shown in Figure \[fig:progress\_plots\] (Left). By this metric, [<span style="font-variant:small-caps;">Bustle</span>]{} (orange) performs very well, solving 24 tasks within 10 million candidate expressions, significantly outperforming the simple synthesizer (blue), for example, which only solves 17 tasks.

As for wall-clock time (Figure \[fig:progress\_plots\], right), for any fixed time budget, [<span style="font-variant:small-caps;">Bustle</span>]{} (orange) solves more tasks than the baselines. Despite the overhead of calling the model, [<span style="font-variant:small-caps;">Bustle</span>]{} (orange) still solves 24 tasks within 10 seconds, outperforming the baseline synthesizer (blue) which only solves 17 tasks. [<span style="font-variant:small-caps;">Bustle</span>]{} also outperforms the domain-specific heuristics, despite the model being slower to execute. Furthermore, when using both the model and heuristics (red), the synthesizer performs the best overall, solving 25 tasks within 10 seconds, while the heuristics-only approach only 