non-zero time.[]{data-label="fig:progress_plots"}](figures/time_and_expressions_plot.png){width="0.8\columnwidth"}

![ Histograms of model predictions for expressions seen while solving benchmarks. (Left) for expressions that were sub-expressions of a solution, the vast majority received predictions close to 1, showing that the model can identify the correct expressions to prioritize during search. (Right) for expressions that were not sub-expressions of a solution, predictions skewed close to 0, although a fraction of predictions were close to 1.[]{data-label="fig:prediction_histograms"}](figures/prediction_histograms.png){width="0.7\columnwidth"}

We conduct two additional analyses to better understand the performance of [<span style="font-variant:small-caps;">Bustle</span>]{}. First, we investigate the predictions of the model when it is run on the intermediate values actually encountered during synthesis of the human-written benchmarks. We compute separate histograms for the model’s predictions on expressions that do appear in the solution and expressions that don’t appear in the solution, for all benchmarks that were solved. Predictions for true sub-expressions skew positive and predictions for negative sub-expressions skew negative. This provides further evidence the our model generalizes well to human benchmarks, despite the domain mismatch to the synthetic data used in training. The detailed results are in Figure \[fig:prediction\_histograms\]. Finally, we evaluated whether all benchmarks solved by the baseline (no model, no heuristics) were also solved by [<span style="font-variant:small-caps;">Bustle</span>]{}. The answer is 