signature is a fixed length representation, it can be fed into a simple feed-forward neural network, rather than requiring a recurrent model, as would be necessary if we passed in e.g. the AST representation. Third, because of this fixed length representation, it is easy to batch many calls to the machine learning model and process them using CPU vector instructions. Inference calls to the machine learning model could, in principle, be done in parallel to the rest of the synthesizer, either on a separate CPU core or on an accelerator, which would further improve wall-clock results. Due to these optimizations, computing property signatures and running the model on them accounts for only roughly 20% of the total time spent; the rest is spent by the symbolic portions of the bottom-up search.

Experiments {#section:results}
===========

We evaluate [<span style="font-variant:small-caps;">Bustle</span>]{} on a suite of human-written benchmarks inspired by string transformation questions asked by users on online help-forums. (See Supplementary Material for the full list of benchmarks.) We measure performance in two ways. First, we consider the number of benchmarks successfully solved as a function of number of candidates considered, which gives insight into how much the machine learning model has been able 