<p>It depends on the method you're using. Usually the methods search for architecture for some epochs and at the end you train from scratch the found architecture. The main architecture search algorithms search for cells because iterating through whole architecture is way too expensive(computationally). Cells are the dynamic part of the architecture while the rest of it is a backbone that is static. The NAS(Neural Architecture Search) searches for cells and replaces them in the whole architecture.Usually the NAS searches for 2 kinds of cells, normal, that keeps the output channel the same as the input channel and reduction cells that reduce the input channel size. Here you can see the most popular methods <a href="https://arxiv.org/abs/1808.05377" rel="nofollow noreferrer">https://arxiv.org/abs/1808.05377</a> . </p>
What is the output of the neural architecture search?
<p>After training, what should be the output of the neural architecture search? </p>

<p>Is that the best one ever during the training or the last architecture produced by the model?</p>