The objective was to verify whether convolutional neural networks can help sweet potato phenotyping for qualitative traits. We evaluated 16 families of sweet potato half-sibs in a randomized block design with four replications. We obtained the images at the plant level and used the ExpImage package of the R software to reduce the resolution and individualize one root per image. We grouped them according to their classifications regarding shape, peel color, and damage caused by insects. 600 roots of each class were destined for training the networks, while the rest was used to verify the quality of the fit. We used the python language on the Google Colab platform and the Keras library, considering the VGG-16, Inception-v3, ResNet-50, InceptionResNetV2, and EfficientNetB3 architectures. The InceptionResNetV2 architecture stood out with high accuracy in classifying individuals according to shape, insect damage, and peel color. Image analysis associated with deep learning may help develop applications used by rural producers and improve sweet potatoes, reducing subjectivity, labor, time, and financial resources in phenotyping.