The advancement in the field of 3D integration circuit technology leads to new challenges for quality assessment of interconnects such as through silicon vias (TSVs) in terms of automated and time-efficient analysis. In this paper, we develop a fully automated high-efficient End-to-End Convolutional Neural Network (CNN) model, utilizing two sequentially linked CNN architectures, suitable to classify and locate thousands of TSVs as well as provide statistical information. In particular, we generate interference patterns of the TSVs by conducting a unique concept of Scanning Acoustic Microscopy (SAM) imaging. Scanning Electron Microscopy (SEM) is used to validate and also disclose the characteristic pattern in the SAM C-scan images. By comparing the model with semi-automated machine learning approaches its outstanding performance is illustrated, indicating a localisation and classification accuracy of 100% and greater than 96%, respectively. The approach is not limited to SAM-image data and presents an important step towards zero defect strategies.