Image transfer plays a significant role in the manufacture of PCB; it affects the production speed and quality of the manufacturing process. This study proposes a surface-framework structure, which divides the network into two parts: surface and framework. The surface part does not include subsampling to extract the detailed features of the image, thereby improving the segmentation effect when the computing power requirement is not large. Meanwhile, a semantic segmentation method based on Unet and surface-framework structure, called pure efficient Unet (PE Unet), is proposed. A comparative experiment is conducted on our mark-point dataset (MPRS). The proposed model achieved good results in various metrics. The proposed network’s IoU attained 84.74%, which is 3.15% higher than Unet. The GFLOPs is 34.0 which shows that the network model balances performance and speed. Furthermore, comparative experiments on MPRS, CHASE_DB1, TCGA-LGG datasets for Surface-Framework structure are introduced, the IoU promotion clipped means on these datasets are 2.38%, 4.35% and 0.78% respectively. The Surface-Framework structure can weaken the gridding effect and improve the performance of semantic segmentation network.