Abstract: Surface defect detection(SDD) is the crucial way to ensure product quality. Deep learning(DL) has been extensively adopted in industry to support the automatic surface defect detection. However, DL model is vulnerable to the menace of adversarial attack, which would seriously affect the accuracy of DL model or even cause the failure of the DL model. To overcome the drawback, a new union adversarial training(UNT) based DL model is proposed to promote its defense ability on adversarial attack. As adversarial attack includes the single-step adversarial attack and iterative adversarial attack, various adversarial samples are generated by adversarial attack methods are used for training the DL models to enhance the defense ability against both the single-step and iterative adversarial attack algorithms. The experiments are conducted on the CIFAR 10 dataset and magnetic-tile surface detect detection dataset. The results show that the proposed UNT method for DL based SDD can significantly promote the robustness of DL model to the single-step adversarial attack and iterative adversarial attack, which is superior to traditional methods.