<p>As far as I am aware, adversarial training (i.e., continuously training / fine-tuning on new adversarial images with correct labels) is the only robust defense to adversarial examples that cannot be completely overcome by some form of adversarial attack, (Please correct me if I'm wrong). There have been many other attempts to defend against adversarial examples, but typically there is a way around them if the attacker has an idea of what the defense is (For instance, see <a href="https://arxiv.org/pdf/1802.00420.pdf" rel="nofollow noreferrer">Obfuscated Gradients Give a False Sense of Security:
Circumventing Defenses to Adversarial Examples</a>).</p>
<p>Note that to truly obtain robustness with adversarial training, you have to generate adversarial examples during training, or continue updating with new adversarial images. As I understand it, this is because once you train on some adversarial examples, your model changes slightly, and while it is made robust to your initial adversarial examples, there still exist other adversarial examples that still target your newly trained / fine-tuned model. Adversarial training gradually changes your model to minimize the availability of effective adversarial perturbations.</p>
<p>However, doing this is can be at odds with accuracy, (See <a href="https://arxiv.org/pdf/1805.12152.pdf" rel="nofollow noreferrer">Robustness May Be at Odds with Accuracy</a>). A model that is truly robust to adversarial examples may have a significantly lower accuracy for non-adversarial examples. Additionally, adversarial training may be hard to scale to datasets with larger images.</p>
Adversarial Training and Testing
<p>Maybe this is more a conceptual problem, but I hope you can give me your opinions. I do understand that adversarial training means introducing some corrupted instances into the training process in order to confuse the model and produce false prediction when testing. However, is this model applicable in the following scenario?:
Let's assume an adversarial patch is created to fool a classifier that detects a stop sign, so a normal object detector will not be able to distinguish a real stop sign in presence of this patch. But what if the model trains both instances with and without patches? This is not so difficult to perform for the object classificator and the attack looses all chances to succeed, right?. 
I do not get the idea why those attacks can be successful if for the model it would take only a bit more of training to include those adversarial samples.</p>