Today in CV Dazzle news: making self-driving cars read stop signs as speed limit signs.

Robust Physical-World Attacks on Machine Learning Models

Our algorithm can create spatially- constrained perturbations that mimic vandalism or art to reduce the likelihood of detection by a casual observer. We show that adversarial examples generated by RP2 achieve high success rates under various conditions for real road sign recognition by using an evaluation methodology that captures physical world conditions. We physically realized and evaluated two attacks, one that causes a Stop sign to be misclassified as a Speed Limit sign in 100% of the testing conditions, and one that causes a Right Turn sign to be misclassified as either a Stop or Added Lane sign in 100% of the testing conditions.

Previously, previously, previously, previously, previously, previously.

Tags: , , , , , , , ,

7 Responses:

  1. Tolomea says:

    Neural networks... if you don't know why it works then probably there are going to be situations where it fails to work in very surprising ways.

    • jwz says:

      Um... who in this scenario do you imagine don't understand how neural networks work? The people building the vision systems for the cars, the people attacking those systems, or me?

      • Tolomea says:

        It's not how neural networks in general work but rather how any particular neural network works.
        You look at a picture and say it's a stop sign because it has a red octagon with "stop" in it.
        The neural network looks at the picture and says it's a stop sign because... (shrug) it passed on the training data...
        In fairness there are ways to look inside the network and try and get a sense of what it's actually doing, but most of the time most people aren't going to bother.

    • legolas says:

      Yep. And I still wonder how such a thing can ever pass all the regulatory mess. There's infinite rules about writing software for cars, but once you say 'neural network', everything goes it seems.

  2. apm74 says:

    So you just have to ban defacing signs. Easy peasy.

    • Barak Pearlmutter says:

      This methodology also allows the construction of posters and commercial signs and such which are misinterpreted as traffic signs by these networks. "Garage sale" or "lost kitten" on a telephone pole, seen as "begin one way" or "minimum speed 50mph".

  • Previously