-The scenarios that success of the attack is less than 50%, a simple ensemble method could be used to defend the attack. It seems that the success of attack in attacking the Google model is around 20% which could be circumvented by using multiple models. -The attack seems to be unstable when changing the architecture. For instance the attack on VGG does not succeed as much as the attack on other architectures. -On novelty of the paper: The ideas behind the attack seem to be simple and borrow ideas from the Metalearing literature. However, this is not necessary a bad thing as it shows simple ideas can be used to attack models. - The experiments of the paper are done only on neural networks and image classification tasks. It would be interesting to see the performance of attack on other architecture and classification tasks.
- The experiments of the paper are done only on neural networks and image classification tasks. It would be interesting to see the performance of attack on other architecture and classification tasks.