Don’t be wrong because you might be fooled: Tips on how secure your ML model

  • An untargeted attack, where an attack is considered successful when the predicted class label is changed (to any other label)
  • A targeted attack with the least-likely target, where we have a successful attack when the predicted class label is changed specifically to the label that the model has the least confidence for the specific instance.

1. Miss-labeled/Confusing Training Data

Data collection is probably the most costly and time-consuming part of most machine learning projects. It’s perfectly reasonable to expect that this arduous process will entail some mistakes. We discovered that the CIFAR10 training set contains some images that either are miss-labeled or they are themselves confusing even for humans (Fig. 4).

2. Is this a “cat” or a “dog”?

The “cat” class has the worst miss-rate of all the classes, followed by the “dog” class which has the third worst miss-rate.

3. Is this a “bird” or an “airplane”?

A similar situation is happening with the “bird” and “airplane” classes. The model in this case is confused by the blue background, since most airplane images contain an object in a blue background (Fig. 7).

👉 Good data means a good model: spend some time investigating your data and try to identify if there are any systematic errors in your training set.

👉 Use explanation methods as a debugger, in order to understand why your model model misses certain groups of instances more than others

👉 Adversarial attacks are a cost-effective way to check the adversarial robustness of your model.

--

--

code4thought is a technology company with a unique purpose: to render technology transparent for large scale software and AI-based systems.

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
code4thought

code4thought

7 Followers

code4thought is a technology company with a unique purpose: to render technology transparent for large scale software and AI-based systems.