Outlier Detection with CIFAR10 Image Classifier

Launch an image classifier model and detect outlier prediction requests.

This demo is based on VAE outlier detection in the alibi detect project. Here we will :

  • Launch an image classifier model trianed on the CIFAR10 dataset
  • Setup an outlier detector for this particular model
  • Send a request to get a image classification
  • Send a perturbed request to get a outlier detection

Create Model

Use the following model url with tensorflow runtime. Set the protocol to ‘tensorflow’:

gs://seldon-models/tfserving/cifar10/resnet32

create_model

Setup Outlier detector

Setup an outlier detector with model name cifar10 using the default settings (which sets Reply URL as seldon-request-logger in the logger’s default namespace - change if you modified this at install time) and storage URI as follows:

gs://seldon-models/alibi-detect/od/OutlierVAE/cifar10

setup_detector

Make Predictions

Run a single prediction using the tensorflow payload format of an image truck. Also a perturbed image of the truck in the same format at outlier truck image. Make a couple of these requests at random using the predict tool in the UI.

classif_images

View outliers on the Requests Screen

Go to the requests screen to view all the historical requests. You can see the outlier value on each instance. Also you can highlight outliers based on this score and also use the filter to see only the outliers as needed.

outlier_detection

Monitor outliers on the Monitor Screen

Under the ‘Monitor’ section you can see a timeline of outlier requests.

outlier_detection

Last modified November 5, 2020: fix outlier gifs and add monitor section (0c7febb)