Istio Ingress

Istio is a production-ready service mesh solution, which Seldon Deploy uses for routing of traffic. SImilarly, traffic can also be managed by Istio for dynamically created seldon models, which simplifies how users can send requests to a single endpoint.

You can use the istio CLI to install the istio containers into your cluster. Istio provides multiple installation configuration profiles, it is important to choose the most appropriate configuration, especially as Istio is often used for applications beyond just Seldon itself. You can read more about the profiles and configurations in the Istio documentation.

Installing Istio Pods

First, obtain istioctl CLI that will help in installation of Istio. We will be installing version 1.7.3 which is compatible with kubernetes clusters version 1.16-1.18.

export ISTIO_VERSION=1.7.3
curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${ISTIO_VERSION} sh -

Once you have downloaded the command line interface, we can configure a cluster local gateway and other istio components for our installation with:

   cat << EOF > ./local-cluster-gateway.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  values:
    global:
      proxy:
        autoInject: disabled
      useMCP: false
  addonComponents:
    pilot:
      enabled: true
    prometheus:
      enabled: false
  components:
    ingressGateways:
      - name: cluster-local-gateway
        enabled: true
        label:
          istio: cluster-local-gateway
          app: cluster-local-gateway
        k8s:
          service:
            type: ClusterIP
            ports:
            - port: 15020
              name: status-port
            - port: 80
              targetPort: 8080
              name: http2
            - port: 443
              targetPort: 8443
              name: https
  values:
    gateways:
      istio-ingressgateway:
        debug: error
EOF

The installation can then be performed with:

./istioctl install -f local-cluster-gateway.yaml

It’s worth emphasising that Istio in itself is quite a feature-rich open source project, which is often leveraged as central service mesh for clusters. If the use-cases are more specific, there are quite a lot of different configurations that can be chosen, examples covered above include the different installation profiles, but also there are instruction like the documentation in the istio section for KNative.

Installing Seldon with Istio Configuration Enabled

In order to install Seldon with Istio enabled there are a couple of requirements, namely:

• An Istio gateway needs to be created
• Seldon Core Helm Chart needs to be installed with the istio values
• Seldon Deploy Helm Chart needs to be installed with istio values

Setting up Istio Gateway

Istio Gateway is required to access deployments and Seldon Deploy itself

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: seldon-gateway
  namespace: istio-system
spec:
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  servers:
  - hosts:
    - '*'
    port:
      name: http
      number: 80
      protocol: HTTP

Example of https-enabled gateway can be found under

seldon-deploy-install/prerequisites-setup/istio/seldon-gateway.yaml

The Istio gateway can be created in the Istio namespace, but it can also be created in a different namespace, you will need to make sure that the configuration of Seldon Deploy and Core are also aligned to the namespace and gateway name.

Install Seldon Core with Istio Enabled

You can now add the following values in your core-values.yaml file. You need to make sure that the value for the gateway is <namespace>/<gatewayname><namespace>.

istio:
  enabled: true
  gateway: "istio-system/seldon-gateway"

Install Seldon Deploy with Istio Enabled

Similarly you would add following entries to your deploy-values.yaml file

ingressGateway:
  seldonIngressService: "istio-ingressgateway"
  kfServingIngressService: "istio-ingressgateway"
  ingressNamespace: "istio-system"

virtualService:
  create: true
  gateways:
    - istio-system/seldon-gateway

Find address of your Seldon Deploy

Following script can help you find address of your Seldon Deploy instance running with Istio:

ISTIO_INGRESS=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
ISTIO_INGRESS+=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

echo "seldon deploy: http://$ISTIO_INGRESS/seldon-deploy/"