I understand that we'll not have envoy proxy related metrics, but I'd expect to see some metrics about how istiod works (pilot metrics maybe) and possibly also how many requests are going through the istio gateway, since IIUC it runs envoy behind the scenes as well. Flying completely blind about Istio seems to be strange, but I am not expert in kubernetes so if the metrics are already published by other means it will be ok as well. My point is that we should investigate what metrics are offered since the start, bootstrapping a service in production without any trace of what it is doing makes me uncomfortable.

To help you answer the question, these images are mostly model servers (tensorflow, pytorch, sklearn, etc) and explainers. The current models built by @kevinbazira and @ACraze are based on a custom image rather than one of the popular model servers. So if you need to have the minimum list of images cloned in your registry to make kfserving work for the current capabilities of the platform, as you said these would be the controller and the agent.

These are the current locations of the images published and/or used by kfserving:

Since we're not planning to use the service-mesh functionality of istio but the ingress-gateway only, there is no reason to connect istio with prometheus.

office hours video recording on model cards

I noticed you are using a timestamp for the version, is this to ensure a unique key? We currently use semver with ORES but I could see how that could lead to naming collisions.

We chose to use S3 object storage over Swift, which comes with:

• Bucket names can have only lowercase letters, number, dots and hyphens. Typically names are in kebab-case, see AWS S3 bucket naming rules.
• Object key is the path to a file, a combination of the dir/subdir names and file name. It has more freedom on the choice of special characters (see AWS Creating object key names), although we can follow the bucket naming rules for simplicity.
• Object key can have tags. This can be useful as meta information, but it can't be used when accessing the objects as the object name has to be unique.

It should work but I've never tested it. Here is an example of how to configure the endpoint_url across all inferenceservices.

I've made a version compatibility matrix from our recent tests (kfserving#1334, kfserving#1482):

The service to service communication can be enhanced with a service mesh if we require for example security policies across the services of that cluster. By having Istio sidecar injection in a namespace, each pod will have an extra container with the envoy proxy that brings the access control, the logging, the tracing, etc in the services that these pods provide.

Hey @elukey, this is the script we're using for e2e testing on kfserving community. It is using the most recent versions of Istio and knative, with their operators and having the sidecar injection disabled as per the requirement above.

Also these large files are not used:

Some thoughts on building a custom image per model.

I've started a WIP PR which supports knative 0.20 and Istio 1.7.1 here.

In T272919#6785099, @ACraze wrote:

I found an example of using helm to install the KFServing dependencies:
https://github.com/kubeflow/kfserving/blob/8a8f33a85a12ea4813f726b41581cbf9335ede9b/test/scripts/run-e2e-tests.sh#L57