Istio manages an internal registry of all services it knows about in the environment. As the workloads get added, removed, or updated, so does Istio's internal service registry.

Istio's control plane exposes a couple of debug endpoints we can use to gather information about the state of the mesh, including the services Istio is aware of. For example, the endpoint /debug/registryz returns the information about all services Istio is aware of:

ISTIOD=$(kubectl get -A pods --selector=app=istiod -o jsonpath='{.items[*].metadata.name}')
kubectl exec -it $ISTIOD -n istio-system -- curl localhost:8080/debug/registryz | jq '.[] | .hostname'

"httpbin.test-ns.svc.cluster.local"
"istio-egressgateway.istio-system.svc.cluster.local"
"istio-ingressgateway.istio-system.svc.cluster.local"
"istiod.istio-system.svc.cluster.local"
"kube-dns.kube-system.svc.cluster.local"
"kubernetes.default.svc.cluster.local"
"metrics-server.kube-system.svc.cluster.local"
"sleep.default.svc.cluster.local"

Note that the above entries don't necessarily have a proxy injected; they list all services Istio knows about. Another endpoint called /debug/instancesz is the one that shows the instance with the proxy injected - it shows the proxies that are connected to the Istios' control plane:

 kubectl exec -it $ISTIOD -n istio-system -- curl localhost:8080/debug/instancesz | jq 'keys'

[
  "istio-egressgateway-7475c75b68-v87zj.istio-system",
  "istio-ingressgateway-6688c7f65d-8rmw2.istio-system",
  "sleep-75bbc86479-m8n6c.default"
]

By default, Envoy proxies will pass through all requests made to external services, regardless of whether they are part of Istio's service registry.

For example, if we try to send a request to https://httpbin.org/headers from within any of the containers that are part of the mesh, we'll get the response back, even though the httpbin.org host isn't part of the Istio's service registry.

The way Envoy proxies handle outbound traffic is configurable using the meshConfig.outboundTrafficPolicy.mode settings, that's set to ALLOW_ANY by default. If we change the outbound traffic policy to REGISTRY_ONLY, Envoy proxies will block calls made to unknown services.

If we'd repeat the above experiment with https://httpbin.org/headers using the mesh with the REGISTRY_ONLY mode, we'll get back an HTTP 502 response because the Envoy proxy doesn't know anything about the httpbin.org host. This is where we can use the ServiceEntry resource to add entries to the service registry manually.

Setting the traffic policy to REGISTRY_ONLY and blocking all external traffic by default is similar to configuring a deny-all authorization policy. We can be explicit about which external traffic we allow and include them in the mesh individually instead of using ALLOW_ALL mode.

However, be careful with blindly setting the REGISTRY_ONLY mode in an existing mesh, as that might cause many problems and HTTP 502 responses. You might be better off continuing with ALLOW_ALL mode while adding external services to the mesh. Once you've configured all services, switch to the REGISTRY_ONLY.

ServiceEntry for external services

Service entry allows us to add an external service and make it a part of Istio's service registry. In addition to explicitly allowing outbound traffic to external services, we can also use other Istio resources to configure traffic policies (DestinationRule) or traffic routing, failure injection, or retries.

Here's an example of a ServiceEntry for the httpbin.org host:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-httpbin
spec:
  hosts:
  - httpbin.org
  location: MESH_EXTERNAL
  ports:
  - name: http
    number: 80
    protocol: HTTP
  resolution: DNS

Deploying the above ServiceEntry makes the httpbin.org host part of the Istio's service registry and tells Envoy proxies to pass through the requests.

The location field specifies whether the service is external to the mesh, typically used for external services consumed through APIs, or whether the service is considered a part of the mesh, used for services running on VMs, for example. The difference is in how the mTLS authentication and policy enforcement works. With MESH_EXTERNAL services, the mTLS authentication is disabled, and the policy enforcement is performed on the callers' side instead of the server side (as we don't know if there's a proxy running there or not).

The second interesting field is the resolution field. We're setting it to DNS because we want to resolve the IP address of the httpbin.org host by querying the DNS. There are a couple of other options we have here that are related to different fields that can be set in the ServiceEntry resource, and we'll look at them through other scenarios.

With the ServiceEntry defined, we can now use a VirtualService to configure things such as retries, aborts or delays, or even route to different services. The key is to use the same hostname (httpbin.org) in VirtualService as in the ServiceEntry resource.

For example, here's how we could configure an HTTP abort:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - httpbin.org
  http:
  - fault:
      abort:
        httpStatus: 500
        percentage:
          value: 100
    route:
    - destination:
        host: httpbin.org

What if I have an IP address?

Another common scenario is when your existing services use IP addresses to access external services. The question is, how do you create a ServiceEntry to bring those endpoints to the mesh?

Let's say we have a service running outside the mesh with an IP 1.2.3.4, and we want to make it part of the mesh. In this case, we don't have a hostname, we have an IP, but we still want to use Istios' features and be able to access the service.

There's a concept of endpoints in the service entry that allows us to specify a network endpoint (domain names, IP addresses, or Unix domain sockets). Because we have an actual IP address and there's nothing to resolve, we can set the resolution to STATIC, and this tells Envoy to use the addresses specified in the endpoints field:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-ip
spec:
  hosts:
  - "thisdoesnt.matter"
  addresses:
    - 1.2.3.4
  location: MESH_INTERNAL
  ports:
  - name: http
    number: 80
    protocol: HTTP
  resolution: STATIC
  endpoints:
    - address: 1.2.3.4

One of the confusing parts here might be the addresses field. These addresses are virtual IPs that are associated with the service. Practically, you can read the above ServiceEntry as "if I send a request to 1.2.3.4 (addresses field), the request will go to 1.2.3.4 (endpoints field)".

This opens up interesting scenarios as we can set any IP address in the addresses field or even a CIDR and then use those VIPs to access the desired endpoints.

Applying a VirtualService is more or less the same - make sure you use the same hostname:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ip-vs
spec:
  hosts:
  - thisdoesnt.matter
  http:
  - fault:
      abort:
        httpStatus: 500
        percentage:
          value: 100
    route:
    - destination:
        host: thisdoesnt.matter

I have an IP address, but I want to use a DNS name

Suppose you wanted to use curl thisdoesnt.matter instead of curl 1.2.3.4, we can do that! To do that, we can create a headless Kubernetes service with an endpoint object. Then, in the ServiceEntry, we won't have any IP addresses, as the address would be part of the endpoint object, and we could use the DNS resolution and the hostname from the headless Kubernetes service.

Let's see how to set that up, starting with the Kubernetes headless service and an endpoints object:

apiVersion: v1
kind: Service
metadata:
  name: headless-svc
spec:
  clusterIP: None 
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: v1
kind: Endpoints
metadata:
  # Same as the Service name
  name: headless-svc
  namespace: default
subsets:
- addresses:
  - hostname: headless-svc-0
    ip: 1.2.3.4
  ports:
  - name: http
    port: 80
    protocol: TCP

With the above deployed, we can now use the headless-svc.default.svc.cluster.local to access IP 1.2.3.4. Setting up the ServiceEntry for this is equivalent to the first scenario we explained:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-httpbin
spec:
  hosts:
  - headless-svc
  location: MESH_EXTERNAL
  ports:
  - name: http
    number: 80
    protocol: HTTP
  resolution: DNS

We've pulled out the addresses and endpoints from the ServiceEntry, moved the endpoints part to the Endpoints resource, and replaced the addresses part with an actual Kubernetes Service.

ServiceEntry resolution modes: DNS vs. STATIC vs. NONE

The resolution field is the most common source of confusion with ServiceEntry, so here's a quick reference for choosing the right mode:

A few rules of thumb that resolve most ServiceEntry issues:

• resolution: STATIC requires the endpoints field. Each endpoint is an IP address (with an optional port). If you omit endpoints with STATIC resolution, the configuration is rejected.
• resolution: DNS with no endpoints resolves the hostnames from the hosts field directly. If you do specify endpoints, the proxy resolves those DNS names instead, and the hosts field becomes the name you use to call the service.
• Wildcard hosts (e.g. *.example.com) can't use DNS resolution with endpoints that need resolving — use NONE for passthrough, or list specific hosts.
• The addresses field is a virtual IP (VIP), not an endpoint. It's the address clients send traffic to; the endpoints field is where the traffic actually goes. For HTTP services on port 80/443, the proxy can route based on the Host header, so addresses is optional. For opaque TCP traffic, you need addresses (or a unique port) so the proxy can tell services apart.

If you see HTTP 502 responses after switching outboundTrafficPolicy to REGISTRY_ONLY, the first things to check are: is there a ServiceEntry for the host, does the resolution mode match what you actually have (hostname → DNS, fixed IPs → STATIC), and do the ports and protocol match the request you're making.