跳到主要内容

在 GCP 上为 Milvus 设置七层负载均衡器

与四层负载均衡器相比，七层负载均衡器提供智能负载均衡和缓存功能，是云原生服务的绝佳选择。

本指南将指导您为已在四层负载均衡器后运行的 Milvus 集群设置七层负载均衡器。

开始之前​

调整 Milvus 配置​

本指南假设您已经在 GCP 上在四层负载均衡器后部署了 Milvus 集群。

在为此 Milvus 集群设置七层负载均衡器之前，运行以下命令移除四层负载均衡器。

helm upgrade my-release milvus/milvus --set service.type=ClusterIP

作为七层负载均衡器的后端服务，Milvus 必须满足某些加密要求，以便它能够理解来自负载均衡器的 HTTP/2 请求。因此，您需要在 Milvus 集群上启用 TLS，如下所示。

helm upgrade my-release milvus/milvus -f tls.yaml

tls.yaml 内容：

extraConfigFiles:
  user.yaml: |+
    common:
      security:
        tlsMode: 1

Set up a health check endpoint​

To ensure service availability, Layer-7 load balancing on GCP requires probing the health conditions of the backend service. Therefore, we need to set up a BackendConfig to wrap up the health check endpoint and associate the BackendConfig with the Milvus service through annotations.

The following snippet is the BackendConfig settings. Save it as backendconfig.yaml for later use.

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: my-release-backendconfig
  namespace: default
spec:
  healthCheck:
    port: 9091
    requestPath: /healthz
    type: HTTP

Then run the following command to create the health check endpoint.

kubectl apply -f backendconfig.yaml

Finally, update the annotations of the Milvus service to ask the Layer-7 load balancer that we will create later to perform health checks using the endpoint just created.

kubectl annotate service my-release-milvus \
    cloud.google.com/app-protocols='{"milvus":"HTTP2"}' \
    cloud.google.com/backend-config='{"default": "my-release-backendconfig"}' \
    cloud.google.com/neg='{"ingress": true}' --overwrite

  • As to the first annotation,

    Milvus is native to gRPC, which is built upon HTTP/2. Therefore, we can use HTTP/2 as the communication protocol between the Layer-7 load balancer and Milvus.

  • As to the second annotation,

    Milvus only offers the health check endpoint over gRPC and HTTP/1. We need to set up a BackendConfig to wrap the health check endpoint and associate it with the Milvus service so that the Layer-7 load balancer probes this endpoint for the health condition of Milvus.

  • As to the third annotation,

    It asks for the creation of a network endpoint group (NEG) after an Ingress is created. When NEGs are used with GKE Ingress, the Ingress controller facilitates the creation of all aspects of the load balancer. This includes creating the virtual IP address, forwarding rules, health checks, firewall rules, and more. For details, refer to Google Cloud docs.

Prepare TLS certificates​

TLS requires certificates to work. There are two ways to create certificates, namely self-managed and Google-managed.

This guide uses my-release.milvus.io as the domain name to access our Milvus service.

Create self-managed certificates​

Run the following commands to create a certificate.

# Generates a tls.key.
openssl genrsa -out tls.key 2048

# Creates a certificate and signs it with the preceding key.
openssl req -new -key tls.key -out tls.csr \
    -subj "/CN=my-release.milvus.io"

openssl x509 -req -days 99999 -in tls.csr -signkey tls.key \
    -out tls.crt

Then create a secret in your GKE cluster with these files for later use.

kubectl create secret tls my-release-milvus-tls --cert=./tls.crt --key=./tls.key

Create Google-managed certificates​

The following snippet is a ManagedCertificate setting. Save it as managed-crt.yaml for later use.

apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
  name: my-release-milvus-tls
spec:
  domains:
    - my-release.milvus.io

Create a managed certificate by applying the setting to your GKE cluster as follows:

kubectl apply -f ./managed-crt.yaml

This could last for a while. You can check the progress by running

kubectl get -f ./managed-crt.yaml -o yaml -w

The output should be similar to the following:

status:
  certificateName: mcrt-34446a53-d639-4764-8438-346d7871a76e
  certificateStatus: Provisioning
  domainStatus:
  - domain: my-release.milvus.io
    status: Provisioning

Once certificateStatus turns to Active, you are ready to set up the load balancer.

Create an Ingress to generate a Layer-7 Load Balancer​

Create a YAML file with one of the following snippets.

  • Using self-managed certificates

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: my-release-milvus
      namespace: default
    spec:
      tls:
      - hosts:
        - my-release.milvus.io
        secretName: my-release-milvus-tls
      rules:
      - host: my-release.milvus.io
        http:
          paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-release-milvus
                port:
                  number: 19530

  • Using Google-managed certificates

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: my-release-milvus
      namespace: default
      annotations:
        networking.gke.io/managed-certificates: "my-release-milvus-tls"
    spec:
      rules:
      - host: my-release.milvus.io
        http:
          paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-release-milvus
                port:
                  number: 19530

Then you can create the Ingress by applying the file to your GKE cluster.

kubectl apply -f ingress.yaml

Now, wait for Google to set up the Layer-7 load balancer. You can check the progress by running

kubectl  -f ./config/samples/ingress.yaml get -w

The output should be similar to the following:

NAME                CLASS    HOSTS                  ADDRESS   PORTS   AGE
my-release-milvus   <none>   my-release.milvus.io             80      4s
my-release-milvus   <none>   my-release.milvus.io   34.111.144.65   80, 443   41m

Once an IP address is displayed in the ADDRESS field, the Layer-7 load balancer is ready to use. Both port 80 and port 443 are displayed in the above output. Remember, you should always use port 443 for your own good.

Verify the connection through the Layer-7 load balancer​

This guide uses PyMilvus to verify the connection to the Milvus service behind the Layer-7 load balancer we have just created. For detailed steps, read this.

Notice that connection parameters vary with the way you choose to manage the certificates in Prepare TLS certificates.

from pymilvus import (
    connections,
    utility,
    FieldSchema,
    CollectionSchema,
    DataType,
    Collection,
)

# For self-managed certificates, you need to include the certificate in the parameters used to set up the connection.
connections.connect("default", host="34.111.144.65", port="443", server_pem_path="tls.crt", secure=True, server_name="my-release.milvus.io")

# For Google-managed certificates, there is not need to do so.
connections.connect("default", host="34.111.144.65", port="443", secure=True, server_name="my-release.milvus.io")
  • The IP address and port number in host and port should match those listed at the end of Create an Ingress to generate a Layer-7 Load Balancer.
  • If you have set up a DNS record to map domain name to the host IP address, replace the IP address in host with the domain name and omit server_name.