跳到主要内容

使用 Docker Compose 或 Helm 配置消息存储

Milvus 使用 Pulsar 或 Kafka 来管理最近更改的日志、输出流日志并提供日志订阅。Pulsar 是默认的消息存储系统。本主题介绍如何使用 Docker Compose 或 Helm 配置消息存储。

您可以使用 Docker Compose 或在 K8s 上配置 Pulsar，以及在 K8s 上配置 Kafka。

使用 Docker Compose 配置 Pulsar​

1. 配置 Pulsar​

要使用 Docker Compose 配置 Pulsar，请在 milvus/configs 路径下的 milvus.yaml 文件中为 pulsar 部分提供您的值。

pulsar:
  address: localhost # Address of pulsar
  port: 6650 # Port of pulsar
  maxMessageSize: 5242880 # 5 * 1024 * 1024 Bytes, Maximum size of each message in pulsar.

See Pulsar-related configurations for more information.

2. Run Milvus​

Run the following command to start Milvus that uses the Pulsar configurations.

docker compose up
Configurations only take effect after Milvus starts. See Start Milvus for more information.

Configure Pulsar with Helm​

For Milvus clusters on K8s, you can configure Pulsar in the same command that starts Milvus. Alternatively, you can configure Pulsar using the values.yml file on the /charts/milvus path in the milvus-helm repository before you start Milvus.

For details on how to configure Milvus using Helm, refer to Configure Milvus with Helm Charts. For details on Pulsar-related configuration items, refer to Pulsar-related configurations. |

Using the YAML file​

  1. Configure the externalConfigFiles section in the values.yaml file.
extraConfigFiles:
  user.yaml: |+
    pulsar:
      address: localhost # Address of pulsar
      port: 6650 # Port of Pulsar
      webport: 80 # Web port of pulsar, if you connect direcly without proxy, should use 8080
      maxMessageSize: 5242880 # 5 * 1024 * 1024 Bytes, Maximum size of each message in pulsar.
      tenant: public
      namespace: default
  1. After configuring the preceding sections and saving the values.yaml file, run the following command to install Milvus which uses the Pulsar configurations.
helm install <your_release_name> milvus/milvus -f values.yaml

Configure Kafka with Helm​

For Milvus clusters on K8s, you can configure Kafka in the same command that starts Milvus. Alternatively, you can configure Kafka using the values.yml file on the /charts/milvus path in the milvus-helm repository before you start Milvus.

For details on how to configure Milvus using Helm, refer to Configure Milvus with Helm Charts. For details on Pulsar-related configuration items, refer to Pulsar-related configurations.

Using the YAML file​

  1. Configure the externalConfigFiles section in the values.yaml file if you want to use Kafka as the message storage system.
extraConfigFiles:
  user.yaml: |+
    kafka:
      brokerList:
        -  <your_kafka_address>:<your_kafka_port>
      saslUsername:
      saslPassword:
      saslMechanisms: PLAIN
      securityProtocol: SASL_SSL
  1. After configuring the preceding sections and saving the values.yaml file, run the following command to install Milvus that uses the Kafka configurations.
helm install <your_release_name> milvus/milvus -f values.yaml

Configure RocksMQ with Helm​

Milvus standalone uses RocksMQ as the default message storage. For detailed steps on how to configure Milvus with Helm, refer to Configure Milvus with Helm Charts. For details on RocksMQ-related configuration items, refer to RocksMQ-related configurations.

  • If you start Milvus with RocksMQ and want to change its settings, you can run helm upgrade -f with the changed settings in the following YAML file.

  • If you have installed Milvus standalone using Helm with a message store other than RocksMQ and want to change it back to RocksMQ, run helm upgrade -f with the following YAML file after you have flushed all collections and stopped Milvus.

extraConfigFiles:
  user.yaml: |+
    rocksmq:
      # The path where the message is stored in rocksmq
      # please adjust in embedded Milvus: /tmp/milvus/rdb_data
      path: /var/lib/milvus/rdb_data
      lrucacheratio: 0.06 # rocksdb cache memory ratio
      rocksmqPageSize: 67108864 # 64 MB, 64 * 1024 * 1024 bytes, The size of each page of messages in rocksmq
      retentionTimeInMinutes: 4320 # 3 days, 3 * 24 * 60 minutes, The retention time of the message in rocksmq.
      retentionSizeInMB: 8192 # 8 GB, 8 * 1024 MB, The retention size of the message in rocksmq.
      compactionInterval: 86400 # 1 day, trigger rocksdb compaction every day to remove deleted data
      # compaction compression type, only support use 0,7.
      # 0 means not compress, 7 will use zstd
      # len of types means num of rocksdb level.
      compressionTypes: [0, 0, 7, 7, 7]

Changing the message store is not recommended. If this is you want to do this, stop all DDL operations, then call the FlushAll API to flush all collections, and finally stop Milvus in the end before you actually change the message store.

Configure NATS with Helm​

NATS is an experimental message store alternative to RocksMQ. For detailed steps on how to configure Milvus with Helm, refer to Configure Milvus with Helm Charts. For details on RocksMQ-related configuration items, refer to NATS-related configurations.

  • If you start Milvus with NATS and want to change its settings, you can run helm upgrade -f with the changed settings in the following YAML file.

  • If you have installed Milvus standalone with a message store other than NATS and want to change it to NATS, run helm upgrade -f with the following YAML file after you flushed all collections and stopped Milvus.

extraConfigFiles:
  user.yaml: |+
    mq:
      type: natsmq
    natsmq:
      # server side configuration for natsmq.
      server: 
        # 4222 by default, Port for nats server listening.
        port: 4222 
        # /var/lib/milvus/nats by default, directory to use for JetStream storage of nats.
        storeDir: /var/lib/milvus/nats 
        # (B) 16GB by default, Maximum size of the 'file' storage.
        maxFileStore: 17179869184 
        # (B) 8MB by default, Maximum number of bytes in a message payload.
        maxPayload: 8388608 
        # (B) 64MB by default, Maximum number of bytes buffered for a connection applies to client connections.
        maxPending: 67108864 
        # (√ms) 4s by default, waiting for initialization of natsmq finished.
        initializeTimeout: 4000 
        monitor:
          # false by default, If true enable debug log messages.
          debug: false 
          # true by default, If set to false, log without timestamps.
          logTime: true 
          # no log file by default, Log file path relative to.. .
          logFile: 
          # (B) 0, unlimited by default, Size in bytes after the log file rolls over to a new one.
          logSizeLimit: 0 
        retention:
          # (min) 3 days by default, Maximum age of any message in the P-channel.
          maxAge: 4320 
          # (B) None by default, How many bytes the single P-channel may contain. Removing oldest messages if the P-channel exceeds this size.
          maxBytes:
          # None by default, How many message the single P-channel may contain. Removing oldest messages if the P-channel exceeds this limit.    
          maxMsgs:

Choose between RocksMQ and NATS?

RockMQ uses CGO to interact with RocksDB and manages the memory by itself, while the pure-GO NATS embedded in the Milvus installation delegates its memory management to Go's garbage collector (GC).

In the scenario where the data packet is smaller than 64 kb, RocksDB outperforms in terms of memory usage, CPU usage, and response time. On the other hand, if the data packet is greater than 64 kb, NATS excels in terms of response time with sufficient memory and ideal GC scheduling.

Currently, you are advised to use NATS only for experiments.

What's next​

Learn how to configure other Milvus dependencies with Docker Compose or Helm: