目录

k8s 集群暴露 kafka 端口

李斯文收录于 Middleware

2020-03-27 约 642 字预计阅读 2 分钟

目录

部署方式

使用 bitnami 的 kafka chart 来部署。

暴露端口到集群外

仔细阅读文档可知，标准的 chart 通过配置 values.yaml 文件的 externalAccess 就能通过主机端口来访问到 kafka，部分配置如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


externalAccess:
  enabled: true
  service:
    ## Type of service for external access. It can be LoadBalancer or NodePort.
    ##
    type: NodePort
    ## Port used when service type is LoadBalancer
    ##
    port: 19092
    ## Array of load balancer IPs for each Kafka broker. Length must be the same as replicas
    ##
    loadBalancerIP: []
    ## When service type is NodePort, you can specify the domain used for Kafka advertised listeners.
    ## If not specified, the container will try to get the kubernetes node external IP using: 'curl -s https://ipinfo.io/ip'
    ## 这里填写一个域名，该域名会配置到 advertised listeners 里，集群外连接 kafka 拿到的 meta 信息就是该域名加上下面的端口列表
    domain: kafka.test.com
    ## Array of node ports used for each Kafka broker. Length must be the same as replicas
    ## 根据自己的端口规划填写
    nodePort: [39090, 39091, 39092]

    ## Service annotations done as key:value pairs
    annotations: {}

这里给外部访问的是 kafka.test.com:39090，kafka.test.com:39091，kafka.test.com:39092 ，如果 kafka.test.com 对应的主机挂了，则服务不可用。

暴露 3 个节点

直接修改 templates/scripts-configmap.yaml 文件，修改 EXTERNAL_ACCESS_IP 如下：

1
2


    # Configure external ip and port
    export EXTERNAL_ACCESS_IP=$(echo '[ip_or_domain-1 ip_or_domain-2 ip_or_domain-3]' | tr -d '[]' | cut -d ' ' -f "$(($ID + 1))")

外部访问添加 SASL_PLAINTEXT 认证

修改 values.yaml 文件

主要修改项如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


allowPlaintextListener: true
deleteTopicEnable: true
numPartitions: 30
extraEnvVars:
  - name: KAFKA_CFG_SECURITY_INTER_BROKER_PROTOCOL
    value: "SASL_PLAINTEXT"
  - name: KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL
    value: "PLAIN"
  - name: KAFKA_CFG_SASL_ENABLED_MECHANISMS
    value: "PLAIN"
  - name: KAFKA_CFG_ALLOW_EVERYONE_IF_NO_ACL_FOUND
    value: "true"
auth:
  enabled: true
  ssl: false
  ## Kafka client user.
  brokerUser: brokeruser
  ## Kafka client password.
  brokerPassword: brokerpassword
  ## Kafka inter broker communication user.
  interBrokerUser: adminuser
  ## Kafka inter broker communication password.
  interBrokerPassword: adminpassword
  ## Kafka Zookeeper user.
  zookeeperUser: admin
  ## Kafka Zookeeper password.
  zookeeperPassword: adminpassword

修改 templates/statefulset.yaml 文件

修改 SASL_SSL 为 SASL_PLAINTEXT

1
2
3
4
5
6


value: "SASL_SSL://:$(KAFKA_PORT_NUMBER)"
value: "SASL_PLAINTEXT://:$(KAFKA_PORT_NUMBER)"

value: 'SASL_SSL://$(MY_POD_NAME).{{ template "kafka.fullname" . }}-headless.{{.Release.Namespace}}.svc.{{ .Values.clusterDomain }}:$(KAFKA_PORT_NUMBER)'
value: 'SASL_PLAINTEXT://$(MY_POD_NAME).{{ template "kafka.fullname" . }}-headless.{{.Release.Namespace}}.svc.{{ .Values.clusterDomain }}:$(KAFKA_PORT_NUMBER)'

移除证书相关配置

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


-            {{- if .Values.auth.enabled }}
-            - name: kafka-certificates
-              mountPath: /opt/bitnami/kafka/conf/certs/
-              readOnly: true
-            {{- end }}

-        {{ if .Values.auth.enabled }}
-        - name: kafka-certificates
-          secret:
-            secretName: {{ required "A secret containing the Kafka JKS certificates is required when authentication in enabled" .Values.auth.certificatesSecret }}
-            defaultMode: 256
-        {{ end }}

修改 templates/scripts-configmap.yaml 文件

1
2
3
4


# 原始配置
export KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
# 添加 EXTERNAL 认证
export KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT