k8s中部署zookeeper和kafka

环境：单节点的k8s环境，版本v1.21.5

部署zookeeper

zookeeper-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: zookeeper
  name: zookeeper
spec:
  selector:
    matchLabels:
     app: zookeeper
  replicas: 1
  template:
    metadata:
      labels:
        app: zookeeper
    spec:
      containers:
      - image: wurstmeister/zookeeper
        imagePullPolicy: IfNotPresent
        name: zookeeper
        ports:
        - containerPort: 2181

zookeeper-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: zookeeper-service
  name: zookeeper-service
spec:
  type: NodePort
  ports:
  - name: zookeeper-port
    port: 2181
    nodePort: 30181
    targetPort: 2181
  selector:
    app: zookeeper

$ kubectl create ns kafka
$ kubectl create -f zookeeper-service.yaml,zookeeper-deployment.yaml -n kafka

部署kafka

kafka-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: kafka
  name: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
     app: kafka
  template:
    metadata:
      labels:
        app: kafka
    spec:
      containers:
      - env:
        - name: KAFKA_ADVERTISED_HOST_NAME
          value: "10.0.4.9" #指定为自己的IP地址
        - name: KAFKA_ADVERTISED_PORT
          value: "30092"
        - name: KAFKA_BROKER_ID
          value: "1"
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: 10.0.4.9:30181  #指定为自己的IP地址
        - name: KAFKA_CREATE_TOPICS
          value: "test:1:1"
        image: wurstmeister/kafka
        imagePullPolicy: IfNotPresent
        name: kafka
        ports:
        - containerPort: 9092

kafka-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: kafka-service
  name: kafka-service
spec:
  type: NodePort
  ports:
  - name: kafka-port
    port: 9092
    nodePort: 30092
    targetPort: 9092
  selector:
    app: kafka

$ kubectl create -f kafka-deploy.yaml,kafka-service.yaml -n kafka

查看所创建的资源：

[root@master ~]# kubectl get svc -n kafka
NAME                TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
kafka-service       NodePort   10.233.24.121           9092:30092/TCP   4m38s
zookeeper-service   NodePort   10.233.6.121            2181:30181/TCP   12m
[root@master ~]# kubectl get pods -n kafka
NAME                         READY   STATUS    RESTARTS   AGE
kafka-755d75d8f7-k7lbt       1/1     Running   0          96s
zookeeper-75fd7fdb96-h9t2z   1/1     Running   0          12m
[root@master ~]# 

发送和消费消息

#进入到容器内部
[root@master ~]# kubectl exec -it kafka-755d75d8f7-k7lbt -n kafka -- sh
#查看kafka的topic数量
/opt/kafka_2.13-2.8.1/bin # ./kafka-topics.sh --zookeeper 10.0.4.9:30181 --list
test
/opt/kafka_2.13-2.8.1/bin # 
#也可也使用zookeeper的svc的地址和端口
/opt/kafka_2.13-2.8.1/bin # ./kafka-topics.sh --zookeeper 10.233.6.121:2181 --list
test
/opt/kafka_2.13-2.8.1/bin #
#或者使用域名zookeeper-service.kafka，来访问zookeeper
/opt/kafka_2.13-2.8.1/bin # ./kafka-topics.sh --zookeeper zookeeper-service.kafka:2181 --list
test
/opt/kafka_2.13-2.8.1/bin 

#创建topic
$ /opt/kafka_2.13-2.8.1/bin # ./kafka-topics.sh --zookeeper 10.0.4.9:30181 --create --replication-factor 1 --partitions 1 --topic web_log
#查看topic详情：
$ /opt/kafka_2.13-2.8.1/bin # ./kafka-topics.sh --zookeeper 10.0.4.9:30181 --describe --topic web_log
Topic: web_log  TopicId: V_c_OFkHRI-z5MyO_Za_nA PartitionCount: 1       ReplicationFactor: 1       Configs: 
        Topic: web_log  Partition: 0    Leader: 1       Replicas: 1     Isr: 1

#发送消息
$ ./opt/kafka_2.13-2.8.1/bin # ./kafka-console-producer.sh --broker-list 10.0.4.9:30092  --topic web_log

#消费消息
$ ./kafka-console-consumer.sh --bootstrap-server 10.0.4.9:30092  --from-beginning --topic web_log

关于容器内时间和宿主机时间不一致的问题

在yaml文件里面增加环境变量或者挂载宿主机时间

      - env: #在env下添加TZ
        - name: TZ
          value: Asia/Shanghai

修改容器的配置文件，增加env的tz配置
kafka ： kubectl edit deploy kafka -n kafka

zookeeper: kubectl edit deploy/zookeeper -n kafka

更多解决方法见：k8s集群下pod时间、虚拟机时间不一致的解决办法

参考链接：
添加 Kafka 作为日志接收者