关于docker版本的kafka,社区使用得比较多的是 wurstmeister/kafka-docker,而它所给出的docker-compose.yaml示例,个人觉得不是很友好。

通常来说,我们选择了docker版本的kafka,多数时候,也是在docker容器中的其他程序会使用到kafka,而不是暴露到外网中使用。同时,也希望容器重启了之后,数据能够继续维持,这当然也包括zookeeper的配置也需要保留。

因此,在项目中我是使用了如下的配置以进行快速搭建环境:

version: "2"
services:
  zookeeper:
    image: zookeeper
    restart: always
    ports:
      - "2181:2181"
    volumes:
      - ./zookeeper/data:/opt/zookeeper/data
      - ./zookeeper/logs:/opt/zookeeper/logs
    environment:
      - ZOO_DATA_DIR=/opt/zookeeper/data
      - ZOO_DATA_LOG_DIR=/opt/zookeeper/data
  kafka:
    image: wurstmeister/kafka:2.12-2.1.1
    restart: always
    links:
      - zookeeper
    ports:
      - "9092"
    volumes:
      - ./kafka/logs:/kafka
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      HOSTNAME_COMMAND: ip addr show dev eth0 | awk '/inet/ {print $$2}' | awk -F'/' '{print $$1}'

日志数据将会保留在 ./kafka/logs./zookeeper/data,并且容器重启之后也不会丢失。

在项目中,完整的配置文件如下:

version: "2"

services:
  xxlive:
   build: .
   links:
     - kafka
     - mysql
     - redis
     - consul
   image: xxlive:centos7
   privileged: yes
   ports:
     - 9600:9600
     - 9602:9602
     - 9680:8080
     - 9633:9633
     - 32345:2345
     - 9631:9631
     - 9632:9632
   volumes:
     - ./bin:/app/golang/bin
     - ./etc:/app/golang/etc
     - ./logs:/data/golang
     - ./python:/app/python
     - ./app/golang/start.sh:/app/golang/start.sh
     - ./app/golang/startdata:/app/golang/startdata
   working_dir: /app/golang/
   command: sh -c '/app/start_live.sh && while true; do sleep 7d; done'
   depends_on:
    - redis
    - mysql
    - kafka
    - consul
  consul:
    image: consul:1.2.1
    restart: always
    ports:
      - 8500:8500
    environment:
      - CONSUL_BIND_INTERFACE=eth0
    command: consul agent -client 0.0.0.0 -dev -ui -datacenter=sg**-test
  redis:
    image: redis:4
    restart: always
    ports:
      - 26379:6379
  mysql:
    image: mysql:8
    restart: always
    ports:
      - 23306:3306
    environment:
      - MYSQL_ROOT_PASSWORD=****
    working_dir: /work
    volumes:
      - ./mysql/db:/work
      - ./mysql/data:/var/lib/mysql
    command: --default-authentication-plugin=mysql_native_password
  zookeeper:
    image: zookeeper
    restart: always
    ports:
      - "2181:2181"
    volumes:
      - ./zookeeper/data:/opt/zookeeper/data
      - ./zookeeper/logs:/opt/zookeeper/logs
    environment:
      - ZOO_DATA_DIR=/opt/zookeeper/data
      - ZOO_DATA_LOG_DIR=/opt/zookeeper/data
  kafka:
    image: wurstmeister/kafka:2.12-2.1.1
    restart: always
    links:
      - zookeeper
    ports:
      - "9092"
    volumes:
      - ./kafka/logs:/kafka
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      HOSTNAME_COMMAND: ip addr show dev eth0 | awk '/inet/ {print $$2}' | awk -F'/' '{print $$1}'

在这个完整的docker-compose.yaml文件中,也包含了 mysql、redis、consul等服务,这些都大家常见的服务了,并且这里我所使用的是官方的zookeeper镜像,它明显比wurstmeister/zookeeper要小得多。