Answers: <p>To begin with, note the following text from the <a href="https://redis.io/docs/management/scaling/#redis-cluster-and-docker" rel="nofollow noreferrer">Redis documentation</a>:</p>
  <blockquote>
  <p>Currently, Redis Cluster does not support NATted environments and in general environments where IP addresses or TCP ports are remapped.</p>
  <p>Docker uses a technique called port mapping: programs running inside Docker containers may be exposed with a different port compared to the one the program believes to be using. This is useful for running multiple containers using the same ports, at the same time, in the same server.</p>
  <p>To make Docker compatible with Redis Cluster, you need to use Docker's host networking mode. Please see the --net=host option in the Docker documentation for more information.</p>
  </blockquote>
  <p>If your goal is to expose the redis cluster outside of your compose environment, then you're going to need to run everything with <code>network_mode: host</code>. When running in host network mode, you cannot use port publishing, so you'll need to (a) get rid of the <code>ports</code> section in your services and (b) ensure that each instance is listening on a different port.</p>
  <p>A second problem is that your <code>redis-cluster-creator</code> service runs before any of your redis containers are ready. The legacy form the <code>depends_on</code> option...</p>
  <pre><code>depends_on:
    - redis-node-1
    - redis-node-2
    - redis-node-3
    .
    .
    .
  </code></pre>
  <p>...is effectively useless; it only waits for the <em>container</em> to start; it doesn't know anything about the state of the application running inside the container. We can (a) add a healthcheck to the containers and then (b) use the <a href="https://docs.docker.com/compose/compose-file/05-services/#long-syntax-1" rel="nofollow noreferrer">extended form of depends_on</a> to wait until redis is actually up before attempting to create the cluster.</p>
  <p>Lastly, a bunch of little things:</p>
  <ul>
  <li>You probably don't need an individual configuration file for each node; they should all be using identical configurations, other than the port, which can be set on the command line.</li>
  <li>You probably want to use persistent storage for the redis data directory in <code>/data</code>.</li>
  </ul>
  <p>Putting all of that together, we get something like this:</p>
  <pre><code>services:
    redis-cluster-creator:
      command:
        - redis-cli
        - --cluster
        - create
        - localhost:7001
        - localhost:7002
        - localhost:7003
        - localhost:7004
        - localhost:7005
        - localhost:7006
        - --cluster-yes
        - --cluster-replicas
        - '1'
      depends_on:
        redis-node-1:
          condition: service_healthy
        redis-node-2:
          condition: service_healthy
        redis-node-3:
          condition: service_healthy
        redis-node-4:
          condition: service_healthy
        redis-node-5:
          condition: service_healthy
        redis-node-6:
          condition: service_healthy
      image: docker.io/redis:latest
      network_mode: host
    redis-node-1:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7001'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7001'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7001:7000
      volumes:
        - redis-data-1:/data
        - ./redis:/redis
    redis-node-2:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7002'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7002'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7002:7000
      volumes:
        - redis-data-2:/data
        - ./redis:/redis
    redis-node-3:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7003'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7003'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7003:7000
      volumes:
        - redis-data-3:/data
        - ./redis:/redis
    redis-node-4:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7004'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7004'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7004:7000
      volumes:
        - redis-data-4:/data
        - ./redis:/redis
    redis-node-5:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7005'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7005'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7005:7000
      volumes:
        - redis-data-5:/data
        - ./redis:/redis
    redis-node-6:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7006'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7006'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7006:7000
      volumes:
        - redis-data-6:/data
        - ./redis:/redis
  volumes:
    redis-data-1: {}
    redis-data-2: {}
    redis-data-3: {}
    redis-data-4: {}
    redis-data-5: {}
    redis-data-6: {}
  </code></pre>
  <p>The contents of <code>redis/redis.conf</code> is:</p>
  <pre><code>cluster-enabled yes
  cluster-config-file nodes.conf
  cluster-node-timeout 5000
  appendonly yes
  </code></pre>
  <p>You can see a successful <code>docker compose up</code> using this environment <a href="https://asciinema.org/a/P1LZ7O6jOwHupjWOZvEn7t8PZ" rel="nofollow noreferrer">here</a>.</p>
  <p>There is a lot of repetition in that compose file, so to make my life easier while experimenting I used <a href="https://jsonnet.org/" rel="nofollow noreferrer">jsonnet</a> to template things. You can find my complete environment at <a href="https://github.com/larsks/redis-cluster-example" rel="nofollow noreferrer">https://github.com/larsks/redis-cluster-example</a>.</p>
  
Answers: CC BY-SA 4.0
Answers: 2024-01-06T02:06:24.663Z
Answers: 2024-01-06T02:06:24.663Z
Answers: larsks
Body: <p>I need to create a Redis cluster using Docker (with Kubernetes afterwards) and didn't find any official documentation on that matter. Could I have guidance on that subject ?</p>
  <p>I have the below yaml content but I am not sure if it follows the best practices:</p>
  <pre><code>version: '3'
  
  networks:
    redis-cluster-compose:
      driver: bridge
  
  services:
    redis-node-1:
      image: redis:latest
      ports:
        - 7000:7000
      networks:
        - redis-cluster-compose
      hostname: redis-node-1
      volumes:
        - ./7000:/redis
      command: redis-server /redis/redis.conf
    redis-node-2:
      image: redis:latest
      ports:
        - 7001:7001
      networks:
        - redis-cluster-compose
      hostname: redis-node-2
      volumes:
        - ./7001:/redis
      command: redis-server /redis/redis.conf
  
    redis-node-3:
      image: redis:latest
      ports:
        - 7002:7002
      networks:
        - redis-cluster-compose
      hostname: redis-node-3
      volumes:
        - ./7002:/redis
      command: redis-server /redis/redis.conf
  
    redis-node-4:
      image: redis:latest
      ports:
        - 7003:7003
      networks:
        - redis-cluster-compose
      hostname: redis-node-4
      volumes:
        - ./7003:/redis
      command: redis-server /redis/redis.conf
  
    redis-node-5:
      image: redis:latest
      ports:
        - 7004:7004
      networks:
        - redis-cluster-compose
      hostname: redis-node-5
      volumes:
        - ./7004:/redis
      command: redis-server /redis/redis.conf
    
    redis-node-6:
      image: redis:latest
      ports:
        - 7005:7005
      networks:
        - redis-cluster-compose
      hostname: redis-node-6
      volumes:
        - ./7005:/redis
      command: redis-server /redis/redis.conf
    
    redis-cluster-creator:
      image: redis:latest
      ports:
        - 6999:6999
      networks:
        - redis-cluster-compose
      command: redis-cli -p 7000 --cluster create redis-node-1:7000 redis-node-2:7001 redis-node-3:7002 redis-node-4:7003 redis-node-5:7004 redis-node-6:7005 --cluster-replicas 1 --cluster-yes
      depends_on:
        - redis-node-1
        - redis-node-2
        - redis-node-3
        - redis-node-4
        - redis-node-5
        - redis-node-6
  </code></pre>
  
Tags: <docker><kubernetes><redis>
Title: Deploy a Redis cluster with Docker
UserDisplayName: Jérémy Ferfer
ThreadText: # Deploy a Redis cluster with Docker
  
  (Asked by: username_0 on 2024-01-05)
  
  I need to create a Redis cluster using Docker (with Kubernetes afterwards) and didn't find any official documentation on that matter. Could I have guidance on that subject ?
  
  I have the below yaml content but I am not sure if it follows the best practices:
  
  ```
  version: '3'
  
  networks:
    redis-cluster-compose:
      driver: bridge
  
  services:
    redis-node-1:
      image: redis:latest
      ports:
        - 7000:7000
      networks:
        - redis-cluster-compose
      hostname: redis-node-1
      volumes:
        - ./7000:/redis
      command: redis-server /redis/redis.conf
    redis-node-2:
      image: redis:latest
      ports:
        - 7001:7001
      networks:
        - redis-cluster-compose
      hostname: redis-node-2
      volumes:
        - ./7001:/redis
      command: redis-server /redis/redis.conf
  
    redis-node-3:
      image: redis:latest
      ports:
        - 7002:7002
      networks:
        - redis-cluster-compose
      hostname: redis-node-3
      volumes:
        - ./7002:/redis
      command: redis-server /redis/redis.conf
  
    redis-node-4:
      image: redis:latest
      ports:
        - 7003:7003
      networks:
        - redis-cluster-compose
      hostname: redis-node-4
      volumes:
        - ./7003:/redis
      command: redis-server /redis/redis.conf
  
    redis-node-5:
      image: redis:latest
      ports:
        - 7004:7004
      networks:
        - redis-cluster-compose
      hostname: redis-node-5
      volumes:
        - ./7004:/redis
      command: redis-server /redis/redis.conf
    
    redis-node-6:
      image: redis:latest
      ports:
        - 7005:7005
      networks:
        - redis-cluster-compose
      hostname: redis-node-6
      volumes:
        - ./7005:/redis
      command: redis-server /redis/redis.conf
    
    redis-cluster-creator:
      image: redis:latest
      ports:
        - 6999:6999
      networks:
        - redis-cluster-compose
      command: redis-cli -p 7000 --cluster create redis-node-1:7000 redis-node-2:7001 redis-node-3:7002 redis-node-4:7003 redis-node-5:7004 redis-node-6:7005 --cluster-replicas 1 --cluster-yes
      depends_on:
        - redis-node-1
        - redis-node-2
        - redis-node-3
        - redis-node-4
        - redis-node-5
        - redis-node-6
  
  ```
  
  **Tags:** docker, kubernetes, redis
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-01-06. Score: 2
  
  To begin with, note the following text from the [Redis documentation](https://redis.io/docs/management/scaling/#redis-cluster-and-docker):
  
  > Currently, Redis Cluster does not support NATted environments and in general environments where IP addresses or TCP ports are remapped.
  >
  > Docker uses a technique called port mapping: programs running inside Docker containers may be exposed with a different port compared to the one the program believes to be using. This is useful for running multiple containers using the same ports, at the same time, in the same server.
  >
  > To make Docker compatible with Redis Cluster, you need to use Docker's host networking mode. Please see the --net=host option in the Docker documentation for more information.
  
  If your goal is to expose the redis cluster outside of your compose environment, then you're going to need to run everything with `network_mode: host`. When running in host network mode, you cannot use port publishing, so you'll need to (a) get rid of the `ports` section in your services and (b) ensure that each instance is listening on a different port.
  
  A second problem is that your `redis-cluster-creator` service runs before any of your redis containers are ready. The legacy form the `depends_on` option...
  
  ```
  depends_on:
    - redis-node-1
    - redis-node-2
    - redis-node-3
    .
    .
    .
  
  ```
  
  ...is effectively useless; it only waits for the *container* to start; it doesn't know anything about the state of the application running inside the container. We can (a) add a healthcheck to the containers and then (b) use the [extended form of depends\_on](https://docs.docker.com/compose/compose-file/05-services/#long-syntax-1) to wait until redis is actually up before attempting to create the cluster.
  
  Lastly, a bunch of little things:
  
  * You probably don't need an individual configuration file for each node; they should all be using identical configurations, other than the port, which can be set on the command line.
  * You probably want to use persistent storage for the redis data directory in `/data`.
  
  Putting all of that together, we get something like this:
  
  ```
  services:
    redis-cluster-creator:
      command:
        - redis-cli
        - --cluster
        - create
        - localhost:7001
        - localhost:7002
        - localhost:7003
        - localhost:7004
        - localhost:7005
        - localhost:7006
        - --cluster-yes
        - --cluster-replicas
        - '1'
      depends_on:
        redis-node-1:
          condition: service_healthy
        redis-node-2:
          condition: service_healthy
        redis-node-3:
          condition: service_healthy
        redis-node-4:
          condition: service_healthy
        redis-node-5:
          condition: service_healthy
        redis-node-6:
          condition: service_healthy
      image: docker.io/redis:latest
      network_mode: host
    redis-node-1:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7001'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7001'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7001:7000
      volumes:
        - redis-data-1:/data
        - ./redis:/redis
    redis-node-2:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7002'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7002'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7002:7000
      volumes:
        - redis-data-2:/data
        - ./redis:/redis
    redis-node-3:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7003'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7003'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7003:7000
      volumes:
        - redis-data-3:/data
        - ./redis:/redis
    redis-node-4:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7004'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7004'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7004:7000
      volumes:
        - redis-data-4:/data
        - ./redis:/redis
    redis-node-5:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7005'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7005'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7005:7000
      volumes:
        - redis-data-5:/data
        - ./redis:/redis
    redis-node-6:
      command:
        - redis-server
        - /redis/redis.conf
        - --port
        - '7006'
      healthcheck:
        interval: 10s
        retries: '3'
        test:
          - CMD
          - redis-cli
          - -p
          - '7006'
          - -c
          - ping
        timeout: 5s
      image: docker.io/redis:latest
      network_mode: host
      ports:
        - 7006:7000
      volumes:
        - redis-data-6:/data
        - ./redis:/redis
  volumes:
    redis-data-1: {}
    redis-data-2: {}
    redis-data-3: {}
    redis-data-4: {}
    redis-data-5: {}
    redis-data-6: {}
  
  ```
  
  The contents of `redis/redis.conf` is:
  
  ```
  cluster-enabled yes
  cluster-config-file nodes.conf
  cluster-node-timeout 5000
  appendonly yes
  
  ```
  
  You can see a successful `docker compose up` using this environment [here](https://asciinema.org/a/P1LZ7O6jOwHupjWOZvEn7t8PZ).
  
  There is a lot of repetition in that compose file, so to make my life easier while experimenting I used [jsonnet](https://jsonnet.org/) to template things. You can find my complete environment at <https://github.com/username_1/redis-cluster-example>.
  
  
  ---
  
  