Answers: <p>You can connect to your Redis container port without exposing it and calling it through localhost if they are both in the same network. You just have to call the container name along with the port number as the URL in your Cronjob container and you will be able to achieve this.</p>
  <p>Since you haven't provided any details of your docker-compose file, I will give you an example:</p>
  <pre><code>version: '3.8'
  
  services:
    redis:
      image: my-redis-image
      container_name: myredis
      expose:
        - &quot;6379&quot;
  
    cronjob:
      image: my-cronjob-image
      container_name: mycronjob
      environment:
        - URL=http://myredis:6379
  
  networks:
    default:
      driver: bridge
  </code></pre>
  <p>I have passed the container URL through as an environment variable in this docker-compose file. You can change that and give it in your <code>Dockerfile</code> or within your image. The containers are in the same network so you can connect to the Redis container without mapping it to the localhost.</p>
  <hr />
  <h2>Edit:</h2>
  <p>If you cannot include the cronjob container in the docker-compose file, then you can run the container under the same network used by the Redis container and use the URL <code>http://myredis:6379</code>. Check the containers are in the same network using this command:</p>
  <pre><code> docker container inspect --format '{{range $net,$v := .NetworkSettings.Networks}}{{printf &quot;%s\n&quot; $net}}{{end}}' &lt;container_name&gt;
  </code></pre>
  <p>Replace the <em>container_name</em> with your Redis container and the Cronjob container. If they are not under the same network, run your cronjob using this command:</p>
  <pre><code>docker run -d --name mycronjob --network &lt;network_name&gt; my-cronjob-image
  </code></pre>
  <p>Replace the <em>network_name</em> with the Redis container's network.</p>
  
Answers: CC BY-SA 4.0
Answers: 2024-07-08T07:28:49.797Z
Answers: Thanks a lot, I will test it, seems `expose`-ing the 6379 may solve the problem. But the cronjob cannot be part of the compose file, as it is not a long running service. The cronjob container is intended to start every hour, execute its `CMD ["./run_manager.sh"]`, the last line of the `Dockerfile` and terminate (taking about ~5 minutes). So basically, a container outside the compose file must be able to reach `myredis:6379` Also, what do the last three lines (network options) mean?
Answers: Della
Answers: CC BY-SA 4.0
Answers: 2024-07-08T07:39:30.817Z
Answers: I added some more details in the question as well.
Answers: Della
Answers: CC BY-SA 4.0
Answers: 2024-07-08T10:55:22.720Z
Answers: @Della The three lines define a network that will be used for the containers. If you didn't provide that, the docker-compose file will create one for you
Answers: Ajay
Answers: CC BY-SA 4.0
Answers: 2024-07-08T11:11:39.713Z
Answers: @Della I have edited my answer. Let me know what works for you.
Answers: Ajay
Answers: CC BY-SA 4.0
Answers: 2024-07-08T06:28:20.613Z
Answers: 2024-07-08T11:11:02.287Z
Answers: 2024-07-08T11:11:02.287Z
Answers: Ajay
Body: <h5>Context</h5>
  <p>I hope the following diagram makes it clear, ask if you still need any answer.</p>
  <p><a href="https://i.sstatic.net/TUvtYXJj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/TUvtYXJj.png" alt="enter image description here" /></a></p>
  <p>Basically, the cronjob itself (a container) will run every hour, that needs to use a long running redis service. (The redis service acts as an interface to some other worker containers, but the cronjob will not connect directly to them). Everything running inside the same Ubuntu instance with docker set up.</p>
  <h5>Initial Idea</h5>
  <p>I have the docker compose file that can fire up the workers and the redis service. But can I put an <em>hourly</em> cronjob running on a container (not a long running service) in the same file, to take advantage of the docker-compose networking? In that case, I don't have to expose 6379 to the localhost, am I correct?</p>
  <p>Alternatively, if I handle the worker and redis service inside compose, can I make the cronjob container run inside the same network without exposing/publishing post 6379?</p>
  <p>Some ideas and options available in docker's networking/compose stack around this scenario will be appreciated.</p>
  <p>And I don't have a K8S cluster available, and it's not feasible. So have to keep the solution within standalone docker stack.</p>
  <h5>Further Clarifications</h5>
  <p>Based on @ajay's helpful answer, I decided to add some more clarifications on how I am envisioning the <code>docker-compose</code> and the cron schedule.</p>
  <pre class="lang-yaml prettyprint-override"><code>services:
    redis: # This becomes the hostname
      image: &quot;redis:alpine&quot; # port 6379 of this runs the service. 
  
    workers:
      # The workers must not be directly visible to the cronjob. 
      # But each worker container must access redis:6379
      image: della/worker # From Dockerhub registry
      depends_on:
        - redis
      deploy:
        replicas: 5 # Scale out 
  </code></pre>
  <p>The Cron schedule looks like this</p>
  <pre class="lang-bash prettyprint-override"><code># The cronjob_container must be able to reach redis:6379
  # Takes ~5 minutes to run. 
  @hourly docker container start cronjob_container 
  </code></pre>
  <p>The <code>cronjob_container</code> <em>cannot</em> be part of the <code>docker-compose</code> because it's not intended as a long running service. But it needs to be part of the network defined by docker compose.</p>
  <p>However, if the compose file allows a cronjob instead of a service, something akin to Kubernetes <code>CronJob</code> resource, that would be the ideal solution.</p>
  
Comments: CC BY-SA 4.0
Comments: 2024-07-09T20:22:28.040Z
Comments: If you can't use k8s on a single node, how about dokku?  Can you clarify why k8s on the node is not practical?
Comments: chicks
Comments: CC BY-SA 4.0
Comments: 2024-07-10T02:26:36.077Z
Comments: 1. I have an ec2 instance, the workload is not that high warranting K8S, although I can fire up a minikube cluster (the only way I have used K8s before). 2. The docker images rest in Google Artefact repo. Fetching them in the minikube pods? I am not sure how to do it (although there must be ways).
  I am not a DevOps, but ML engineer, so trying to find my way around situation. The options are based on tools I understand and confident with, rather than what is objectively the best.
Comments: Della
Comments: CC BY-SA 4.0
Comments: 2024-07-10T02:29:48.013Z
Comments: https://dokku.com/ is easier to get started with than k8s.  K8s requires an ongoing investment in maintenance and updates so not choosing to have that much complexity is completely reasonable.
Comments: chicks
Tags: <docker><docker-compose><networking><redis>
Title: Best Way to Call a Service Running in a Docker Container from another Docker Container in the Same EC2 Instance?
UserDisplayName: Della
ThreadText: # Best Way to Call a Service Running in a Docker Container from another Docker Container in the Same EC2 Instance?
  
  (Asked by: username_0 on 2024-07-07)
  
  ##### Context
  
  I hope the following diagram makes it clear, ask if you still need any answer.
  
  [![enter image description here](https://i.sstatic.net/TUvtYXJj.png)](https://i.sstatic.net/TUvtYXJj.png)
  
  Basically, the cronjob itself (a container) will run every hour, that needs to use a long running redis service. (The redis service acts as an interface to some other worker containers, but the cronjob will not connect directly to them). Everything running inside the same Ubuntu instance with docker set up.
  
  ##### Initial Idea
  
  I have the docker compose file that can fire up the workers and the redis service. But can I put an *hourly* cronjob running on a container (not a long running service) in the same file, to take advantage of the docker-compose networking? In that case, I don't have to expose 6379 to the localhost, am I correct?
  
  Alternatively, if I handle the worker and redis service inside compose, can I make the cronjob container run inside the same network without exposing/publishing post 6379?
  
  Some ideas and options available in docker's networking/compose stack around this scenario will be appreciated.
  
  And I don't have a K8S cluster available, and it's not feasible. So have to keep the solution within standalone docker stack.
  
  ##### Further Clarifications
  
  Based on @username_2's helpful answer, I decided to add some more clarifications on how I am envisioning the `docker-compose` and the cron schedule.
  
  ```yaml
  services:
    redis: # This becomes the hostname
      image: "redis:alpine" # port 6379 of this runs the service. 
  
    workers:
      # The workers must not be directly visible to the cronjob. 
      # But each worker container must access redis:6379
      image: username_0/worker # From Dockerhub registry
      depends_on:
        - redis
      deploy:
        replicas: 5 # Scale out 
  
  ```
  
  The Cron schedule looks like this
  
  ```bash
  # The cronjob_container must be able to reach redis:6379
  # Takes ~5 minutes to run. 
  @hourly docker container start cronjob_container 
  
  ```
  
  The `cronjob_container` *cannot* be part of the `docker-compose` because it's not intended as a long running service. But it needs to be part of the network defined by docker compose.
  
  However, if the compose file allows a cronjob instead of a service, something akin to Kubernetes `CronJob` resource, that would be the ideal solution.
  
  **Tags:** docker, docker-compose, networking, redis
  
  > **Comments:**
  > **username_1**: If you can't use k8s on a single node, how about dokku?  Can you clarify why k8s on the node is not practical?  
  > **username_0**: 1. I have an ec2 instance, the workload is not that high warranting K8S, although I can fire up a minikube cluster (the only way I have used K8s before). 2. The docker images rest in Google Artefact repo. Fetching them in the minikube pods? I am not sure how to do it (although there must be ways). I am not a DevOps, but ML engineer, so trying to find my way around situation. The options are based on tools I understand and confident with, rather than what is objectively the best.  
  > **username_1**: https://dokku.com/ is easier to get started with than k8s.  K8s requires an ongoing investment in maintenance and updates so not choosing to have that much complexity is completely reasonable.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-07-08. Score: 1 | Accepted answer
  
  You can connect to your Redis container port without exposing it and calling it through localhost if they are both in the same network. You just have to call the container name along with the port number as the URL in your Cronjob container and you will be able to achieve this.
  
  Since you haven't provided any details of your docker-compose file, I will give you an example:
  
  ```
  version: '3.8'
  
  services:
    redis:
      image: my-redis-image
      container_name: myredis
      expose:
        - "6379"
  
    cronjob:
      image: my-cronjob-image
      container_name: mycronjob
      environment:
        - URL=http://myredis:6379
  
  networks:
    default:
      driver: bridge
  
  ```
  
  I have passed the container URL through as an environment variable in this docker-compose file. You can change that and give it in your `Dockerfile` or within your image. The containers are in the same network so you can connect to the Redis container without mapping it to the localhost.
  
  ---
  
  ## Edit:
  
  If you cannot include the cronjob container in the docker-compose file, then you can run the container under the same network used by the Redis container and use the URL `http://myredis:6379`. Check the containers are in the same network using this command:
  
  ```
   docker container inspect --format '{{range $net,$v := .NetworkSettings.Networks}}{{printf "%s\n" $net}}{{end}}' <container_name>
  
  ```
  
  Replace the *container\_name* with your Redis container and the Cronjob container. If they are not under the same network, run your cronjob using this command:
  
  ```
  docker run -d --name mycronjob --network <network_name> my-cronjob-image
  
  ```
  
  Replace the *network\_name* with the Redis container's network.
  
  > **Comments:**
  > **username_0**: Thanks a lot, I will test it, seems `expose`-ing the 6379 may solve the problem. But the cronjob cannot be part of the compose file, as it is not a long running service. The cronjob container is intended to start every hour, execute its `CMD ["./run_manager.sh"]`, the last line of the `Dockerfile` and terminate (taking about ~5 minutes). So basically, a container outside the compose file must be able to reach `myredis:6379` Also, what do the last three lines (network options) mean?  
  > **username_0**: I added some more details in the question as well.  
  > **username_2**: @username_0 The three lines define a network that will be used for the containers. If you didn't provide that, the docker-compose file will create one for you  
  > **username_2**: @username_0 I have edited my answer. Let me know what works for you.  
  
  ---
  
  