Answers: <p>Docker images are after all VM templates, i.e. they have to be more or less self-contained: you get the image, you run the environment with all the dependencies.</p>
  
  <p>To approach the challenge <strong>operationally</strong>, Docker supports reuse of environments through the <code>FROM</code> statement i.e. you could maintain a base image and on top a much smaller image with the app itself. </p>
  
  <p>But, from the <strong>strategic point of view</strong>, I would say your architects might want consider splitting the whole thing into smaller microservices - that would allow for 5 smaller images. Of course I cannot judge whether it's really possible.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-08-17T11:46:20.927Z
Answers: Docker images aren't "VM templates" as containerisation isn't the same as VM.
Answers: Hauleth
Answers: CC BY-SA 3.0
Answers: 2017-08-17T12:57:16.400Z
Answers: I yet have to search for the right term, but indeed with a Docker container you can simulate a host, therefore I consider containers to be lightweight VMs (in terms of Turing machine but also with a virtual network adapter and mountable virtual storage)
Answers: Ta Mu
Answers: CC BY-SA 3.0
Answers: 2017-08-17T13:01:58.837Z
Answers: Turing machine has nothing to do with either Docker or VMs. Virtual Machine simulate (nomen omen) machine, while containers (like Docker) only restraint user space. The difference is crucial. And if something Docker wasn't meant as "lightweight VMs", this is role of LXC.
Answers: Hauleth
Answers: CC BY-SA 3.0
Answers: 2017-08-17T15:07:51.170Z
Answers: then I need some better and correct wording to explain mgmt why and how Docker could be better than VMWare.. worth of another SE question :-)
Answers: Ta Mu
Answers: CC BY-SA 3.0
Answers: 2017-08-19T11:15:05.610Z
Answers: you can think about Docker containers as a fat JARs for something other than Java. So you pack all your dependencies and provide single file that can be ran as an application (from end user viewpoint). LXC and VM are rather system-like environments where you run your init system and all the things that are normally ran during startup.
Answers: Hauleth
Answers: CC BY-SA 3.0
Answers: 2017-08-21T14:05:05.567Z
Answers: what if I need to have a system-like environment in Docker? i.e. many middleware services
Answers: Ta Mu
Answers: CC BY-SA 3.0
Answers: 2017-08-21T14:06:09.637Z
Answers: Then use either multiple Docker containers or LXC.
Answers: Hauleth
Answers: CC BY-SA 3.0
Answers: 2017-08-21T14:20:32.787Z
Answers: which effectively replaces set of VMs. So I can't understand why I can't tell to business, "let's deploy instead of that MySQL VM just a MySQL container" => clever guys use VMs as service containers as well
Answers: Ta Mu
Answers: CC BY-SA 3.0
Answers: 2017-08-21T14:23:00.757Z
Answers: You should never run DB in container.
Answers: Hauleth
Answers: CC BY-SA 3.0
Answers: 2017-08-21T15:45:28.850Z
Answers: Integration environment neither - just for CI testing?
Answers: Ta Mu
Answers: CC BY-SA 3.0
Answers: 2017-08-21T16:02:22.307Z
Answers: In that case you shouldn't care. But in CI do you run multiple things in container?
Answers: Hauleth
Answers: CC BY-SA 3.0
Answers: 2017-11-20T22:14:19.943Z
Answers: @Hauleth docker being more or less lxc with aufs (non persistent file system), are you really speaking about LinuXContainer or something else? I fully agree with the difference between vm, VMware, xen, kvm or else doing hardware emulation against containers doing process isolation but I've a hard time to relate that to lxc)
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2017-11-20T22:18:55.353Z
Answers: @PeterMuryshkin à mysql server is not just the mysql service when you want to run it out of toy projects. To run properly in production, you need to give mysql access to low level io for caching, let it swap properly indexes and a bunch of other things suffering a lot from namespace isolation in a container. Added to the complexity of running maintenance jobs (hourly/weekly) which need to work in sync between the mysql server and filesystem. Containers are not lightweight VMs
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2017-11-20T22:20:47.807Z
Answers: BTW the point of Uber jar stands and is a proper comparison, I think I get the confusion between Java vm and hardware vm from this point of view.
Answers: Tensibai
Answers: CC BY-SA 3.0
Answers: 2017-08-14T05:47:43.220Z
Answers: 2017-11-20T18:58:55.923Z
Answers: 2017-11-20T18:58:55.923Z
Answers: Ta Mu
Answers: <p>I'm thinking about different approach to your problem. Instead of adding maven repository into the image, I will mount a volume to $HOME/.m2/repository, and another volume to $PROJECT_DIR</p>
  
  <pre><code>ENV PROJECT_DIR=/project
  VOLUME /repository
  VOLUME ${PROJECT_DIR}
  RUN mkdir -p $HOME/.m2 /repository &amp;&amp; ln -s /repository $HOME/.m2/repository
  </code></pre>
  
  <p>After that, add a condition check in component.sh whether this is the first run and then build the required component.</p>
  
  <pre><code>...
  if [[ ! -e $PROJECT_DIR/app_is_ready ]]; then
    cd $PROJECT_DIR
    mvn clean install -P test
    touch $PROJECT_DIR/app_is_ready
  fi
  &lt;&lt;actual run commands&gt;&gt;
  </code></pre>
  
  <p>Please be note that the example code is not handling the race condition of two (or more) instances using the same volume for PROJECT_DIR started at the same time.</p>
  
  <p>Hope this help!</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-09-21T16:11:06.360Z
Answers: 2017-09-21T16:11:06.360Z
Answers: Isaac A. Nugroho
Body: <p>I am trying to make images from our company Maven based applications. These apps are using some libraries which are included in a Maven repository, but fills a lot of disk space. They also have some dependencies which should be downloaded. I have seen <a href="https://devops.stackexchange.com/questions/1039/using-gitlab-ci-to-cache-maven-repository">this question</a> which is talking about downloading dependencies (and unfortunately has no approved answer), but for me it is more important to use my local repository without adding it to the image (which increases image size from 300M to 4G) and caching downloading dependencies is in the second place.</p>
  
  <p>If I wanted to <code>ADD</code> my local repository to the image, I would have make this <code>Dockerfile</code>:</p>
  
  <pre><code>FROM java:8.0_31
  MAINTAINER Zeinab Abbasimazar (zeinab.abbasi@peykasa.ir)
  ADD .m2/ $HOME
  WORKDIR .
  RUN apt-get update;apt-get install -y maven;cd Development/;mvn clean install -P test
  CMD ["./Component.sh &amp;"]
  </code></pre>
  
  <p>Which makes this image:</p>
  
  <pre><code>sudo docker images
  REPOSITORY                                   TAG                 IMAGE ID            CREATED             SIZE
  maven                                        3.3.9               44ab79a0bd9e        2 hours ago         4.678 GB
  </code></pre>
  
  <p>This size for an image is not affordable for me; could anyone please help  me on this? Is there any workaround using <code>gitlab</code> or <code>Jenkins</code> to play <code>Maven Repository</code> role?</p>
  
Tags: <docker><dockerfile><maven><repository>
Title: Make use of CIs as Maven Repository in order to use it for building Docker images without including dependencies
UserDisplayName: Zeinab Abbasimazar
ThreadText: # Make use of CIs as Maven Repository in order to use it for building Docker images without including dependencies
  
  (Asked by: username_0 on 2017-08-12)
  
  I am trying to make images from our company Maven based applications. These apps are using some libraries which are included in a Maven repository, but fills a lot of disk space. They also have some dependencies which should be downloaded. I have seen [this question](https://devops.stackexchange.com/questions/1039/using-gitlab-ci-to-cache-maven-repository) which is talking about downloading dependencies (and unfortunately has no approved answer), but for me it is more important to use my local repository without adding it to the image (which increases image size from 300M to 4G) and caching downloading dependencies is in the second place.
  
  If I wanted to `ADD` my local repository to the image, I would have make this `Dockerfile`:
  
  ```
  FROM java:8.0_31
  MAINTAINER username_0 username_0 (username_0.abbasi@peykasa.ir)
  ADD .m2/ $HOME
  WORKDIR .
  RUN apt-get update;apt-get install -y maven;cd Development/;mvn clean install -P test
  CMD ["./Component.sh &"]
  
  ```
  
  Which makes this image:
  
  ```
  sudo docker images
  REPOSITORY                                   TAG                 IMAGE ID            CREATED             SIZE
  maven                                        3.3.9               44ab79a0bd9e        2 hours ago         4.678 GB
  
  ```
  
  This size for an image is not affordable for me; could anyone please help me on this? Is there any workaround using `gitlab` or `Jenkins` to play `Maven Repository` role?
  
  **Tags:** docker, dockerfile, maven, repository
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2017-08-14. Score: 2
  
  Docker images are after all VM templates, i.e. they have to be more or less self-contained: you get the image, you run the environment with all the dependencies.
  
  To approach the challenge **operationally**, Docker supports reuse of environments through the `FROM` statement i.e. you could maintain a base image and on top a much smaller image with the app itself.
  
  But, from the **strategic point of view**, I would say your architects might want consider splitting the whole thing into smaller microservices - that would allow for 5 smaller images. Of course I cannot judge whether it's really possible.
  
  > **Comments:**
  > **username_2**: Docker images aren't "VM templates" as containerisation isn't the same as VM.  
  > **username_1**: I yet have to search for the right term, but indeed with a Docker container you can simulate a host, therefore I consider containers to be lightweight VMs (in terms of Turing machine but also with a virtual network adapter and mountable virtual storage)  
  > **username_2**: Turing machine has nothing to do with either Docker or VMs. Virtual Machine simulate (nomen omen) machine, while containers (like Docker) only restraint user space. The difference is crucial. And if something Docker wasn't meant as "lightweight VMs", this is role of LXC.  
  > **username_1**: then I need some better and correct wording to explain mgmt why and how Docker could be better than VMWare.. worth of another SE question :-)  
  > **username_2**: you can think about Docker containers as a fat JARs for something other than Java. So you pack all your dependencies and provide single file that can be ran as an application (from end user viewpoint). LXC and VM are rather system-like environments where you run your init system and all the things that are normally ran during startup.  
  > **username_1**: what if I need to have a system-like environment in Docker? i.e. many middleware services  
  > **username_2**: Then use either multiple Docker containers or LXC.  
  > **username_1**: which effectively replaces set of VMs. So I can't understand why I can't tell to business, "let's deploy instead of that MySQL VM just a MySQL container" => clever guys use VMs as service containers as well  
  > **username_2**: You should never run DB in container.  
  > **username_1**: Integration environment neither - just for CI testing?  
  > **username_2**: In that case you shouldn't care. But in CI do you run multiple things in container?  
  > **username_3**: @username_2 docker being more or less lxc with aufs (non persistent file system), are you really speaking about LinuXContainer or something else? I fully agree with the difference between vm, VMware, xen, kvm or else doing hardware emulation against containers doing process isolation but I've a hard time to relate that to lxc)  
  > **username_3**: @PeterMuryshkin à mysql server is not just the mysql service when you want to run it out of toy projects. To run properly in production, you need to give mysql access to low level io for caching, let it swap properly indexes and a bunch of other things suffering a lot from namespace isolation in a container. Added to the complexity of running maintenance jobs (hourly/weekly) which need to work in sync between the mysql server and filesystem. Containers are not lightweight VMs  
  > **username_3**: BTW the point of Uber jar stands and is a proper comparison, I think I get the confusion between Java vm and hardware vm from this point of view.  
  
  ---
  
  ### Answer by username_4 on 2017-09-21. Score: 0
  
  I'm thinking about different approach to your problem. Instead of adding maven repository into the image, I will mount a volume to $HOME/.m2/repository, and another volume to $PROJECT\_DIR
  
  ```
  ENV PROJECT_DIR=/project
  VOLUME /repository
  VOLUME ${PROJECT_DIR}
  RUN mkdir -p $HOME/.m2 /repository && ln -s /repository $HOME/.m2/repository
  
  ```
  
  After that, add a condition check in component.sh whether this is the first run and then build the required component.
  
  ```
  ...
  if [[ ! -e $PROJECT_DIR/app_is_ready ]]; then
    cd $PROJECT_DIR
    mvn clean install -P test
    touch $PROJECT_DIR/app_is_ready
  fi
  <<actual run commands>>
  
  ```
  
  Please be note that the example code is not handling the race condition of two (or more) instances using the same volume for PROJECT\_DIR started at the same time.
  
  Hope this help!
  
  
  ---
  
  