Answers: <p>Eventually, I chose to use docker daemon in docker as a long running pod and Jenkins agents that have docker client inside triggered on demand. They interact with containerized Docker daemon.</p>
  <p>I summarized my choice of <a href="https://www.rokpoto.com/jenkins-docker-in-docker-agent/" rel="nofollow noreferrer">Jenkins Docker in Docker Agent</a>.</p>
  
Answers: CC BY-SA 4.0
Answers: 2022-12-10T20:39:00.223Z
Answers: 2022-12-10T20:39:00.223Z
Answers: rokpoto.com
Answers: <p>There are ways to do it but I've not been able to manage it. I would recommend sidestepping the problem by using <a href="https://github.com/GoogleContainerTools/kaniko" rel="nofollow noreferrer">kaniko</a>. You can add a purpose built kaniko container into your 'build' agent k8s podTemplate and let it handle the container build (and push) part of the workload with no docker daemons or sockets necessary.</p>
  <p>It's working well for me and I am also using jenkins via helm on k8s.
  Here is a snippet of my helm chart <code>values.yaml</code> agent.podTemplates section.</p>
  <p>Note that the 'builder' agent contains 2 containers (actually 3 including the default 'jnlp' container which handles communicating with the jenkins controller) and they share a common workspace:</p>
  <pre><code>agent:
    namespace: jenkins
    podTemplates:
      builder: |
      - name: 'builder'
        label: 'builder'
        serviceAccount: jenkins
        nodeUsageMode: EXCLUSIVE
        yamlMergeStrategy: &quot;merge&quot;
        yaml: |-
          apiVersion: v1
          kind: Pod
          metadata:
            name: builder
            namespace: jenkins
          spec:
            serviceAccountName: jenkins
            containers:
            - name: builder
              image: builder:latest
              imagePullPolicy: IfNotPresent
              command: [&quot;sleep&quot;]
              args: [&quot;99d&quot;]
              tty: true
              resources:
                requests:
                  cpu: 1
                  memory: 1Gi
                limits:
                  cpu: 2
                  memory: 2Gi
              workingDir: /home/jenkins/agent
            - name: kaniko
              image: gcr.io/kaniko-project/executor:debug
              imagePullPolicy: IfNotPresent
              command: [&quot;sleep&quot;]
              args: [&quot;99d&quot;]
              tty: true
              volumeMounts:
              - name: docker-config
                mountPath: /kaniko/.docker
              workingDir: /home/jenkins/agent
              env:
              - name: DOCKER_REPOSITORY
                value: &quot;private_docker_repository&quot;
            volumes:
            - name: docker-config
              configMap:
                name: docker-config
  </code></pre>
  <p>Note that I used the <code>yaml: |-</code> field  because it was the only way I could find to mount a volume containing the configMap needed by kaniko for docker authorization, <a href="https://github.com/jenkinsci/kubernetes-plugin#configuration-reference" rel="nofollow noreferrer">configuration reference</a>.
  So I also kubectl applied a simple docker-config configMap in the jenkins namespace first before applying this helm chart values.yaml.</p>
  <pre><code>apiVersion: v1
  kind: ConfigMap
  metadata:
    name: docker-config
    namespace: jenkins
  data:
    config.json: |-
      {
         &quot;credsStore&quot;: &quot;ecr-login&quot;
      }
  </code></pre>
  <p>Find more details on what that docker-config needs to contain for you <a href="https://github.com/GoogleContainerTools/kaniko#pushing-to-different-registries" rel="nofollow noreferrer">here</a>.</p>
  <p>And then here is an example of using the different containers in a declarative Jenkinsfile script:</p>
  <pre><code>pipeline {
    
    agent {
      label 'builder' // Refers to the custom k8s pod template defined in the jenkins helm chart values.yaml
    }
  
    environment {
      RELEASE_TAG = &quot;1.0.0&quot;
    }
    
    stages {
      stage('Build') {
        steps {
          container('builder') { // refers to the 'builder' container within the 'builder' agent pod
            echo &quot;do a maven build or whatever other steps you need to take here&quot;
          }
          container('kaniko') { // refers to the 'kaniko' container within the 'builder' agent pod sharing the same workspace
            /kaniko/executor --dockerfile `pwd`/dockerfile --context `pwd` --destination=${DOCKER_REPOSITORY}:${RELEASE_TAG}
          }
        }
      }
    }
  }
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2022-03-17T12:51:29.493Z
Answers: 2022-03-17T12:57:17.263Z
Answers: 2022-03-17T12:57:17.263Z
Answers: david_beauchamp
Body: <p>Jenkins is installed using official helm chart on Kubernetes cluster that uses <code>containerd</code> as container runtime.</p>
  <p>How to configure containerized agents which will run docker client commands e.g. for building docker images.</p>
  <p>There's no docker daemon on Kubernetes nodes. Is there an option to use containerd client which will interact with containerd server on the nodes? The idea is similar to case where docker client inside the container interacts with docker daemon running on the node.</p>
  <p>I'm also thinking about below options for Jenkins agents:</p>
  <ul>
  <li><a href="https://hub.docker.com/_/docker/" rel="nofollow noreferrer">docker daemon in docker</a></li>
  <li><a href="https://github.com/nestybox/sysbox" rel="nofollow noreferrer">sysbox containers</a> having docker daemon</li>
  </ul>
  <p>Is there a best practice?</p>
  
Comments: CC BY-SA 4.0
Comments: 2022-04-13T14:11:18.780Z
Comments: Very similar to this existing question https://devops.stackexchange.com/questions/13179/how-to-run-a-container-from-within-another-container-while-also-mounting-a-volum
Comments: Levi
Tags: <docker><jenkins><kubernetes><docker-in-docker><containerd>
Title: Running builds requiring docker daemon in Jenkins installed using helm and running as containerd container
UserDisplayName: rokpoto.com
ThreadText: # Running builds requiring docker daemon in Jenkins installed using helm and running as containerd container
  
  (Asked by: username_0 on 2022-02-08)
  
  Jenkins is installed using official helm chart on Kubernetes cluster that uses `containerd` as container runtime.
  
  How to configure containerized agents which will run docker client commands e.g. for building docker images.
  
  There's no docker daemon on Kubernetes nodes. Is there an option to use containerd client which will interact with containerd server on the nodes? The idea is similar to case where docker client inside the container interacts with docker daemon running on the node.
  
  I'm also thinking about below options for Jenkins agents:
  
  * [docker daemon in docker](https://hub.docker.com/_/docker/)
  * [sysbox containers](https://github.com/nestybox/sysbox) having docker daemon
  
  Is there a best practice?
  
  **Tags:** docker, jenkins, kubernetes, docker-in-docker, containerd
  
  > **Comments:**
  > **username_1**: Very similar to this existing question https://devops.stackexchange.com/questions/13179/how-to-run-a-container-from-within-another-container-while-also-mounting-a-volum  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2022-12-10. Score: 0 | Accepted answer
  
  Eventually, I chose to use docker daemon in docker as a long running pod and Jenkins agents that have docker client inside triggered on demand. They interact with containerized Docker daemon.
  
  I summarized my choice of [Jenkins Docker in Docker Agent](https://www.username_0/jenkins-docker-in-docker-agent/).
  
  
  ---
  
  ### Answer by username_2 on 2022-03-17. Score: 1
  
  There are ways to do it but I've not been able to manage it. I would recommend sidestepping the problem by using [kaniko](https://github.com/GoogleContainerTools/kaniko). You can add a purpose built kaniko container into your 'build' agent k8s podTemplate and let it handle the container build (and push) part of the workload with no docker daemons or sockets necessary.
  
  It's working well for me and I am also using jenkins via helm on k8s.
  Here is a snippet of my helm chart `values.yaml` agent.podTemplates section.
  
  Note that the 'builder' agent contains 2 containers (actually 3 including the default 'jnlp' container which handles communicating with the jenkins controller) and they share a common workspace:
  
  ```
  agent:
    namespace: jenkins
    podTemplates:
      builder: |
      - name: 'builder'
        label: 'builder'
        serviceAccount: jenkins
        nodeUsageMode: EXCLUSIVE
        yamlMergeStrategy: "merge"
        yaml: |-
          apiVersion: v1
          kind: Pod
          metadata:
            name: builder
            namespace: jenkins
          spec:
            serviceAccountName: jenkins
            containers:
            - name: builder
              image: builder:latest
              imagePullPolicy: IfNotPresent
              command: ["sleep"]
              args: ["99d"]
              tty: true
              resources:
                requests:
                  cpu: 1
                  memory: 1Gi
                limits:
                  cpu: 2
                  memory: 2Gi
              workingDir: /home/jenkins/agent
            - name: kaniko
              image: gcr.io/kaniko-project/executor:debug
              imagePullPolicy: IfNotPresent
              command: ["sleep"]
              args: ["99d"]
              tty: true
              volumeMounts:
              - name: docker-config
                mountPath: /kaniko/.docker
              workingDir: /home/jenkins/agent
              env:
              - name: DOCKER_REPOSITORY
                value: "private_docker_repository"
            volumes:
            - name: docker-config
              configMap:
                name: docker-config
  
  ```
  
  Note that I used the `yaml: |-` field because it was the only way I could find to mount a volume containing the configMap needed by kaniko for docker authorization, [configuration reference](https://github.com/jenkinsci/kubernetes-plugin#configuration-reference).
  So I also kubectl applied a simple docker-config configMap in the jenkins namespace first before applying this helm chart values.yaml.
  
  ```
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: docker-config
    namespace: jenkins
  data:
    config.json: |-
      {
         "credsStore": "ecr-login"
      }
  
  ```
  
  Find more details on what that docker-config needs to contain for you [here](https://github.com/GoogleContainerTools/kaniko#pushing-to-different-registries).
  
  And then here is an example of using the different containers in a declarative Jenkinsfile script:
  
  ```
  pipeline {
    
    agent {
      label 'builder' // Refers to the custom k8s pod template defined in the jenkins helm chart values.yaml
    }
  
    environment {
      RELEASE_TAG = "1.0.0"
    }
    
    stages {
      stage('Build') {
        steps {
          container('builder') { // refers to the 'builder' container within the 'builder' agent pod
            echo "do a maven build or whatever other steps you need to take here"
          }
          container('kaniko') { // refers to the 'kaniko' container within the 'builder' agent pod sharing the same workspace
            /kaniko/executor --dockerfile `pwd`/dockerfile --context `pwd` --destination=${DOCKER_REPOSITORY}:${RELEASE_TAG}
          }
        }
      }
    }
  }
  
  ```
  
  
  ---
  
  