Answers: <p>To ensure that the new upload pod mounts the same volume as the previous pod and continues uploading the remaining files, you can use a persistent volume claim (PVC) to claim the volume and mount it to the new pod.</p>
  <p>A PVC is a resource in Kubernetes that allows a pod to request and consume a persistent volume (PV) in the cluster. When you create a PVC, you can specify the size of the volume and the access mode (e.g. read-write). The PVC is then bound to a PV that meets the specified criteria.</p>
  <p>To use a PVC to claim the volume in your scenario, you can do the following:</p>
  <ol>
  <li><p>Create a PV to represent the volume on the on-premise system. The PV should specify the path to the volume on the on-premise system, as well as the access mode (e.g. read-write).</p>
  </li>
  <li><p>Create a PVC that references the PV. The PVC should specify the same size and access mode as the PV.</p>
  </li>
  <li><p>Mount the PVC to the upload pod using a volume mount in the pod's spec.</p>
  </li>
  </ol>
  <p>Here is an example of how you can create a PV, PVC, and pod that mounts the PVC:</p>
  <pre><code>apiVersion: v1
  kind: PersistentVolume
  metadata:
    name: my-pv
  spec:
    capacity:
      storage: 10Gi
    accessModes:
    - ReadWriteMany
    hostPath:
      path: /path/to/volume
  
  ---
  
  apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: my-pvc
  spec:
    accessModes:
    - ReadWriteMany
    resources:
      requests:
        storage: 10Gi
    volumeName: my-pv
  
  ---
  
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: upload
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: upload
    template:
      metadata:
        labels:
          app: upload
      spec:
        containers:
        - name: upload
          image: upload-image
          volumeMounts:
          - name: my-volume
            mountPath: /path/to/volume
        volumes:
        - name: my-volume
          persistentVolumeClaim:
            claimName: my-pvc
  </code></pre>
  <p>This configuration will create a PV and PVC that represent the volume on the on-premise system and claim it for the upload pod. The pod will then mount the volume to the specified path using the PVC.</p>
  <p>If the upload pod fails and is restarted, the PVC will still be bound to the PV and the new pod will be able to mount the same volume and continue uploading the remaining files.</p>
  <p>I hope this helps!</p>
  
Answers: CC BY-SA 4.0
Answers: 2023-01-05T13:07:55.927Z
Answers: May worth explaining that both pod MUST be on the same worker node to share the same host's path mounted. In any cluster this will not work properly if both pods are not scheduled on the same node as each pod will get its own node directory totaly unrelated to the other.
Answers: Tensibai
Answers: CC BY-SA 4.0
Answers: 2023-01-05T08:04:03.860Z
Answers: 2023-01-05T08:04:03.860Z
Answers: Rami M. Nassar
Answers: <p>I'm picturing a use-case like a log-exporter on network with a very flaky external internet connection. I take it you want to ensure that even if there are considerable delays accessing S3, your client applications are not blocked, and you also do not risk data loss from local failures (of cluster nodes or storage drives).</p>
  <p>You will need a <em>Persistent Volume Claim</em>, which both of your pods can then mount.</p>
  <p>Since you are using k3s you may be limited in what underlying types of persistent volume are supported, unless you install additional components.</p>
  
Answers: CC BY-SA 4.0
Answers: 2023-01-04T04:09:39.330Z
Answers: 2023-01-04T04:09:39.330Z
Answers: benjimin
Body: <p>I have an application which consists of 2 containers which are running on an on-premise system. The responsibility of the application is to accept data from other on-premise systems and upload it to cloud. First container exposes a <code>store</code> service which accepts data and writes the data to a volume mounted on the on-premise system. There is another <code>upload</code> service which gets triggered whenever a new file is available in this volume and uploads the data to S3 in the background.</p>
  <p>The problem I am trying to solve is to host these containers in a <code>K3S</code> cluster to provide HA capabilities. I do not want to combine <code>store</code> and <code>upload</code> into a single POD as failure detection and recovery becomes complex. I want to host each container in a separate POD. In such a scenario how do I ensure that data is not  lost whenever one container fails and new POD is started? For example, lets say <code>store</code> service received 100 files and written it to the volume. <code>Upload</code> started uploading but crashed while uploading 50th file. Now K3S will ensure that new <code>upload</code> POD is started, but how do I ensure that it mounts the same volume which had 50 pending files?</p>
  
Tags: <docker><kubernetes><high-availability><k3s>
Title: Persistent Volume with Kubernetes
UserDisplayName: No Name
ThreadText: # Persistent Volume with Kubernetes
  
  (Asked by: username_0 on 2023-01-03)
  
  I have an application which consists of 2 containers which are running on an on-premise system. The responsibility of the application is to accept data from other on-premise systems and upload it to cloud. First container exposes a `store` service which accepts data and writes the data to a volume mounted on the on-premise system. There is another `upload` service which gets triggered whenever a new file is available in this volume and uploads the data to S3 in the background.
  
  The problem I am trying to solve is to host these containers in a `K3S` cluster to provide HA capabilities. I do not want to combine `store` and `upload` into a single POD as failure detection and recovery becomes complex. I want to host each container in a separate POD. In such a scenario how do I ensure that data is not lost whenever one container fails and new POD is started? For example, lets say `store` service received 100 files and written it to the volume. `Upload` started uploading but crashed while uploading 50th file. Now K3S will ensure that new `upload` POD is started, but how do I ensure that it mounts the same volume which had 50 pending files?
  
  **Tags:** docker, kubernetes, high-availability, k3s
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-01-05. Score: 0 | Accepted answer
  
  To ensure that the new upload pod mounts the same volume as the previous pod and continues uploading the remaining files, you can use a persistent volume claim (PVC) to claim the volume and mount it to the new pod.
  
  A PVC is a resource in Kubernetes that allows a pod to request and consume a persistent volume (PV) in the cluster. When you create a PVC, you can specify the size of the volume and the access mode (e.g. read-write). The PVC is then bound to a PV that meets the specified criteria.
  
  To use a PVC to claim the volume in your scenario, you can do the following:
  
  1. Create a PV to represent the volume on the on-premise system. The PV should specify the path to the volume on the on-premise system, as well as the access mode (e.g. read-write).
  2. Create a PVC that references the PV. The PVC should specify the same size and access mode as the PV.
  3. Mount the PVC to the upload pod using a volume mount in the pod's spec.
  
  Here is an example of how you can create a PV, PVC, and pod that mounts the PVC:
  
  ```
  apiVersion: v1
  kind: PersistentVolume
  metadata:
    name: my-pv
  spec:
    capacity:
      storage: 10Gi
    accessModes:
    - ReadWriteMany
    hostPath:
      path: /path/to/volume
  
  ---
  
  apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: my-pvc
  spec:
    accessModes:
    - ReadWriteMany
    resources:
      requests:
        storage: 10Gi
    volumeName: my-pv
  
  ---
  
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: upload
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: upload
    template:
      metadata:
        labels:
          app: upload
      spec:
        containers:
        - name: upload
          image: upload-image
          volumeMounts:
          - name: my-volume
            mountPath: /path/to/volume
        volumes:
        - name: my-volume
          persistentVolumeClaim:
            claimName: my-pvc
  
  ```
  
  This configuration will create a PV and PVC that represent the volume on the on-premise system and claim it for the upload pod. The pod will then mount the volume to the specified path using the PVC.
  
  If the upload pod fails and is restarted, the PVC will still be bound to the PV and the new pod will be able to mount the same volume and continue uploading the remaining files.
  
  I hope this helps!
  
  > **Comments:**
  > **username_2**: May worth explaining that both pod MUST be on the same worker node to share the same host's path mounted. In any cluster this will not work properly if both pods are not scheduled on the same node as each pod will get its own node directory totaly unrelated to the other.  
  
  ---
  
  ### Answer by username_3 on 2023-01-04. Score: 0
  
  I'm picturing a use-case like a log-exporter on network with a very flaky external internet connection. I take it you want to ensure that even if there are considerable delays accessing S3, your client applications are not blocked, and you also do not risk data loss from local failures (of cluster nodes or storage drives).
  
  You will need a *Persistent Volume Claim*, which both of your pods can then mount.
  
  Since you are using k3s you may be limited in what underlying types of persistent volume are supported, unless you install additional components.
  
  
  ---
  
  