Answers: <p>AzureFile didn't support that operation, so I tried AzureDisk and it worked. This is what I used:</p>
  
  <pre><code>apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    name: cockroach-disk
    namespace: cockroach
  provisioner: kubernetes.io/azure-disk
  parameters:
    storageaccounttype: Standard_LRS
    kind: Managed
    #resourceGroup: CockroachStorage
  reclaimPolicy: Retain
  allowVolumeExpansion: true
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2019-12-03T19:14:07.400Z
Answers: 2019-12-03T19:14:07.400Z
Answers: RQDQ
Body: <p>I'm getting an error trying to run CockroachDB (v19.2.1) on AKS. I'm able to spin up cluster using the default storage class. However, when I attempt to specify AzureFile storage, I get errors.</p>
  
  <p>Storage class:</p>
  
  <pre><code>kind: StorageClass
  apiVersion: storage.k8s.io/v1
  metadata:
    name: cockroach
    namespace: cockroach
  provisioner: kubernetes.io/azure-file
  mountOptions:
    - mfsymlinks
    - dir_mode=0777
    - file_mode=0777
    - uid=1000
    - gid=1000
    - nobrl
    - cache=none
  parameters:
    skuName: Standard_LRS
  </code></pre>
  
  <p>Helm command:</p>
  
  <pre><code>helm install cockroach stable/cockroachdb `
      --namespace cockroach `
      --set StorageClass="cockroach" `
      --set Storage="100Gi"
  </code></pre>
  
  <p>The first node starts up, but the second node crashes with the following:</p>
  
  <pre><code>F191203 17:07:35.042824 214 storage/store_raft.go:322  [n3,s3,r10/2:{-}] while applying snapshot: IO error: while link file to /cockroach/cockroach-data/000049.sst: /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/0.sst: Operation not supported
  while ingesting [/cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/0.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/1.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/2.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/3.sst]
  </code></pre>
  
  <p>Full log:</p>
  
  <pre><code>CockroachDB node starting at 2019-12-03 17:07:34.142182444 +0000 UTC (took 6.1s)
  build:               CCL v19.2.1 @ 2019/11/18 23:23:55 (go1.12.12)
  webui:               http://0.0.0.0:8080
  sql:                 postgresql://root@cockroach-cockroachdb-1.cockroach-cockroachdb.cockroach.svc.cluster.local:26257?sslmode=disable
  RPC client flags:    /cockroach/cockroach &lt;client cmd&gt; --host=cockroach-cockroachdb-1.cockroach-cockroachdb.cockroach.svc.cluster.local:26257 --insecure
  logs:                /cockroach/cockroach-data/logs
  temp dir:            /cockroach/cockroach-data/cockroach-temp261131495
  external I/O path:   /cockroach/cockroach-data/extern
  store[0]:            path=/cockroach/cockroach-data
  status:              restarted pre-existing node
  clusterID:           dd364f3f-7900-45f2-9a65-7a105cd7a147
  nodeID:              3
  I191203 17:07:34.153647 232 server/server_update.go:53  [n3] no need to upgrade, cluster already at the newest version
  I191203 17:07:34.191164 234 sql/event_log.go:130  [n3] Event: "node_restart", target: 3, info: {Descriptor:{NodeID:3 Address:cockroach-cockroachdb-1.cockroach-cockroachdb.cockroach.svc.cluster.local:26257 Attrs: Locality: ServerVersion:19.2 BuildTag:v19.2.1 StartedAt:1575392852575521409 LocalityAddress:[] ClusterName: SQLAddress:cockroach-cockroachdb-1.cockroach-cockroachdb.cockroach.svc.cluster.local:26257} ClusterID:dd364f3f-7900-45f2-9a65-7a105cd7a147 StartedAt:1575392852575521409 LastUp:1575392800598285198}
  I191203 17:07:34.360523 214 storage/replica_raftstorage.go:794  [n3,s3,r10/2:{-}] applying LEARNER snapshot [id=ea03bb31 index=18]
  I191203 17:07:34.487170 214 storage/engine/rocksdb.go:119  [db/version_set.cc:3086] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed.
  I191203 17:07:34.508684 84 gossip/gossip.go:1531  [n3] node has connected to cluster via gossip
  I191203 17:07:34.508951 84 storage/stores.go:259  [n3] wrote 2 node addresses to persistent storage
  I191203 17:07:34.711691 214 storage/engine/rocksdb.go:119  [db/external_sst_file_ingestion_job.cc:262] AddFile() clean up for file  failed : IO error: No such file or directorywhile unlink() file: No such file or directory
  I191203 17:07:34.711725 214 storage/engine/rocksdb.go:119  [db/external_sst_file_ingestion_job.cc:262] AddFile() clean up for file  failed : IO error: No such file or directorywhile unlink() file: No such file or directory
  I191203 17:07:34.711873 214 storage/engine/rocksdb.go:119  [db/external_sst_file_ingestion_job.cc:262] AddFile() clean up for file  failed : IO error: No such file or directorywhile unlink() file: No such file or directory
  I191203 17:07:34.711893 214 storage/engine/rocksdb.go:119  [db/external_sst_file_ingestion_job.cc:262] AddFile() clean up for file  failed : IO error: No such file or directorywhile unlink() file: No such file or directory
  I191203 17:07:34.711990 214 storage/replica_raftstorage.go:815  [n3,s3,r10/2:{-}] applied LEARNER snapshot [total=351ms ingestion=4@-9223372036855ms id=ea03bb31 index=18]
  E191203 17:07:35.042651 214 util/log/crash_reporting.go:537  [n3,s3,r10/2:{-}] Reported as error 2cfee4bb679a4beb8a489e30d87ab8ea
  F191203 17:07:35.042824 214 storage/store_raft.go:322  [n3,s3,r10/2:{-}] while applying snapshot: IO error: while link file to /cockroach/cockroach-data/000049.sst: /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/0.sst: Operation not supported
  while ingesting [/cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/0.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/1.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/2.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/3.sst]
  github.com/cockroachdb/cockroach/pkg/storage.(*Replica).applySnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raftstorage.go:908
  github.com/cockroachdb/cockroach/pkg/storage.(*Replica).handleRaftReadyRaftMuLocked
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:526
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:321
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).withReplicaForRequest
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:200
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:262
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).receiveSnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_snapshot.go:854
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:59
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:300
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:49
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1.1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:415
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:416
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:321
  runtime.goexit
          /usr/local/go/src/runtime/asm_amd64.s:1337
  while applying snapshot
  github.com/cockroachdb/cockroach/pkg/storage.(*Replica).handleRaftReadyRaftMuLocked
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:528
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:321
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).withReplicaForRequest
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:200
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:262
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).receiveSnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_snapshot.go:854
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:59
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:300
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:49
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1.1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:415
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:416
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:321
  runtime.goexit
          /usr/local/go/src/runtime/asm_amd64.s:1337
  goroutine 214 [running]:
  github.com/cockroachdb/cockroach/pkg/util/log.getStacks(0xc000438300, 0xc000438360, 0x0, 0xc0031ba280)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1017 +0xb1
  github.com/cockroachdb/cockroach/pkg/util/log.(*loggingT).outputLogEntry(0x7508900, 0xc000000004, 0x6cd15ce, 0x15, 0x142, 0xc00051ac00, 0x1087)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:871 +0x95b
  github.com/cockroachdb/cockroach/pkg/util/log.addStructured(0x4a54f40, 0xc003763560, 0xc000000004, 0x3, 0x40d626f, 0x7, 0xc0035b5170, 0x2, 0x2)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/structured.go:66 +0x2cc
  github.com/cockroachdb/cockroach/pkg/util/log.logDepth(0x4a54f40, 0xc003763560, 0x2, 0xc000000004, 0x40d626f, 0x7, 0xc0035b5170, 0x2, 0x2)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:69 +0x8c
  github.com/cockroachdb/cockroach/pkg/util/log.FatalfDepth(...)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:199
  github.com/cockroachdb/cockroach/pkg/storage.maybeFatalOnRaftReadyErr(0x4a54f40, 0xc003763560, 0x4104925, 0x17, 0x49e0600, 0xc003a98600, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:859 +0x1de
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest.func1(0x4a54f40, 0xc003763560, 0xc003694000, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:322 +0x25f
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).withReplicaForRequest(0xc000c48300, 0x4a54f40, 0xc003763560, 0xc00077dbc8, 0xc0035b5418, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:200 +0x16b
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest(0xc000c48300, 0x4a54f40, 0xc0035d5bf0, 0xc00077db80, 0xe5401ee131bb03ea, 0x3f9d1493005c399, 0xc000545340, 0x0, 0x0, 0x0, ...)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:262 +0x1a4
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).receiveSnapshot(0xc000c48300, 0x4a54f40, 0xc0035d5bf0, 0xc00077db80, 0x7f30769bb2c8, 0xc000369540, 0x0, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_snapshot.go:854 +0x6d3
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot.func1(0x4a54f40, 0xc0035d5bf0, 0xc00054d320, 0x411c3aa)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:59 +0x1d9
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc00054d320, 0x4a54f40, 0xc0035d5bf0, 0x411c3aa, 0x1e, 0xc00331bc30, 0x0, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:300 +0xeb
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot(0xc000c48300, 0xc00077db80, 0x7f30769bb298, 0xc000369540, 0xc000369540, 0x30)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:49 +0xe6
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1.1(0x4aa3720, 0xc000369540, 0xc000ca40f0, 0x4a54f40, 0xc0035d5b30, 0x78c3df, 0xc000544ff0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:415 +0x138
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1(0x4a54f40, 0xc0035d5b30)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:416 +0x5d
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00054d320, 0x4a54f40, 0xc0035d5b30, 0xc000075780, 0x32, 0x0, 0x0, 0xc0035d5b60)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:321 +0xe6
  created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:316 +0x131
  
  </code></pre>
  
  <p>Do I need to specify different options in my storage class?</p>
  
Tags: <kubernetes><azure-kubernetes-service>
Title: Error while using AzureFile storage with CockroachDB
UserDisplayName: RQDQ
ThreadText: # Error while using AzureFile storage with CockroachDB
  
  (Asked by: username_0 on 2019-12-03)
  
  I'm getting an error trying to run CockroachDB (v19.2.1) on AKS. I'm able to spin up cluster using the default storage class. However, when I attempt to specify AzureFile storage, I get errors.
  
  Storage class:
  
  ```
  kind: StorageClass
  apiVersion: storage.k8s.io/v1
  metadata:
    name: cockroach
    namespace: cockroach
  provisioner: kubernetes.io/azure-file
  mountOptions:
    - mfsymlinks
    - dir_mode=0777
    - file_mode=0777
    - uid=1000
    - gid=1000
    - nobrl
    - cache=none
  parameters:
    skuName: Standard_LRS
  
  ```
  
  Helm command:
  
  ```
  helm install cockroach stable/cockroachdb `
      --namespace cockroach `
      --set StorageClass="cockroach" `
      --set Storage="100Gi"
  
  ```
  
  The first node starts up, but the second node crashes with the following:
  
  ```
  F191203 17:07:35.042824 214 storage/store_raft.go:322  [n3,s3,r10/2:{-}] while applying snapshot: IO error: while link file to /cockroach/cockroach-data/000049.sst: /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/0.sst: Operation not supported
  while ingesting [/cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/0.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/1.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/2.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/3.sst]
  
  ```
  
  Full log:
  
  ```
  CockroachDB node starting at 2019-12-03 17:07:34.142182444 +0000 UTC (took 6.1s)
  build:               CCL v19.2.1 @ 2019/11/18 23:23:55 (go1.12.12)
  webui:               http://0.0.0.0:8080
  sql:                 postgresql://root@cockroach-cockroachdb-1.cockroach-cockroachdb.cockroach.svc.cluster.local:26257?sslmode=disable
  RPC client flags:    /cockroach/cockroach <client cmd> --host=cockroach-cockroachdb-1.cockroach-cockroachdb.cockroach.svc.cluster.local:26257 --insecure
  logs:                /cockroach/cockroach-data/logs
  temp dir:            /cockroach/cockroach-data/cockroach-temp261131495
  external I/O path:   /cockroach/cockroach-data/extern
  store[0]:            path=/cockroach/cockroach-data
  status:              restarted pre-existing node
  clusterID:           dd364f3f-7900-45f2-9a65-7a105cd7a147
  nodeID:              3
  I191203 17:07:34.153647 232 server/server_update.go:53  [n3] no need to upgrade, cluster already at the newest version
  I191203 17:07:34.191164 234 sql/event_log.go:130  [n3] Event: "node_restart", target: 3, info: {Descriptor:{NodeID:3 Address:cockroach-cockroachdb-1.cockroach-cockroachdb.cockroach.svc.cluster.local:26257 Attrs: Locality: ServerVersion:19.2 BuildTag:v19.2.1 StartedAt:1575392852575521409 LocalityAddress:[] ClusterName: SQLAddress:cockroach-cockroachdb-1.cockroach-cockroachdb.cockroach.svc.cluster.local:26257} ClusterID:dd364f3f-7900-45f2-9a65-7a105cd7a147 StartedAt:1575392852575521409 LastUp:1575392800598285198}
  I191203 17:07:34.360523 214 storage/replica_raftstorage.go:794  [n3,s3,r10/2:{-}] applying LEARNER snapshot [id=ea03bb31 index=18]
  I191203 17:07:34.487170 214 storage/engine/rocksdb.go:119  [db/version_set.cc:3086] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed.
  I191203 17:07:34.508684 84 gossip/gossip.go:1531  [n3] node has connected to cluster via gossip
  I191203 17:07:34.508951 84 storage/stores.go:259  [n3] wrote 2 node addresses to persistent storage
  I191203 17:07:34.711691 214 storage/engine/rocksdb.go:119  [db/external_sst_file_ingestion_job.cc:262] AddFile() clean up for file  failed : IO error: No such file or directorywhile unlink() file: No such file or directory
  I191203 17:07:34.711725 214 storage/engine/rocksdb.go:119  [db/external_sst_file_ingestion_job.cc:262] AddFile() clean up for file  failed : IO error: No such file or directorywhile unlink() file: No such file or directory
  I191203 17:07:34.711873 214 storage/engine/rocksdb.go:119  [db/external_sst_file_ingestion_job.cc:262] AddFile() clean up for file  failed : IO error: No such file or directorywhile unlink() file: No such file or directory
  I191203 17:07:34.711893 214 storage/engine/rocksdb.go:119  [db/external_sst_file_ingestion_job.cc:262] AddFile() clean up for file  failed : IO error: No such file or directorywhile unlink() file: No such file or directory
  I191203 17:07:34.711990 214 storage/replica_raftstorage.go:815  [n3,s3,r10/2:{-}] applied LEARNER snapshot [total=351ms ingestion=4@-9223372036855ms id=ea03bb31 index=18]
  E191203 17:07:35.042651 214 util/log/crash_reporting.go:537  [n3,s3,r10/2:{-}] Reported as error 2cfee4bb679a4beb8a489e30d87ab8ea
  F191203 17:07:35.042824 214 storage/store_raft.go:322  [n3,s3,r10/2:{-}] while applying snapshot: IO error: while link file to /cockroach/cockroach-data/000049.sst: /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/0.sst: Operation not supported
  while ingesting [/cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/0.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/1.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/2.sst /cockroach/cockroach-data/auxiliary/sstsnapshot/10/ea03bb31-e11e-40e5-99c3-053049d1f903/3.sst]
  github.com/cockroachdb/cockroach/pkg/storage.(*Replica).applySnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raftstorage.go:908
  github.com/cockroachdb/cockroach/pkg/storage.(*Replica).handleRaftReadyRaftMuLocked
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:526
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:321
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).withReplicaForRequest
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:200
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:262
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).receiveSnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_snapshot.go:854
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:59
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:300
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:49
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1.1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:415
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:416
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:321
  runtime.goexit
          /usr/local/go/src/runtime/asm_amd64.s:1337
  while applying snapshot
  github.com/cockroachdb/cockroach/pkg/storage.(*Replica).handleRaftReadyRaftMuLocked
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:528
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:321
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).withReplicaForRequest
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:200
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:262
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).receiveSnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_snapshot.go:854
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:59
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:300
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:49
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1.1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:415
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:416
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:321
  runtime.goexit
          /usr/local/go/src/runtime/asm_amd64.s:1337
  goroutine 214 [running]:
  github.com/cockroachdb/cockroach/pkg/util/log.getStacks(0xc000438300, 0xc000438360, 0x0, 0xc0031ba280)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1017 +0xb1
  github.com/cockroachdb/cockroach/pkg/util/log.(*loggingT).outputLogEntry(0x7508900, 0xc000000004, 0x6cd15ce, 0x15, 0x142, 0xc00051ac00, 0x1087)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:871 +0x95b
  github.com/cockroachdb/cockroach/pkg/util/log.addStructured(0x4a54f40, 0xc003763560, 0xc000000004, 0x3, 0x40d626f, 0x7, 0xc0035b5170, 0x2, 0x2)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/structured.go:66 +0x2cc
  github.com/cockroachdb/cockroach/pkg/util/log.logDepth(0x4a54f40, 0xc003763560, 0x2, 0xc000000004, 0x40d626f, 0x7, 0xc0035b5170, 0x2, 0x2)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:69 +0x8c
  github.com/cockroachdb/cockroach/pkg/util/log.FatalfDepth(...)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:199
  github.com/cockroachdb/cockroach/pkg/storage.maybeFatalOnRaftReadyErr(0x4a54f40, 0xc003763560, 0x4104925, 0x17, 0x49e0600, 0xc003a98600, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_raft.go:859 +0x1de
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest.func1(0x4a54f40, 0xc003763560, 0xc003694000, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:322 +0x25f
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).withReplicaForRequest(0xc000c48300, 0x4a54f40, 0xc003763560, 0xc00077dbc8, 0xc0035b5418, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:200 +0x16b
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).processRaftSnapshotRequest(0xc000c48300, 0x4a54f40, 0xc0035d5bf0, 0xc00077db80, 0xe5401ee131bb03ea, 0x3f9d1493005c399, 0xc000545340, 0x0, 0x0, 0x0, ...)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:262 +0x1a4
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).receiveSnapshot(0xc000c48300, 0x4a54f40, 0xc0035d5bf0, 0xc00077db80, 0x7f30769bb2c8, 0xc000369540, 0x0, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_snapshot.go:854 +0x6d3
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot.func1(0x4a54f40, 0xc0035d5bf0, 0xc00054d320, 0x411c3aa)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:59 +0x1d9
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc00054d320, 0x4a54f40, 0xc0035d5bf0, 0x411c3aa, 0x1e, 0xc00331bc30, 0x0, 0x0)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:300 +0xeb
  github.com/cockroachdb/cockroach/pkg/storage.(*Store).HandleSnapshot(0xc000c48300, 0xc00077db80, 0x7f30769bb298, 0xc000369540, 0xc000369540, 0x30)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/store_raft.go:49 +0xe6
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1.1(0x4aa3720, 0xc000369540, 0xc000ca40f0, 0x4a54f40, 0xc0035d5b30, 0x78c3df, 0xc000544ff0)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:415 +0x138
  github.com/cockroachdb/cockroach/pkg/storage.(*RaftTransport).RaftSnapshot.func1(0x4a54f40, 0xc0035d5b30)
          /go/src/github.com/cockroachdb/cockroach/pkg/storage/raft_transport.go:416 +0x5d
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00054d320, 0x4a54f40, 0xc0035d5b30, 0xc000075780, 0x32, 0x0, 0x0, 0xc0035d5b60)
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:321 +0xe6
  created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
          /go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:316 +0x131
  
  ```
  
  Do I need to specify different options in my storage class?
  
  **Tags:** kubernetes, azure-kubernetes-service
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2019-12-03. Score: 1
  
  AzureFile didn't support that operation, so I tried AzureDisk and it worked. This is what I used:
  
  ```
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    name: cockroach-disk
    namespace: cockroach
  provisioner: kubernetes.io/azure-disk
  parameters:
    storageaccounttype: Standard_LRS
    kind: Managed
    #resourceGroup: CockroachStorage
  reclaimPolicy: Retain
  allowVolumeExpansion: true
  
  ```
  
  
  ---
  
  