title: How to create Ceph user & capabilities for Kubernetes CSI rbdplugin
body: <p>When I follow the guide for Kubernetes block devices on Ceph RBD at <a href="https://docs.ceph.com/docs/master/rbd/rbd-kubernetes/" rel="nofollow noreferrer">https://docs.ceph.com/docs/master/rbd/rbd-kubernetes/</a> I get the following error at the user creation.</p>
  
  <p>~# ceph auth get-or-create client.kubernetes mon 'profile rbd' osd 'profile rbd pool=kubernetes' mgr 'profile rbd pool=kubernetes'
  Error EINVAL: mon capability parse failed, stopped at 'pool=kubernetes' of 'profile rbd pool=kubernetes'</p>
  
  <p>I can create the user if I remove the mgr capabilities. Does Kubernetes Ceph client needs mgr capabilities ? </p>
  
  <p>If yes, how should I configured that user ?</p>
  
  <p>With the above capabilities definition, I have the below error from the provisioner:</p>
  
  <pre><code>$ kubectl logs csi-rbdplugin-provisioner-6956bdfdf9-knpvt csi-provisioner
  
  I0403 21:37:12.449780       1 leaderelection.go:282] successfully renewed lease default/rbd-csi-ceph-com
  I0403 21:37:15.750596       1 controller.go:1199] provision "default/rbd-pvc" class "csi-rbd-sc": started
  I0403 21:37:15.753508       1 controller.go:494] CreateVolumeRequest {Name:pvc-857dbdb6-fc82-40b0-b78c-d3ca675741b0 CapacityRange:required_bytes:1073741824  VolumeCapabilities:[mount:&lt;fs_type:"ext4" mount_flags:"discard" &gt; access_mode:&lt;mode:SINGLE_NODE_WRITER &gt; ] Parameters:map[adminId:admin clusterID:3ce42410-da82-4467-bc36-258e1f2217b1 csi.storage.k8s.io/node-stage-secret-name:csi-rbd-secret csi.storage.k8s.io/node-stage-secret-namespace:default csi.storage.k8s.io/provisioner-secret-name:csi-rbd-secret csi.storage.k8s.io/provisioner-secret-namespace:default monitors:10.250.20.21:6789,10.250.20.22:6789,10.250.20.23:6789 pool:kubernetes userId:kubernetes] Secrets:map[] VolumeContentSource:&lt;nil&gt; AccessibilityRequirements:&lt;nil&gt; XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
  I0403 21:37:15.753733       1 event.go:255] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"rbd-pvc", UID:"857dbdb6-fc82-40b0-b78c-d3ca675741b0", APIVersion:"v1", ResourceVersion:"6475509", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/rbd-pvc"
  I0403 21:37:15.758339       1 connection.go:180] GRPC call: /csi.v1.Controller/CreateVolume
  I0403 21:37:15.758356       1 connection.go:181] GRPC request: {"capacity_range":{"required_bytes":1073741824},"name":"pvc-857dbdb6-fc82-40b0-b78c-d3ca675741b0","parameters":{"adminId":"admin","clusterID":"3ce42410-da82-4467-bc36-258e1f2217b1","monitors":"10.250.20.21:6789,10.250.20.22:6789,10.250.20.23:6789","pool":"kubernetes","userId":"kubernetes"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4","mount_flags":["discard"]}},"access_mode":{"mode":1}}]}
  I0403 21:37:16.736899       1 connection.go:183] GRPC response: {}
  I0403 21:37:16.737220       1 connection.go:184] GRPC error: rpc error: code = Internal desc = failed to get IOContext: failed to get connection: connecting failed: rados: ret=1, Operation not permitted
  I0403 21:37:16.737260       1 controller.go:1016] Final error received, removing PVC 857dbdb6-fc82-40b0-b78c-d3ca675741b0 from claims in progress
  W0403 21:37:16.737273       1 controller.go:887] Retrying syncing claim "857dbdb6-fc82-40b0-b78c-d3ca675741b0", failure 60
  E0403 21:37:16.737289       1 controller.go:910] error syncing claim "857dbdb6-fc82-40b0-b78c-d3ca675741b0": failed to provision volume with StorageClass "csi-rbd-sc": rpc error: code = Internal desc = failed to get IOContext: failed to get connection: connecting failed: rados: ret=1, Operation not permitted
  I0403 21:37:16.737338       1 event.go:255] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"rbd-pvc", UID:"857dbdb6-fc82-40b0-b78c-d3ca675741b0", APIVersion:"v1", ResourceVersion:"6475509", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "csi-rbd-sc": rpc error: code = Internal desc = failed to get IOContext: failed to get connection: connecting failed: rados: ret=1, Operation not permitted
  I0403 21:37:17.486310       1 leaderelection.go:282] successfully renewed lease default/rbd-csi-ceph-com
  </code></pre>
  
  <p>Thanks !</p>