<p dir="auto"><strong>Is this a request for help?</strong> (If yes, you should use our troubleshooting guide and community support channels, see <a href="http://kubernetes.io/docs/troubleshooting/" rel="nofollow">http://kubernetes.io/docs/troubleshooting/</a>.): No</p>
<p dir="auto"><strong>What keywords did you search in Kubernetes issues before filing this one?</strong> (If you have found any duplicates, you should instead reply there.): deployment</p>
<p dir="auto"><strong>Is this a BUG REPORT or FEATURE REQUEST?</strong> (choose one): BUG REPORT</p>
<p dir="auto"><strong>Kubernetes version</strong> (use <code class="notranslate">kubectl version</code>):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Client Version: version.Info{Major:&quot;1&quot;, Minor:&quot;4&quot;, GitVersion:&quot;v1.4.1&quot;, GitCommit:&quot;33cf7b9acbb2cb7c9c72a10d6636321fb180b159&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-10-10T18:19:49Z&quot;, GoVersion:&quot;go1.6.3&quot;, Compiler:&quot;gc&quot;, Platform:&quot;windows/amd64&quot;}
Server Version: version.Info{Major:&quot;1&quot;, Minor:&quot;4&quot;, GitVersion:&quot;v1.4.5&quot;, GitCommit:&quot;5a0a696437ad35c133c0c8493f7e9d22b0f9b81b&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-10-29T01:32:42Z&quot;, GoVersion:&quot;go1.6.3&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}"><pre class="notranslate"><code class="notranslate">Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.1", GitCommit:"33cf7b9acbb2cb7c9c72a10d6636321fb180b159", GitTreeState:"clean", BuildDate:"2016-10-10T18:19:49Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.5", GitCommit:"5a0a696437ad35c133c0c8493f7e9d22b0f9b81b", GitTreeState:"clean", BuildDate:"2016-10-29T01:32:42Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
</code></pre></div>
<p dir="auto"><strong>Environment</strong>:</p>
<ul dir="auto">
<li><strong>Cloud provider or hardware configuration</strong>: Google Cloud</li>
<li><strong>OS</strong> (e.g. from /etc/os-release): GCE Container VM</li>
<li><strong>Kernel</strong> (e.g. <code class="notranslate">uname -a</code>): N/A</li>
<li><strong>Install tools</strong>: N/A</li>
<li><strong>Others</strong>: N/A</li>
</ul>
<p dir="auto"><strong>What happened</strong>:<br>
We use <code class="notranslate">kubectl replace -f deployment.json</code> to update deployment resources in our cluster.  We do this because we generate <code class="notranslate">deployment.json</code>, and pass it to either <code class="notranslate">create</code> or <code class="notranslate">replace</code> depending on whether or not the resource already exists</p>
<p dir="auto"><strong>What you expected to happen</strong>:<br>
We expected that <code class="notranslate">replace</code> will trigger the rollout logic, however <code class="notranslate">kubectl rollout status deployment ...</code> always seems to return immediately available, even when <code class="notranslate">kubectl get pods</code> shows pods creating / being terminated (in violation of the rolling update strategy as well).</p>
<p dir="auto"><strong>How to reproduce it</strong> (as minimally and precisely as possible):<br>
<code class="notranslate">kubectl replace -f deployment.json</code> where <code class="notranslate">deployment.json</code> describes a deployment resource.  You can try changing the image version in a container in <code class="notranslate">deployment.json</code> and running the <code class="notranslate">replace</code> command.</p>
<p dir="auto"><strong>Anything else do we need to know</strong>:<br>
N/A</p>
<p dir="auto"><strong>Is this a request for help?</strong> (If yes, you should use our troubleshooting guide and community support channels, see <a href="http://kubernetes.io/docs/troubleshooting/" rel="nofollow">http://kubernetes.io/docs/troubleshooting/</a>.): No, filing after reproducing the issue in discussion on sig-auth with <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ericchiang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ericchiang">@ericchiang</a> and <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/whitlockjc/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/whitlockjc">@whitlockjc</a> and figured out a workaround</p>
<p dir="auto"><strong>What keywords did you search in Kubernetes issues before filing this one?</strong> (If you have found any duplicates, you should instead reply there.): kubernetes rbac policies</p>
<hr>
<p dir="auto"><strong>Is this a BUG REPORT or FEATURE REQUEST?</strong> (choose one): BUG</p>

<p dir="auto"><strong>Kubernetes version</strong> (use <code class="notranslate">kubectl version</code>):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Client Version: version.Info{Major:&quot;1&quot;, Minor:&quot;3&quot;, GitVersion:&quot;v1.3.0&quot;, GitCommit:&quot;283137936a498aed572ee22af6774b6fb6e9fd94&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-07-01T19:26:38Z&quot;, GoVersion:&quot;go1.6.2&quot;, Compiler:&quot;gc&quot;, Platform:&quot;darwin/amd64&quot;}
Server Version: version.Info{Major:&quot;1&quot;, Minor:&quot;3&quot;, GitVersion:&quot;v1.3.4+coreos.0&quot;, GitCommit:&quot;be9bf3e842a90537e48361aded2872e389e902e7&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-08-02T00:54:53Z&quot;, GoVersion:&quot;go1.6.2&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}"><pre class="notranslate"><code class="notranslate">Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.0", GitCommit:"283137936a498aed572ee22af6774b6fb6e9fd94", GitTreeState:"clean", BuildDate:"2016-07-01T19:26:38Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.4+coreos.0", GitCommit:"be9bf3e842a90537e48361aded2872e389e902e7", GitTreeState:"clean", BuildDate:"2016-08-02T00:54:53Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
</code></pre></div>
<p dir="auto"><strong>Environment</strong>:</p>
<ul dir="auto">
<li><strong>Cloud provider or hardware configuration</strong>: CoreOS Single node - <a href="https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant-single.html" rel="nofollow">https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant-single.html</a></li>
<li><strong>OS</strong> (e.g. from /etc/os-release):</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="localhost manifests # cat /etc/os-release 
NAME=CoreOS
ID=coreos
VERSION=1122.0.0
VERSION_ID=1122.0.0
BUILD_ID=2016-07-27-0739
PRETTY_NAME=&quot;CoreOS 1122.0.0 (MoreOS)&quot;
ANSI_COLOR=&quot;1;32&quot;
HOME_URL=&quot;https://coreos.com/&quot;
BUG_REPORT_URL=&quot;https://github.com/coreos/bugs/issues&quot;"><pre class="notranslate"><code class="notranslate">localhost manifests # cat /etc/os-release 
NAME=CoreOS
ID=coreos
VERSION=1122.0.0
VERSION_ID=1122.0.0
BUILD_ID=2016-07-27-0739
PRETTY_NAME="CoreOS 1122.0.0 (MoreOS)"
ANSI_COLOR="1;32"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"
</code></pre></div>
<ul dir="auto">
<li><strong>Kernel</strong> (e.g. <code class="notranslate">uname -a</code>):</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="localhost manifests # uname -a
Linux localhost 4.7.0-coreos #1 SMP Wed Jul 27 07:30:04 UTC 2016 x86_64 Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz GenuineIntel GNU/Linux"><pre class="notranslate"><code class="notranslate">localhost manifests # uname -a
Linux localhost 4.7.0-coreos #1 SMP Wed Jul 27 07:30:04 UTC 2016 x86_64 Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz GenuineIntel GNU/Linux
</code></pre></div>
<ul dir="auto">
<li><strong>Install tools</strong>:</li>
<li><strong>Others</strong>:</li>
</ul>
<p dir="auto"><strong>What happened</strong>:</p>
<p dir="auto">After configuring both OIDC and RBAC on the api server and getting a JWT for the user named as the super user k8s refuses to authorize any api calls.  In order to work around the issue, I had to configure an X509 client as the super user, at which point I was able to bootstrap the policies and my oidc user and user's groups were properly recognized by k8s.</p>
<p dir="auto"><strong>What you expected to happen</strong>:</p>
<p dir="auto">Since the user is a super-user, all commands should execute without checking the rbac authorization system.  The x509 login should not have been necessary.</p>
<p dir="auto"><strong>How to reproduce it</strong> (as minimally and precisely as possible):</p>
<ul dir="auto">
<li>Deploy k8s</li>
<li>Add the openid connect flags to the api server, here was my config with KeyCloak:</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="   - --oidc-issuer-url=https://kcdev.tremolosecurity.com:8443/auth/realms/kubernetes
    - --oidc-client-id=kubernetes
    - --oidc-username-claim=preferred_username
    - --oidc-groups-claim=user_role
    - --oidc-ca-file=/etc/kubernetes/ssl/kc-ca.pem"><pre class="notranslate"><code class="notranslate">   - --oidc-issuer-url=https://kcdev.tremolosecurity.com:8443/auth/realms/kubernetes
    - --oidc-client-id=kubernetes
    - --oidc-username-claim=preferred_username
    - --oidc-groups-claim=user_role
    - --oidc-ca-file=/etc/kubernetes/ssl/kc-ca.pem
</code></pre></div>
<ul dir="auto">
<li>Verify login with OAuth2 bearer token succeeds</li>
<li>Add RBAC Authorization parameters:</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="- --runtime-config=extensions/v1beta1/networkpolicies=true,rbac.authorization.k8s.io/v1alpha1
    - --oidc-issuer-url=https://kcdev.tremolosecurity.com:8443/auth/realms/kubernetes
    - --oidc-client-id=kubernetes
    - --oidc-username-claim=preferred_username
    - --oidc-groups-claim=user_role
    - --oidc-ca-file=/etc/kubernetes/ssl/kc-ca.pem
    - --authorization-rbac-super-user=mmosley.ent2k12.domain.com
    - --authorization-mode=RBAC"><pre class="notranslate"><code class="notranslate">- --runtime-config=extensions/v1beta1/networkpolicies=true,rbac.authorization.k8s.io/v1alpha1
    - --oidc-issuer-url=https://kcdev.tremolosecurity.com:8443/auth/realms/kubernetes
    - --oidc-client-id=kubernetes
    - --oidc-username-claim=preferred_username
    - --oidc-groups-claim=user_role
    - --oidc-ca-file=/etc/kubernetes/ssl/kc-ca.pem
    - --authorization-rbac-super-user=mmosley.ent2k12.domain.com
    - --authorization-mode=RBAC
</code></pre></div>
<p dir="auto">and here is an example JWT:</p>
<div class="highlight highlight-source-json notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="{
  &quot;jti&quot;: &quot;34d2d176-1f33-4522-ac75-c1f1e2ba291d&quot;,
  &quot;exp&quot;: 1472061895,
  &quot;nbf&quot;: 0,
  &quot;iat&quot;: 1472061595,
  &quot;iss&quot;: &quot;https://kcdev.tremolosecurity.com:8443/auth/realms/kubernetes&quot;,
  &quot;aud&quot;: &quot;kubernetes&quot;,
  &quot;sub&quot;: &quot;cc28e060-4c63-4380-94a7-63e2aad1f372&quot;,
  &quot;typ&quot;: &quot;Bearer&quot;,
  &quot;azp&quot;: &quot;kubernetes&quot;,
  &quot;auth_time&quot;: 1472061358,
  &quot;session_state&quot;: &quot;8b3773bc-23ca-4f75-8c35-d2bf197d4057&quot;,
  &quot;acr&quot;: &quot;0&quot;,
  &quot;client_session&quot;: &quot;85550ebd-fe8c-4212-bf37-3579b3532f76&quot;,
  &quot;allowed-origins&quot;: [],
  &quot;realm_access&quot;: {
    &quot;roles&quot;: [
      &quot;uma_authorization&quot;
    ]
  },
  &quot;resource_access&quot;: {
    &quot;account&quot;: {
      &quot;roles&quot;: [
        &quot;manage-account&quot;,
        &quot;view-profile&quot;
      ]
    }
  },
  &quot;user_role&quot;: [
    &quot;admin&quot;
  ],
  &quot;name&quot;: &quot;mmosley@ent2k12.domain.com Mosley&quot;,
  &quot;preferred_username&quot;: &quot;mmosley.ent2k12.domain.com&quot;,
  &quot;given_name&quot;: &quot;mmosley@ent2k12.domain.com&quot;,
  &quot;family_name&quot;: &quot;Mosley&quot;,
  &quot;email&quot;: &quot;matt.mosley@tremolosecurity-test.com&quot;
}"><pre class="notranslate">{
  <span class="pl-ent">"jti"</span>: <span class="pl-s"><span class="pl-pds">"</span>34d2d176-1f33-4522-ac75-c1f1e2ba291d<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"exp"</span>: <span class="pl-c1">1472061895</span>,
  <span class="pl-ent">"nbf"</span>: <span class="pl-c1">0</span>,
  <span class="pl-ent">"iat"</span>: <span class="pl-c1">1472061595</span>,
  <span class="pl-ent">"iss"</span>: <span class="pl-s"><span class="pl-pds">"</span>https://kcdev.tremolosecurity.com:8443/auth/realms/kubernetes<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"aud"</span>: <span class="pl-s"><span class="pl-pds">"</span>kubernetes<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"sub"</span>: <span class="pl-s"><span class="pl-pds">"</span>cc28e060-4c63-4380-94a7-63e2aad1f372<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"typ"</span>: <span class="pl-s"><span class="pl-pds">"</span>Bearer<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"azp"</span>: <span class="pl-s"><span class="pl-pds">"</span>kubernetes<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"auth_time"</span>: <span class="pl-c1">1472061358</span>,
  <span class="pl-ent">"session_state"</span>: <span class="pl-s"><span class="pl-pds">"</span>8b3773bc-23ca-4f75-8c35-d2bf197d4057<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"acr"</span>: <span class="pl-s"><span class="pl-pds">"</span>0<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"client_session"</span>: <span class="pl-s"><span class="pl-pds">"</span>85550ebd-fe8c-4212-bf37-3579b3532f76<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"allowed-origins"</span>: [],
  <span class="pl-ent">"realm_access"</span>: {
    <span class="pl-ent">"roles"</span>: [
      <span class="pl-s"><span class="pl-pds">"</span>uma_authorization<span class="pl-pds">"</span></span>
    ]
  },
  <span class="pl-ent">"resource_access"</span>: {
    <span class="pl-ent">"account"</span>: {
      <span class="pl-ent">"roles"</span>: [
        <span class="pl-s"><span class="pl-pds">"</span>manage-account<span class="pl-pds">"</span></span>,
        <span class="pl-s"><span class="pl-pds">"</span>view-profile<span class="pl-pds">"</span></span>
      ]
    }
  },
  <span class="pl-ent">"user_role"</span>: [
    <span class="pl-s"><span class="pl-pds">"</span>admin<span class="pl-pds">"</span></span>
  ],
  <span class="pl-ent">"name"</span>: <span class="pl-s"><span class="pl-pds">"</span>mmosley@ent2k12.domain.com Mosley<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"preferred_username"</span>: <span class="pl-s"><span class="pl-pds">"</span>mmosley.ent2k12.domain.com<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"given_name"</span>: <span class="pl-s"><span class="pl-pds">"</span>mmosley@ent2k12.domain.com<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"family_name"</span>: <span class="pl-s"><span class="pl-pds">"</span>Mosley<span class="pl-pds">"</span></span>,
  <span class="pl-ent">"email"</span>: <span class="pl-s"><span class="pl-pds">"</span>matt.mosley@tremolosecurity-test.com<span class="pl-pds">"</span></span>
}</pre></div>
<ul dir="auto">
<li>Try to use kubectl, all requests results in a 403 (unauthorized)</li>
<li>Change the api server to use a ca client certificate</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="- --runtime-config=extensions/v1beta1/networkpolicies=true,rbac.authorization.k8s.io/v1alpha1
    - --oidc-issuer-url=https://kcdev.tremolosecurity.com:8443/auth/realms/kubernetes
    - --oidc-client-id=kubernetes
    - --oidc-username-claim=preferred_username
    - --oidc-groups-claim=user_role
    - --oidc-ca-file=/etc/kubernetes/ssl/kc-ca.pem
    - --authorization-rbac-super-user=kube-admin
    - --authorization-mode=RBAC"><pre class="notranslate"><code class="notranslate">- --runtime-config=extensions/v1beta1/networkpolicies=true,rbac.authorization.k8s.io/v1alpha1
    - --oidc-issuer-url=https://kcdev.tremolosecurity.com:8443/auth/realms/kubernetes
    - --oidc-client-id=kubernetes
    - --oidc-username-claim=preferred_username
    - --oidc-groups-claim=user_role
    - --oidc-ca-file=/etc/kubernetes/ssl/kc-ca.pem
    - --authorization-rbac-super-user=kube-admin
    - --authorization-mode=RBAC
</code></pre></div>
<ul dir="auto">
<li>Try using kubectl with the client certificate</li>
</ul>
<p dir="auto"><strong>Anything else do we need to know</strong>:</p>
<p dir="auto">OpenID Connect provider is KeyCloak 2.1</p>