Answers: <p>I would be inclined to implement the <a href="https://www.openpolicyagent.org/docs/latest/kubernetes-introduction/" rel="nofollow noreferrer">Open Policy Agent</a> as this will give you the ability to define a policy similar to this:</p>
  
  <pre><code>apiVersion: constraints.gatekeeper.sh/v1beta1
  kind: K8sContainerLimits
  metadata:
    name: container-must-have-limits
  spec:
    match:
      kinds:
        - apiGroups: [""]
          kinds: ["Pod"]
    parameters:
      cpu: "200m"
      memory: "1Gi"
  </code></pre>
  
  <p>OPA will enforce any policy you defined via an <a href="https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/" rel="nofollow noreferrer">Admission Controller</a>, beware though it's very powerful and I have bricked clusters by not thinking through the options carefully.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-11-12T09:34:09.087Z
Answers: 2019-11-12T09:34:09.087Z
Answers: Richard Slater
Body: <p>I have built a self-service platform based on Kubernetes, where we create namespaces for each team and allow them to 'do whatever they want within the namespace' (we set resource limits so no one can kill the whole cluster).</p>
  
  <p>However, now I want to implement some kind of standard across the organization. For example, I want every PodSpec to define its own resource limits, and I want every resource to have a label that specifies what application it belongs to.</p>
  
  <p>Is there a mechanism that will allow the API server to check the manifests being applied against a set of rules, and if it fails the check the manifest is rejected.</p>
  
  <p>For example, the following manifest would be rejected because it has neither a label nor are resource limits set.</p>
  
  <pre><code>apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx-deployment
  spec:
    selector:
      matchLabels:
        app: nginx
    replicas: 2
    template:
      metadata:
        labels:
          app: nginx
      spec:
        containers:
        - name: nginx
          image: nginx:1.7.9
          ports:
          - containerPort: 80
  </code></pre>
  
  <p>But the following manifest would succeed because it satisfies all the rules:</p>
  
  <pre>
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx-deployment
    <b>labels:</b>
      <b>app: foobar</b>
  spec:
    selector:
      matchLabels:
        app: nginx
    replicas: 2
    template:
      metadata:
        labels:
          app: nginx
      spec:
        containers:
        - name: nginx
          image: nginx:1.7.9
          ports:
          - containerPort: 80
          <b>resources:</b>
            <b>limits:</b>
              <b>cpu: "1"</b>
            <b>requests:</b>
              <b>cpu: "0.5"</b>
  </pre>
  
Comments: CC BY-SA 4.0
Comments: 2019-11-11T21:47:58.640Z
Comments: Why not using RBAC?
Comments: 030
Comments: CC BY-SA 4.0
Comments: 2019-11-11T21:52:18.643Z
Comments: As far as I know RBAC limits what resources you can interact with and what actions you can perform on those resources. I am not sure it is able to perform validation on whether the manifest has a specified field (like `metadata.labels.app`, for example)
Comments: dayuloli
Comments: CC BY-SA 4.0
Comments: 2019-11-11T22:42:57.947Z
Comments: And what about writing a custom resource?
Comments: 030
Comments: CC BY-SA 4.0
Comments: 2019-11-12T03:10:39.130Z
Comments: Have you thought about creating resource quotas per namespace instead?
  https://kubernetes.io/docs/concepts/policy/resource-quotas/#enabling-resource-quota
Comments: kaizenCoder
Comments: CC BY-SA 4.0
Comments: 2019-11-12T09:35:36.360Z
Comments: @kaizenCoder have you considered posting your comment as an answer, it's perfectly valid - expand it a little to include a basic summary of the link.  This question would benefit from multiple answers as there are many ways to solve this problem.
Comments: Richard Slater
Tags: <kubernetes>
Title: How to enforce policies for manifests in Kubernetes?
UserDisplayName: dayuloli
ThreadText: # How to enforce policies for manifests in Kubernetes?
  
  (Asked by: username_0 on 2019-11-11)
  
  I have built a self-service platform based on Kubernetes, where we create namespaces for each team and allow them to 'do whatever they want within the namespace' (we set resource limits so no one can kill the whole cluster).
  
  However, now I want to implement some kind of standard across the organization. For example, I want every PodSpec to define its own resource limits, and I want every resource to have a label that specifies what application it belongs to.
  
  Is there a mechanism that will allow the API server to check the manifests being applied against a set of rules, and if it fails the check the manifest is rejected.
  
  For example, the following manifest would be rejected because it has neither a label nor are resource limits set.
  
  ```
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx-deployment
  spec:
    selector:
      matchLabels:
        app: nginx
    replicas: 2
    template:
      metadata:
        labels:
          app: nginx
      spec:
        containers:
        - name: nginx
          image: nginx:1.7.9
          ports:
          - containerPort: 80
  
  ```
  
  But the following manifest would succeed because it satisfies all the rules:
  
  ```
  
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx-deployment
    labels:
      app: foobar
  spec:
    selector:
      matchLabels:
        app: nginx
    replicas: 2
    template:
      metadata:
        labels:
          app: nginx
      spec:
        containers:
        - name: nginx
          image: nginx:1.7.9
          ports:
          - containerPort: 80
          resources:
            limits:
              cpu: "1"
            requests:
              cpu: "0.5"
  
  ```
  
  **Tags:** kubernetes
  
  > **Comments:**
  > **username_1**: Why not using RBAC?  
  > **username_0**: As far as I know RBAC limits what resources you can interact with and what actions you can perform on those resources. I am not sure it is able to perform validation on whether the manifest has a specified field (like `metadata.labels.app`, for example)  
  > **username_1**: And what about writing a custom resource?  
  > **username_2**: Have you thought about creating resource quotas per namespace instead? https://kubernetes.io/docs/concepts/policy/resource-quotas/#enabling-resource-quota  
  > **username_3**: @username_2 have you considered posting your comment as an answer, it's perfectly valid - expand it a little to include a basic summary of the link.  This question would benefit from multiple answers as there are many ways to solve this problem.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-11-12. Score: 1
  
  I would be inclined to implement the [Open Policy Agent](https://www.openpolicyagent.org/docs/latest/kubernetes-introduction/) as this will give you the ability to define a policy similar to this:
  
  ```
  apiVersion: constraints.gatekeeper.sh/v1beta1
  kind: K8sContainerLimits
  metadata:
    name: container-must-have-limits
  spec:
    match:
      kinds:
        - apiGroups: [""]
          kinds: ["Pod"]
    parameters:
      cpu: "200m"
      memory: "1Gi"
  
  ```
  
  OPA will enforce any policy you defined via an [Admission Controller](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/), beware though it's very powerful and I have bricked clusters by not thinking through the options carefully.
  
  
  ---
  
  