title: Horizontal Pod Autoscale unable to read metrics
body: <p>I am using Kafka Helm charts from <a href="https://github.com/helm/charts/tree/master/incubator/kafka" rel="nofollow noreferrer">here</a>.
  I was trying Horizontal Pod Autoscaler for the same.</p>
  
  <p>I added a hpa.yaml file as given below inside the templates folder.</p>
  
  <pre><code>apiVersion: autoscaling/v2beta1
  kind: HorizontalPodAutoscaler
  metadata:
    name: kafka-hpa
  spec:
    scaleTargetRef:
      apiVersion: extensions/v1beta1
      kind: Deployment
      name: {{ include "kafka.fullname" . }}
  minReplicas: {{ .Values.replicas }}
  maxReplicas: 5
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 50
  - type: Resource
    resource:
      name: memory
      targetAverageValue: 8000Mi
  </code></pre>
  
  <p>I have also tried the above YAML with <strong>kind: StatefulSet</strong> but the same issue persists.</p>
  
  <p>My intention is to have 3 Kafka pods initially and scale it up to 5 based on CPU and memory targetValues as mentioned above.</p>
  
  <p>However, the hpa gets deployed but it is unable to read the metrics as per my understanding as the current usage shows unknown as mentioned below.</p>
  
  <pre><code>NAME        REFERENCE                          TARGETS                          MINPODS   MAXPODS   REPLICAS   AGE
  kafka-hpa   Deployment/whopping-walrus-kafka   &lt;unknown&gt;/8000Mi, &lt;unknown&gt;/50%   3         5         0          1h . 
  </code></pre>
  
  <p>I am new to helm and Kubernetes, so I am assuming there might be some issue with my understanding.</p>
  
  <p>I have also deployed metrics-server.</p>
  
  <pre><code>$ kubectl get deployments
  NAME                             DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
  metrics-server                   1         1         1            1           1d
  whopping-walrus-kafka-exporter   1         1         1            1           1h
  </code></pre>
  
  <p>Pods output</p>
  
  <pre><code>$ kubectl get pods
  NAME                                              READY     STATUS    RESTARTS   AGE
  metrics-server-55cbf87bbb-vm2v5                   1/1       Running   0          15m
  whopping-walrus-kafka-0                           1/1       Running   1          1h
  whopping-walrus-kafka-1                           1/1       Running   0          1h
  whopping-walrus-kafka-2                           1/1       Running   0          1h
  whopping-walrus-kafka-exporter-5c66b5b4f9-mv5kv   1/1       Running   1          1h
  whopping-walrus-zookeeper-0                       1/1       Running   0          1h
  </code></pre>
  
  <p>I want the <strong>whopping-walrus-kafka</strong> pod to scale up to 5 on load, however, there's no deployment corresponding to it.</p>
  
  <p>StatefulSet Output</p>
  
  <pre><code>$ kubectl get statefulset
  NAME                        DESIRED   CURRENT   AGE
  original-bobcat-kafka       3         2         2m
  original-bobcat-zookeeper   1         1         2m
  </code></pre>
  
  <p>Output of describe hpa when kind in hpa.yaml is <strong>StatefulSet</strong>.</p>
  
  <pre><code>$ kubectl describe hpa
  Name:                                                  kafka-hpa
  Namespace:                                             default
  Labels:                                                &lt;none&gt;
  Annotations:                                           &lt;none&gt;
  CreationTimestamp:                                     Fri, 18 Jan 2019 12:13:59 +0530
  Reference:                                             StatefulSet/original-bobcat-kafka
  Metrics:                                               ( current / target )
    resource memory on pods:                             &lt;unknown&gt; / 8000Mi
    resource cpu on pods  (as a percentage of request):  &lt;unknown&gt; / 5%
  Min replicas:                                          3
  Max replicas:                                          5
  Conditions:
    Type         Status  Reason          Message
    ----         ------  ------          -------
    AbleToScale  False   FailedGetScale  the HPA controller was unable to get the target's current scale: no matches for kind "StatefulSet" in group "extensions"
  Events:
    Type     Reason          Age                From                       Message
    ----     ------          ----               ----                       -------
    Warning  FailedGetScale  15s (x17 over 8m)  horizontal-pod-autoscaler  no matches for kind "StatefulSet" in group "extensions"
  </code></pre>
  
  <p>Output of describe hpa when kind in hpa.yaml is <strong>Deployment</strong>.</p>
  
  <pre><code>$ kubectl describe hpa
  Name:                                                  kafka-hpa
  Namespace:                                             default
  Labels:                                                &lt;none&gt;
  Annotations:                                           &lt;none&gt;
  CreationTimestamp:                                     Fri, 18 Jan 2019 12:30:07 +0530
  Reference:                                             Deployment/good-elephant-kafka
  Metrics:                                               ( current / target )
    resource memory on pods:                             &lt;unknown&gt; / 8000Mi
    resource cpu on pods  (as a percentage of request):  &lt;unknown&gt; / 5%
  Min replicas:                                          3
  Max replicas:                                          5
  Conditions:
    Type         Status  Reason          Message
    ----         ------  ------          -------
    AbleToScale  False   FailedGetScale  the HPA controller was unable to get the target's current scale: could not fetch the scale for deployments.extensions good-elephant-kafka: deployments/scale.extensions "good-elephant-kafka" not found
  Events:
    Type     Reason          Age   From                       Message
    ----     ------          ----  ----                       -------
    Warning  FailedGetScale  9s    horizontal-pod-autoscaler  could not fetch the scale for deployments.extensions good-elephant-kafka: deployments/scale.extensions "good-elephant-kafka" not found
  </code></pre>
  
  <p>Output from metrics server pod</p>
  
  <pre><code>$ kubectl describe pods metrics-server-55cbf87bbb-vm2v5
  Name:           metrics-server-55cbf87bbb-vm2v5
  Namespace:      default
  Node:           docker-for-desktop/192.168.65.3
  Start Time:     Fri, 18 Jan 2019 11:26:33 +0530
  Labels:         app=metrics-server
              pod-template-hash=1176943666
              release=metrics-server
  Annotations:    &lt;none&gt;
  Status:         Running
  IP:             10.1.0.119
  Controlled By:  ReplicaSet/metrics-server-55cbf87bbb
  Containers:
    metrics-server:
      Container ID:  docker://ee4b3d9ed1b15c2c8783345b0ffbbc565ad25f1493dec0148f245c9581443631
      Image:         gcr.io/google_containers/metrics-server-amd64:v0.3.1
      Image ID:      docker-pullable://gcr.io/google_containers/metrics-server-amd64@sha256:78938f933822856f443e6827fe5b37d6cc2f74ae888ac8b33d06fdbe5f8c658b
      Port:          &lt;none&gt;
      Host Port:     &lt;none&gt;
      Command:
        /metrics-server
        --kubelet-insecure-tls
        --kubelet-preferred-address-types=InternalIP
        --logtostderr
      State:          Running
    Started:      Fri, 18 Jan 2019 11:26:35 +0530
      Ready:          True
      Restart Count:  0
      Environment:    &lt;none&gt;
      Mounts:
    /var/run/secrets/kubernetes.io/serviceaccount from metrics-server-token-d2g7b (ro)
  Conditions:
    Type           Status
    Initialized    True 
    Ready          True 
    PodScheduled   True 
  Volumes:
    metrics-server-token-d2g7b:
      Type:        Secret (a volume populated by a Secret)
      SecretName:  metrics-server-token-d2g7b
      Optional:    false
      QoS Class:       BestEffort
  Node-Selectors:  &lt;none&gt;
  Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
               node.kubernetes.io/unreachable:NoExecute for 300s
  Events:          &lt;none&gt;
  </code></pre>
  
  <p>Guys please feel free to clear my understanding as well if I am going wrong somewhere.</p>
  
  <p>Help on this would be highly appreciated.</p>