title: How can prometheus-operator be configured to collect from cAdvisor on Kubernetes?
body: <p>I am using <a href="https://github.com/coreos/prometheus-operator" rel="nofollow noreferrer">prometheus-operator</a> to manage a <a href="https://prometheus.io/" rel="nofollow noreferrer">Prometheus</a> deployment on my <a href="https://kubernetes.io/" rel="nofollow noreferrer">Kubernetes</a> cluster.  The setup is working fine to extract metrics from a number of my application pods, using several ServiceMonitors which select the Kubernetes endpoints giving the network address at which metrics are published.  As seems to be typical (required?) with prometheus-operator, the Prometheus configuration is generated for me based on the Kubernetes endpoints discovered based on the ServiceMonitors.</p>
  
  <p>I would also like my Prometheus deployment to retrieve <a href="http://blog.kubernetes.io/2015/05/resource-usage-monitoring-kubernetes.html" rel="nofollow noreferrer">the cAdvisor metrics published by kubelet</a> on each cluster node.  I've verified that kubelet on my cluster has cAdvisor and that it is enabled (by visiting port 4194 and observing the native cAdvisor web interface).  However, what I'm missing is how to tell prometheus-operator to configure my Prometheus deployment with targets including each of these kubelet/cAdvisor servers.</p>
  
  <p>The only "documentation" I've found on this is <a href="https://github.com/coreos/prometheus-operator/issues/261" rel="nofollow noreferrer">a prometheus-operator github issue</a> asking why some cAdvisor metrics <em>are</em> being discovered on the poster's cluster.  The explanation suggests that Kubernetes endpoints for kubelet/cAdvisor gets created by prometheus-operator somehow and then an additional ServiceMonitor finds them and causes Prometheus to be configured with additional targets.  However, these Kubernetes endpoints do not exist on my Kubernetes cluster and I haven't found any information about why they ever would.</p>
  
  <p>What do I need to configure so that my prometheus-operator-configured Prometheus deployment can get these metrics?</p>