Answers: <p>If you could let us know what helm chart you have used to install prom-operator, that might help refine the answer.</p>
  <p>But in case of <a href="https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack" rel="nofollow noreferrer">kube-prometheus-stack</a>, in the values.yaml of the , use the <code>additionalScrapeConfigs</code> section to describe jobs that are external to k8s system and re-deploy.</p>
  <p>Or use serviceMonitors as described in this <a href="https://devops.college/prometheus-operator-how-to-monitor-an-external-service-3cb6ac8d5acb" rel="nofollow noreferrer">article</a></p>
  
Answers: CC BY-SA 4.0
Answers: 2021-07-07T21:23:58.783Z
Answers: 2021-07-07T21:36:57.197Z
Answers: 2021-07-07T21:36:57.197Z
Answers: rohatgisanat
Body: <p>I am learning rancher (still) and after deploying the monitoring chart (prom, grafana etc) i need to add more targets to the prometheus config (such as other bare-metal server and some QEMU too), I know I can update the config by entering the pods via shell but these changes would be lost after redeploy or upgrade of the chart.</p>
  <p>I tends to think that the chart uses configMap to handle the prometheus config but (and i'm feeling noobish here) I cannot find a way to update this configMap, well I just cannot find this specific configMap.</p>
  <p>I went through the helm quickstart and chart update docs but this did not help, same goes for the rancher docs.</p>
  <p>Any hint will be highly apreciated.</p>
  <p>For example in the cattle-monitoring-system namespace I can see these configmaps :</p>
  <pre><code>pierre@latpopo:~$ k get cm -n cattle-monitoring-system
  NAME                                                   DATA   AGE
  grafana-nginx-proxy-config                             1      2d23h
  kube-root-ca.crt                                       1      2d23h
  prometheus-nginx-proxy-config                          1      2d23h
  prometheus-rancher-monitoring-prometheus-rulefiles-0   26     2d22h
  rancher-monitoring-crd-manifest                        1      2d22h
  rancher-monitoring-grafana                             1      2d23h
  rancher-monitoring-grafana-config-dashboards           1      2d23h
  rancher-monitoring-grafana-datasource                  1      2d23h
  rancher-monitoring-grafana-test                        1      2d23h
  rancher-monitoring-prometheus-adapter                  1      2d23h
  </code></pre>
  <p>None of them seems to be hosting the prometheus config file</p>
  <p>In the describe for what I think is the actual prom server deployment I can see there is a use of a configMap :</p>
  <pre><code>k describe statefulsets prometheus-rancher-monitoring-prometheus -n cattle-monitoring-system
  [...]
  --config.file=/etc/prometheus/config_out/prometheus.env.yaml
  [...]
      Mounts:
        /etc/prometheus/config from config (rw)
  [...]
  
  </code></pre>
  <p>But still, I can't find how to change in a persistent manner</p>
  <p>EDIT :
  Ok by reviewing again the rancher docs i finally saw :</p>
  <blockquote>
  <p>For information on configuring custom scrape targets and rules for Prometheus, please refer to the upstream documentation for the Prometheus Operator.</p>
  </blockquote>
  <p>But following that documentation path led me to the prometheus-operator which seems to be the correct place for editing the prometheus config, but this also seems to be only usable for kubernetes based resources: <em>Looks like this is not the correct place to add external targets and services monitoring</em>.</p>
  <p>So the question still stand:</p>
  <p><strong>How can I add new targets (external to kubernetes) to the prometheus &quot;chart deployed&quot; ?</strong></p>
  
Tags: <kubernetes><helm><rancher>
Title: how to change a rancher UI installed prometheus server config
UserDisplayName: Pier
ThreadText: # how to change a rancher UI installed prometheus server config
  
  (Asked by: username_0 on 2021-07-02)
  
  I am learning rancher (still) and after deploying the monitoring chart (prom, grafana etc) i need to add more targets to the prometheus config (such as other bare-metal server and some QEMU too), I know I can update the config by entering the pods via shell but these changes would be lost after redeploy or upgrade of the chart.
  
  I tends to think that the chart uses configMap to handle the prometheus config but (and i'm feeling noobish here) I cannot find a way to update this configMap, well I just cannot find this specific configMap.
  
  I went through the helm quickstart and chart update docs but this did not help, same goes for the rancher docs.
  
  Any hint will be highly apreciated.
  
  For example in the cattle-monitoring-system namespace I can see these configmaps :
  
  ```
  pierre@latpopo:~$ k get cm -n cattle-monitoring-system
  NAME                                                   DATA   AGE
  grafana-nginx-proxy-config                             1      2d23h
  kube-root-ca.crt                                       1      2d23h
  prometheus-nginx-proxy-config                          1      2d23h
  prometheus-rancher-monitoring-prometheus-rulefiles-0   26     2d22h
  rancher-monitoring-crd-manifest                        1      2d22h
  rancher-monitoring-grafana                             1      2d23h
  rancher-monitoring-grafana-config-dashboards           1      2d23h
  rancher-monitoring-grafana-datasource                  1      2d23h
  rancher-monitoring-grafana-test                        1      2d23h
  rancher-monitoring-prometheus-adapter                  1      2d23h
  
  ```
  
  None of them seems to be hosting the prometheus config file
  
  In the describe for what I think is the actual prom server deployment I can see there is a use of a configMap :
  
  ```
  k describe statefulsets prometheus-rancher-monitoring-prometheus -n cattle-monitoring-system
  [...]
  --config.file=/etc/prometheus/config_out/prometheus.env.yaml
  [...]
      Mounts:
        /etc/prometheus/config from config (rw)
  [...]
  
  ```
  
  But still, I can't find how to change in a persistent manner
  
  EDIT :
  Ok by reviewing again the rancher docs i finally saw :
  
  > For information on configuring custom scrape targets and rules for Prometheus, please refer to the upstream documentation for the Prometheus Operator.
  
  But following that documentation path led me to the prometheus-operator which seems to be the correct place for editing the prometheus config, but this also seems to be only usable for kubernetes based resources: *Looks like this is not the correct place to add external targets and services monitoring*.
  
  So the question still stand:
  
  **How can I add new targets (external to kubernetes) to the prometheus "chart deployed" ?**
  
  **Tags:** kubernetes, helm, rancher
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-07-07. Score: 1 | Accepted answer
  
  If you could let us know what helm chart you have used to install prom-operator, that might help refine the answer.
  
  But in case of [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack), in the values.yaml of the , use the `additionalScrapeConfigs` section to describe jobs that are external to k8s system and re-deploy.
  
  Or use serviceMonitors as described in this [article](https://devops.college/prometheus-operator-how-to-monitor-an-external-service-3cb6ac8d5acb)
  
  
  ---
  
  