Answers: <p>Load can mean a lot with an ELK stack.</p>
  
  <p>For <strong>ingestion</strong> load, the <strong>logstash parsing</strong> side, you're looking at a combination of one or two things:</p>
  
  <ul>
  <li>CPU load on the ingestion boxes.</li>
  <li>If used, back-pressure in your ingestion-queue (such as Redis, AMQP, or others)
  
  <ul>
  <li>This can be simple stuff like queue-size, or a fancy number like age of the oldest member of the queue.</li>
  </ul></li>
  </ul>
  
  <p>If you're in a cloud-provider that gives you a way to auto-scale your ingestion nodes, keying on CPU functions pretty well in recent Logstash versions.</p>
  
  <p>For <strong>ingestion and searching</strong>, the <strong>ElasticSearch</strong> side, it can depend on a couple of things due to your architecture. If you're using physical servers, it is very easy to have far more CPU capacity than I/O capacity to store the processed results. If you're using physical servers, keep an eye on <strong>I/O queue depth</strong> in addition to CPU loading. Also beware of the Compressed Object Pointer barrier, around 32GB of JVM HEAP. Stay under 32GB. Just do it.</p>
  
  <p>If you're on cloudy instances, its harder to out-strip your I/O due to the reduced core-counts. Even so, queue-depth and cpu-loading are good things to keep an eye on. If your JVM monitoring tools are up to it, tracking Garbage Collection frequency is worth it until you have a feel for what you need for RAM.</p>
  
  <p>For all of this, keeping an eye on consumed disk-space will tell you when you need to add more ES data-nodes or adjust your index retention settings.</p>
  
  <p>For <strong>display and searching</strong>, such as with Kibana or Grafana, I've found this to be the part of the stack I never bother looking at. At least in any way other than up/down tracking. We track CPU, but we don't alarm anything since in our experience even a 2 core box with 7GB of RAM is enough to keep up without breaking a sweat. Both of these work OK with horizontal scaling if you do run into problems.</p>
  
  <hr>
  
  <p>ELK stacks come in many sizes, from all in one boxes, all the way up to systems with routing logic to bring events to parsing and ES clusters owned by completely different parts of the organization. The monitoring needs of a small architecture (all in one) are rather different than medium architectures (discrete parsing tier, usually queue-mediated), and are different again from large architectures (multiple parsing-tiers with multiple event-stores).</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-08-24T14:42:32.407Z
Answers: 2018-08-24T14:42:32.407Z
Answers: sysadmin1138
Answers: <p>I'm using Datadog which has a really great EKS plugin that works really well with the other AWS integrations.</p>
  
  <p>Downside is cost, which is $15 per node on the basic plan.</p>
  
Answers: CC BY-SA 4.0
Answers: 2018-08-21T14:15:07.087Z
Answers: 2018-08-21T14:15:07.087Z
Answers: Caledonia91
Body: <p>We have tried Prometheus and Grafana. Currently it is not set up on our cluster and we are tight with resources.</p>
  
  <ul>
  <li>What are the best ways to monitor load on an ELK stack?</li>
  <li>Is there anything built in?</li>
  </ul>
  
Comments: CC BY-SA 4.0
Comments: 2018-08-21T13:53:52.147Z
Comments: There's various way, a bunch of plugins like head, cerebro and official pay plugin x-pack for elastic.
Comments: Tensibai
Tags: <monitoring><elastic>
Title: How to monitor ELK Stack Load?
UserDisplayName: David West
ThreadText: # How to monitor ELK Stack Load?
  
  (Asked by: username_0 on 2018-08-21)
  
  We have tried Prometheus and Grafana. Currently it is not set up on our cluster and we are tight with resources.
  
  * What are the best ways to monitor load on an ELK stack?
  * Is there anything built in?
  
  **Tags:** monitoring, elastic
  
  > **Comments:**
  > **username_1**: There's various way, a bunch of plugins like head, cerebro and official pay plugin x-pack for elastic.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2018-08-24. Score: 1
  
  Load can mean a lot with an ELK stack.
  
  For **ingestion** load, the **logstash parsing** side, you're looking at a combination of one or two things:
  
  * CPU load on the ingestion boxes.
  * If used, back-pressure in your ingestion-queue (such as Redis, AMQP, or others)
    + This can be simple stuff like queue-size, or a fancy number like age of the oldest member of the queue.
  
  If you're in a cloud-provider that gives you a way to auto-scale your ingestion nodes, keying on CPU functions pretty well in recent Logstash versions.
  
  For **ingestion and searching**, the **ElasticSearch** side, it can depend on a couple of things due to your architecture. If you're using physical servers, it is very easy to have far more CPU capacity than I/O capacity to store the processed results. If you're using physical servers, keep an eye on **I/O queue depth** in addition to CPU loading. Also beware of the Compressed Object Pointer barrier, around 32GB of JVM HEAP. Stay under 32GB. Just do it.
  
  If you're on cloudy instances, its harder to out-strip your I/O due to the reduced core-counts. Even so, queue-depth and cpu-loading are good things to keep an eye on. If your JVM monitoring tools are up to it, tracking Garbage Collection frequency is worth it until you have a feel for what you need for RAM.
  
  For all of this, keeping an eye on consumed disk-space will tell you when you need to add more ES data-nodes or adjust your index retention settings.
  
  For **display and searching**, such as with Kibana or Grafana, I've found this to be the part of the stack I never bother looking at. At least in any way other than up/down tracking. We track CPU, but we don't alarm anything since in our experience even a 2 core box with 7GB of RAM is enough to keep up without breaking a sweat. Both of these work OK with horizontal scaling if you do run into problems.
  
  ---
  
  ELK stacks come in many sizes, from all in one boxes, all the way up to systems with routing logic to bring events to parsing and ES clusters owned by completely different parts of the organization. The monitoring needs of a small architecture (all in one) are rather different than medium architectures (discrete parsing tier, usually queue-mediated), and are different again from large architectures (multiple parsing-tiers with multiple event-stores).
  
  
  ---
  
  ### Answer by username_3 on 2018-08-21. Score: 0
  
  I'm using Datadog which has a really great EKS plugin that works really well with the other AWS integrations.
  
  Downside is cost, which is $15 per node on the basic plan.
  
  
  ---
  
  