Body: <p>Our goal is to have StatsD deployed on k8s cluster running on AWS.
  Currently all existing applications using UDP to send StatsD metrics.
  As we understood from the kubernetes docs and tests we performed, currently there's no support for UDP listeners while using external loadbalancer (CLB, ALB, NLB), we could use NodePort to expose UDP port but we need support for multiple nodes and failover scenarios when nodes replaced or failing.
  Please share your suggestions or experience of running such setup.</p>
  
Comments: CC BY-SA 4.0
Comments: 2020-06-23T16:24:39.420Z
Comments: Why does it need an external load-balancer? Most metric collection services I've seen running on container orchestration platforms like EKS and ECS are colocated and use service discovery (DNS-based load-balancing) for container-to-container communication. Would that be feasible for your stack?
Comments: Woodland
Comments: CC BY-SA 4.0
Comments: 2020-06-23T16:29:40.613Z
Comments: @WoodlandHunter Thank you for your suggestion, the service should be publicly exposed as metrics reported from different clusters or instances, not necessarily from the same cluster.
Comments: rombob
Comments: CC BY-SA 4.0
Comments: 2020-06-23T16:44:22.043Z
Comments: Totally understandable. I had a similar issue (collecting StatsD in ECS Fargate across multiple AWS accounts, no UDP support https://github.com/aws/containers-roadmap/issues/445 ) and opted to deploy small collector "services" in each of those accounts instead of a centralized one, but it isn't ideal.
  Another not-ideal-but-might-work-okay option would be to use Spot.io's Managed Instance. It would effectively be a pet server (boo) but it persists the root volume and maintains its ENI despite failure. I've used it as a helpful interim solution  https://help.spot.io/managed-instances
Comments: Woodland
Comments: CC BY-SA 4.0
Comments: 2020-06-23T17:19:35.577Z
Comments: @WoodlandHunter we considered deploying pet UDP to TCP proxy, still feels like ugly hack, could be temporary solution until UDP listener support is ready
Comments: rombob
Tags: <kubernetes><amazon-web-services><monitoring><aws-elb>
Title: How to deploy StatsD listening on UDP in kubernetes with external loadbalancer on AWS?
UserDisplayName: rombob
ThreadText: # How to deploy StatsD listening on UDP in kubernetes with external loadbalancer on AWS?
  
  (Asked by: username_0 on 2020-06-23)
  
  Our goal is to have StatsD deployed on k8s cluster running on AWS.
  Currently all existing applications using UDP to send StatsD metrics.
  As we understood from the kubernetes docs and tests we performed, currently there's no support for UDP listeners while using external loadbalancer (CLB, ALB, NLB), we could use NodePort to expose UDP port but we need support for multiple nodes and failover scenarios when nodes replaced or failing.
  Please share your suggestions or experience of running such setup.
  
  **Tags:** kubernetes, amazon-web-services, monitoring, aws-elb
  
  > **Comments:**
  > **username_1**: Why does it need an external load-balancer? Most metric collection services I've seen running on container orchestration platforms like EKS and ECS are colocated and use service discovery (DNS-based load-balancing) for container-to-container communication. Would that be feasible for your stack?  
  > **username_0**: @WoodlandHunter Thank you for your suggestion, the service should be publicly exposed as metrics reported from different clusters or instances, not necessarily from the same cluster.  
  > **username_1**: Totally understandable. I had a similar issue (collecting StatsD in ECS Fargate across multiple AWS accounts, no UDP support https://github.com/aws/containers-roadmap/issues/445 ) and opted to deploy small collector "services" in each of those accounts instead of a centralized one, but it isn't ideal. Another not-ideal-but-might-work-okay option would be to use Spot.io's Managed Instance. It would effectively be a pet server (boo) but it persists the root volume and maintains its ENI despite failure. I've used it as a helpful interim solution  https://help.spot.io/managed-instances  
  > **username_0**: @WoodlandHunter we considered deploying pet UDP to TCP proxy, still feels like ugly hack, could be temporary solution until UDP listener support is ready  
  
  ---
  
  ## 0 answers
  
  