FreeBSD/EC2 on C5 instances
While the new C5 instances are certainly welcome - I've been hoping for their release since their announcement in November <i>2016</i> (and they were already late for Skylake at that point) - we have encountered a number of show-stopping problems that point to this project being just a bit too ambitious.<p>To name a few:<p>1. EBS volumes attached to C5 instances show completely bogus CloudWatch metrics, over an order of magnitude higher than reality (e.g. average read/write latency prints at 100-60,000ms depending on load)<p>2. C5 instances don't work - at all - behind an NLB with a Target Group pointing to it as an "instance". You have to put it in "IP" mode.<p>3. OpsWorks, as always, lags way behind AWS offerings. You can't launch C5 instances. This was true of even R4 instances for a while, but you could at least change them via API. Not so with the C5 instances; unless you want to lose track of their type completely, you just have to abstain for now.<p>3a. As a result, we have to run R4 instances for some of our web tier - despite not needing the memory - because they have the highest network allocation. To make matters worse, AWS won't tell you the network allocation. You don't know until you start dropping packets.<p>4. ZFS on C5 instances can behave strangely. We've been unable to resize drives (zpool online -e <pool> <drive>) if they're identified by ID ("unable to read disk capacity"). Moving the instance back to any other type fixes the issue.<p>As always, you expect a couple of quirks with a new architecture, but I found myself wishing they had just stuck a new board with a Skylake chip into a rack and launched it.<p>Compared to this, GCE has a far more attractive offering: even ignoring all these issues, we simply can't get the instance size we want (a few fast CPUs + lots of memory). It just doesn't exist.