Proposed server purchase for GitLab.com
For server hardware, the Supermicro 2U Twins are a reasonable choice, but I prefer their 4U FatTwin chassis. The engineering quality is a little better IMO, and the cost increase isn&#x27;t too big. Absolutely do not buy their 1U Twin systems, they are hot garbage.<p>The FatTwin chassis has similar density, and can support either 1U half width or 2U half width systems in a particular chassis. Typically I use 1U&#x27;s for app &#x2F; web servers and 2U&#x27;s for lower end database &#x2F; storage. Separate 2U servers for higher end database and 4U servers for bulk storage.<p>HPE has the Apollo 2600 &#x2F; XL170r 2U chassis, which I think is somewhat inferior but still a reasonable choice. Dell sells the same thing as the C6300. I really prefer a 4U chassis though from a cooling and power supply perspective, but Dell or HPE may have a better international story for you.<p>You absolutely should not buy the 2630v4 CPU. I say that because the lower-end Intel CPUs do not support maximum throughput for memory and QPI. The 2630v4 is 8.0GT&#x2F;s QPI and 2133MHz DDR4. A better solution low-watt part is 2650Lv4 (9.6GT&#x2F;s QPI and 2400MHz DDR4). I have a guide that I created (and use myself) to determine comparative $&#x2F;performance of Intel CPUs based on SPEC numbers [1]. If you can go up to 105W the 2660v4 is probably your best bet. Presuming that you&#x27;re targeting 12-15kW per rack, a 105W part should allow you to deploy between 60-80 hosts per rack.<p>Also, don&#x27;t use a W-series CPU that draws 160W. That&#x27;s crazy power draw per-socket. If you want a super high-end CPU for your database server, I suggest a 2698v4 -- but normally I would go with 2680v4 or 2683v4 depending on the part cost.<p>In terms of hard drives, absolutely you should specify HGST over Seagate. At some point you may want to dual source this, but if you&#x27;re only going with one vendor right now HGST is the best option. He8 or He10 8TB are your best bet in terms of cost and availability right now, although start thinking about He10 10T drives. The newly announced He12 drives shouldn&#x27;t be on your radar until Q2 2017. Stock spares, maybe 2-3% of your total drives deployed, but at least 5-6 drives per site. You don&#x27;t want to get caught out if there is a supply shortage when you need it most. Your business depends on ready access to considerable quantities of storage.<p>The P-series Intel SSDs are probably not going to be cost effective for your use case. But they are considerably better in terms of IOPS and remove the need for an HBA or RAID controller. Consider a Supermicro 2U chassis with 2.5&quot; NVMe support, which will allow you to go considerably denser than the PCIe form factor. However, I think it&#x27;s too early to go with NVMe unless you truly need beyond-SSD performance.<p>Don&#x27;t PXE boot every time you boot. This creates a single point of failure (even if your architecture is redundant), and you will regret this at some point. However, DO PXE boot to install the OS.<p>Don&#x27;t use 128GB DIMMs. They are not cost effective today.<p>There is only one solution for database scaling: shard. You&#x27;ll either shard it today, or you&#x27;ll shard it tomorrow when the problem is much harder. Scale up each host to what is easily achievable with today&#x27;s hardware, and if push comes to shove retrofit to get over a hump that arises in the future, but know that you MUST shard in order to keep up with demand. Scaling up simply does not work.<p>There&#x27;s a lot more to say, but without doing my job for free in a HN comment, the best advice I can offer is:<p>1. Simplify what you hope to accomplish in the first round. This is a lot to achieve at once. I think you&#x27;ll have a hard time achieving the fanciness you want from a software perspective while also forklifting the entire stack over to physical hardware. It&#x27;s perfectly fine to have something be good enough for now.<p>2. Find people who have done this before and get their advice. Find a VAR you can trust.<p>3. Plan, plan, plan, plan. Don&#x27;t commit until you have a plan, make sure the plan is flexible enough to change course without tossing everything out, and plan to do a good enough job to survive long enough to figure out a better plan next time.<p>4. Get eval gear, qualify and test things, and make sure that what you think will work does work.<p>[1] <a href="https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;1bbbeMCmqt5pZCb_x2QMWVSL9_e8YGovZthHNGqr5H1A&#x2F;edit?usp=sharing" rel="nofollow">https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;1bbbeMCmqt5pZCb_x2QMW...</a>