Answers: <p>As I see it, the basic problem you have is that you aren't looking at the Total Cost of Ownership (TCO). While you may eventually save on the <em>hardware</em> costs, this is costing you in man-hours - either yours or your employees - a cost which you <em>also</em> pay out of pocket until you begin generating revenue.</p>
  
  <p>The reason for this is this: Instead of using a ready to go system, you have to set up, install and configure the following:</p>
  
  <ul>
  <li>Hardware and RAID</li>
  <li>Version and source control</li>
  <li>DNS</li>
  <li>Firewall</li>
  <li>Operating system</li>
  <li>Database(es)</li>
  <li>Web server</li>
  <li>Build controller / test agent</li>
  <li>Backups</li>
  </ul>
  
  <p>Then you have to deal with the operational maintenance of that as well. While <em>some</em> operational maintenance is required with in-house equipment, There is less required if you host. Consider:</p>
  
  <p>Instead of sharing enterprise-grade infrastructure, you are buying white box home desktops from Walmart. This means you are accepting all of the liability and reliability issues that goes with that - drive failures, motherboards that fry, CPUs that overheat and toast, PSUs that give out and so forth - making your backups (which you don't even have to deal with if you host) that much more important. Odds are at those prices, these machines are built off of "seconds" - refurbished parts that have <em>already</em> failed once. There is a reason my organization purchases proper servers from Dell or HP - it's reliability.</p>
  
  <p>This early in your software shop's life it is important to have <em>focus</em> on a <em>single</em> strategic vision - to produce great software (whatever software that is). In order to do that most effectively, you need to minimize distractions. You don't want to be in the hosting business too. It will distract you from making great software and that setup and management will cost you a great deal more than it would cost to simply have it hosted. </p>
  
  <p>Figure out an estimate of how long it would take for you to set up and build what your hosting provider already has using the components you are looking at. Then multiply times your (or your employee's) hourly rate and factor that into your cost.  At 300 a month, you are nearly halfway through your year. If your figured set up costs are greater than 1500 to 2000 dollars, then it is actually cheaper to host - plus less headache <em>and</em> you got a true year of development time instead of losing 2 weeks to a month (or more!) to setting up your environment.</p>
  
  <p>After that year, you can still move in-house on the hardware - you were planning on possibly needing to migrate anyway.</p>
  
  <hr>
  
  <p>But finally, to answer your question, use the three smaller hosts. You are clearly virtualizing anyway and this provides you redundancy. If your single large hosts crashes or reboots, it affects your entire system. If you have a problem with one instance, it affects 1/3rd of your environment and you might even be able to seamlessly migrate your VM to another host if your system is properly configured.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-09-03T09:18:33.217Z
Answers: While you are right, and using off the shelf tools will significantly reduce maintenance overhead, I'm only dealing with a start-up cost of 2500. I don't want to be stuck in the cloud provider model. What that means is that once you code for Azure/Amazon it is rally hard to switch provides thereafter. I have to build lots of abstractions and wrappers around cloud services in order to make the code generic enough. That's why I opt out for my own infrastructure. The technologies for hosting (Apache) and DB (Postgre) are free, scalable and ready to use. The only concern is to have
Answers: bleepzter
Answers: CC BY-SA 3.0
Answers: 2017-09-03T09:25:08.287Z
Answers: a repeatable build and deploy process that is tied to a requirements management system (rms). In addition the rms should provide a testing environment which can produce a traceability matrix. The matrix associates requirements, bugs, and code check-ins with unit tests and functional tests. Such systems cost a pretty penny regardless of provider, while an MSDN Pro membership provides the software licenses for a measly 500 a year.
Answers: bleepzter
Answers: CC BY-SA 3.0
Answers: 2017-09-04T20:41:08.123Z
Answers: @bleepzter - it looks like you've already made your purchase, but you have to remember that most cloud providers are using open source tools and slapping their branding on it. If they can do it, you can do it - it's just not always most lucrative to DIY at your scale. I can export my web site from any web server to Apache, from MariaDB, MySQL, Galara or Percona to Postgres (or vice-versa) - regardless of which cloud provider I do (or don't use) - The technologies for hosting (Apache) and DB (Postgre) are *not* free - they require setup.
Answers: James Shewey
Answers: CC BY-SA 3.0
Answers: 2017-09-04T20:43:06.033Z
Answers: I'm also willing to bet I can build a repeatable deploy system with RMS including testing env (jenkins), bug tracker and code check-ins (git) with functional tests for zero licensing cost. But again, these aren't free - the require setup and integration.
  
  Good luck with your new endeavor though!
Answers: James Shewey
Answers: CC BY-SA 3.0
Answers: 2017-08-30T16:43:52.413Z
Answers: 2017-08-30T16:43:52.413Z
Answers: James Shewey
Answers: <p>A good rule of thumb is to try and avoid using one bit of logical kit (be it a single VM or a single physical host) for more than one thing.</p>
  
  <p>If a physical server is going to be a VM host then that's all it should be. If a VM or a physical host is going to be a source code repository then that's all it should be etc. You can squeeze a bit extra out of your infrastructure if you consider everything to be a VM host and you then partition up those VMs to be single purpose hosts themselves.</p>
  
  <p>The rationale behind this is that</p>
  
  <ol>
  <li>A single host going down shouldn't impact your entire stack. Trust me this will hurt you badly as soon as you need downtime for an upgrade.</li>
  <li>The cost to scale any individual component increases drastically if you want to try and scale on multiple similar servers (which will save you management dollars by reducing complexity) or it will cost you to scale out adhoc hardware as needed because you then need to deal with many different-sized hosts that are a pain to manage and require anyone managing them to know the platform intimately.</li>
  </ol>
  
  <p><strong>recommendation:</strong> split your infrastructure into as small a unit as is logical. If you can only afford to buy a single large bit of kit then make sure you split it into logically separate VMs or containers so that if you need to pop a bit of your infrastructure onto another server, or you need to scale stuff at different rates you can easily expand out but investing in a smaller upgrade.</p>
  
  <p>Obviously the realities of software licencing, especially for windows OS's might make this cost prohibitive but, while I haven't caught up on windows licensing models in a while, I'm sure there are development centric licensing models that can suit your specific needs.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-09-03T09:30:40.160Z
Answers: Thank you. I just bought a small refurbished server with dual Intel Xeon 5650, 192GB of RAM and 4TB of storage for mere $1200 That should be plenty to run individual small dedicated VMs as you suggested.  Now I can have a domain controller, smtp server, build agent, release agent, tfs+sql, and the dev, beta, prod environments in individual virtual machines. Overall no more than 12 machines for 12 dedicated cores (24 with hyper threading)
Answers: bleepzter
Answers: CC BY-SA 3.0
Answers: 2017-09-02T08:23:13.297Z
Answers: 2017-09-02T08:23:13.297Z
Answers: hvindin
Body: <p>First, I apologize if this is the wrong forum to ask this question. I was wondering if this is a ServerStack or DevOps question. In either case here it goes:</p>
  
  <p>I am building a small software shop and I am trying to figure out the infrastructure and machines required for it. </p>
  
  <p>I bought MSDN Pro Subscription ($600/year) as well as JetBrains All Product Subscription ($600) for the development tools, desktop and server OS licenses, etc etc...</p>
  
  <p>At the moment I have a budget of roughly $1500 (Walmart Credit Card) and I am entertaining the idea of buying 3x refurbished machines (each with Core i7-3770 cpu and 16GB of RAM) at $452 a pop or buying a large server at $1120 and maxing out it's RAM capacity for another $300-$400.</p>
  
  <p>The background info on this is that I want to develop a product and have a full blown development environment (Source Control, Build/Deploy/Test and Release Management, Dev, Beta, and Prod environments). Cloud services (the cheapest) will run me about $200-$300 a month, and over period of 6 months it will become a recurring liability rather than an asset (which I pay out of pocket btw). </p>
  
  <p>If I buy the hardware - it will have an initial cost of about $1500 but it will last me for the duration of the development process which is at least a year or so. Beyond that, if I outgrow it's capacity, I can move to docker containers and maybe a cloud solution for the Dev, Beta, and Prod environments.</p>
  
  <p>So the question is - which is better - multiple dedicated cpu's or a single large server?</p>
  
  <p><strong>1x Large VM Host (64gb RAM) to have the following VMS (<a href="https://www.walmart.com/ip/Hp-Proliant-Ml30-G9-4u-Micro-Tower-Server-1-X-Intel-Xeon-E3-1240-V5-Quad-core-4-Core-3-50-Ghz-1-Processor-Support-8-Gb-Standard-64-Gb-Ddr4-Sdram-Maxi/49334749" rel="nofollow noreferrer">from here</a>) (4GB used by Hyper-V 2016 Server):</strong></p>
  
  <ul>
  <li>1x - 8GB RAM - TFS - (Git, Active Directory, SQL2016, Sharepoint, IIS)</li>
  <li>1x -  4GB RAM - TFS - Build Controller</li>
  <li>1x -  4GB RAM - TFS - Release Agent, Test Agent</li>
  <li>1x -  4GB RAM - DEV - Linux Mint (Postgre SQL)</li>
  <li>1x -  4GB RAM - DEV - Linux Mint (Web Server/Apache)</li>
  <li>1x -  4GB RAM - BETA - Linux Mint (Postgre SQL)</li>
  <li>1x -  4GB RAM - BETA - Linux Mint (Web Server/Apache)</li>
  <li>1x -  4GB RAM - PROD - Linux Mint (Postgre SQL)</li>
  <li>1x -  4GB RAM - PROD - Linux Mint (Web Server/Apache)</li>
  </ul>
  
  <p><strong>Or having multiple smaller hosts:(<a href="https://www.walmart.com/ip/Refurbished-Dell-OptiPlex-7010-Desktop-PC-with-Intel-Core-i7-3770-Processor-16GB-Memory-2TB-Hard-Drive-and-Windows-10-Pro-Monitor-Not-Included/52030501" rel="nofollow noreferrer">from here</a>)</strong></p>
  
  <p>1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):</p>
  
  <ul>
  <li>1x - 6GB RAM - TFS - (Git, Active Directory, SQL2016, Sharepoint, IIS)</li>
  <li>1x - 3GB RAM - TFS - Build Controller</li>
  <li>1x - 3GB RAM - TFS - Release Agent, Test Agent</li>
  </ul>
  
  <p>1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):</p>
  
  <ul>
  <li>1x -  4GB RAM - DEV - Linux Mint (Postgre SQL)</li>
  <li>1x -  4GB RAM - BETA - Linux Mint (Postgre SQL)</li>
  <li>1x -  4GB RAM - PROD - Linux Mint (Postgre SQL)</li>
  </ul>
  
  <p>1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):</p>
  
  <ul>
  <li>1x -  4GB RAM - DEV - Linux Mint (Web Server/Apache)</li>
  <li>1x -  4GB RAM - BETA - Linux Mint (Web Server/Apache)</li>
  <li>1x -  4GB RAM - PROD - Linux Mint (Web Server/Apache)</li>
  </ul>
  
Comments: CC BY-SA 3.0
Comments: 2017-08-30T07:56:03.553Z
Comments: Memory is not the only requirement, you're planning to have only 9 machines, one host with a single i7-3770 will only have 4 cores, keep one for the hypervisor and you won't have really more than 3 machines running at the same time, under low load this should work but I'd better go on multiple small hosts and spread the types (don't set all DB on same host, you'll experiment IO congestions). That sound better suited toward serverfault btw.
Comments: Tensibai
Tags: <environments><servers>
Title: VM Hosts: Single Large Host or Few Smaller Hosts?
UserDisplayName: bleepzter
ThreadText: # VM Hosts: Single Large Host or Few Smaller Hosts?
  
  (Asked by: username_0 on 2017-08-30)
  
  First, I apologize if this is the wrong forum to ask this question. I was wondering if this is a ServerStack or DevOps question. In either case here it goes:
  
  I am building a small software shop and I am trying to figure out the infrastructure and machines required for it.
  
  I bought MSDN Pro Subscription ($600/year) as well as JetBrains All Product Subscription ($600) for the development tools, desktop and server OS licenses, etc etc...
  
  At the moment I have a budget of roughly $1500 (Walmart Credit Card) and I am entertaining the idea of buying 3x refurbished machines (each with Core i7-3770 cpu and 16GB of RAM) at $452 a pop or buying a large server at $1120 and maxing out it's RAM capacity for another $300-$400.
  
  The background info on this is that I want to develop a product and have a full blown development environment (Source Control, Build/Deploy/Test and Release Management, Dev, Beta, and Prod environments). Cloud services (the cheapest) will run me about $200-$300 a month, and over period of 6 months it will become a recurring liability rather than an asset (which I pay out of pocket btw).
  
  If I buy the hardware - it will have an initial cost of about $1500 but it will last me for the duration of the development process which is at least a year or so. Beyond that, if I outgrow it's capacity, I can move to docker containers and maybe a cloud solution for the Dev, Beta, and Prod environments.
  
  So the question is - which is better - multiple dedicated cpu's or a single large server?
  
  **1x Large VM Host (64gb RAM) to have the following VMS ([from here](https://www.walmart.com/ip/Hp-Proliant-Ml30-G9-4u-Micro-Tower-Server-1-X-Intel-Xeon-E3-1240-V5-Quad-core-4-Core-3-50-Ghz-1-Processor-Support-8-Gb-Standard-64-Gb-Ddr4-Sdram-Maxi/49334749)) (4GB used by Hyper-V 2016 Server):**
  
  * 1x - 8GB RAM - TFS - (Git, Active Directory, SQL2016, Sharepoint, IIS)
  * 1x - 4GB RAM - TFS - Build Controller
  * 1x - 4GB RAM - TFS - Release Agent, Test Agent
  * 1x - 4GB RAM - DEV - Linux Mint (Postgre SQL)
  * 1x - 4GB RAM - DEV - Linux Mint (Web Server/Apache)
  * 1x - 4GB RAM - BETA - Linux Mint (Postgre SQL)
  * 1x - 4GB RAM - BETA - Linux Mint (Web Server/Apache)
  * 1x - 4GB RAM - PROD - Linux Mint (Postgre SQL)
  * 1x - 4GB RAM - PROD - Linux Mint (Web Server/Apache)
  
  **Or having multiple smaller hosts:([from here](https://www.walmart.com/ip/Refurbished-Dell-OptiPlex-7010-Desktop-PC-with-Intel-Core-i7-3770-Processor-16GB-Memory-2TB-Hard-Drive-and-Windows-10-Pro-Monitor-Not-Included/52030501))**
  
  1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):
  
  * 1x - 6GB RAM - TFS - (Git, Active Directory, SQL2016, Sharepoint, IIS)
  * 1x - 3GB RAM - TFS - Build Controller
  * 1x - 3GB RAM - TFS - Release Agent, Test Agent
  
  1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):
  
  * 1x - 4GB RAM - DEV - Linux Mint (Postgre SQL)
  * 1x - 4GB RAM - BETA - Linux Mint (Postgre SQL)
  * 1x - 4GB RAM - PROD - Linux Mint (Postgre SQL)
  
  1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):
  
  * 1x - 4GB RAM - DEV - Linux Mint (Web Server/Apache)
  * 1x - 4GB RAM - BETA - Linux Mint (Web Server/Apache)
  * 1x - 4GB RAM - PROD - Linux Mint (Web Server/Apache)
  
  **Tags:** environments, servers
  
  > **Comments:**
  > **username_1**: Memory is not the only requirement, you're planning to have only 9 machines, one host with a single i7-3770 will only have 4 cores, keep one for the hypervisor and you won't have really more than 3 machines running at the same time, under low load this should work but I'd better go on multiple small hosts and spread the types (don't set all DB on same host, you'll experiment IO congestions). That sound better suited toward serverfault btw.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2017-09-02. Score: 3 | Accepted answer
  
  A good rule of thumb is to try and avoid using one bit of logical kit (be it a single VM or a single physical host) for more than one thing.
  
  If a physical server is going to be a VM host then that's all it should be. If a VM or a physical host is going to be a source code repository then that's all it should be etc. You can squeeze a bit extra out of your infrastructure if you consider everything to be a VM host and you then partition up those VMs to be single purpose hosts themselves.
  
  The rationale behind this is that
  
  1. A single host going down shouldn't impact your entire stack. Trust me this will hurt you badly as soon as you need downtime for an upgrade.
  2. The cost to scale any individual component increases drastically if you want to try and scale on multiple similar servers (which will save you management dollars by reducing complexity) or it will cost you to scale out adhoc hardware as needed because you then need to deal with many different-sized hosts that are a pain to manage and require anyone managing them to know the platform intimately.
  
  **recommendation:** split your infrastructure into as small a unit as is logical. If you can only afford to buy a single large bit of kit then make sure you split it into logically separate VMs or containers so that if you need to pop a bit of your infrastructure onto another server, or you need to scale stuff at different rates you can easily expand out but investing in a smaller upgrade.
  
  Obviously the realities of software licencing, especially for windows OS's might make this cost prohibitive but, while I haven't caught up on windows licensing models in a while, I'm sure there are development centric licensing models that can suit your specific needs.
  
  > **Comments:**
  > **username_0**: Thank you. I just bought a small refurbished server with dual Intel Xeon 5650, 192GB of RAM and 4TB of storage for mere $1200 That should be plenty to run individual small dedicated VMs as you suggested.  Now I can have a domain controller, smtp server, build agent, release agent, tfs+sql, and the dev, beta, prod environments in individual virtual machines. Overall no more than 12 machines for 12 dedicated cores (24 with hyper threading)  
  
  ---
  
  ### Answer by username_2 on 2017-08-30. Score: 1
  
  As I see it, the basic problem you have is that you aren't looking at the Total Cost of Ownership (TCO). While you may eventually save on the *hardware* costs, this is costing you in man-hours - either yours or your employees - a cost which you *also* pay out of pocket until you begin generating revenue.
  
  The reason for this is this: Instead of using a ready to go system, you have to set up, install and configure the following:
  
  * Hardware and RAID
  * Version and source control
  * DNS
  * Firewall
  * Operating system
  * Database(es)
  * Web server
  * Build controller / test agent
  * Backups
  
  Then you have to deal with the operational maintenance of that as well. While *some* operational maintenance is required with in-house equipment, There is less required if you host. Consider:
  
  Instead of sharing enterprise-grade infrastructure, you are buying white box home desktops from Walmart. This means you are accepting all of the liability and reliability issues that goes with that - drive failures, motherboards that fry, CPUs that overheat and toast, PSUs that give out and so forth - making your backups (which you don't even have to deal with if you host) that much more important. Odds are at those prices, these machines are built off of "seconds" - refurbished parts that have *already* failed once. There is a reason my organization purchases proper servers from Dell or HP - it's reliability.
  
  This early in your software shop's life it is important to have *focus* on a *single* strategic vision - to produce great software (whatever software that is). In order to do that most effectively, you need to minimize distractions. You don't want to be in the hosting business too. It will distract you from making great software and that setup and management will cost you a great deal more than it would cost to simply have it hosted.
  
  Figure out an estimate of how long it would take for you to set up and build what your hosting provider already has using the components you are looking at. Then multiply times your (or your employee's) hourly rate and factor that into your cost. At 300 a month, you are nearly halfway through your year. If your figured set up costs are greater than 1500 to 2000 dollars, then it is actually cheaper to host - plus less headache *and* you got a true year of development time instead of losing 2 weeks to a month (or more!) to setting up your environment.
  
  After that year, you can still move in-house on the hardware - you were planning on possibly needing to migrate anyway.
  
  ---
  
  But finally, to answer your question, use the three smaller hosts. You are clearly virtualizing anyway and this provides you redundancy. If your single large hosts crashes or reboots, it affects your entire system. If you have a problem with one instance, it affects 1/3rd of your environment and you might even be able to seamlessly migrate your VM to another host if your system is properly configured.
  
  > **Comments:**
  > **username_0**: While you are right, and using off the shelf tools will significantly reduce maintenance overhead, I'm only dealing with a start-up cost of 2500. I don't want to be stuck in the cloud provider model. What that means is that once you code for Azure/Amazon it is rally hard to switch provides thereafter. I have to build lots of abstractions and wrappers around cloud services in order to make the code generic enough. That's why I opt out for my own infrastructure. The technologies for hosting (Apache) and DB (Postgre) are free, scalable and ready to use. The only concern is to have  
  > **username_0**: a repeatable build and deploy process that is tied to a requirements management system (rms). In addition the rms should provide a testing environment which can produce a traceability matrix. The matrix associates requirements, bugs, and code check-ins with unit tests and functional tests. Such systems cost a pretty penny regardless of provider, while an MSDN Pro membership provides the software licenses for a measly 500 a year.  
  > **username_2**: @username_0 - it looks like you've already made your purchase, but you have to remember that most cloud providers are using open source tools and slapping their branding on it. If they can do it, you can do it - it's just not always most lucrative to DIY at your scale. I can export my web site from any web server to Apache, from MariaDB, MySQL, Galara or Percona to Postgres (or vice-versa) - regardless of which cloud provider I do (or don't use) - The technologies for hosting (Apache) and DB (Postgre) are *not* free - they require setup.  
  > **username_2**: I'm also willing to bet I can build a repeatable deploy system with RMS including testing env (jenkins), bug tracker and code check-ins (git) with functional tests for zero licensing cost. But again, these aren't free - the require setup and integration.  Good luck with your new endeavor though!  
  
  ---
  
  