Practical advice for analysis of large, complex data sets
I hope some experienced data analytics people will read this thread so here is a slightly unrelated question: We have a data set of 1 TB  growing at 1 TB&#x2F;year we need to analyze. Our IT is pushing for Hadoop but this involves a lot of integration work because they have no plumbing ready. The whole thing just feels way too complex for our use case.<p>The data is reasonably structured so I think we can easily use a SQL database with possibly some XML or JSON columns. This would be much easier and quicker to set up.<p>Is 1TB a size that makes sense for Hadoop? Are there any alternatives like Google BigQuery, MongoDB or others? Sorry, I am not up to date with the latest cloud offerings. Also, we are in the medical field so this raises some security questions.