Im trying to design a system that can periodic "download" a large amount of data from an outside api..
This user could have around 600,000 records of data that I need once, then to check back every hour or so to reconcile both datasets.
Im thinking about doing this in python or ruby in background tasks eventually but I'm curious about how to store the data.
Would it possible/good idea to store everything in one record hashed as json vs copying each record individually?
It would be nice to be able to index or search the data without anything failing so I was wondering what would be the best implementation memory wise.
For example if the a user has 500,000 tweet records and I want to store all of them, which would be a better implementation?  
one record as JSON =>   user_1 = {id:1 twt:"blah"},{id:2 twt:"blah"},.....{id:600,000 twt:"blah"}
vs 
many records =>
id:1 outside_id=1 twt:"blah"
id:2 outside_id=1  twt:"blah"
id:3 outside_id=1 twt:"blah"
I'm curious how I would find out how memory intensive each method is or what is the best solution.
The records are alot more complex with maybe 40 attributes per record I wanted to store.
Also would MySQL or MongoDB be a better solution for fastest copy/storage?
I think this all boils down to what the most important needs are for the project. These are some of the questions I would try to answer before selecting the technology:

Will I need to access records individually after inserting into the database?
Will I ever need to aggregate the data when reading it (for reporting, for instance)?
Is it more important to the project goals to have the data written quickly or read quickly?
How large do I anticipate the data will grow and will the database technology I select scale easily, cheaply and reliably to support the data volume?
Will the schema of the data change? Do I need a schemaless database solution like MongoDB?
Where are the trade offs between development time/cost, maintenance time/cost and time/cost for running the program?

Without knowing much about the particulars or your project or its goals I would say it's generally not a good idea to store a single JSON object for the entirety of the data. This would likely make it more difficult to read the data and append to it in the future. You should probably apply some more thought on how to model your data and represent it in the database in a way that will make sense when you actually need to use it later.
Best Way to Store Large Amount of Outside API Data... using Ruby or Python