How to build model in DynamoDB if each night I need to process the daily records and then delete them?
You can use RabbitMQ to schedule jobs asynchronously. This would be faster than multiple DB queries. Basically, this tool allows you to create a job queue (Containing UserID, StoreID & Timestamp) where workers can remove (at midnight if you want) and create your reports (or whatever your heart desires).
This also allows you to scale your system horizontally across nodes. Your workers can be different machines executing these tasks. You will also be safe if your DB crashes (though you may still have to design redundancy for a machine running RabbitMQ service).
DB should be used for persistent storage and not as a queue for processing.
I need to store some daily information in DynamoDB. Basically, I need to store user actions: UserID, StoreID, ActionID and Timestamp.
Each night I would like to process the information generated that day, do some aggregations, some reports, and then I can safely deleted those records.
How should I model this? I mean the hash key and the sort key... I need to have the full timestamp of each action for the reports but in order to query DynamoDB I guess it is easier to also save the date only.
I have some PKs as UserID and StoreID but anyhow I need to process all data each night, not the data related to one user or one store...
Thanks!
Patricio