title: Best way to process SQS messages
body: <p>I have a client that is constantly pouring semi-real-time data into an sqs queue, and want to process and store the messages. My first thought was to use a CloudWatch scheduler that prompts a Lambda with the approximate number of messages that lambda then spawns worker lambdas to process and push the data into a Firehose. The problem is that there will be hundreds of thousands of messages put into the queue every day. I could also use EC2 to do this, but is there any other cost-effective way to process the queue semi-real-time.</p>