This is just my 5 cents, and it's probably not a official, perfect, or even great way of doing things. But here we go:

Create a SQS queue
Have a lambda listen to messages on that queue
Run the lambda, have it read the first chunk of items from the database.
From the query you get a LastEvaluatedKey, store this in a separate dynamodb item/table
Put the LastEvaluatedKey to the queue
When the Lambda is invoked again, use the LastEvaluatedKey passed in from the queue and pass it to the ExclusiveStartKey  in your query, so that your query returns the next items from DynamoDB that you haven't processed yet

This setup means your lambda will only process small chunks at a time. It also means that when things go wrong, you have your item in DynamoDB with the LastEvaluatedKey stored, which allows you to continue from where you left off.
Hopefully this was useful, good luck :)
I'm trying to update a dynamodb table having some million of rows. Tried updating it with a lambda function but it eventually times out after 15mins. I want to update a single attribute value for all the records present in the table with an efficient way. Can someone suggest how to achieve this?
How to update millions of dynamodb records in an efficient way