I have a long item to write to a DynamoDB table.
If the size of the item exceeds the 400KB limit, I want to split the item into multiple chucks and store it as multiple items in the same table with different range keys. Range key here can be monotonically increasing number. How do I do this? Im using boto3
There is no way to have boto library or the DynamoDB API automatically ‘split’ your items.
You will have to split up the data yourself and essentially put multiple items that logically make up your original item.
Depending on the actual structure of your item you will have to decide on a split schema that makes sense for your use case.
If your system can tolerate eventual consistency, you can use the PutItem api to write each of the splits. Otherwise, you can use the new transactions API to put all splits in a transaction with the caveat that it costs more and that each transaction is limited to 25 requests and up to 4 MB of data.
Split a DynamoDB item that exceeds the size limit