Answers: <p>A variation of method 2 is infinitely scalable:</p>
  
  <ul>
  <li>write the data to s3.</li>
  <li>from s3 trigger lambda to break s3 file into sqs queue events of good size</li>
  <li>from sqs queue trigger lambdas to insert</li>
  <li>sqs can be fifo queue if it matters.</li>
  </ul>
  
Answers: CC BY-SA 4.0
Answers: 2018-12-11T06:49:27.947Z
Answers: 2018-12-11T06:49:27.947Z
Answers: jdog
Body: <p>I received a timeout when trying to perform 7000 * 3 inserts into MariaDB in 30 seconds with AWS Python Lambda. I have three ideas how to get around the 30 second API Gateway timeout:</p>
  
  <ol>
  <li><p>Convert the data to a file which MariaDB can import using <code>LOAD_DATA_INFILE</code> and let the Lambda do that. </p></li>
  <li><p>Write the data in a file to S3 and let another function write the data from S3 to MariaDB which then will not have the API Gateway timeout limit. </p></li>
  <li><p>Let the web client convert the file in JavaScript. </p></li>
  </ol>
  
  <p>I am leaning towards the first strategy, to try and use <code>LOAD_DATA_INFILE</code> from the Lambda function. Do you agree?</p>
  
Tags: <databases><aws-lambda>
Title: Strategies to work around 30 sec timeout
UserDisplayName: Niklas Rosencrantz
ThreadText: # Strategies to work around 30 sec timeout
  
  (Asked by: username_0 on 2018-12-11)
  
  I received a timeout when trying to perform 7000 \* 3 inserts into MariaDB in 30 seconds with AWS Python Lambda. I have three ideas how to get around the 30 second API Gateway timeout:
  
  1. Convert the data to a file which MariaDB can import using `LOAD_DATA_INFILE` and let the Lambda do that.
  2. Write the data in a file to S3 and let another function write the data from S3 to MariaDB which then will not have the API Gateway timeout limit.
  3. Let the web client convert the file in JavaScript.
  
  I am leaning towards the first strategy, to try and use `LOAD_DATA_INFILE` from the Lambda function. Do you agree?
  
  **Tags:** databases, aws-lambda
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-12-11. Score: 2 | Accepted answer
  
  A variation of method 2 is infinitely scalable:
  
  * write the data to s3.
  * from s3 trigger lambda to break s3 file into sqs queue events of good size
  * from sqs queue trigger lambdas to insert
  * sqs can be fifo queue if it matters.
  
  
  ---
  
  