Serverless Full-Text Search with AWS Lambda and EFS
For a long time, I was looking for a serverless full-text search solution. Of course, there are managed solutions for this, like Elastic Cloud or Amazon Elasticsearch Service. But all these solutions have a fixed cost and do not scale automatically.

I also saw a couple of attempts to run Apache Lucene with Lambda and S3, but these had a couple of issues. Firstly, Lucene is a Java library, and JVM had terrible cold start issues. Well, not anymore with GraalVM, which generates a native executable out of a JAR, reducing cold start and improving overall performance. Secondly, S3 is a bit too slow. Guess what? You can use EFS with a Lambda now, and it has much better latency.

So I thought to myself, why not build a simple proof of concept and see how it goes.

Approach

I used the Quarkus framework because of its small footprint and the ease of creating a native executable. This will help me solving the cold start.

Reading a Lucene index is thread and process safe. It means that read throughput is only limited by the AWS Lambda and EFS service limits. Writing, however, is a different story. An IndexWriter needs to obtain an exclusive lock on an index and will fail if another process is holding it.
['Arseny Yankovski']
['AWS', 'Scalability', 'Java', 'Open Source', 'Serverless']