## Caching Aided Multi-Tenant Serverless Computing

Chu Qiao University of Delaware qiaochu@udel.edu

Cong Wang Zhejiang University cwang85@zju.edu.cn

Zhenkai Zhang Clemson University zhenkai@clemson.edu

Yuede Ji University of Texas at Arlington yuede.ji@uta.edu

Abstract -One key to enabling high-performance serverless computing is to mitigate cold-starts. Current solutions utilize a warm pool to keep function alive: a warm-start can be analogous to a CPU cache-hit. However, modern cache has multiple hierarchies and the last-level cache is shared among cores, whereas the warm pool is limited to a single tenant for security concerns. Also, the warm pool keep-alive policy can be further optimized using cache replacement algorithms. In this paper, we borrow practical optimizations from caching, and design FaasCamp, a caching-aided multi-tenant serverless computing framework. FaasCamp extends the single-tier warm pool into multi-tiers, with a reclaim pool introduced enabling secure function instance sharing among tenants. Also, FaasCamp leverages machine learning to approximate the optimal cache replacement policy to improve the warm rate. We have implemented a prototype and conducted extensive experiments under multiple scenarios. The results show that FaasCamp can outperform existing platforms with minimal overhead.

## I. INTRODUCTION

Serverless computing, also known as Functions as a Service (FaaS), is an emerging cloud computing paradigm enabling cost-efficiency and elasticity for high-productive software development. Serverless computing decomposes computation into functions (e.g. small applications dedicated to specific tasks). It provides users with great scalability and flexibility by completely hiding the underlying server management, and thus tenants can focus on function development. It also allows service providers to efficiently manage hardware resources. To date, serverless computing has been adopted not only by major cloud platforms [1], [2], [3], but also in resource-limited environments (e.g., edge/mobile servers [4], [5], [6], [7]).

One challenge for high-performance serverless computing is to mitigate the effect of cold start, which refers to the process of initiating a new execution instance. Reducing cold start is essential for both tenants (e.g., user experiences) and vendors (e.g., resource management). While extensive research efforts (e.g., reducing startup latency [8], [9], [10] or using a lightweight sandbox [11], [12]) have been devoted to optimizing the cold start process, existing cold start start-up latency still ranges from hundreds of milliseconds to several seconds [13], whereas a typical serverless function only lasts 1 second [14]. To battle against cold start, one common approach deployed by major platforms is to maintain a warm pool [15], [16], [17]. After a function is executed, the underlying sandbox (e.g.,

Xing Gao University of Delaware xgao@udel.edu container) remains idle in the warm pool and can be reused for the same type of request later (i.e., a warm start). A warm start avoids building a sandbox from scratch, and thus can save a substantial amount of time. To further enhance warm pool efficiency, state-of-the-art approaches have treated warm pool as CPU caching: a warm function execution is equivalent to a cache hit, while a cold start is similar to a cache miss. Based on this insight, previous works such as FaasCache [15] have employed cache replacement algorithms to optimize the warm pool keep-alive policy.

While modern CPU caching utilizes a tiered memory system (e.g., L1/L2 and last-level cache), existing serverless computing approaches only maintain a single-tier warm pool for each individual tenant. Even for the same type of request, a sandbox (e.g., container) used by one tenant in the warm pool cannot be shared by another tenant. The reason is mainly for security concerns: even though most containers in serverless computing are stateless, intermediate data can still be stored inside. However, such a design may not perform well under multi-tenant serverless computing scenarios, especially in a resource limited environment (e.g., edge computing), where the server resources (e.g., memory) are inadequate. As the total size of the warm pool is limited, different tenants need to compete for spaces in the warm pool, significantly increasing the number of container evictions. A new user will almost certainly get a cold start even if the warm pool contains available containers. Unfortunately, most existing approaches are not optimized for resource limited environments. Meanwhile, existing cache replacement algorithms, including Least Recently Used (LRU), Least Frequently Used (LFU), and Greedy-Dual-Size-Frequency [15], still cannot achieve optimized performance for serverless computing.

In this paper, we present FaasCamp, a cache-aided framework for enabling high performance multi-tenant serverless computing. The main insight of FaasCamp is to borrow practical optimizations from the modern caching system. First, motivated by the hierarchy caching architecture, FaasCamp extends the existing single-tier warm pool to multiple layers, by adding a second tier pool (i.e., reclaim pool). Different from containers in the warm pool which are specific to tenants, containers in the reclaim pool can be shared for the same type of request among tenants. Specifically, the reclaim pool leverages the checkpoint and restore technique to enable secured container sharing: every time a new type of function is executed, FaasCamp makes a checkpoint at the beginning, and restores the container when moving it to the reclaim pool so that all intermediate data is erased.

Additionally, motivated by learning-based cache replacement optimizations [18], [19], [20], FaasCamp applies deep learning techniques to evict containers guided by the optimal B´ el´ ady's algorithm for CPU caching. We first develop a simulator to efficiently generate training traces, perform feature engineering, and train a machine learning model. A neural network based classifier is chosen to approximate B´ el´ ady's algorithm. Then, FaasCamp collects multiple system runtime statistics and percontainer information for each warm container. When multiple warm containers exist in the reclaim pool, FaasCamp feeds the collected information to a loaded deep learning model for inference, and selects the optimal one for eviction.

We have developed a prototype of FaasCamp on top of OpenWhisk. Specifically, FaasCamp employs a modular design with several key components incorporated. The reclaim pool extends the finite state machine of OpenWhisk so that containers are carefully provisioned among different pools. FaasCamp further develops a machine learning (ML) based invoker for container dispatching: a StateTracker component collects all desired features and transmits the corresponding state vectors to an independent ML module for inference.

We conduct extensive real-world trace-driven experiments on a small cluster simulating a resource limited environment, to evaluate the effectiveness of FaasCamp under different scenarios. The experimental results show that FaasCamp can outperform existing approaches (e.g., vanilla OpenWhisk and FaasCache). The reclaim pool can increase the warm ratio for multi-tenant scenarios, while the ML-guided invoker can make better container eviction decisions compared with multiple policies (i.e., LRU, LFU, and greedy-dual-size-frequency). Particularly, FaasCamp can significantly enhance the performance for mobile users, who utilize a provided service a few times and then leave. We also evaluate the overhead of FaasCamp by using a series of microbenchmarks and measuring overall system utilization. The results indicate that FaasCamp incurs minimal overhead.

## II. BACKGROUND

## A. Serverless Computing

Serverless Computing Platforms (SCPs) have largely taken the responsibilities of configuring and managing underlying system resources, compared with traditional cloud computing. Users only need to specify minimal configurations (e.g., desired memory size and programming language) and upload source code (i.e., function) to the platform. SCPs then deploy isolated and lightweight sandboxes to serve user requests and run each function. After the user code is executed, the sandbox is reclaimed/destroyed to free the resources. A function is usually short-lived: Microsoft Azure observes that 50% of functions execute for less than 1 second on average [14]. Thus, SCPs typically choose containers (e.g., Docker [21])

or lightweight virtual machine (e.g., Firecracker [11]), so that they can spawn a large number of sandboxes simultaneously. In this paper, we consider Linux containers as the underlying ephemeral execution sandbox, which can achieve near-native performance [22], [23], [24], [25].

Cold Start. Executing serverless functions with low latency is critical for user experience [26], [27], [28], [29]. Thus, container latency especially startup latency is one key metric to enhance performance. Cold start latency is the time to initialize a new container in response to the request for a function. Particularly, when a request arrives, SCP spawns a sandbox instance from a pre-built image with the desired language runtime, which further downloads the required packages and libraries, with user code injected. While extensive research has been devoted to reducing the cold startup time, the initialization process still takes non-negligible overhead, and can last in the order of seconds [8], [9], [10].

Warm Pool. One common approach for battling the cold start is to maintain a pool of warm containers [15], [16], [17], [13]. After a function is executed, the underlying container is paused in the warm pool for a period of time without being destroyed immediately. As a result, the warm container can be reused for the same type of request later. The duration for a container to stay in the warm pool is referred to as the keep-alive time. If no request (of the same type) arrives during the keep-alive period, the warm container is destroyed and the corresponding resources (e.g., memory) are released. Most existing SCPs adopt a fixed-time keep-alive policy: for example, AWS Lambda used to keep containers alive for 5 minutes [30]. Rather than having a fixed-time keep-alive policy, Shahrad et al. [14] proposed a hybrid policy to dynamically set the duration based on the histograms of intervals of consecutive function invocations.

Eviction Policy. The warm pool size is limited by the amount of allocated memory. If the warm pool is full and a container arrives, either the arriving container or an idle container from the warm pool needs to be evicted (i.e., destroyed). The container eviction can be analogous to CPU caching: a warm function execution is equivalent to a cache hit, while a cold start is similar to a cache miss. Most SCPs adopt the least recently used (LRU) or the least frequently used (LFU) policies as the warm container eviction policy. However, both are not versatile enough to handle different workload invocation patterns. FaasCache [15] then applies the greedy-dual-sizefrequency caching policy [31] to evict containers by considering multiple factors like object sizes and frequency.

OpenWhisk Architecture . Many open-source SCP frameworks (e.g., Apache OpenWhisk [32], OpenFaaS [33], and OpenLambda [34]) have been recently developed to facilitate serverless computing adoption on private clouds and dedicated clusters. Among them, OpenWhisk has been widely used in academia [15], [35], [36], [37], [38], [39] and industry (e.g., IBM Cloud Function [40]). OpenWhisk processes user requests with a logically centralized controller , which manages all entities. The invoker is in charge of executing the actions: it actively monitors the message pool in Kafka, and assigns workload to existing containers or spawns new containers

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-5e812831-a18c-4750-9ee5-9defa87d38b6.jpg)

Fig. 1: B´ el´dy's cache replacement policy example: Object B a with a reuse distance 4 should be evicted.

| Cache   | Cache   | Access Sequence   | Access Sequence   | Access Sequence   |    |    |
|---------|---------|-------------------|-------------------|-------------------|----|----|
| Line 1  | A       | Time              | 0                 | 1                 | 2  | 3  |
| Line 2  | B       |                   | D                 | C                 | A  | B  |

accordingly. By default, OpenWhisk sets the keep-alive time to 10 minutes with LRU as the eviction policy.

## B. CPU Caching

Modern computer systems utilize a tiered memory system to enhance their performance (e.g., increase cache hit ratio). The lower level cache (e.g., L1 and L2) is small with low access latency, but not shared among cores. Instead, the high level cache (e.g., L3) is typically slower, but shared by all cores in a CPU socket. Besides the multi-level caching architecture, the cache replacement algorithm is crucial to improve the hit ratio. While LRU and LFU are common cache replacement policies, extensive research has been conducted to optimize the cache replacement algorithm, from naive heuristicbased solutions [41], [42], [43] to sophisticated learning-based solutions [18], [19], [20], [44], [45].

The idea of many learning-based approaches is to train a machine (deep) learning model to learn the optimal policy, which is referred to as B´ el´dy's a algorithm [46]. Basically, B´ el´dy's a algorithm evicts objects based on the reuse distance, which is the time difference between the current and the next time when the same object is accessed again. The one with the furthest reuse distance will be evicted first. Figure 1 illustrates a simple example. Supposing the cache size is 2, the cache is fully occupied by objects A and B at time point 0. Thus, when object D arrives, the cache replacement policy needs to evict one object (e.g., A or B) in the cache. According to the access sequence, the reuse distances of A and B are 3 and 4, respectively, as shown in Figure 1. As a result, object B is evicted based on B´ el´ ady's algorithm.

B´ el´dy's a algorithm is infeasible in practice because it requires information from the future. Thus, recent works focus on developing cache replacement policies to approximate B´ el´dy's a algorithm using machine learning methods [20]. For example, Hawkeye [18] and Glider [19] build classifiers to predict whether a cache is cache-friendly or cache-averse, where cache-averse lines are evicted with high priority.

## C. Multi-Tenant Edge Computing

Edge computing has been used to offload workloads and reduce latency from a centralized computation approach in modern and next-generation networks (e.g., 5G) and IoT infrastructures. Unlike the cloud clusters, edge servers typically contain limited resources, but still need to provide multitenancy with proper isolation. One example is Vehicular Edge Computing (VEC) [47], [48], [49], [50], [51], [52], where cars need to offload computing tasks to edge clusters for accelerated computing. With thousands of different automobile manufacturers, even though the offloaded tasks might be similar

(e.g., functions with similar/identical packages or developed with same programming languages), they cannot be shared among tenants. Thus, as resource management is crucial in edge computing, both industry [53], [54] and academia [55], [56], [57], [58], [59] have conducted extensive researches to support multi-tenancy on edge servers.

## III. MOTIVATION

Existing approaches for maintaining a warm pool of containers works well in cloud computing environments with abundant computing resources. However, it is challenging to directly adopt it in multi-tenant resource-limit environments (e.g., edge computing). There are multiple reasons. First, for security and isolation considerations, a warm pool is separately maintained for each tenant [60], and warm containers cannot be shared among tenants. As warm container eviction can be analogous to CPU caching, this design is similar to low-level cache (e.g. L1/L2) which is not shared among cores.

The problem is that, in the edge environment with a high density of users (for example, 5G enables up to thousands of devices existing in one square kilometer), it is impossible to maintain this number of warm pools in edge servers with limited resources. Thus, in resource limited environments, container eviction happens frequently to make room for new requests. The performance will be significantly degraded with multiple tenants sharing the same server, as the container of one tenant can evict containers from other tenants. Second, a good eviction policy is important to keep the warm rate high. But there is still a non-negligible performance gap between the optimal B´ el´dy's a method and existing approaches (e.g., LRU, LFU, and Greedy-Dual [15]).

Preliminary Study Setup. We have conducted several tracedriven preliminary studies using OpenWhisk to demonstrate the above challenges when deploying a single-tier warm pool design in a multi-tenant scenario. We use the dataset from Azure Functions [14], which includes a real-world trace of serverless functions running in the Azure cloud for 14 days. In this trace, each user could deploy multiple apps, and each app may be composed of many functions. The Azure trace includes the invocation count for each function during each minute of a day (i.e., 1,440 minutes in total), without the detailed timestamp of function invocation. We thus assume all invocations are evenly distributed in the one minute period. A small time adjustment is added to guarantee there are no two requests arriving at the same time.

Following previous work [15], we select eight workloads from FunctionBench [61]. We create OpenWhisk actions (i.e., functions) for each workload with 256 MB memory and prebuilt docker image that includes all required packages. Note that the workload selection is not crucial for this preliminary study, and we expect any functions (e.g., different programming runtimes) will produce similar results. A detailed introduction of selected workloads are presented in Section V.

Multi-Tenant Performance Degradation. All tenants share the same warm pool and need to fight for memory spaces. With more tenants using the same server, the warm rate will

Number of Tenants

Fig. 2: Number of cold starts (dots) with respect to tenants.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-23f44c33-0147-4fd2-9ea1-c1b646ff8516.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-64f477bb-5500-4167-8083-3966861a6f45.jpg)

Fig. 3: Average Spike Interval.

be decreased due to the competition. We run an experiment using a fixed trace sequence (e.g., serialize multiple traces as mentioned above) with the same workload, but with different numbers of tenants. OpenWhisk by default sets a user-level namespace to each tenant, and containers are shareable within a namespace. By theory, since all containers are running the same workload, with only one tenant, the majority of cold starts should happen in the beginning when there are no or only a few warm containers. All following requests should be warm starts. However, with multiple tenants, warm containers from one tenant may be evicted by other tenants, and thus cold start happens.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-4b768fe1-4829-48c1-9cfa-c539dda31dfd.jpg)

45

Fig. 4: Performance Gap.

We define a spike as the time point when the number of invocations is at least twice as the average counts. Figure 3 shows the CDF of the average spike interval in minute. The result shows that 80% of the spikes have the interval larger than 5 minutes and at least 50% of the interval is larger than 15 minutes. It indicates that if the warm pool is not optimized for handling flashing events, many cold starts can potentially happen due to invocation spikes.

In particular, we evenly distribute 100 traces of the same workload to different numbers of tenants (i.e., 1, 5, 10, 15, 20). Each trace is sampled from the Azure dataset with 3,034 invocations in total. The experiment is conducted on our local Linux server (running Ubuntu 20.04) with 16 GB RAM and Intel i5-6600K 3.50 GHz CPU. OpenWhisk is allocated 2 GB of memory, and the workload is configured with a maximum of 256 MB. Thus, at most eight containers can exist at the same time.

Figure 2 shows the number of cold start with respect to time. Each colored dot in the figure represents a cold start and each line represents one scenario (e.g., different numbers of tenants). For a better visualization, we omit some warm starts in sparse regions to shorten the time axis. In all scenarios, many cold starts happen in the beginning, indicating that the system keeps provisioning containers. After the initial spike, the number of cold starts decreases as there are available containers in the warm pool. Overall, with more tenants, more cold starts happen as the system needs to provision new containers due to container evictions. The situation will become worse with more types of workload involved.

Invocation Spikes. Invocation spikes will further reduce the warm rate in a multi-tenant resource-limited scenario. In serverless computing, user requests may not arrive evenly. Instead, the bursty invocation pattern (e.g., flash events) is quite common in the wild [13]. It is entirely possible that one tenant receives many requests within a short period of time, and becomes idle then. In this case, the warm pool will be swarmed by containers of one tenant (caused by invocation spikes), and containers of other tenants will be evicted. As a result, all the following requests require a cold start.

We analyze the spike pattern using the Azure dataset [14].

Performance Gap. With the cache analogy, algorithms from caching can be customized to enhance warm rate in serverless computing. The optimal solution, B´ el´dy's a algorithm, still outperforms existing methods (e.g., LRU, LFU, Greedy-Dual from FaasCache [15]) with a non-negligible performance gap. We conduct an experiment to demonstrate this gap. we use OpenWhisk on commit 3e5b8b2, which utilizes LRU as the default policy. We implement a LFU version on top of OpenWhisk. We also run FaasCache (available through GitHub). The warm rate of B´ el´ ady's algorithm can be calculated with a serialized trace sequence. The performance of B´ el´dy's a algorithm depends on how far it can look into the future: looking ahead to a large number of requests may actually diminish benefit returns [18]. Thus, we empirically choose 30 as the window size to look ahead. When calculating, we consider the running time of containers: a busy container will not be considered for eviction. We also drop incoming requests if all containers are busy and treat them as cold starts. Using the exact same trace on different algorithms, Figure 4 clearly indicates that there is still a performance gap between existing approaches and B´ el´dy's algorithm. We should mention that the a real gap should be even larger, as many requests are dropped by us and considered as cold starts.

## IV. SYSTEM OVERVIEW

This section presents the overview of FaasCamp. The main insight of FaasCamp is to borrow practical optimizations from the caching system. Particularly, similar to the tiered caching architecture used by modern computer systems, FaasCamp extends existing single-tier warm pool to multiple layers, by adding a second tier pool (i.e., reclaim pool) enabling secure container sharing among tenants. Additionally, motivated by learning-based cache replacement optimizations, FaasCamp applies deep learning techniques to evict container guided by the optimal B´ el´dy's algorithm. a

Figure 5 illustrates the high-level architecture of FaasCamp, which adopts a modular design with several key components.

Fig. 5: FaasCamp System Overview.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-5c28252c-6234-4fa0-b133-9a0b201d2278.jpg)

Reclaim Pool is the second tier pool that extends the finite state machine (FSM) of vanilla OpenWhisk so that containers in the reclaim pool can be reused by the same type of request from different tenants. The StateTracker component collects multiple system runtime statistics, and per-container information for each warm container. Both are fed into a deep-learning based invoker. When multiple warm containers (i.e., more than one) exist in Reclaim Pool, FaasCamp invoker calculates a score for each container instance, and selects the optimal one for eviction. The detailed machine learning model is loaded in a separate container and communicates with the invoker, so that the model can be updated without interfering with the system. To facilitate the training process, we have developed a simulator to generate training traces and conduct the feature engineering job. While FaasCamp is built on top of OpenWhisk, the idea can be easily applied to other serverless computing platforms.

## A. Reclaim Pool Design

FaasCamp extends the existing warm pool into a multi-tier architecture, by adding a reclaim pool where containers can be shared among tenants. The idea is similar to the last-level cache (LLC), which is shared among all cores. However, we need to take multiple factors into consideration. The reason that the warm pool is limited to a single tenant is for security concerns [60]. Even though most containers in serverless computing are stateless, intermediate data can still be stored inside containers [62]. Thus, to enable sharing containers among tenants, it is critical to ensure that the container has been safely cleaned: a previously executed function cannot affect future functions, and the currently running function should not infer previous functions.

The overall idea of the reclaim pool is to extend the FSM of Openwhisk by employing the checkpoint-restore technique for ensuring security. Every time a new type of function (which needs a new container image) is executed, FaasCamp makes a checkpoint at the beginning. This checkpoint ensures that the container is stateless at this point regardless of whether the function is stateful or stateless. Specifically, the checkpoint procedure will run in a separate process to reduce system overhead and function waiting time. Plus, FaasCamp only needs to create checkpoints for new (e.g., checkpoint does not exist) and unique workloads (e.g., requires specific extra

TABLE I: State Vector Features.

| Category   | Feature                                                                                         | Description                                                                                                                                                                                                                                                                                          |
|------------|-------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| System     | Current request PII 1, 2 Past 200                                                               | The workload ID of the current request The intervals between past 2 requests The history of past 200 requests                                                                                                                                                                                        |
| Container  | Workload ID Idle time rank Frequency Frequency rank Alive count Warm count Past { 10, 50, 100 } | The ID of the workload The ranking of idle time among all containers Total number of container being accessed The ranking of freq. among all containers # of requests passed since the first invocation # of requests passed since the prev. invocation # of invocation of past 10, 50, 100 requests |

libraries in containers). Thus, the total number of checkpoints stored in a server is limited.

After executing the function, the container will be moved to the warm pool as usual. If the container has been idle for a period of time in the warm pool (e.g., 5 minutes), FaasCamp moves this container into the reclaim pool by two operations. It first removes the ownership (e.g., tenant information) of this container. Then the container will be restored to the previous checkpoint so that all intermediate data is erased. For a new incoming request, FaasCamp will first check the warm pool, and then the shared reclaim pool for warm instances. If a warm container exists in the shared reclaim pool, FaasCamp adds user-specific ownership to the container.

FaasCamp does not reserve any memory space for the reclaim pool. Instead, administrators can set the upper limit (e.g., memory size) for the number of containers in the reclaim pool. For example, the size of the reclaim pool can change dynamically from 0 to 25% of the total memory allocated to OpenWhisk. If the system supports at most 16 containers, at most 4 containers can be placed in the reclaim pool. The specific implementation details (e.g., extending container FSM in Openwhisk) are presented in Section VI.

## B. Machine Learning Based Invoker

While machine learning has been used to optimize the cache replacement policy, the models were never deployed inside real CPUs due to the obviously high overhead and resource restriction. Instead, with a similar design principle as cache, the container eviction policy serves as an ideal place for implementing learning-based optimizations, as serving serverless requests are much more resilient (and slow) than object caching in hardware.

While the idea is simple: developing a machine (deep) learning model to approximate the optimal B´ el´ ady's algorithm, FaasCamp still needs to address several challenges. First, the performance of machine learning models heavily depends on the quality and size of training data, as well as the selection of features. The feature engineering work is tedious, requiring many repetitive attempts to try different combinations. It is impractical to run workloads in a real server for months just to collect training data and extract corresponding features for model training. Thus, FaasCamp first adopts a similar approach as FaasCache by implementing a trace-driven discrete event simulator for feature engineering and training data generation.

Based on the simulation, FaasCamp implements StateTracker to collect various system information (e.g., request history) and a wide-spectrum of container-dependent states (e.g., invocation frequency and idle counts). StateTracker constructs state vectors for each container, and transmits them to a machine learning server. A TensorFlow model performs inferences on all state vectors, and decides the container to evict. The machine learning model runs in a separate container, and delivers the decision to the invoker, which is responsible for finally evicting the target container. Such a design enables FaasCamp to update the machine learning model online without re-compiling the invoker. For example, combining the simulator, FaasCamp can easily re-train the model based on recent request patterns (e.g., request distributions).

## V. MACHINE LEARNING MODEL DESIGN

This section presents the detailed design of our machine learning model. We first implement a trace-driven simulator to emulate the warm pool activities. The simulator uses two lists to represent a busy pool of containers and a free (warm) pool. The simulator supports different numbers of containers in the warm pool, and various workloads with configured running time and cold start latency. Given an incoming request from traces, the simulator calculates the end running time based on arrival time, execution duration, and cold start time. Multiple eviction policies (e.g. LRU, LFU, FaasCache, our machine learning model) can run in our simulator, which further decides whether to evict an idle container in the warm pool or discard the request. With a fixed sequence of traces, we can also obtain the optimal choice from B´ el´ ady's algorithm. The simulator is written in Python with around 700 LoC, and significantly shortens the time to run the whole sequence in a real server. Note that the simulator is not intended to precisely replicate OpenWhisk. Instead, the simulator enables us to quickly generate training samples for selecting features and comparing different ML algorithms and models. Therefore, algorithms (e.g., FaasCache, LRU, and LFU) might perform differently between the simulation and the actual system.

Setup . We select eight workloads from FunctionBench [61], including basic computation operations (e.g., float operation), complex mathematics calculations (e.g., linear equation solving, matrix multiplication, and AES encryption), machine learning workloads (e.g., model training, model serving), and real-world applications (e.g., HTML rendering, image processing). We use the same method described in Section III to generate traces. As the Azure dataset includes a significant amount of data from 14 consecutive days, we randomly select a portion of data (roughly 4,400 traces) from the first day as the training data. Particularly, some traces contain overmuch invocations (e.g., thousands per minute) causing the training process inefficient. Meanwhile, if we select multiple traces with very few invocations (e.g., less than 10 per day), the generated data might not be enough for model training. Thus, we mark these traces as outliers, and conduct experiments without them (i.e., selecting traces from 10 to 10k).

Our plan is to deploy the ML based invoker on the reclaim pool, which is simply a portion of the entire warm pool. Thus the size is limited in a resource-limited server. Each time, we randomly feed 40 traces into our simulator, with state vectors (e.g., features) and labels (based on B´ el´ ady's algorithm with 30 requests into the future) generated. We only use day 1's data for training. State vectors and labels are input-output pairs to train the machine learning model. We randomly generate 100 sets (each set contains 40 traces) as our training data. Because the trace length varies in a large range, even with a fixed number of traces (i.e., 40), the input-output training pairs for each set still significantly differ from each other. Note that these 100 sets of data are only used for model training, and we generate new data for model evaluation (as described later).

## A. Eviction Model Details

This subsection presents the details of FaasCamp's ML architecture, with its choice and rationale described. Our exploration might not be the optimal choice as it can change in different scenarios. Particularly, we use the simulation results to show that these features have the potential to outperform traditional non-learning based methods. The detailed evaluation is conducted on real servers in Section VII.

## 1) Features.

As Table I, FaasCamp uses two different types of features: one set describing the overall state information of the warm pool (i.e., system state vector), and another set describing the information of particular workload (i.e., container state vector). System state vector provides the historical information for the server (e.g., past 200 requests). The rationale is that request invocations might exhibit similar patterns in a short amount of time. The past information can serve as an estimate of the future invocation pattern, which is learned by B´ el´dy's algorithm. For a each type of workload, we also record past invocation intervals (PIIs), indicating the number of requests between the two requests on the same workload. As the eviction happens when a new request arrives, the interval is described as the number of requests, rather than a period of time. Specifically, we track two most recent intervals (e.g., PII1 and PII2) for each type of workload. For example, if the current request is for workload A, PII1 is the number of fulfilled requests since A was served last time, and PII2 is the number of requests before A was served last time.

Container state vector contains per-container information required to make a replacement decision. Multiple classes of features are extracted for each container, such as the current access information (e.g., workload type), the frequency information of each type of container, and the amount of idle time stayed in the warm pool. We also collect the ranking information (e.g., frequency rank) as features. In addition to the overall historical information in the system state vector, we also track the number of historical invocation for each type of container. Statistics are updated on every request: the container's alive count is reset to zero if it is newly created, and is incremented on every incoming request to the system.

Fig. 6: Simulation Results.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-49766249-6090-4389-8cc2-f1feb5642f31.jpg)

## 2) ML Architecture.

When designing the ML architecture, one of the challenges is to select a model that is general to different configurations. For example, the model should have competitive performance with different memory sizes of the warm pool. Also, the number of available containers is dynamic in the warm pool: we cannot evict a busy container while it is serving a request. In both cases, the varying number of containers results in different lengths of inputs, which contradicts a multi-class neural network classifier with a fixed-length input. We have explored multiple choices, including deep neural networks and convolutional neural networks with zero padding, as well as recurrent neural networks for time series classification, but none of them are satisfactory.

For example, in a realistic scenario, after deploying the system for one day, we can update our model based on the collected historical data. To simulate this scenario, we adopt the following method to generate testing datasets: we keep day 1's newly generated testing traces unchanged, and further extract data of the same traces from the following days (e.g., days 2-8) for evaluation. In some traces, requests might disappear in the next day. This is quite normal as one tenant might leave the service on the other day. Thus, we adopt two strategies on these traces: (1) we ignore the missing requests; (2) we randomly add more traces to the evaluation data to make them 40.

Finally, we perform a prediction for each available container instance to decide whether it is warm-friendly or warm-averse. Warm-friendly indicates that this type of container should be kept in the warm pool with high priority, and warm-averse container is with low priority. Particularly, we train a neural network with 5 hidden layers and 256 nodes in each layer, using the binary cross-entropy loss function. For each container, the model utilizes the overall system state vector and the corresponding container state vector, and outputs a probability that this container should be evicted. Then, the container with the highest probability is evicted.

## B. Model Performance

We evaluate our model (via the simulator) using newly generated datasets. We randomly generate another 20 sets of traces (each set contains 40 Azure traces), ensuring that the testing data is unseen by the model. Particularly, we randomly generate such testing data for eight days (i.e., days 1-8) independently. This allows us to test the feature generality, as the model is trained using only day 1's data. Figure 6(a) shows the results of our machine learning model compared with LRU, LFU, and FaasCache. Our machine learning model outperforms them on day 1's data. This is expected as our model is also trained with day 1's data. Also, our model can achieve competitive (even the best) performance in the following days, indicating the potential and effectiveness of our selected features.

Model Updating. One advantage of our design is that the model can be updated frequently based on recent request patterns.

We further adopt two model retraining strategies. First, when evaluating the performance of the next day (e.g., 3rd day), we retrain the model using the data of the previous day (e.g., 2nd day) in the testing trace. Second, to avoid the catastrophic forgetting phenomenon (the retrained model performs poorly on the old data because the new data may have a different distribution), we mix the data from the base training traces (i.e., from day 1) and day 2 testing traces to retrain and evaluate the new model with data from day 3. The same approach also applies to other days.

In both cases, model retraining can outperform all previous methods (i.e., LRU, LFU, FaasCache). With a modular design, FaasCamp allows administrators to update the machine learning model for achieving satisfactory performance.

## VI. IMPLEMENTATION DETAILS

This section describes some non-trivial implementation details for integrating FaasCamp on top of OpenWhisk. The major modification is to develop our own invoker to dispatch workloads and manage the life-cycles of containers.

## A. Reclaim Pool

In the vanilla OpenWhisk, the invoker manages containers through ContainerPool and ContainProxy . ContainerPool oversees two pools, busyPool (contains busy containers) and freePool (i.e., warm pool). ContainProxy is a FSM created for each newly spawned container. The state changes when the corresponding container moves to another pool. FaasCamp extends the FSM transition and the logic of ContainerPool to support the second tier pool (Reclaim Pool).

Checkpoint and Restore Optimization. We adopt the CRIU [63] project for container checkpoint/restore functionality.

Fig. 7: Container Finite State Machine.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-efde9f8a-6fa4-40b0-8442-65b95aee771b.jpg)

CRIU creates checkpoints by gathering information of the container processes from the /proc file system, and restores the processes through the fork() system call with previously collected data. The data created after the checkpoint will be removed when the container is restored, and thus the container can be safely used by a new user.

When a new container is created, we add another state, Checkpoint , in the ContainProxy (i.e., FSM). The detailed FSM is illustrated in Figure 7, where states marked with the grey color are added by FaasCamp. If a checkpoint is needed (e.g., no previous checkpoint exists for this type of workload), FaasCamp builds a checkpoint by spawning an extra container in another thread, while the original one follows the procedure in the FSM and is provided to the user. The extra container is not counted towards the container limit in the OpenWhisk container pool, and will be removed after the checkpoint is successfully created. This method, although consumes extra memory, allows us to cause minimal impact on users. Also, the number of checkpoints is determined by the number of unique workloads, thus generating them is often a one-time effort.

Containers are restored when they are moved from the free pool to the reclaim pool. Previously the container enters into a Paused state for staying in the warm pool, where all processes of this container are suspended. We add an extra state PausedRP for containers staying at the reclaim pool. The restore process takes place when the state transitions from Paused to PausedRP . This process requires to shutdown the container first, and then restore/restart the container. Although the restore process causes a cold start, this cold start is hidden from users (as it is not happened for serving a request).

Namespace Handling. OpenWhisk add a user-level namespace to all containers ensuring that warm containers are isolated among tenants. When moving a container between pools, FaasCamp also modifies the namespace. Specifically, we add a namespace for the reclaim pool, and label containers with this namespace when they enter into the reclaim pool, indicating the container is free to all tenants. The namespace is set to the specific tenant if the container is taken to serve new a request (i.e., becomes busy). Compared with using containers in the warm pool, the namespace operation is the only overhead added for containers in the reclaim pool. This overhead is negligible as we evaluate it in Section VII.

Container Provision Algorithm. One challenge for implementing the reclaim pool is to ensure that the total number of containers in all pools does not exceed the memory limit in

## Algorithm 1 Container Provision

- 1: if WP container &gt; 0 or RP container &gt; 0 then

2:

Assign a free container to the new request

3:

4:

5:

6:

7:

8:

9:

10:

11:

12:

13:

14:

15:

16:

17:

18:

19:

else maxFreeSpace = maxContainerCount - busyPool.size

if maxFreeSpace == 0

then

Send the current request to buffer

## else

freeSpace = maxFreeSpace - WP.size - RP.size if

freeSpace

&gt;

0

then createContainer()

else if

WP.size

&gt;

0

if then

RP

&gt;

0

then

Remove a container from RP

createContainer()

else

Remove a container from WP

createContainer()

else if

WP.size == 0

then

Remove a container from RP

20:

createContainer()

the whole time. These include busy containers in the busy pool and idle (warmed) containers in both warm and reclaim pools. The invoker needs to carefully handle the interaction (e.g., move a container) among different pools when provisioning containers and finishing workloads. Algorithm 1 presents the details of how FaasCamp reacts to a new request. The invoker searches idle containers in the warm pool and reclaim pool (Lines 1-2). If there is no available container, our invoker needs to create a new container if there is still space. First, the invoker checks if the memory space is entirely occupied by running containers. If the number of running containers is already reached the maximum limit, no memory is available for serving the current request. Thus, this request will be buffered in a queue for storing unfulfilled requests (Lines 5-6). Next, the invoker checks whether all free spaces are occupied by warm containers (in both warm and reclaim pools). if not, a new container can be directly created (Lines 9-10). Otherwise, FaasCamp needs to evict an idle container to make space for serving the current request. FaasCamp prioritizes reserving space in the warm pool, so the invoker will first check and evict idle containers in the reclaim pool (Lines 12-14 and 18-20). Instead, if all idle containers are in the warm pool (Lines 15-17), a warm pool container will be destroyed. When a container exits the busy pool (e.g., finish serving a request), this container will be placed in the warm pool (only belongs to the corresponding tenant). Meanwhile, if the warm pool is already full, we will select and move a container from the warm pool to the reclaim pool.

## B. ML Module

The high-level idea of our ML module has been introduced in Section IV. We add a StateTracker in the invoker to collect all desired features and transmit the corresponding state vectors to an independent ML server for inference. The invoker then evicts one container based on the results from the ML server. Currently, we only evict containers using machine learning in

TABLE II: Reclaim Pool Performance.

| System                                                           | Avg. Warm Rate (win counts)            | Avg. Warm Rate (win counts)                                       | Avg. Warm Rate (win counts)                         |
|------------------------------------------------------------------|----------------------------------------|-------------------------------------------------------------------|-----------------------------------------------------|
|                                                                  | S1                                     | S2                                                                | S3                                                  |
| OpenWhisk (32-0) FaasCamp (24-8) FaasCamp (26-6) FaasCamp (28-4) | 85.43 87.92 (15) 86.42 (13) 85.85 (11) | 10.25 36.12 (15) 27.27 (15) 18.36 (15) 9.89 39.92 (15) 29.91 (15) | 81.17 85.28 (15) 82.98 81.88 (11) 77.41 82.14 78.42 |
|                                                                  |                                        |                                                                   | (13)                                                |
| OpenWhisk(24-0)                                                  | 78.8                                   |                                                                   |                                                     |
| FaasCamp (16-8)                                                  | 82.7 (15)                              |                                                                   | (14)                                                |
| FaasCamp (18-6)                                                  | 80.95 (15)                             |                                                                   | (13)                                                |
| FaasCamp (20-4)                                                  | 79.65 (11)                             | 18.1 (15)                                                         | 77.63 (10)                                          |

Fig. 8: Eviction Policy Comparison.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-6adc9690-5004-4cbb-834d-4def6fc3272e.jpg)

the reclaim pool, while it can be easily extended into the warm pool as well.

StateTracker. The StateTracker contains two components, SystemTracker and ContainerTracker . SystemTracker is initialized with the invoker and tracks all invocation history. It also manages and triggers ContainerTrackers to update statistical data (e.g. increment idle count). For each newly spawned container, a ContainerTracker is created to track all container features. Each container tracker will be removed when the corresponding container is destroyed.

SystemTracker is responsible for generating state vectors when a container needs to be evicted. It constructs state vectors by gathering information from each ContainerTracker and calculating PIIs using the recorded invocation history.

ML Server . The ML model is trained using TensorFlow with Python in the simulator. OpenWhisk is developed with Scala, which is not officially supported by TensorFlow, so we cannot directly integrate the ML model in the invoker. Thus, we seek a modular design by creating an independent ML server in a separate container. While multiple methods (e.g., shared memory) can be used for information exchange among two containers, currently we utilize network sockets to transfer state vectors and model decisions. The overhead of data transfer is not the bottleneck (evaluated in Section VII). The model computes a score for each idle container, and returns the one with the highest score for eviction. In this way, for a cluster with multiple nodes, each node can deploy its own ML server for independent inferences, or rely on a centralized ML server by transmitting state vectors.

## VII. EVALUATION

## A. Experimental Setup

Real-world serverless computing typically contains a widerange number of requests from different users, from single digit to hundreds of thousands per day. For example, in the Azure trace, 47% of users only have 1 function deployed, and over 60% of the functions are only invoked once per day. We thus consider two types of users to emulate real-world scenarios. The first type is regular users who deploy their applications on the server with frequent invocations. The second type is mobile users . Such users might visit this location, utilize the provided services (e.g., from an edge server) a few times and then leave. Based on the characteristics of both users, we divide the dataset

Fig. 9: FaasCamp Overall Performance (circles indicate cold starts).

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-ba7a323e-eab5-424f-912e-8bf7492fd56c.jpg)

into two sections: mobile users are assigned with traces that have less than 100 invocations, and the rest are regular users. We further consider three scenarios:

- · Regular User Only (scenario 1): all tenants are regular users who might run multiple different applications. So each tenant has multiple traces assigned.
- · Mobile User Only (scenario 2): all tenants are mobile users, and each tenant has only one trace assigned.
- · Mixed (scenario 3): Both mobile and regular users exist with different ratios (e.g., 1:1 or 3:1).

To reduce the impact of trace selection (the performance is not tied to specific traces), we first conduct each experiment with different traces (randomly generated) multiple times but over short periods. Particularly, for evaluating reclaim pool (§ VII-B) and ML-based invoker (§ VII-C), we randomly select a short period (10 ∼ 15 minutes), while keeping a wide range of invocation counts. Sampling mobile users might cause 0 invocation in our randomly selected period. In this case, one invocation is randomly added. We also conduct a long-term trace-driven experiment to evaluate the performance (§ VII-D) and overhead (§ VII-E) of FaasCamp on a single server. Finally, we evaluate FaasCamp in a small cluster (§ VII-F).

## B. Effectiveness of Reclaim Pool

We first compare the warm rate of the reclaim pool against vanilla OpenWhisk, with the default LRU as the eviction policy for both systems. With each container's memory limited in 256MB, we conduct experiments with two setups: (1) 6G maximum memory with 24 containers and (2) 8G maximum memory with 32 containers in total. These setups ensure a reasonable representation of a server with limited resources. For each setup, we further test two different configurations by adjusting the memory limit of the reclaim pool. For example, for the 8G setup, we set idle containers limit in the reclaim pool to 4 or 8. The details of our configuration are presented in Table II: FaasCamp (24-8) indicates that at most 24 and 8 containers can exist in the warm pool and reclaim pool, respectively. For each setup, we conduct 15 experiments, with 360 experiments performed in total.

Table II shows the results: FaasCamp robustly outperforms vanilla OpenWhisk on almost all experiments. Particularly, the reclaim pool can greatly increase the warm start ratio for mobile users (scenarios 2 and 3). In some cases, the increase can reach 4x better. The reason is obvious: with less invocations, mobile

Fig. 10: Microbenchmarks and Overhead.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-cf1b1887-e79d-4436-b766-7523ed7acc21.jpg)

users will almost certainly get cold start, while the reclaim pool enables them to use idle containers from other tenants.

## C. ML Eviction Accuracy

Next, we compare the warm rate of our ML-based invoker with other eviction algorithms (i.e., LRU, LFU, and FaasCache). To reduce the impact of tenants, we distribute all traces to only one tenant, so that containers will not be evicted due to the impact of cross-tenant. We conduct experiments 30 times for each setup (360 experiments in total) using random traces from the regular users. As FaasCamp only adopts the ML eviction in the reclaim pool, we evaluate different maximum container counts (i.e., 4, 6, 8), which are the sizes evaluated in § VII-B. Note that the machine learning model is trained with 8 maximum containers. Also, the reclaim pool is not enabled. Figure 8 shows the results, and the bar represents the average warm rate of all experiments. Our ML model clearly outperforms all others policies in the 8 container case (e.g., 11% better than FaasCache). Meanwhile, even for other cases, the model still exhibits competitive performance (e.g., the best one for the 4 container case). The result shows the generality of our model and selected features. We should emphasize that, system administrators can always train the model based on their setups (e.g., number of maximum containers) for best performance.

## D. Overall Performance

In this experiment, we enable both modules (e.g., reclaim pool and ML module) and evaluate the overall performance using a long trace. We use the same server mentioned in Section III with a Nvidia 3080 GPU. Specifically, we compare FaasCamp with LRU, LFU, FaasCache using a 6-hour traces distributed to 50 mobile users and 20 regular users (i.e., scenario 3). The same set of traces are fed into four systems running on the same hardware testbed independently for evaluation. We set the maximum of containers to 32, and FaasCamp is configured with 24-8 (warm pool - reclaim pool) setup. For other systems, we keep the default keep-alive period (i.e., 10 minutes). For FaasCamp, we set keep-alive policy as 5 minutes for both warm pool and reclaim pool. During the evaluation, we also measure the CPU and memory utilization (the results are presented in the following subsection).

Figure 9 illustrates the number of cold start for each system. We calculate the number of cold start in every 20 minutes, and draw circles with the radius proportional to the number of

Fig. 11: System Utilization.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-53acf6b2-e719-4c81-a972-3378eb5274ca.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00957v1-figure-1fdd0370-3391-41d7-a217-8615d070bded.jpg)

cold start. Basically, the larger circles are, the more cold starts occur in a 20-minute period. All four systems have many cold starts in first 20 minutes because the warm pool is empty at the beginning. However, FaasCamp has much better performance with much less cold start over time (i.e., the circle size is smaller than others). Indeed, the number of cold start is much lower than other systems (e.g., 364 for FaasCamp versus 566 for FaasCache).

## E. Implementation Overhead

Micro-benchmarks. The two modules of FaasCamp, reclaim pool and ML module, both introduces extra operations. We employ a series of micro-benchmarks to measure the time consumption of multiple operations, including (1) cold-start container + make a checkpoint; (2) cold-start container without creating checkpoint; (3) move a container from the warm pool to reclaim pool (with the restore operation); (4) use a container from the reclaim pool (add namespace information); (5) use a container from the warm pool; (6) State vector creation for one container; (7) ML inference per state vector; and (8) network transmission for eviction results.

We measure the timing information using the model training workload for 200 times, as this is the most complicated workload from FunctionBench. Figure 10a shows the duration of all operations involved with the reclaim pool, presented in milliseconds. There is almost no difference between a warm start from the warm pool and a warm start from the reclaim pool. This is expected as the only overhead is adding namespace (tenant) information to containers, which is almost instantaneous. Cold start a new image with a new checkpoint created consumes slightly more time than a normal cold start. Note that this is not the time for creating a checkpoint, which is done by a separate thread (thus has no impact on users). As the overhead is only in millisecond level, we believe this overhead is acceptable. Finally, the restore operation when moving a container from the warm pool to a reclaim pool definitely adds extra overhead (430 ms on average). However, this operation is transparent to users and thus will not affect user experiences.

The timing measurements of machine learning related operations are presented in Figure 10b. All individual operations, (e.g., generating the state vector for each container, ML inference for each case, and the network transmission) take negligible overhead (less than 10 ms). But the total time for making an eviction decision is decided by the number of

TABLE III: The average warm rate, response time, and queue size of the different systems in the multi-invoker evaluations. High warm rate and low response time, queue size indicates high throughput.

| System    | Avg. Warm Rate   | Avg. Warm Rate   | Avg. Warm Rate   | Avg. Response Time (s)   | Avg. Response Time (s)   | Avg. Response Time (s)   | Avg. Queue Size   | Avg. Queue Size   | Avg. Queue Size   |
|-----------|------------------|------------------|------------------|--------------------------|--------------------------|--------------------------|-------------------|-------------------|-------------------|
|           | S1               | S2               | S3               | S1                       | S2                       | S3                       | S1                | S2                | S3                |
| LRU       | 86.55            | 52.07            | 73.36            | 4.86                     | 10.69                    | 10.4                     | 4.87              | 5.73              | 9.99              |
| LFU       | 78.52            | 47.33            | 66.53            | 11.27                    | 11.59                    | 16.02                    | 10.32             | 6.27              | 17.04             |
| FaasCache | 73.1             | 46.83            | 59.47            | 14.36                    | 12.6                     | 23.02                    | 13.46             | 7.68              | 25.7              |
| FaasCamp  | 94.85            | 70.32            | 91.13            | 0.59                     | 7.97                     | 1.17                     | 0                 | 3.58              | 0.22              |

available containers. In our experiments, for a maximum of 8 containers, the average is only 38.63ms.

## VIII. RELATED WORK

System Overhead. We also compare the system overhead (e.g., CPU and memory consumption) of FaasCamp with other systems (e.g., vanilla Openwhisk and FaasCache). The CPU and memory utilization is collected through a system monitor sar from the package sysstat [64]. Figures 11a and 11b demonstrate the CPU load and the memory consumption for running the 6-hour trace. While the system utilization changes based on the number of concurrent requests, there is almost no difference for the CPU utilization among four systems. Also, FaasCamp shows a similar memory usage pattern as others, with the average memory consumption less than 1% difference compared with LRU and LFU. The results indicate that the overhead of FaasCamp is acceptable.

## F. Multi-Invoker Performance

Finally, we evaluate our system in a multi-invoker cluster setup, with three physical servers included. Each server runs one invoker on Ubuntu 20.04 with 16GB of memory. Among them, one is the main server hosting the controller and dispatching incoming requests to invokers based on the default load balancing strategy of OpenWhisk. Similarly, we conduct 15 experiments with randomly selected traces for each scenario (as mentioned in § VII-A) per system. Each invoker is capped with 32 containers, and we choose 24-8 as the configuration for FaasCamp.

In realistic scenarios, flash events regularly occur, generating bursty incoming request patterns that are beyond the cluster's capacity. Requests are then queued, incurring queuing delays. To demonstrate that our system can achieve high throughput, we use three different metrics: (1) Warm rate; (2) Request response time; and (3) System queue size. Particularly, for each request, the response time includes three components: the initialization (cold start) overhead, the wait time in the queue (i.e., queuing delay), and the execution duration. The wait time represents the interval between when a request is received by the controller and when a container is provisioned by an invoker. A shorter response time indicates that the system is more efficient in provisioning containers, highlighting the effectiveness of the reclaim pool.

The system queue size (i.e., the number of queued requests) is another indicator of throughput: a smaller queue size suggests that the system can handle more requests within a given time frame. Table III presents the results, demonstrating that FaasCamp outperforms other systems in all scenarios.

Cold Start Optimization. Extensive research has been conducted to optimize serverless computing [65], [66], [67], [68], [69], [70], particularly the cold start process [71], [11], [72], [12], [15], [10]. SOCK [9] and Catalyzer [8] build template images to shorten the package download and install time. Firecracker [11] and FaaSnap [12] utilize lightweight VMs for improving function execution. Our work can be combined with them to further improve the sandbox warm ratio.

Another common approach is to reduce the sandbox provisioning frequency [15], [14], [73]. Romero et al. [74] developed a transparent auto-scaling distributed cache: the cache is pre-warmed following data access patterns on Azure. IceBreaker [70] dynamically warm up a function based on the function's time-varying probability of the next invocation. Unlike them, we adopt multiple optimizations from the cache to enhance serverless computing performance.

ML in Serverless Computing. ML has also be applied in optimizing serverless computing [75], [76], [36], [77], [35], [78]. For example, neural networks have been utilized to predict request so that a container can be pre-warmed to avoid cold start [35], [78]. Others use reinforcement learning to adjust system parameters to maximum resource utilization [75], [76], [36], [77]. Unlike these prior efforts, FaasCamp develop a framework enabling ML-based container eviction, and any deep learning models can be integrated into our system.

Approximating B´ el´ ady's Algorithm. Many works attempt to approximate B´ el´ ady's algorithm using machine learning techniques in both the cache design and other computing scenarios. Both Hawkeye [18] and Glider [19] construct neural networks using program counters as input features for cache replacement. RLR [20] adopts reinforcement learning with neural network using time and frequency of cache lines. B´ el´dy's algorithm has also been utilized in guiding caching in a the content distribution network (CDN) [79], [80], [81]. Song et al. [81] developed Learning Relaxed Belady to mimic a Relaxed Belady algorithm in CDN cache design. All these prior works indicate that approximating B´ el´ ady's algorithm is possible and feasible, and further motivate our works.

Serverless Edge Computing. Serverless computing has been combined with edge computing in past years [82], [6], [83], [4]. Mittal et al. [84] adapted Knative [85] to edge devices with limited computing resources, and Cicconetti et al. [86] proposed a decentralized architecture to manage edge computing nodes to mitigate network congestion. Our work can be integrated with previous works to further improve the performance of serverless edge computing.

## IX. CONCLUSION

This paper presents FaasCamp, a serverless computing framework with a multi-tier warm pool design and MLenabled invoker. FaasCamp enables secure cross-tenant container sharing via the checkpoint and restore techniques. It can improve the warm ratio compared with the existing singertier warm pool. Also, FaasCamp leverages deep learning techniques as the container keep-alive policy, guided by the optimal cache replacement algorithms. We have implemented FaasCamp and conducted extensive experiments on evaluating both performance and overhead using real-world traces and workloads. Our results show that the FaasCamp can enhance performance with minimal overhead.

## ACKNOWLEDGMENTS

This work is partially supported by National Science Foundation (NSF) grants CNS-2054657 and OAC-2319975.

## REFERENCES

- [1] 'Aws lambda,' https://aws.amazon.com/lambda/, Referenced February 2022.
- [2] 'Google cloud functions,' https://cloud.google.com/functions, Referenced February 2022.
- [3] 'Azure functions,' https://azure.microsoft.com/en-us/products/functions/, Referenced February 2022.
- [4] R. Xie, Q. Tang, S. Qiao, H. Zhu, F. R. Yu, and T. Huang, 'When serverless computing meets edge computing: Architecture, challenges, and open issues,' IEEE Wireless Communications , vol. 28, no. 5, pp. 126-133, 2021.
- [5] S. Nastic, T. Rausch, O. Scekic, S. Dustdar, M. Gusev, B. Koteska, M. Kostoska, B. Jakimovski, S. Ristov, and R. Prodan, 'A serverless real-time data analytics platform for edge computing,' IEEE Internet Computing , vol. 21, no. 4, pp. 64-71, 2017.
- [6] L. Baresi and D. F. Mendonc ¸a, 'Towards a serverless platform for edge computing,' in 2019 IEEE International Conference on Fog Computing (ICFC) . IEEE, 2019, pp. 1-10.
- [7] M. S. Aslanpour, A. N. Toosi, C. Cicconetti, B. Javadi, P. Sbarski, D. Taibi, M. Assuncao, S. S. Gill, R. Gaire, and S. Dustdar, 'Serverless edge computing: vision and challenges,' in 2021 Australasian Computer Science Week Multiconference , 2021, pp. 1-10.
- [8] D. Du, T. Yu, Y. Xia, B. Zang, G. Yan, C. Qin, Q. Wu, and H. Chen, Catalyzer: Sub-Millisecond Startup for Serverless Computing with Initialization-Less Booting . New York, NY, USA: Association for Computing Machinery, 2020, p. 467-481. [Online]. Available: https://doi.org/10.1145/3373376.3378512
- [9] E. Oakes, L. Yang, D. Zhou, K. Houck, T. Harter, A. ArpaciDusseau, and R. Arpaci-Dusseau, 'SOCK: Rapid task provisioning with serverless-optimized containers,' in 2018 USENIX Annual Technical Conference (USENIX ATC 18) . Boston, MA: USENIX Association, Jul. 2018, pp. 57-70. [Online]. Available: https: //www.usenix.org/conference/atc18/presentation/oakes
- [10] S. Shillaker and P. Pietzuch, 'Faasm: Lightweight isolation for efficient stateful serverless computing,' in 2020 USENIX Annual Technical Conference (USENIX ATC 20) , 2020, pp. 419-433.
- [11] A. Agache, M. Brooker, A. Iordache, A. Liguori, R. Neugebauer, P. Piwonka, and D.-M. Popa, 'Firecracker: Lightweight virtualization for serverless applications,' in 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20) . Santa Clara, CA: USENIX Association, Feb. 2020, pp. 419-434. [Online]. Available: https://www.usenix.org/conference/nsdi20/presentation/agache
- [12] L. Ao, G. Porter, and G. M. Voelker, 'Faasnap: Faas made fast using snapshot-based vms,' in Proceedings of the Seventeenth European Conference on Computer Systems , 2022, pp. 730-746.
- [13] A. Wang, S. Chang, H. Tian, H. Wang, H. Yang, H. Li, R. Du, and Y. Cheng, 'Faasnet: Scalable and fast provisioning of custom serverless container runtimes at alibaba cloud function compute,' in 2021 USENIX Annual Technical Conference (USENIX ATC 21) , 2021.
- [14] M. Shahrad, R. Fonseca, I. Goiri, G. Chaudhry, P. Batum, J. Cooke, E. Laureano, C. Tresness, M. Russinovich, and R. Bianchini, 'Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider,' in 2020 USENIX Annual Technical Conference (USENIX ATC 20) . USENIX Association, Jul. 2020, pp. 205-218. [Online]. Available: https://www.usenix.org/conference/atc20/ presentation/shahrad
- [15] A. Fuerst and P. Sharma, 'Faascache: Keeping serverless computing alive with greedy-dual caching,' ser. ASPLOS 2021. New York, NY, USA: Association for Computing Machinery, 2021, p. 386-400. [Online]. Available: https://doi.org/10.1145/3445814.3446757
- [16] 'Cold start / warm start with aws lambda,' https://blog.octo.com/en/ cold-start-warm-start-with-aws-lambda/, Referenced December 2022.
- [17] P. Silva, D. Fireman, and T. E. Pereira, 'Prebaking functions to warm the serverless cold start,' in Proceedings of the 21st International Middleware Conference , 2020, pp. 1-13.
- [18] A. Jain and C. Lin, 'Back to the future: Leveraging belady's algorithm for improved cache replacement,' in 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) , 2016, pp. 78-89.
- [19] Z. Shi, X. Huang, A. Jain, and C. Lin, 'Applying deep learning to the cache replacement problem,' in Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture , ser. MICRO '52. New York, NY, USA: Association for Computing Machinery, 2019, p. 413-425. [Online]. Available: https://doi.org/10.1145/3352460. 3358319
- [20] S. Sethumurugan, J. Yin, and J. Sartori, 'Designing a cost-effective cache replacement policy using machine learning,' in 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , 2021, pp. 291-303.
- [21] 'Docker,' https://www.docker.com/, Referenced May 2022.
- [22] K.-T. Seo, H.-S. Hwang, I.-Y. Moon, O.-Y. Kwon, and B.-J. Kim, 'Performance comparison analysis of linux container and virtual machine for building cloud,' Advanced Science and Technology Letters , vol. 66, no. 105-111, p. 2, 2014.
- [23] W. Felter, A. Ferreira, R. Rajamony, and J. Rubio, 'An updated performance comparison of virtual machines and linux containers,' in 2015 IEEE international symposium on performance analysis of systems and software (ISPASS) . IEEE, 2015, pp. 171-172.
- [24] R. Morabito, J. Kj¨ allman, and M. Komu, 'Hypervisors vs. lightweight virtualization: a performance comparison,' in 2015 IEEE International Conference on cloud engineering . IEEE, 2015, pp. 386-393.
- [25] M. Eder, 'Hypervisor-vs. container-based virtualization,' Future Internet (FI) and Innovative Internet Technologies and Mobile Communications (IITM) , vol. 1, 2016.
- [26] S. Boucher, A. Kalia, D. G. Andersen, and M. Kaminsky, 'Putting the' micro' back in microservice,' in 2018 USENIX Annual Technical Conference (USENIX ATC 18) , 2018, pp. 645-650.
- [27] J. M. Hellerstein, J. Faleiro, J. E. Gonzalez, J. Schleier-Smith, V. Sreekanti, A. Tumanov, and C. Wu, 'Serverless computing: One step forward, two steps back,' arXiv preprint arXiv:1812.03651 , 2018.
- [28] A. Klimovic, Y. Wang, P. Stuedi, A. Trivedi, J. Pfefferle, and C. Kozyrakis, 'Pocket: Elastic ephemeral storage for serverless analytics,' in 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18) , 2018, pp. 427-444.
- [29] M. Liu, S. Peter, A. Krishnamurthy, and P. M. Phothilimthana, 'E3:energyefficient microservices on smartnic-accelerated servers,' in 2019 USENIX Annual Technical Conference (USENIX ATC 19) , 2019, pp. 363-378.
- [30] 'Cold start / warm start with aws lambda,' https://blog.octo.com/en/ cold-start-warm-start-with-aws-lambda/, Referenced January 2023.
- [31] L. Cherkasova, Improving WWW proxies performance with greedy-dualsize-frequency caching policy . Hewlett-Packard Laboratories Palo Alto, CA, 1998.
- [32] 'Openwhisk,' https://openwhisk.apache.org/, Referenced February 2022.
- [33] 'Openfaas,' https://www.openfaas.com/, Referenced January 2023.
- [34] S. Hendrickson, S. Sturdevant, T. Harter, V. Venkataramani, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau, 'Serverless computation with openlambda,' in 8th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 16) . Denver, CO: USENIX Association, Jun. 2016. [Online]. Available: https://www.usenix.org/conference/hotcloud16/ workshop-program/presentation/hendrickson
- [35] P. Vahidinia, B. Farahani, and F. S. Aliee, 'Mitigating cold start problem in serverless computing: A reinforcement learning approach,' IEEE Internet of Things Journal , pp. 1-1, 2022.
- [36] H. Yu, H. Wang, J. Li, and S.-J. Park, 'Harvesting idle resources in serverless computing via reinforcement learning,' arXiv preprint arXiv:2108.12717 , 2021.
- [37] Y. Zhang, ´ I. Goiri, G. I. Chaudhry, R. Fonseca, S. Elnikety, C. Delimitrou, and R. Bianchini, 'Faster and cheaper serverless computing on harvested resources,' in Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles , 2021, pp. 724-739.
- [38] K. Kaffes, N. J. Yadwadkar, and C. Kozyrakis, 'Hermod: Principled and practical scheduling for serverless functions,' in Proceedings of the 13th Symposium on Cloud Computing , ser. SoCC '22. New York, NY, USA: Association for Computing Machinery, 2022, p. 289-305. [Online]. Available: https://doi.org/10.1145/3542929.3563468
- [39] D. Mvondo, M. Bacou, K. Nguetchouang, L. Ngale, S. Pouget, J. Kouam, R. Lachaize, J. Hwang, T. Wood, D. Hagimont et al. , 'Ofc: an opportunistic caching system for faas platforms,' in Proceedings of the Sixteenth European Conference on Computer Systems , 2021, pp. 228-244.
- [40] 'Ibm cloud functions,' https://www.ibm.com/cloud/functions, Referenced December 2022.
- [41] A. Jaleel, K. B. Theobald, S. C. Steely Jr, and J. Emer, 'High performance cache replacement using re-reference interval prediction (rrip),' ACM SIGARCH computer architecture news , vol. 38, no. 3, pp. 60-71, 2010.
- [42] N. Duong, D. Zhao, T. Kim, R. Cammarota, M. Valero, and A. V. Veidenbaum, 'Improving cache management policies using dynamic reuse distances,' in 2012 45Th annual IEEE/ACM international symposium on microarchitecture . IEEE, 2012, pp. 389-400.
- [43] P. Faldu and B. Grot, 'Leeway: Addressing variability in dead-block prediction for last-level caches,' in 2017 26th International Conference on Parallel Architectures and Compilation Techniques (PACT) . IEEE, 2017, pp. 180-193.
- [44] C.-J. Wu, A. Jaleel, W. Hasenplaugh, M. Martonosi, S. C. Steely Jr, and J. Emer, 'Ship: Signature-based hit predictor for high performance caching,' in Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture , 2011, pp. 430-441.
- [45] D. A. Jim´ enez and E. Teran, 'Multiperspective reuse prediction,' in 2017 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, 2017, pp. 436-448.
- [46] L. A. Belady, 'A study of replacement algorithms for a virtual-storage computer,' IBM Systems journal , vol. 5, no. 2, pp. 78-101, 1966.
- [47] J. Du, F. R. Yu, X. Chu, J. Feng, and G. Lu, 'Computation offloading and resource allocation in vehicular networks based on dual-side cost minimization,' IEEE Transactions on Vehicular Technology , vol. 68, no. 2, pp. 1079-1092, 2018.
- [48] G. Qiao, S. Leng, K. Zhang, and Y. He, 'Collaborative task offloading in vehicular edge multi-access networks,' IEEE Communications Magazine , vol. 56, no. 8, pp. 48-54, 2018.
- [49] Y. Liu, S. Wang, J. Huang, and F. Yang, 'A computation offloading algorithm based on game theory for vehicular edge networks,' in 2018 IEEE International Conference on Communications (ICC) . IEEE, 2018, pp. 1-6.
- [50] M. M. K. Tareq, O. Semiari, M. A. Salehi, and W. Saad, 'Ultra reliable, low latency vehicle-to-infrastructure wireless communications with edge computing,' in 2018 IEEE Global Communications Conference (GLOBECOM) . IEEE, 2018, pp. 1-7.
- [51] X. Chen, L. Jiao, W. Li, and X. Fu, 'Efficient multi-user computation offloading for mobile-edge cloud computing,' IEEE/ACM transactions on networking , vol. 24, no. 5, pp. 2795-2808, 2015.
- [52] T. Zhao, S. Zhou, X. Guo, and Z. Niu, 'Tasks scheduling and resource allocation in heterogeneous cloud for delay-bounded mobile edge computing,' in 2017 IEEE international conference on communications (ICC) . IEEE, 2017, pp. 1-7.
- [53] 'Providing multi-tenancy and network isolation to the edge,' https://cloud.redhat.com/blog/ providing-multi-tenancy-and-network-isolation-to-the-edge.
- [54] 'Smarter: A multi-tenant framework for ml at the edge,' https://www. nvidia.com/en-us/on-demand/session/gtcspring21-s33118/.
- [55] T. H. T. Le, N. H. Tran, T. LeAnh, T. Z. Oo, K. Kim, S. Ren, and C. S. Hong, 'Auction mechanism for dynamic bandwidth allocation in multitenant edge computing,' IEEE Transactions on Vehicular Technology , vol. 69, no. 12, pp. 15 162-15 176, 2020.
- [56] L. Zanzi, F. Giust, and V. Sciancalepore, 'M 2 ec: A multi-tenant resource orchestration in multi-access edge computing systems,' in 2018 IEEE wireless communications and networking conference (wcnc) . IEEE, 2018, pp. 1-6.
- [57] N. Wang, M. Matthaiou, D. S. Nikolopoulos, and B. Varghese, 'Dyverse: dynamic vertical scaling in multi-tenant edge environments,' Future Generation Computer Systems , vol. 108, pp. 598-612, 2020.
- [58] D. Willis, A. Dasgupta, and S. Banerjee, 'Paradrop: a multi-tenant platform to dynamically install third party services on wireless gateways,' in Proceedings of the 9th ACM workshop on Mobility in the evolving internet architecture , 2014, pp. 43-48.
- [59] B. C. S ¸enel, M. Mouchet, J. Cappos, O. Fourmaux, T. Friedman, and R. McGeer, 'Edgenet: a multi-tenant and multi-provider edge cloud,' in Proceedings of the 4th International Workshop on Edge Systems, Analytics and Networking , 2021, pp. 49-54.
- [60] L. Wang, M. Li, Y. Zhang, T. Ristenpart, and M. Swift, 'Peeking behind the curtains of serverless platforms,' in 2018 USENIX Annual Technical Conference (USENIX ATC 18) . Boston, MA: USENIX Association, Jul. 2018, pp. 133-146. [Online]. Available: https://www.usenix.org/conference/atc18/presentation/wang-liang
- [61] J. Kim and K. Lee, 'Practical cloud workloads for serverless faas,' in Proceedings of the ACM Symposium on Cloud Computing , ser. SoCC '19. New York, NY, USA: Association for Computing Machinery, 2019, p. 477. [Online]. Available: https://doi.org/10.1145/3357223.3365439
- [62] O. Stenbom, 'Refunction: Eliminating serverless cold starts through container reuse,' MEng Individual Project, Department Of Computing Imperial College London , 2019.
- [63] 'Criu docker,' https://criu.org/Docker, Referenced April 2022.
- [64] 'Sysstat,' http://sebastien.godard.pagesperso-orange.fr/, Referenced January 2023.
- [65] A. Tariq, A. Pahl, S. Nimmagadda, E. Rozner, and S. Lanka, 'Sequoia: Enabling quality-of-service in serverless computing,' in Proceedings of the 11th ACM Symposium on Cloud Computing , 2020, pp. 311-327.
- [66] V. M. Bhasi, J. R. Gunasekaran, P. Thinakaran, C. S. Mishra, M. T. Kandemir, and C. Das, 'Kraken: Adaptive container provisioning for deploying dynamic dags in serverless platforms,' in Proceedings of the ACM Symposium on Cloud Computing , 2021, pp. 153-167.
- [67] R. S. Kannan, L. Subramanian, A. Raju, J. Ahn, J. Mars, and L. Tang, 'Grandslam: Guaranteeing slas for jobs in microservices execution frameworks,' in Proceedings of the Fourteenth EuroSys Conference 2019 , 2019, pp. 1-16.
- [68] N. Daw, U. Bellur, and P. Kulkarni, 'Xanadu: Mitigating cascading cold starts in serverless function chain deployments,' in Proceedings of the 21st International Middleware Conference , 2020, pp. 356-370.
- [69] J. Stojkovic, T. Xu, H. Franke, and J. Torrellas, 'Specfaas: Accelerating serverless applications with speculative function execution,' in 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA) . IEEE, 2023, pp. 814-827.
- [70] R. B. Roy, T. Patel, and D. Tiwari, 'Icebreaker: warming serverless functions better with heterogeneity,' in Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems , 2022, pp. 753-767.
- [71] A. Singhvi, A. Balasubramanian, K. Houck, M. D. Shaikh, S. Venkataraman, and A. Akella, 'Atoll: A scalable low-latency serverless platform,' in Proceedings of the ACM Symposium on Cloud Computing , 2021, pp. 138-152.
- [72] I. E. Akkus, R. Chen, I. Rimac, M. Stein, K. Satzke, A. Beck, P. Aditya, and V. Hilt, 'Sand: Towards high-performance serverless computing,' in 2018 Usenix Annual Technical Conference (USENIX ATC 18) , 2018, pp. 923-935.
- [73] Z. Li, L. Guo, Q. Chen, J. Cheng, C. Xu, D. Zeng, Z. Song, T. Ma, Y. Yang, C. Li et al. , 'Help rather than recycle: Alleviating cold startup in serverless computing through { Inter-Function } container sharing,' in 2022 USENIX Annual Technical Conference (USENIX ATC 22) , 2022, pp. 69-84.
- [74] F. Romero, G. I. Chaudhry, ´ I. Goiri, P. Gopa, P. Batum, N. J. Yadwadkar, R. Fonseca, C. Kozyrakis, and R. Bianchini, 'Faa$t: A transparent autoscaling cache for serverless applications,' in Proceedings of the ACM Symposium on Cloud Computing , 2021, pp. 122-137.
- [75] G. Somma, C. Ayimba, P. Casari, S. P. Romano, and V. Mancuso, 'When less is more: Core-restricted container provisioning for serverless computing,' in IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) , 2020, pp. 11531159.
- [76] L. Schuler, S. Jamil, and N. K¨ uhl, 'Ai-based resource allocation: Reinforcement learning for adaptive auto-scaling in serverless environments,' in 2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid) . IEEE, 2021, pp. 804-811.
- [77] H. Wang, D. Niu, and B. Li, 'Distributed machine learning with a serverless architecture,' in IEEE INFOCOM 2019 - IEEE Conference on Computer Communications , 2019, pp. 1288-1296.
- [78] Z. Xu, H. Zhang, X. Geng, Q. Wu, and H. Ma, 'Adaptive function launching acceleration in serverless computing platforms,' in 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS) , 2019, pp. 9-16.
- [79] G. Yan and J. Li, 'Rl-b´ el´dy: A unified learning framework for content a caching,' in Proceedings of the 28th ACM International Conference on Multimedia , 2020, pp. 1009-1017.
- [80] --, 'Towards latency awareness for content delivery network caching,' in 2022 USENIX Annual Technical Conference (USENIX ATC 22) , 2022, pp. 789-804.
- [81] Z. Song, D. S. Berger, K. Li, and W. Lloyd, 'Learning relaxed belady for content distribution network caching,' in Proceedings of the 17th Usenix Conference on Networked Systems Design and Implementation , ser. NSDI'20. USA: USENIX Association, 2020, p. 529-544.
- [82] A. Aske and X. Zhao, 'Supporting multi-provider serverless computing on the edge,' in Workshop Proceedings of the 47th International Conference on Parallel Processing , 2018, pp. 1-6.
- [83] L. Baresi, D. F. Mendonc ¸a, M. Garriga, S. Guinea, and G. Quattrocchi, 'A unified model for the mobile-edge-cloud continuum,' ACM Transactions on Internet Technology (TOIT) , vol. 19, no. 2, pp. 1-21, 2019.
- [84] V. Mittal, S. Qi, R. Bhattacharya, X. Lyu, J. Li, S. G. Kulkarni, D. Li, J. Hwang, K. Ramakrishnan, and T. Wood, 'Mu: an efficient, fair and responsive serverless framework for resource-constrained edge clouds,' in Proceedings of the ACM Symposium on Cloud Computing , 2021, pp. 168-181.
- [85] 'Knative,' https://knative.dev/, Referenced August 2023.
- [86] C. Cicconetti, M. Conti, and A. Passarella, 'A decentralized framework for serverless edge computing in the internet of things,' IEEE Transactions on Network and Service Management , vol. 18, no. 2, pp. 2166-2180, 2020.
Chu Qiao Cong Wang Zhenkai Zhang Yuede Ji Xing Gao
Caching Aided Multi-Tenant Serverless Computing
One key to enabling high-performance serverless computing is to mitigate
cold-starts. Current solutions utilize a warm pool to keep function alive: a
warm-start can be analogous to a CPU cache-hit. However, modern cache has
multiple hierarchies and the last-level cache is shared among cores, whereas
the warm pool is limited to a single tenant for security concerns. Also, the
warm pool keep-alive policy can be further optimized using cache replacement
algorithms. In this paper, we borrow practical optimizations from caching, and
design FaasCamp, a caching-aided multi-tenant serverless computing framework.
FaasCamp extends the single-tier warm pool into multi-tiers, with a reclaim
pool introduced enabling secure function instance sharing among tenants. Also,
FaasCamp leverages machine learning to approximate the optimal cache
replacement policy to improve the warm rate. We have implemented a prototype
and conducted extensive experiments under multiple scenarios. The results show
that FaasCamp can outperform existing platforms with minimal overhead.