SLALOM: FAST, VERIFIABLE AND PRIVATE EXECUTION OF NEURAL NETWORKS IN TRUSTED HARDWARE
As Machine Learning (ML) gets applied to security-critical or sensitive domains, there is a growing need for integrity and privacy for outsourced ML computations. A pragmatic solution comes from Trusted Execution Environments (TEEs), which use hardware and software protections to isolate sensitive computations from the untrusted software stack. However, these isolation guarantees come at a price in performance, compared to untrusted alternatives. This paper initiates the study of high performance execution of Deep Neural Networks (DNNs) in TEEs by efficiently partitioning DNN computations between trusted and untrusted devices. Building upon an efficient outsourcing scheme for matrix multiplication, we propose Slalom, a framework that securely delegates execution of all linear layers in a DNN from a TEE (e.g., Intel SGX or Sanctum) to a faster, yet untrusted, co-located processor. We evaluate Slalom by running DNNs in an Intel SGX enclave, which selectively delegates work to an untrusted GPU. For canonical DNNs (VGG16, MobileNet and ResNet variants) we obtain 6× to 20× increases in throughput for verifiable inference, and 4× to 11× for verifiable and private inference.
The authors propose a new method of securely evaluating neural networks. The approach builds upon existing Trusted Execution Environments (TEE), a combination of hardware and software that isolates sensitive computations from the untrusted software stack. The downside of TEE is that it is expensive and slow to run. This paper proposes outsourcing the linear evaluation portions of the DNN to an untrusted stack that's co-located with the TEE. To achieve privacy (i.e., the input isn't revealed to the untrusted evaluator), the approach adds a random number r to the input vector x, evaluates f(x+r) on the untrusted stack, then subtracts off f(r) from the output. This limits the approach to be applicable to only linear functions. To achieve integrity (verify the correctness of the output), the paper proposes testing with random input vectors (an application of Freivalds theorem, which bounds the error probability). The techniques for integrity and privacy works only on integer evaluations, hence the network weights and inputs need to be quantized. The paper tries to minimize degradation in accuracy by quantizing as finely as numerically allowable, achieving <0.5% drop in accuracy on two example DNNs. Overall, compared to full evaluation in a TEE, this approach is 10x faster on one DNN, and 40x to 64x faster on another network (depending on how the network is formulated).
Disclaimer: I am a complete outsider to the field of HW security and privacy. The paper is very readable, so I think I understand its overall gist. I found the approach to be novel and the results convincing, though I may be missing important context since I'm not familiar with the subject.
To me, the biggest missing piece is a discussion of the limitations of the approach. How big of a network can be evaluated this way? Is it sufficient for most common applications? What are the bottlenecks to scaling this approach?
It's also not clear why integrity checks are required. Is there a chance that the outsourcing could result in incorrect values? (It's not obvious why it would.)
Lastly, a question about quantization. You try to quantize as finely as possible (to minimize quantization errors) by multiplying by the largest power of 2 possible without causing overflow. Since quantization need to be applied to both input and network weights, does this mean that you must also bound the scale of the input? Or do you assume that the inputs are pre-processed to be within a known scale? Is this possible for intermediate outputs (i.e., after the input has been multiplied through a few layers of the DNN)?
Pros:
- Simple yet effective approach to achieve the goals laid out in the problem statement
- Clearly written
- Thorough experiments and benchmarks
- Strong results
Cons:
- No discussion of limitations
- Minor questions regarding quantization and size limits
Disclaimer: reviewer is generally knowledgeable but not familiar with the subject area.