Answers.Body: <p>While <a href="https://ai.stackexchange.com/users/2193/oliver-mason">@Oliver Mason</a>'s comment is correct, and your proposed method won't provide perfect security, you can still protect your models at rest, so that they are stored encrypted in the memory, and your software feed the key at runtime to decrypt it.<br />
  On whatever DL inference engine that you have, once it supports loading the model from a buffer (e.g. void*) rather than a file path, you can do the following: read the encrypted model, decrypt it into a buffer, and initialize your neural network model from the decrypted buffer. <a href="https://github.com/openvinotoolkit/openvino/blob/96875ef4116c5b23b6cec8d7885c6925389031e7/inference-engine/include/ie_core.hpp#L90" rel="nofollow noreferrer">OpenVino supports loading the model from a buffer</a>.<br />
  For encrypt / decrypt the model, any framework such as OpenSSL, or even tiny-AES, can work.<br />
  Mentioning again, how to store and use the key is something that should be handled carefully, and a user with sufficient knowledge can read the model and the keys at runtime from the application's memory.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-01T11:53:33.820Z
Answers.LastActivityDate: 2021-03-01T12:11:44.967Z
Answers.LastEditDate: 2021-03-01T12:11:44.967Z
Answers.UserDisplayName: rkellerm
Body: <p>I want my models to be accessible only by my programs. How do I encrypt and decrypt my model when I run inference on it? Is there any existing technology that is widely used?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-22T08:11:19.633Z
Comments.Text: It's not really possible. If your program runs on the users' machines, they will have access to the encryption keys (as your program will have them). You can make it hard, but if someone is determined enough, they will be able to crack it. I wouldn't waste time on encrypting it.
Comments.UserDisplayName: Oliver Mason
Tags: <machine-learning><models><ai-security>
Title: How do I encrypt and decrypt my model when I run inference on it?
UserDisplayName: 55597
ThreadText: # How do I encrypt and decrypt my model when I run inference on it?
  
  (Asked by: username_0 on 2019-05-22)
  
  I want my models to be accessible only by my programs. How do I encrypt and decrypt my model when I run inference on it? Is there any existing technology that is widely used?
  
  **Tags:** machine-learning, models, ai-security
  > **Comments:**
  > **username_1**: It's not really possible. If your program runs on the users' machines, they will have access to the encryption keys (as your program will have them). You can make it hard, but if someone is determined enough, they will be able to crack it. I wouldn't waste time on encrypting it.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-03-01. Score: 1
  
  While [@Oliver Mason](https://ai.stackexchange.com/users/2193/oliver-mason)'s comment is correct, and your proposed method won't provide perfect security, you can still protect your models at rest, so that they are stored encrypted in the memory, and your software feed the key at runtime to decrypt it.  
  On whatever DL inference engine that you have, once it supports loading the model from a buffer (e.g. void\*) rather than a file path, you can do the following: read the encrypted model, decrypt it into a buffer, and initialize your neural network model from the decrypted buffer. [OpenVino supports loading the model from a buffer](https://github.com/openvinotoolkit/openvino/blob/96875ef4116c5b23b6cec8d7885c6925389031e7/inference-engine/include/ie_core.hpp#L90).  
  For encrypt / decrypt the model, any framework such as OpenSSL, or even tiny-AES, can work.  
  Mentioning again, how to store and use the key is something that should be handled carefully, and a user with sufficient knowledge can read the model and the keys at runtime from the application's memory.
  
  
  ---
  
  