Answers.Body: <p>It will recover the encrypted inputs.</p>
  <p>The algorithm starts with dummy data and dummy labels, and then iteratively optimizes the dummy gradients to be close as to the original. This makes the dummy data close to the real training data:</p>
  <p><span class="math-container">$$\mathbf{x}^{\prime *}, \mathbf{y}^{\prime *}=\underset{\mathbf{x}^{\prime}, \mathbf{y}^{\prime}}{\arg \min }\left\|\nabla W^{\prime}-\nabla W\right\|^{2}=\underset{\mathbf{x}^{\prime}, \mathbf{y}^{\prime}}{\arg \min }\left\|\frac{\partial \ell\left(F\left(\mathbf{x}^{\prime}, W\right), \mathbf{y}^{\prime}\right)}{\partial W}-\nabla W\right\|^{2}$$</span></p>
  <p>As the distance is minimized, the algorithm restores the original training data; in the case of encrypted training data - you should get an encrypted input (up to failures when the resulted 'input' isn't close to the original input).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-28T19:20:38.837Z
Answers.LastActivityDate: 2021-04-22T15:21:26.220Z
Answers.LastEditDate: 2021-04-22T15:21:26.220Z
Answers.UserDisplayName: rkellerm
Body: <p>Some algorithms in the literature allow recovering the input data used to train a neural network. This is done using the gradients (updates) of weights, such as in <a href="https://arxiv.org/abs/1906.08935" rel="noreferrer">Deep Leakage from Gradients</a> (2019) by Ligeng Zhu et al.</p>
  <p>In case the neural network is trained using encrypted (homomorphic) input data, what could be the output of the above algorithm? Will the algorithm recover the data in clear or encrypted (as it was fed encrypted)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-19T23:44:18.733Z
Comments.Text: Just to clarify, although I am not familiar with that paper/work, but are you assuming that the neural network would be trained with an encrypted version of the data?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-20T11:40:31.593Z
Comments.Text: Yes, exactly the neural network is trained with encrypted data using homomorphic encryption ... but the neural network ie. weights and parameters are not encrypted, only the training data is encrypted.
Comments.UserDisplayName: witdev
Tags: <neural-networks><training><gradient-descent><ai-security><training-datasets>
Title: During neural network training, can gradients leak sensitive information in case training data fed is encrypted (homomorphic)?
UserDisplayName: witdev
ThreadText: # During neural network training, can gradients leak sensitive information in case training data fed is encrypted (homomorphic)?
  
  (Asked by: username_0 on 2020-12-19)
  
  Some algorithms in the literature allow recovering the input data used to train a neural network. This is done using the gradients (updates) of weights, such as in [Deep Leakage from Gradients](https://arxiv.org/abs/1906.08935) (2019) by Ligeng Zhu et al.
  
  In case the neural network is trained using encrypted (homomorphic) input data, what could be the output of the above algorithm? Will the algorithm recover the data in clear or encrypted (as it was fed encrypted)?
  
  **Tags:** neural-networks, training, gradient-descent, ai-security, training-datasets
  
  > **Comments:**
  > **username_1**: Just to clarify, although I am not familiar with that paper/work, but are you assuming that the neural network would be trained with an encrypted version of the data?  
  > **username_0**: Yes, exactly the neural network is trained with encrypted data using homomorphic encryption ... but the neural network ie. weights and parameters are not encrypted, only the training data is encrypted.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-02-28. Score: 2 | Accepted answer
  
  It will recover the encrypted inputs.
  
  The algorithm starts with dummy data and dummy labels, and then iteratively optimizes the dummy gradients to be close as to the original. This makes the dummy data close to the real training data:
  
  $$\mathbf{x}^{\prime \*}, \mathbf{y}^{\prime \*}=\underset{\mathbf{x}^{\prime}, \mathbf{y}^{\prime}}{\arg \min }\left\|\nabla W^{\prime}-\nabla W\right\|^{2}=\underset{\mathbf{x}^{\prime}, \mathbf{y}^{\prime}}{\arg \min }\left\|\frac{\partial \ell\left(F\left(\mathbf{x}^{\prime}, W\right), \mathbf{y}^{\prime}\right)}{\partial W}-\nabla W\right\|^{2}$$
  
  As the distance is minimized, the algorithm restores the original training data; in the case of encrypted training data - you should get an encrypted input (up to failures when the resulted 'input' isn't close to the original input).
  
  
  ---
  
  