Answers.Body: <p>We can manipulate a model's test data set if the machine learning model takes user input and uses it to resample test data set. The actual training dataset of the ML model does not get manipulated, but if we figure out the ML model through an exploratory attack (sending a lot of inquiries to the ML model to find out its nature), we can generate a training dataset which was built into the original ML model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-15T03:38:07.430Z
Answers.LastActivityDate: 2019-11-15T03:38:07.430Z
Answers.UserDisplayName: boomselector
Answers.Body: <p>They don't have acces to the original training or test dataset. Machine learning environments are build on the premise of a benign environment. The models are trained on real data (real inputs). When someone sends a made up input (fake input) it is very easy to fool the model. </p>
  
  <p>This is used for example in image recognition. Imagine a fotograph of a panda. the model may correctly identify this fotograph as a panda. With knowledge of the model you can now alter some pixels in the fotograph. To the human Eye, the fotographs will appear exactly the same, but the model can be fooled to believe the fotograph is actually of a gibbon.</p>
  
  <p>This is all done after the training of the model and doesn't require the original datasets.</p>
  
  <p>For more info, visit this site:
  <a href="https://medium.com/@ml.at.berkeley/tricking-neural-networks-create-your-own-adversarial-examples-a61eb7620fd8" rel="nofollow noreferrer">https://medium.com/@ml.at.berkeley/tricking-neural-networks-create-your-own-adversarial-examples-a61eb7620fd8</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T08:32:45.063Z
Answers.LastActivityDate: 2019-11-13T08:32:45.063Z
Answers.UserDisplayName: Lustwelpintje
Answers.Body: <p>In adverserial machine learning, someone (program or human) attempts to fool an existing model with a malicious input. </p>
  
  <p>The best human example would be an optical illusion. The human brain's model for image processing starts outputting wrong information when looking at an optical illusion. So in the end we see wrong colour, shape, etc. In this case, the optical illusion would be considered as the malicious input.</p>
  
  <p>We can trick the human brain’s model through images created with trial and error.</p>
  
  <p>So, if you just have the trained model at hand, you don’t have to know the data it has been trained with. You just need to be able to input a value to the model and get the output.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T08:50:21.343Z
Answers.LastActivityDate: 2019-11-13T08:50:21.343Z
Answers.UserDisplayName: Lively
Body: <p>In the field of adversarial machine learning, machine learning models are vulnerable to attacks both on the test and training data set. However, how does the attacker get access to these datasets? How do these datasets get manipulated/tampered with?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-13T07:01:03.743Z
Comments.Text: They doesn't. That's why adversarial attack is sham
Comments.UserDisplayName: mirror2image
Tags: <machine-learning><reinforcement-learning><training><datasets><adversarial-ml>
Title: In adversarial machine learning, how does an attacker have access to the test and training dataset in order to poison it?
UserDisplayName: boomselector
ThreadText: # In adversarial machine learning, how does an attacker have access to the test and training dataset in order to poison it?
  
  (Asked by: username_0 on 2019-11-13)
  
  In the field of adversarial machine learning, machine learning models are vulnerable to attacks both on the test and training data set. However, how does the attacker get access to these datasets? How do these datasets get manipulated/tampered with?
  
  **Tags:** machine-learning, reinforcement-learning, training, datasets, adversarial-ml
  
  > **Comments:**
  > **username_1**: They doesn't. That's why adversarial attack is sham  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_0 on 2019-11-15. Score: 0 | Accepted answer
  
  We can manipulate a model's test data set if the machine learning model takes user input and uses it to resample test data set. The actual training dataset of the ML model does not get manipulated, but if we figure out the ML model through an exploratory attack (sending a lot of inquiries to the ML model to find out its nature), we can generate a training dataset which was built into the original ML model.
  
  
  ---
  
  ### Answer by username_2 on 2019-11-13. Score: 1
  
  They don't have acces to the original training or test dataset. Machine learning environments are build on the premise of a benign environment. The models are trained on real data (real inputs). When someone sends a made up input (fake input) it is very easy to fool the model.
  
  This is used for example in image recognition. Imagine a fotograph of a panda. the model may correctly identify this fotograph as a panda. With knowledge of the model you can now alter some pixels in the fotograph. To the human Eye, the fotographs will appear exactly the same, but the model can be fooled to believe the fotograph is actually of a gibbon.
  
  This is all done after the training of the model and doesn't require the original datasets.
  
  For more info, visit this site:
  <https://medium.com/@ml.at.berkeley/tricking-neural-networks-create-your-own-adversarial-examples-a61eb7620fd8>
  
  
  ---
  
  ### Answer by username_3 on 2019-11-13. Score: 0
  
  In adverserial machine learning, someone (program or human) attempts to fool an existing model with a malicious input.
  
  The best human example would be an optical illusion. The human brain's model for image processing starts outputting wrong information when looking at an optical illusion. So in the end we see wrong colour, shape, etc. In this case, the optical illusion would be considered as the malicious input.
  
  We can trick the human brain’s model through images created with trial and error.
  
  So, if you just have the trained model at hand, you don’t have to know the data it has been trained with. You just need to be able to input a value to the model and get the output.
  
  
  ---
  
  