Answers.Body: <p>As discussed in this tech <a href="https://bdtechtalks.com/2022/05/23/machine-learning-undetectable-backdoors/" rel="nofollow noreferrer">blog</a>:</p>
  <blockquote>
  <p>Machine learning backdoors are techniques that implant secret behaviors into trained ML models. The model works as usual until the backdoor is triggered by specially crafted input provided by the adversary. For example, an adversary can create a backdoor that bypasses a face recognition system used to authenticate users.</p>
  </blockquote>
  <blockquote>
  <p>A simple and well-known ML backdooring method is data poisoning. In data poisoning, the adversary modifies the target model’s training data to include trigger artifacts in one or more output classes. The model then becomes sensitive to the backdoor pattern and triggers the intended behavior (e.g., the target output class) whenever it sees it.</p>
  </blockquote>
  <blockquote>
  <p>Machine learning backdoors are closely related to adversarial attacks, input data that is perturbed to cause the ML model to misclassify it. Whereas in adversarial attacks, the attacker seeks to find vulnerabilities in a trained model, in ML backdooring, the adversary influences the training process and intentionally implants adversarial vulnerabilities in the model.</p>
  </blockquote>
  <p>Thus in some cases data poisoning could be considered as one of backdoor attack methods which usually has some <em>specific</em> hidden trigger condition like a humanly unnoticeable watermark of a certain characteristic applied during model's training process, while adversarial attacks usually apply to a trained model by <em>trying</em> to exploit its functional vulnerabilities for certain small random perturbations of input data or images.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-16T22:03:36.883Z
Answers.LastActivityDate: 2024-02-16T22:03:36.883Z
Answers.UserDisplayName: cinch
Body: <p>I've recently come across the term &quot;backdoor attack&quot; in the context of machine learning and I'm trying to understand its precise definition and characteristics. From what I gather, backdoor attacks involve embedding malicious behavior or vulnerabilities within a machine learning model, which are triggered under specific conditions. However, I'm unclear about the subtleties that differentiate backdoor attacks from other types of attacks, such as adversarial or data poisoning attacks.</p>
  <p>What exactly defines a backdoor attack in machine learning models? Are there specific criteria that an attack must meet to be considered a &quot;backdoor&quot; attack?</p>
  
Tags: <terminology><definitions><adversarial-ml><adversarial-attacks>
Title: What constitutes a 'backdoor' attack in machine learning models?
UserDisplayName: hanugm
ThreadText: # What constitutes a 'backdoor' attack in machine learning models?
  
  (Asked by: username_0 on 2024-02-16)
  
  I've recently come across the term "backdoor attack" in the context of machine learning and I'm trying to understand its precise definition and characteristics. From what I gather, backdoor attacks involve embedding malicious behavior or vulnerabilities within a machine learning model, which are triggered under specific conditions. However, I'm unclear about the subtleties that differentiate backdoor attacks from other types of attacks, such as adversarial or data poisoning attacks.
  
  What exactly defines a backdoor attack in machine learning models? Are there specific criteria that an attack must meet to be considered a "backdoor" attack?
  
  **Tags:** terminology, definitions, adversarial-ml, adversarial-attacks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-02-16. Score: 1
  
  As discussed in this tech [blog](https://bdtechtalks.com/2022/05/23/machine-learning-undetectable-backdoors/):
  
  > Machine learning backdoors are techniques that implant secret behaviors into trained ML models. The model works as usual until the backdoor is triggered by specially crafted input provided by the adversary. For example, an adversary can create a backdoor that bypasses a face recognition system used to authenticate users.
  
  > A simple and well-known ML backdooring method is data poisoning. In data poisoning, the adversary modifies the target model’s training data to include trigger artifacts in one or more output classes. The model then becomes sensitive to the backdoor pattern and triggers the intended behavior (e.g., the target output class) whenever it sees it.
  
  > Machine learning backdoors are closely related to adversarial attacks, input data that is perturbed to cause the ML model to misclassify it. Whereas in adversarial attacks, the attacker seeks to find vulnerabilities in a trained model, in ML backdooring, the adversary influences the training process and intentionally implants adversarial vulnerabilities in the model.
  
  Thus in some cases data poisoning could be considered as one of backdoor attack methods which usually has some *specific* hidden trigger condition like a humanly unnoticeable watermark of a certain characteristic applied during model's training process, while adversarial attacks usually apply to a trained model by *trying* to exploit its functional vulnerabilities for certain small random perturbations of input data or images.
  
  
  ---
  
  