Answers.Body: <p>I think your assumptions are flawed for step 1. From how I understand it, you have a binary predictor (email no/yes) onto a metric outcome (propensity to buy). However, logistic regression assumes that you have a binary outcome variable, so it is not applicable to your case.</p>
  <p>For what you want to do, you need to do a comparison of group means. Since you have one binary predictor, you could do a t-test, ANOVA or linear regression and should always obtain the same results. You could then interpret the coefficient for the treatment variable as you mentioned above, which also gives the benefit that it is directly interpretable as these methods are linear models and not generalized linear models, thus dropping the need for a link function.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-25T09:03:25.510Z
Answers.LastActivityDate: 2023-01-25T09:03:25.510Z
Answers.UserDisplayName: David
Body: <p>I'm trying to create a pipeline for a <strong>very common business scenerio</strong>. I want to see whats the impact of an <em>intervention</em> on an <em>outcome</em>. For example I want to know if I send a marketing email (<em>intervention</em>) to users, how would it impact their propensity to buy (<em>outcome</em>).
  To be specific there are 3 things I want to know:</p>
  <ol>
  <li>What is the average impact of the <em>intervention</em>(marketing email) on the <em>outcome</em>(purchase)- This will tell me if sending emails is generally a good idea or not?</li>
  <li>Who should we email, and who we shouldn't email (uplift modelling)- This will help me customize the <em>treatment</em> at individual level.</li>
  <li>How can I be confident in the above mentioned model's findings, without having to run the A/B tests? - This will help me understand if the numbers I've calculated are trustworthy or not.</li>
  </ol>
  <p>This is what I've been thinking:</p>
  <ol>
  <li>What is the average impact of the intervention (marketing email) on
  the outcome (purchase)
  <ul>
  <li>Raw conditional probabilities</li>
  <li>Simple logistic regression, to understand what's the coefficient for the
  treatment variable</li>
  <li>Average Treatment Effect (ATE) using
  CausalML</li>
  </ul>
  </li>
  <li>Who should we email, and who we shouldn't email (uplift modelling)
  <ul>
  <li>Uplift Modeling via Class Variable Transformation</li>
  <li>Meta Learners (S learners/t learners etc.)</li>
  <li>Tree based uplift modelling</li>
  </ul>
  </li>
  <li>How can I be confident in the above mentioned model's findings, without having to run the A/B tests?
  <ul>
  <li>AUUC (Area under the uplift curve)</li>
  <li>Sensitivity Analysis</li>
  </ul>
  </li>
  </ol>
  <p>I'll really appreciate any input to improve this pipeline</p>
  
Tags: <machine-learning><ai-design><causation><drug-design>
Title: Causal Inference: Understanding the impact of an intervention
UserDisplayName: anwartheravian
ThreadText: # Causal Inference: Understanding the impact of an intervention
  
  (Asked by: username_0 on 2023-01-24)
  
  I'm trying to create a pipeline for a **very common business scenerio**. I want to see whats the impact of an *intervention* on an *outcome*. For example I want to know if I send a marketing email (*intervention*) to users, how would it impact their propensity to buy (*outcome*).
  To be specific there are 3 things I want to know:
  
  1. What is the average impact of the *intervention*(marketing email) on the *outcome*(purchase)- This will tell me if sending emails is generally a good idea or not?
  2. Who should we email, and who we shouldn't email (uplift modelling)- This will help me customize the *treatment* at individual level.
  3. How can I be confident in the above mentioned model's findings, without having to run the A/B tests? - This will help me understand if the numbers I've calculated are trustworthy or not.
  
  This is what I've been thinking:
  
  1. What is the average impact of the intervention (marketing email) on
     the outcome (purchase)
     * Raw conditional probabilities
     * Simple logistic regression, to understand what's the coefficient for the
       treatment variable
     * Average Treatment Effect (ATE) using
       CausalML
  2. Who should we email, and who we shouldn't email (uplift modelling)
     * Uplift Modeling via Class Variable Transformation
     * Meta Learners (S learners/t learners etc.)
     * Tree based uplift modelling
  3. How can I be confident in the above mentioned model's findings, without having to run the A/B tests?
     * AUUC (Area under the uplift curve)
     * Sensitivity Analysis
  
  I'll really appreciate any input to improve this pipeline
  
  **Tags:** machine-learning, ai-design, causation, drug-design
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-01-25. Score: 1
  
  I think your assumptions are flawed for step 1. From how I understand it, you have a binary predictor (email no/yes) onto a metric outcome (propensity to buy). However, logistic regression assumes that you have a binary outcome variable, so it is not applicable to your case.
  
  For what you want to do, you need to do a comparison of group means. Since you have one binary predictor, you could do a t-test, ANOVA or linear regression and should always obtain the same results. You could then interpret the coefficient for the treatment variable as you mentioned above, which also gives the benefit that it is directly interpretable as these methods are linear models and not generalized linear models, thus dropping the need for a link function.
  
  
  ---
  
  