title: Airflow DAG Triggering
body: <p>We have recently tried to adopt Airflow as our "data workflow" engine, and while I have figure most of the things out, I am still in the grey area about how the scheduler calculates when to trigger DAGs.</p>
  
  <p>Take a look at this simple dag:</p>
  
  <pre><code>from airflow import DAG
  from datetime import datetime
  from airflow.operators.bash_operator import BashOperator
  
  dag_options = {                
              'owner':                'Airflow',  
              'depends_on_past':      False,      
              'start_date':           datetime.now()
  }
  
  with DAG('test_dag1', schedule_interval="5 * * * *", default_args=dag_options) as dag:
                  task1 = BashOperator(      
                  task_id='task1', 
                  bash_command='date',                
                  dag=dag)      
  </code></pre>
  
  <p>The schedule will pick this up, but <strong>will not</strong> execute it. Now if I change the "start_date" to:</p>
  
  <pre><code>datetime(year=xxxx,month=yyyy=day=zzzz) 
  </code></pre>
  
  <p>where xxxx,yyyy,zzzz are today's date, it will start executing. Is the cause of this that the scheduler keeps re-reading this dags from the source dag folder, executing datetime.now() each time, noticing the start date is different from currently queued, re-adding this dag and therefore re-scheduling/pushing the execution date forward (my <strong>dag_dir_list_interval</strong> is 300)?</p>
  
  <p>Also, in airflow, as I understand it, when a dag is un-paused (or added with dags_are_paused_at_creation = False), the scheduler will schedule the execution  as follows: </p>
  
  <ul>
  <li>1st dag execution: instant after (start_date + interval)</li>
  <li>2nd dag execution: instant after (start_date + (interval * 2)) </li>
  <li>3rd dag execution: instant after (start_date + (interval * 3))</li>
  </ul>
  
  <p>Is this correct assumption?</p>
  
  <p><strong>UPDATE (7/30/2017)</strong></p>
  
  <p>Based on the assumption above, I created this dag today (07/30/2017):</p>
  
  <pre><code>from airflow import DAG
  from datetime import datetime
  from airflow.operators.bash_operator import BashOperator
  
  dag_options = {                
              'owner':             'Airflow',  
              'depends_on_past':   False,      
              'start_date':   
  datetime(year=2017,month=7,day=30,hour=20,minute=10)
  }
  
  with DAG('test_dag_100', schedule_interval="*/10 * * * *", 
  default_args=dag_options) as dag:
                  task1 = BashOperator(      
                  task_id='task_100', 
                  bash_command='date',                
                  dag=dag)      
  </code></pre>
  
  <p>which should start on (UTC):</p>
  
  <ul>
  <li>7/30/2017 20:20:00</li>
  <li>7/30/2017 20:30:00</li>
  <li>7/30/2017 20:40:00</li>
  </ul>
  
  <p>Unfortunately this is not happening. 
  Here are some screen shots of my dashboard:</p>
  
  <ul>
  <li><a href="https://ibb.co/cf8kLQ" rel="nofollow noreferrer">Details of the dag</a></li>
  <li><a href="https://ibb.co/fmMuEk" rel="nofollow noreferrer">After when first execution should have happened</a></li>
  </ul>
  
  <p>Can someone explain why on 20:21:00 the dag did not execute? after 20:31:00 it still did not execute... What am i missing here?</p>
  
  <p>By the way, I also noticed that, for some reason, that every time I go and
  kick off a dag manually through the dashboard, it just sits in the "running" stage. Why is this? Does kicking it off manually have anything to do with any of the start timing options (start_date/interval/etc) ??</p>
  
  <p>Thank you for any clarifications you can provide</p>