title: How to pull data by looping over dates in pyspark sql?
body: <p>I have a script where I'm pulling data into a pyspark DataFrame using spark sql. The script is shown below: </p>
  
  <pre><code>from pyspark import SparkContext, SparkConf, HiveContext
  from pyspark.sql import SparkSession
  spark = SparkSession.builder.enableHiveSupport().getOrCreate()
  
  df_query = """
  select 
    *
  from schema.table
  where start_date between date '2019-03-01' and date '2019-03-07'
  """
  df = spark.sql(df_query) 
  </code></pre>
  
  <p>Currently, the script pulls data for a particular week. However, I want to iterate this script over all weeks. How can I do that? </p>