Question: How do you convert a timestamp column to epoch seconds?

```
var df = sc.parallelize(Seq("2018-07-01T00:00:00Z")).toDF("date_string")
df = df.withColumn("timestamp", $"date_string".cast("timestamp"))
df.show(false)

```

DataFrame:

```
+--------------------+---------------------+
|date_string         |timestamp            |
+--------------------+---------------------+
|2018-07-01T00:00:00Z|2018-07-01 00:00:00.0|
+--------------------+---------------------+

```

Answer: 
Use [`unix_timestamp`](https://spark.apache.org/docs/2.0.2/api/java/org/apache/spark/sql/functions.html#unix_timestamp(org.apache.spark.sql.Column)) from `org.apache.spark.functions`. It can a timestamp column or from a string column where it is possible to specify the format. From the documentation:

> 
> 
> ```
> public static Column unix_timestamp(Column s)
> 
> ```
> 
> Converts time string in format yyyy-MM-dd HH:mm:ss to Unix timestamp (in seconds), using the default timezone and the default locale, return null if fail.
> 
> 
> 
> ```
> public static Column unix_timestamp(Column s, String p)
> 
> ```
> 
> Convert time string with given pattern (see <http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html>) to Unix time stamp (in seconds), return null if fail.
> 
> 
> 

Use as follows:

```
import org.apache.spark.functions._

df.withColumn("epoch_seconds", unix_timestamp($"timestamp")))

```

or if the column is a string with other format:

```
df.withColumn("epoch_seconds", unix_timestamp($"date_string", "yyyy-MM-dd'T'HH:mm:ss'Z'")))

```
It can be easily done with `unix_timestamp` function in spark SQL like this:

```
spark.sql("SELECT unix_timestamp(inv_time) AS time_as_long FROM agg_counts LIMIT 10").show()

```

Hope this helps.