Question: Python Q. How to parse an object index in a data frame into its date, time, and time zone when it has multiple time zones?

The format is "YYY-MM-DD HH:MM:SS-HH:MM" where the right "HH:MM" is the timezone.

Example: Midnight Jan 1st, 2020 in Mountain Time, counting up:

```
2020-01-01 00:00:00-07:00
2020-01-01 01:00:00-07:00
2020-01-01 02:00:00-07:00
2020-01-01 04:00:00-06:00

```

I've got code that works for one time zone, but it breaks when a second timezone is introduced.

```
df['Date'] = pd.to_datetime(df.index)
df['year']= df['Date'].dt.year
df['month']= df['Date'].dt.month
df['month_n']= df['Date'].dt.month_name()
df['day']= df['Date'].dt.day
df['day_n']= df['Date'].dt.day_name()
df['h']= df['Date'].dt.hour
df['mn']= df['Date'].dt.minute
df['s']= df['Date'].dt.second

```

> 
> ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc="True"
> 
> 
>

Answer: 
The issue arises with `json.dump()`. For this function to write a JSON file output, a valid JSON object has to be given which is not an RDD (returned by df.toJSON()). I got the same error when I tried using the same code.

[![enter image description here](https://i.imgur.com/rLtOZbo.png)](https://i.imgur.com/rLtOZbo.png)

* To fix the code, you can get the output of your dataframe as a Dictionary. This can be done using `df_json.collect()`. The following will be the output when we use `df_json.collect()` for my sample data

```
print(df_json.collect())

```

![enter image description here](https://i.imgur.com/8yCPHtS.png)

* You can see that above is an array of strings (where each string is json object). You can follow the code below to convert it to a complete JSON dictionary and successfully write it.

```
output = [eval(i) for i in df_json.collect()]
#output variable has the required generated json

import json
file_out="output.json"
#file would be saved in /databricks/driver/
with open(file_out, 'w') as f:
    json.dump(output, f)

```

* Use `dbutils.fs.ls()` to verify. `/databricks/driver/` will be the location of the saved file in Databricks when file path is just filename (file\_out="output.json")

```
display(dbutils.fs.ls("file:/databricks/driver"))

```

![enter image description here](https://i.imgur.com/ciguaxz.png)

* When I read the same file, you can see that it is successful and given json data.

```
with open(file_out, 'r') as k:
    ans = json.load(k)

```

[![enter image description here](https://i.stack.imgur.com/9Vsn9.png)](https://i.stack.imgur.com/9Vsn9.png)
As the alternative:

```py
df = spark.read.format("csv") \
                    .option("inferSchema", False) \
                    .option("header", True) \
                    .option("sep", ",") \
                    .load("path-to-file")

df.toPandas().to_json("/mnt/some-dir/out.json")

```