Question: Is it possible and what would be the most efficient neat method to add a column to Data Frame? 

More specifically, column may serve as Row IDs for the existing Data Frame.

In a simplified case, reading from file and not tokenizing it, I can think of something as below (in Scala), but it completes with errors (at line 3), and anyways doesn't look like the best route possible: 

```
var dataDF = sc.textFile("path/file").toDF() 
val rowDF = sc.parallelize(1 to DataDF.count().toInt).toDF("ID") 
dataDF = dataDF.withColumn("ID", rowDF("ID")) 

```

Answer: 
I took help from above answer. However, I find it incomplete if we want to change a `DataFrame` and current APIs are little different in `Spark 1.6`. 
`zipWithIndex()` returns a `Tuple` of `(Row, Long)` which contains each row and corresponding index. We can use it to create new `Row` according to our need. 

```
val rdd = df.rdd.zipWithIndex()
             .map(indexedRow => Row.fromSeq(indexedRow._2.toString +: indexedRow._1.toSeq))
val newstructure = StructType(Seq(StructField("Row number", StringType, true)).++(df.schema.fields))
sqlContext.createDataFrame(rdd, newstructure ).show

```

I hope this will be helpful.
You can use [row\_number](https://spark.apache.org/docs/2.0.2/api/java/org/apache/spark/sql/functions.html#row_number()) with [Window function](https://databricks.com/blog/2015/07/15/introducing-window-functions-in-spark-sql.html) as below to get the distinct id for each rows in a dataframe.

```
df.withColumn("ID", row_number() over Window.orderBy("any column name in the dataframe"))

```

You can also use `monotonically_increasing_id` for the same as 

```
df.withColumn("ID", monotonically_increasing_id())

```

And there are some [other ways](https://hadoopist.wordpress.com/2016/05/24/generate-unique-ids-for-each-rows-in-a-spark-dataframe/) too.