Question: I'm just starting out learning Python/Spyder. I have completed a few classes in R for data science.

Below is failing with the error: IOError: File APPL.csv does not exist 

However, my script and .csv file are in the same folder. Any thoughts on what I'm doing wrong? Thanks

```
    import pandas as pd

     def test_run():
         df = pd.read_csv("APPL.csv")
         print df

     if __name__ == "__main__":
          test_run()

```

Answer: 
I implemented my own version of k-means on top of spark which uses standard TF-IDF vector representation and (-ve) cosine similarity as the distance metric [code snippet for reference](https://gist.github.com/rajanim/e01d1af591d697202c75ee0a785fa94f). The results from this k-means look right, not as skewed as spark k-means. [figure 1 and 2](https://i.stack.imgur.com/CkcIZ.png)
Additionally, I experimented by plugging in Euclidean distance as a similarity metric (into my own version of k-mean) and results continue to look right, not at all as skewed as spark k-means. Results indicate that its not issue with distance measure but some other case with spark's k-means implementation(scala mllib)
A late update incase anyone runs into this. Current versions of spark kmeans do implement cosine distance function, but the default is *euclidean*. For pyspark, this can be set in the constructor:

```py
from pyspark.ml.clustering import KMeans

km = KMeans(distanceMeasure='cosine', k=2, seed=1.0)
# or via setter
km.setDistanceMeasure('cosine')

```

[pyspark docs](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.clustering.KMeans.html#pyspark.ml.clustering.KMeans.distanceMeasure)

---

For Scala use the setter:

```scala
import org.apache.spark.ml.clustering.KMeans

val km = new KMeans()
km.setDistanceMeasure("cosine")

```

[Scala docs](https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/clustering/KMeans.html#distanceMeasure:org.apache.spark.ml.param.Param%5BString%5D)

Note that an exception will be thrown if there are any zero vectors in the feature vector.