I have this code

```
groupedDF.show()
val window =
      Window.partitionBy($"app_id", $"country_code").orderBy($"rate".desc)

    val windowResult = groupedDF
      .transform(calculateRankOverWindow(window))
      .limit(topN) //change this to .where("rank<=topN") and it works.

    windowResult.show()
    val finalResult = windowResult
      .groupBy("app_id", "country_code")
      .agg(collect_list("advertiser_id").as("recommended_advertiser_ids"))

    finalResult.show()

```

Sample output of just one app\_id and country. as There are many different app\_ids and countries.

GrouedDF

```
+------+------------+-------------+-----------------+
|app_id|country_code|advertiser_id|             rate|
+------+------------+-------------+-----------------+
|    32|          UK|            9|              8.0|
|    32|          UK|            5|              5.5|
|    32|          UK|            4|              5.5|
|    32|          UK|            6|              6.1|
|    32|          UK|            3|              5.5|
|    32|          UK|            2|              2.0|
|    32|          UK|            1|6.099999999999999|
+------+------------+-------------+-----------------+

```

windowResult

```
+------+------------+-------------+-----------------+----+
|app_id|country_code|advertiser_id|             rate|rank|
+------+------------+-------------+-----------------+----+
|    32|          UK|            9|              8.0|   1|
|    32|          UK|            6|              6.1|   2|
|    32|          UK|            1|6.099999999999999|   3|
|    32|          UK|            5|              5.5|   4|
|    32|          UK|            4|              5.5|   4|
+------+------------+-------------+-----------------+----+

```

Desired Output format a list of advertiser\_ids per app\_id and country\_code.

Incorrect(What I get)

```
+------+------------+--------------------------+
|app_id|country_code|recommended_advertiser_ids|
+------+------------+--------------------------+
|    32|          UK|           [9, 5, 4, 6, 3]|
+------+------------+--------------------------+

```

Correct(what I want)

```
+------+------------+--------------------------+
|app_id|country_code|recommended_advertiser_ids|
+------+------------+--------------------------+
|    32|          UK|           [9, 6, 1, 5, 4]|
+------+------------+--------------------------+

```

But when I do groupby and collect the groupby messes the order or previous window operation and hence I collect the list [9,5,4,6,3] instead of [9,6,1,5,4].

How can I do it?

If I do

```
windowResult
      .withColumn(
        "recommended_advertiser_ids",
        collect_list("advertiser_id").over(window)
      )
      .show()

```

it gives

```
+------+------------+-------------+-----------------+----+--------------------------+
|app_id|country_code|advertiser_id|             rate|rank|recommended_advertiser_ids|
+------+------------+-------------+-----------------+----+--------------------------+
|    32|          UK|            9|              8.0|   1|                       [9]|
|    32|          UK|            6|              6.1|   2|                    [9, 6]|
|    32|          UK|            1|6.099999999999999|   3|                 [9, 6, 1]|
|    32|          UK|            5|              5.5|   4|           [9, 6, 1, 5, 4]|
|    32|          UK|            4|              5.5|   4|           [9, 6, 1, 5, 4]|
+------+------------+-------------+-----------------+----+--------------------------+

```

but I just want the last one per group(app\_id, country\_code).

**UPDATE:** I fixed it by reverting to my original code. i.e.
switched back to `df.where(rank<=n) instead of limit(n)`
But I the reason I switched is rank<=n can give more than n results if the column has same values. So the question is how to pick top n from rank(topN per group)?