title: Java-Spark: how to get a Dataset<Row> column's value when iterating in a loop and use it in when().otherwise()?
body: <p>I have a <code>Dataset&lt;Row&gt;</code> with columns having values <code>"null"</code>(null written text).<br>
  I am trying to replace the <strong>"null"</strong> text to text:<strong>\N</strong>.<br>
  For this I am using a logic that I will add a new column with name appended by "_nulled", e.g. column <strong>abc</strong> becomes <strong>abc_nulled</strong> and this new column will have the value <strong>"\N"</strong> if the current value is text <strong>null</strong> other wise the value remains the same.<br>
  For this I have used <code>withColumn(&lt;new name&gt;, when(col.equalTo("null"), "\\N").otherwise(&lt;existing_value&gt;))</code>.
  How do I obtain this <code>&lt;existing_value&gt;</code>.<br>
  When I am passing <code>otherwise(ds.col(col_nm))</code> it is not working, possibly because it it expecting a <code>String</code> in <code>otherwise()</code> and finding a <code>Column</code>.</p>
  
  <p>How should I resolve this? Here is the code:</p>
  
  <pre><code>ArrayList&lt;String&gt; newCols = new ArrayList&lt;String&gt;();
  List&lt;String&gt; reqColListCopy = Arrays.asList(reqCols);
  Dataset&lt;Row&gt; testingDS = DS.selectExpr(JavaConverters.asScalaIteratorConverter(reqColListCopy.iterator()).asScala().toSeq())
  
  //Creating newCols (ArrayList so that I can add/remove column names.
  Iterator itrTmp2 = reqColListCopy.iterator();
  while(itrTmp2.hasNext()){
      newCols.add((String)itrTmp2.next());
  }
  
  //Creating a List reference for newCols ArrayList. This will be used to get Seq(&lt;columns&gt;).
  List&lt;String&gt; newColsList = newCols;
  
  Iterator colListItr = reqColListCopy.iterator();
  while(colListItr.hasNext())
  {
      String col = colListItr.next().toString();
      testingDS = testingDS.selectExpr(convertListToSeq(newColsList))
              .withColumn(col+"_nulled",  functions.when(testingDS.col(col).equalTo("null"), functions.lit("\\N")).otherwise(testingDS.col(col))) //'otherwise' needs a string parameter
              .drop(testingDS.col(col));
  
      newCols.add(col+"_nulled");
      newCols.remove(col);
      newColsList = newCols;
  }
  Dataset&lt;Row&gt; testingDS = DS.selectExpr(JavaConverters.asScalaIteratorConverter(newColsList.iterator()).asScala().toSeq())
  
  testingDS.show(false);
  </code></pre>