title: Cannot resolve column name error for column passed as strings - Spark Scala
body: <p>I have a peculiar problem. I am trying to select some columns from a dataframe using automatically generated strings. While the selects are working fine if I pass them directly, if I put the same expression in strings and pass the string to select, it fails with the error &quot;Cannot resolve column name given list of columns&quot;. It is somehow adding backticks ` to the string and trying to find the full string as a column name (instead of evaluating the string expression and using that as a column name)</p>
  <p>Examples below:</p>
  <p>This works, historyJoinedDF is a dataframe which is an output of an outer join between two dataframes:</p>
  <pre><code>historyJoinedDF.select(
          currentUnmatchedDF(&quot;sessiondetail_sessionid&quot;).as(&quot;sessiondetail_sessionid&quot;)
        ).show()
  </code></pre>
  <p>This does not work:</p>
  <pre><code>val test = &quot;&quot;&quot;currentUnmatchedDF(&quot;sessiondetail_sessionid&quot;).as(&quot;sessiondetail_sessionid&quot;)&quot;&quot;&quot;
        historyJoinedDF.select(
        test
       .show()
  </code></pre>
  <p>Fails with:</p>
  <pre><code>cannot resolve '`currentUnmatchedDF(&quot;sessiondetail_sessionid&quot;).as(&quot;sessiondetail_sessionid&quot;)`' given input columns:
  </code></pre>
  <p>Not sure if it matters but my dataframe is the output of a full outer join between two dataframes with the same schema (thus the need for qualifying the column names with DF names).</p>
  <p>Edit: adding output of currentUnmatchedDF.show(false) &amp; historyJoinedDF .show(false) below, both have few hundred columns (which are same) so showing only the columns mentioned in the question.</p>
  <p><a href="https://i.stack.imgur.com/pdDVz.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pdDVz.png" alt="currentUnmatchedDF" /></a></p>
  <p><a href="https://i.stack.imgur.com/s3mjF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/s3mjF.png" alt="historyJoinedDF" /></a></p>