title: Spark Column Renamed
body: <p>I am just trying to understand why the below "withColumnRenamed" function is not working.I  don't have a reason to do this but i am trying to understand why it fails:</p>
  
  <pre><code>val a = sqlContext.sql("msck repair table db_name.table_name")
  a: org.apache.spark.sql.DataFrame = [result: string]
  scala&gt; a.show()
  +------+
  |result|
  +------+
  +------+
  
  
  scala&gt; a.printSchema
  root
   |-- result: string (nullable = false)
  
  
      a.withColumnRenamed("result","res")
      org.apache.spark.sql.AnalysisException: resolved attribute(s) result#32 missing from result#22 in operator !Project [result#32 AS res#33];
  </code></pre>
  
  <p>But this works:</p>
  
  <pre><code>a.select($"result".as("res"))
  </code></pre>
  
  <p>I looked at the Plan and seems like its getting assigned a different unique column id for the same column at different stages of the plan,the unique id for a given column should be same across the plan ,correct?</p>
  
  <pre><code>a.explain(true)
  == Parsed Logical Plan ==
  HiveNativeCommand msck repair table db_name.table_name
  == Analyzed Logical Plan ==
  result: string
  HiveNativeCommand msck repair table db_name.table_name
  == Optimized Logical Plan ==
  HiveNativeCommand msck repair table db_name.table_name
  == Physical Plan ==
  ExecutedCommand HiveNativeCommand msck repair table db_name.table_name
  scala&gt; a.queryExecution.logical.output
  res76: Seq[org.apache.spark.sql.catalyst.expressions.Attribute] = List(result#140)
  scala&gt; a.queryExecution.analyzed.output
  res77: Seq[org.apache.spark.sql.catalyst.expressions.Attribute] = List(result#141)
  scala&gt; a.queryExecution.optimizedPlan.output
  res78: Seq[org.apache.spark.sql.catalyst.expressions.Attribute] = List(result#142)
  </code></pre>
  
  <p>I am using Spark 1.6</p>