title: spark: convert dataframe to svm labeled point
body: <p>When referring to the spark ml/mllib docs, they all start from a svm stored example. This is really frustrating me since there doesn't seem to be a straightforward way to go from a standard RDD[Row] or Dataframe (taken from a "table" select) to this notation without first storing it.</p>
  
  <p>This is just an inconvenience when dealing with 3 features or so, but when you scale that up to lots and lots of features, it's implying you will be doing a lot of typing and searching.</p>
  
  <p>I ended up with something like this: (where "train" is a random split of a dataset w/ features stored in a table)</p>
  
  <pre><code>val trainLp = train.map(row =&gt; LabeledPoint(row.getInt(0).toDouble, Vectors.dense(row(8).asInstanceOf[Int].toDouble,row(9).asInstanceOf[Int].toDouble,row(10).asInstanceOf[Int].toDouble,row(11).asInstanceOf[Int].toDouble,row(12).asInstanceOf[Int].toDouble,row(13).asInstanceOf[Int].toDouble,row(14).asInstanceOf[Int].toDouble,row(15).asInstanceOf[Int].toDouble,row(18).asInstanceOf[Int].toDouble,row(21).asInstanceOf[Int].toDouble,row(27).asInstanceOf[Int].toDouble,row(28).asInstanceOf[Int].toDouble,row(29).asInstanceOf[Int].toDouble,row(30).asInstanceOf[Int].toDouble,row(31).asInstanceOf[Double],row(32).asInstanceOf[Double],row(33).asInstanceOf[Double],row(34).asInstanceOf[Double],row(35).asInstanceOf[Double],row(36).asInstanceOf[Double],row(37).asInstanceOf[Double],row(38).asInstanceOf[Double],row(39).asInstanceOf[Double],row(40).asInstanceOf[Double],row(41).asInstanceOf[Double],row(42).asInstanceOf[Double],row(43).asInstanceOf[Double])))
  </code></pre>
  
  <p>This is a nightmare to maintain, since these rows tend to change pretty often.</p>
  
  <p>And here I'm only in the stage of getting labeled points, I'm not even at a svm stored version of this data.</p>
  
  <p>What am I missing here that could potentially save me days of misery?</p>
  
  <p>EDIT:</p>
  
  <p>I got one step closer to the solution using something called a <a href="http://spark.apache.org/docs/1.6.0/ml-features.html#vectorassembler" rel="nofollow noreferrer">vectorassembler</a> to build up my vector</p>