title: Is it possible to scale data by group in Spark?
body: <p>I want to scale data with <code>StandardScaler</code> (<code>from pyspark.mllib.feature import StandardScaler</code>), by now I can do it by passing the values of RDD to transform function, but the problem is that I want to preserve the key. is there anyway that I scale my data by preserving its key? </p>
  
  <p>Sample dataset</p>
  
  <pre><code>0,tcp,http,SF,181,5450,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,8,8,0.00,0.00,0.00,0.00,1.00,0.00,0.00,9,9,1.00,0.00,0.11,0.00,0.00,0.00,0.00,0.00,normal.
  0,tcp,http,SF,239,486,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,8,8,0.00,0.00,0.00,0.00,1.00,0.00,0.00,19,19,1.00,0.00,0.05,0.00,0.00,0.00,0.00,0.00,normal.
  0,tcp,http,SF,235,1337,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,8,8,0.00,0.00,0.00,0.00,1.00,0.00,0.00,29,29,1.00,0.00,0.03,0.00,0.00,0.00,0.00,0.00,smurf.
  </code></pre>
  
  <p>Imports</p>
  
  <pre><code>import sys
  import os
  from collections import OrderedDict
  from numpy import array
  from math import sqrt
  try:
      from pyspark import SparkContext, SparkConf
      from pyspark.mllib.clustering import KMeans
      from pyspark.mllib.feature import StandardScaler
      from pyspark.statcounter import StatCounter
  
      print ("Successfully imported Spark Modules")
  except ImportError as e:
      print ("Can not import Spark Modules", e)
      sys.exit(1)
  </code></pre>
  
  <p>Portion of code </p>
  
  <pre><code>    sc = SparkContext(conf=conf)   
      raw_data = sc.textFile(data_file)
      parsed_data = raw_data.map(Parseline)
  </code></pre>
  
  <p><code>Parseline</code> function:</p>
  
  <pre><code>def Parseline(line):
      line_split = line.split(",")
      clean_line_split = [line_split[0]]+line_split[4:-1]
      return (line_split[-1], array([float(x) for x in clean_line_split]))
  </code></pre>