title: optimizing the for loop for faster performance
body: <p>I have a dataframe that contains the similarity scores 100x100 for each 100 products against 100 products(data_neighbours). I have another dataframe that has the data at user and product level(1000x100). I want to go through each product for each user and get top10 similar products from data_neighbours and their corresponding similarity scores and compute a function getScore as below:</p>
  
  <pre><code>def getScore(history, similarities):
     return sum(history*similarities)/sum(similarities)
  
  for i in range(0,len(data_sims.index)):
      for j in range(1,len(data_sims.columns)):
          user = data_sims.index[i]
          product = data_sims.columns[j]
  
          if data.ix[i][j] == 1:
              data_sims.ix[i][j] = 0
          else:
              product_top_names = data_neighbours.ix[product][1:10]
              product_top_sims = data_ibs.ix[product].order(ascending=False)[1:10]
              user_purchases = data_germany.ix[user,product_top_names]
  
              data_sims.ix[i][j] = getScore(user_purchases,product_top_sims)
  </code></pre>
  
  <p>How can I optimize this loop for faster processing. The example has been cited from here: <code>http://www.salemmarafi.com/code/collaborative-filtering-with-python/</code></p>
  
  <p>Sample data:
  Data:(1000x101) user is the 101th column:</p>
  
  <pre><code>Index   user    song1   song2.....
  0        1       0       0
  1        33      0       1
  2        42      1       0
  3        51      0       0
  </code></pre>
  
  <p>data_ibs(similarity scores)--(100x100):</p>
  
  <pre><code>    song1   song2   song3   song4
  song1   1.00    0.00    0.02    0.05
  song2   0.00    1.00    0.05    0.03
  song3   0.02    0.05    1.00    0.11
  song4   0.05    0.03    0.11    1.00
  </code></pre>
  
  <p>data_neighbours(top10 similar songs for each song based on sorted score from data_ibs)--(100x10):</p>
  
  <pre><code>          1       2      3......... 10
  song1   song5   song10  song4
  song2   song8   song11  song5
  song3   song9   song12  song10
  </code></pre>
  
  <p>data germany(user level data for each song as column, except userid)--(1000x100):</p>
  
  <pre><code>index   song1   song2   song3
  1        0        0     0
  2        1        0     0
  3        0        0     1
  </code></pre>
  
  <p>Expected dataset(data_sims)--1000x101:</p>
  
  <pre><code>user    song1   song2   song3
  1      0.00     0.00    0.22
  33     0.09     0.00    0.11
  42     0.00     0.10    0.00
  51     0.09     0.09    0.00
  </code></pre>
  
  <p>where if value is 1 in data for any song, basically its score is set to 0, other cases, top 10 songs are fetched from data_neighbours and corresponding scores from data_ibs. Now it is checked if those songs are already present for the user or not(1,0) in user_purchases dataset. finally similarity scores are computed for the ixj position using user_purchses(1/0 values for each top 10 song) multiply by similarity score from data_ibs and divide by the sum of total top 10 similarity scores. Repeat the same for all the user x song combination.</p>