title: R Best way to merge large dataframes of the same size
body: <p>I have one huge dataframe (400k rows by 8000 cols), another dataframe that's 400k rows by 3 cols, and a third dataframe that's 402k rows by 3 cols. Each df has 2 key columns. Label them:</p>
  
  <p><code>dfBig</code> - biggest df</p>
  
  <p><code>dfSmall1</code> - the df with 402k rows</p>
  
  <p><code>dfSmall2</code> - the df with the same number of rows</p>
  
  <p>when I try to compile all the information with the following <code>dplyr</code> solution my RStudio session gets aborted:</p>
  
  <pre><code>dfCompile &lt;- dfBig %&gt;%
    left_join(dfSmall1, by = c('Key1', 'Key2')) %&gt;% 
    left_join(dfSmall2, by = c('Key1', 'Key2')) %&gt;%
    select(Key1, Key2, newCol1, newCol2, everything())
  </code></pre>
  
  <p>I can't even run <code>dfBig %&gt;% arrange(Key1, Key2)</code> without it blowing up. What I've done instead to make it work is:</p>
  
  <pre><code># Grab only the keys to cut down on size
  dfInfo &lt;- dfBig %&gt;% select(Key1, Key2)
  
  # Merge other dfs to the info DF
  dfInfo &lt;- dfInfo %&gt;%
    left_join(payoutMap, by = c('Key1', 'Key2')) %&gt;% 
    left_join(ranks, by = c('Key1', 'Key2')) %&gt;% 
    select(-Key1, -Key2)
  
  # Cbind the rest of the columns back in since I know they'll be in the same order (arrange(.) blows it up)
  dfBig &lt;- cbind(dfBig, dfInfo) %&gt;% 
    select(Key1, Key2, newCol1, newCol2, everything())
  </code></pre>
  
  <p>It works, but it doesn't feel very eloquent, and I don't want to have to mess with it again if it breaks down with an even bigger dataframe. Can <code>sqldf</code> or <code>data.table</code> handle this better? Just want to understand the best path to go down before I start learning for myself.</p>