title: How to make a summary data.frame in R, with mixed data types
body: <p>Fully aware that this <em>type</em> of question has been asked hundreds of times.<br />
  Still, I could not find an answer to the specific problem I am describing, which is about:</p>
  <ul>
  <li>performance (i.e. I know <em>how</em> to do what I need, but it's too slow in some cases, so I am looking for a faster solution)</li>
  <li>good programming practice (i.e. I am questioning whether the approach I chose is 'clean' rather than roundabout or inefficient for other reasons)</li>
  </ul>
  <p>I have a data.frame with numerical and character columns. I need to create a data.frame summary from it, <em>grouping by one of the character columns (<code>ID</code>)</em>, and reporting 1) some stats about some numerical columns within each group, <em>and</em> 2) some character concatenations (i.e. the report has <em>mixed</em> data types - that's what makes it tricky, at least for me, and that's why I am asking for advice).</p>
  <p>Here is the <code>R</code> script:</p>
  <pre><code># Simulate original data.frame
  
  set.seed(384092)
  
  N &lt;- 10000
  
  d &lt;- data.frame(&quot;ID&quot; = paste0(sample(LETTERS, N, replace = T),  sprintf(&quot;%03.0f&quot;, sample(1:floor(sqrt(N)), N, replace = T )) ), stringsAsFactors = F)
  d[&quot;set&quot;] &lt;- sample(LETTERS, N, replace = T)
  d[&quot;P&quot;] &lt;- runif(N, -20, 120)
  d[&quot;K&quot;] &lt;- rnorm(N, 10, 0.5)
  
  # Make summary
  # For each unique ID, report: ID, number of rows of d, mean of P, sd of P, comma-separated list of unique set's
  
  # Method 1: rbind data.frames from 'by'
  
  time.1 &lt;- system.time({
    d_summary.1 &lt;- do.call(rbind, by(d, d$ID, function(dd) {
      data.frame(&quot;ID&quot; = dd$ID[1], &quot;N&quot; = nrow(dd), &quot;P_mean&quot; = mean(dd$P), &quot;P_sd&quot; = sd(dd$P), &quot;sets&quot; = paste(unique(dd$set), collapse = &quot;,&quot;))
    })
    )
  })
  
  cat(&quot;\ntime.1 =&quot;,time.1,&quot;\n&quot;)
  print(sapply(d_summary.1, class))
  
  # Method 2: create a list of lists and combine them at the end
  # https://stackoverflow.com/a/68162050/6376297
  
  time.2 &lt;- system.time({
    time.2.1 &lt;- system.time({d_summary.2 &lt;- by(d, d$ID, function(dd) {
      list(&quot;ID&quot; = dd$ID[1], &quot;N&quot; = nrow(dd), &quot;P_mean&quot; = mean(dd$P), &quot;P_sd&quot; = sd(dd$P), &quot;sets&quot; = paste(unique(dd$set), collapse = &quot;,&quot;))
    })
    })
    d_summary.2 &lt;- do.call(rbind, lapply(d_summary.2, data.frame))
  })
  
  cat(&quot;\ntime.2.1 =&quot;,time.2.1)
  cat(&quot;\ntime.2 =&quot;,time.2,&quot;\n&quot;)
  print(sapply(d_summary.2, class))
  </code></pre>
  <p>which on my PC produces the following output:</p>
  <pre><code>time.1 = 1.72 0 1.72 NA NA 
           ID           N      P_mean        P_sd        sets 
  &quot;character&quot;   &quot;integer&quot;   &quot;numeric&quot;   &quot;numeric&quot; &quot;character&quot; 
  
  time.2.1 = 0.3 0 0.29 NA NA
  time.2 = 1.79 0 1.82 NA NA 
           ID           N      P_mean        P_sd        sets 
  &quot;character&quot;   &quot;integer&quot;   &quot;numeric&quot;   &quot;numeric&quot; &quot;character&quot;
  </code></pre>
  <p>The linked post <a href="https://stackoverflow.com/a/68162050/6376297">https://stackoverflow.com/a/68162050/6376297</a> specifically mentions that the kind of handling used in method 2 is necessary to avoid coercing all columns to a single data type.<br />
  And indeed, any solution I tried that relies on making an intermediate matrix, as fully expected, results in coercion to character.</p>
  <p>This is really unfortunate, because as shown by <code>time.2.1</code>, the initial formation of a list of lists containing the required information (and still retaining all the original data types) takes only 1/6 - 1/5 of the total time.<br />
  And you need to imagine that I am doing this on <code>d</code>'s at least 10-100 times larger than this example.</p>
  <blockquote>
  <p>Would anyone be able to advise / suggest a faster method to do this?</p>
  </blockquote>
  <p>Thanks!</p>
  <hr />
  <p><strong>EDIT</strong> : follow up from users' feedback</p>
  <p>Trial of the <code>dplyr</code> (4) and <code>data.table</code> (5) method, plus a couple more base <code>R</code> methods (using <code>aggregate</code>, (6) and (7)) that are more involved but might be somewhat competitive with those two.</p>
  <pre><code># Method 4: dplyr
  
  require(dplyr)
  
  time.4 &lt;- system.time({
    d %&gt;% 
      group_by(ID) %&gt;% 
      summarise(N = n(),
                P_mean = mean(P),
                P_sd = sd(P),
                sets = paste(unique(set), collapse = &quot;,&quot;)) -&gt; d_summary.4
  })
  
  cat(&quot;\ntime.4 =&quot;,time.4,&quot;\n&quot;)
  print(sapply(d_summary.4, class))
  
  # Method 5: data.table
  
  require(data.table)
  
  time.5 &lt;- system.time({
    setDT(d)
    
    d_summary.5 &lt;- d[, .(N = .N, 
          P_mean = mean(P), 
          P_sd = sd(P), 
          sets = toString(unique(set))), ID]
    
    d_summary.5 &lt;- as.data.frame(d_summary.5)
    
  })
  
  cat(&quot;\ntime.5 =&quot;,time.5,&quot;\n&quot;)
  print(sapply(d_summary.5, class))
  
  # Method 6: aggregate each column separately and merge
  
  time.6 &lt;- system.time({
    
    d_summary.6 &lt;- setNames(as.data.frame(table(d$ID), stringsAsFactors = F),c(&quot;ID&quot;,&quot;N&quot;))
    d_summary.6 &lt;- merge(d_summary.6, setNames(aggregate(P ~ ID, data = d, FUN = mean),c(&quot;ID&quot;,&quot;P_mean&quot;)), by = &quot;ID&quot;)
    d_summary.6 &lt;- merge(d_summary.6, setNames(aggregate(P ~ ID, data = d, FUN = sd),c(&quot;ID&quot;,&quot;P_sd&quot;)), by = &quot;ID&quot;)
    d_summary.6 &lt;- merge(d_summary.6, setNames(aggregate(set ~ ID, data = d, FUN = function(x) {paste(unique(x),collapse=&quot;,&quot;)}),c(&quot;ID&quot;,&quot;sets&quot;)), by = &quot;ID&quot;)
    
  })
  
  cat(&quot;\ntime.6 =&quot;,time.6,&quot;\n&quot;)
  print(sapply(d_summary.6, class))
  
  # Method 7: aggregate each column separately and cbind (this assumes that both table and aggregate will report all values of ID, sorted)
  
  time.7 &lt;- system.time({
    
    d_summary.7 &lt;- setNames(as.data.frame(table(d$ID), stringsAsFactors = F),c(&quot;ID&quot;,&quot;N&quot;))
    d_summary.7 &lt;- cbind(d_summary.7, &quot;P_mean&quot; = aggregate(P ~ ID, data = d, FUN = mean)[,2])
    d_summary.7 &lt;- cbind(d_summary.7, &quot;P_sd&quot; = aggregate(P ~ ID, data = d, FUN = sd)[,2])
    d_summary.7 &lt;- cbind(d_summary.7, &quot;sets&quot; = aggregate(set ~ ID, data = d, FUN = function(x) {paste(unique(x),collapse=&quot;,&quot;)})[,2])
    
  })
  
  cat(&quot;\ntime.7 =&quot;,time.7,&quot;\n&quot;)
  print(sapply(d_summary.7, class))
  </code></pre>
  <p>Timing:</p>
  <pre><code>time.1 = 1.73 0.02 1.77 NA NA 
           ID           N      P_mean        P_sd        sets 
  &quot;character&quot;   &quot;integer&quot;   &quot;numeric&quot;   &quot;numeric&quot; &quot;character&quot; 
  
  time.2.1 = 0.29 0 0.3 NA NA
  time.2 = 1.83 0.01 1.84 NA NA 
           ID           N      P_mean        P_sd        sets 
  &quot;character&quot;   &quot;integer&quot;   &quot;numeric&quot;   &quot;numeric&quot; &quot;character&quot; 
  
  time.4 = 0.13 0 0.13 NA NA 
           ID           N      P_mean        P_sd        sets 
  &quot;character&quot;   &quot;integer&quot;   &quot;numeric&quot;   &quot;numeric&quot; &quot;character&quot; 
  
  time.5 = 0.08 0 0.08 NA NA 
           ID           N      P_mean        P_sd        sets 
  &quot;character&quot;   &quot;integer&quot;   &quot;numeric&quot;   &quot;numeric&quot; &quot;character&quot; 
  
  time.6 = 0.25 0 0.25 NA NA 
           ID           N      P_mean        P_sd        sets 
  &quot;character&quot;   &quot;integer&quot;   &quot;numeric&quot;   &quot;numeric&quot; &quot;character&quot; 
  
  time.7 = 0.25 0 0.25 NA NA 
           ID           N      P_mean        P_sd        sets 
  &quot;character&quot;   &quot;integer&quot;   &quot;numeric&quot;   &quot;numeric&quot; &quot;character&quot; 
  </code></pre>