title: ggplot fails to draw smooth gam using facet_wrap and group asthetic
body: <p>I am trying to plot a multi-panel and multi-line plot using ggplot with a group aesthetic as well as <code>facet_wrap</code>. However, the <code>geom_smooth</code> fails for all lines in the facet plot when one group has too few data points. </p>
  
  <pre><code>plot1 &lt;- ggplot(data=df1, 
                  aes(x=Year, y=Mean, group=Group2, linetype=Group2, shape=Group2)) +  
    geom_errorbar(aes(ymin=Mean-SE, ymax=Mean+SE), width=0.2) +  
    geom_smooth(method = "gam", se=F, formula = y ~ s(x, k=3), size = 1, colour="black") + 
    geom_point(position=pd, size=2, fill="white") +  
    scale_x_continuous(limits=c(min(df1$Year-0.1), max(df1$Year+0.1)), 
                       breaks=seq(min(df1$Year),max(df1$Year),5)) +  
    facet_wrap(~Group1, scales = "free", ncol=2) +  
    theme_bw() + 
    theme(axis.text.x = element_text(),
          axis.title.x = element_blank(), 
          strip.background = element_blank(), 
          axis.line.x = element_line(colour="black"),
          axis.line.y = element_line(colour="black"), 
          panel.grid.minor = element_blank(), 
          panel.grid.major = element_blank(),
          panel.border = element_blank(), 
          panel.background = element_blank(),
          legend.position="top",
          legend.title = element_blank())
  plot(plot1)
  </code></pre>
  
  <p>to produce plot the following plot. This is only summary data to make it easier. It's as if the error stops ggplot from computing the series smooths in that particular facet. </p>
  
  <p>data</p>
  
  <pre><code>Year    Group1      Group2      Mean        SE
  2011    Factor A    Factor C    30.62089116 3.672624771
  2011    Factor A    Factor D    54.99066324 2.822405771
  2011    Factor B    Factor C    30.48859003 3.748388489
  2011    Factor B    Factor D    45.70410611 4.284244405
  2017    Factor A    Factor C    33.68256601 4.030964172
  2017    Factor A    Factor D    53.43496462 4.687042033
  2017    Factor B    Factor C    23.08799875 5.17753488
  2001    Factor A    Factor C    23.79166667 2.837795432
  2001    Factor A    Factor D    23.75925926 3.688185081
  2001    Factor B    Factor C    29.05555556 4.08597798
  2001    Factor B    Factor D    28.13333333 7.877429079
  2008    Factor A    Factor C    23.3        2.383624691
  2008    Factor A    Factor D    28.83333333 2.750959429
  2008    Factor B    Factor C    34.01666667 5.340999698
  </code></pre>
  
  <p>and plot </p>
  
  <p><a href="https://i.stack.imgur.com/AmjwJ.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/AmjwJ.jpg" alt="plot 1"></a></p>
  
  <p>Clearly there is enough data to draw a smooth for factorC line in group factorB. Any idea? </p>