Answers.Body: <p>In my opinion, no.
  Also images could be interpreted as tabular dataset as well, where certain columns represent different rgb codes of pixels. If you seek to use neural nets opt for image datasets, with large sample size.
  Neural networks generally require large sample sizes to perform, and huge dimension inputs to not be outperformed by boosting.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-11T15:59:59.740Z
Answers.Text: That makes a lot of sense, thank you Matthew! May I ask if you know any complex / large tabular dataset (e.g. perhaps with low-level features as columns) that could be difficult for boosting? Thank you!
Answers.UserDisplayName: Clara
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-12T07:25:37.820Z
Answers.Text: Unfortunately, I don't. But you can always make one for yourself, out of image data sets,
Answers.UserDisplayName: MatthewTuby
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-12T20:18:42.797Z
Answers.Text: I can confirm this anecdotally.  I have tried many times to apply NNs to tabular and/or non-complex data and I cannot recall a NN ever doing better than more "traditional" models.  Ha ha, of course, we all know that the plural of anecdote is *not* data!  So exceptions may exist.  I do, however, disagree with trying to represent an image as column data.
Answers.UserDisplayName: user1269942
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-13T07:38:12.003Z
Answers.Text: Disagree on what ground? Possibility or usefulness?
Answers.UserDisplayName: MatthewTuby
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-15T21:34:17.970Z
Answers.Text: @MatthewTuby  on the grounds that it will *likely* make the typical convolution operators ineffective as "columnified" image data will likely lose the benefit that pixels near-by in the actual image are near-by in the new representation(column data) like they are in the standard RGB layered matrix representation. A pedant could assert that the normal matrix representation is constructed from columns...but that is not what is being implied here. One example of image data in column format, used in GIS, is the .csv format with encodes a single channel image into "x,y,value" rows.
Answers.UserDisplayName: user1269942
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-15T21:36:58.580Z
Answers.Text: @MatthewTuby  a quick note to say that current image modeling success is highly dependent on convolution operations.
Answers.UserDisplayName: user1269942
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-16T14:25:57.410Z
Answers.Text: @user1269942 Do we require convolution operator on tabular data set? Since the question originally focused on tabular data, I somehow implied that the "deep neural network" term refers to multilayer perceptrons and not convolutional networks.
Answers.UserDisplayName: MatthewTuby
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-17T16:12:59.263Z
Answers.Text: @MatthewTuby we're going to get shut down for too many comments!  "are convolution operators required?"  --no.  But in some cases they work(some tabular time series data for example).
Answers.UserDisplayName: user1269942
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-11T12:26:38.217Z
Answers.LastActivityDate: 2019-12-11T12:26:38.217Z
Answers.UserDisplayName: MatthewTuby
Body: <p>Are there (complex) tabular datasets where deep neural networks (e.g. more than 3 layers) outperform traditional methods such as XGBoost by a large margin?</p>
  <p>I'd prefer tabular datasets rather than image datasets, since most image dataset are either too simple that even XGBoost can perform well (e.g. MNIST), or too difficult for XGBoost that its performance is too low (e.g. almost any dataset that is more complex than CIFAR10; please correct me if I'm wrong).</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-12T20:27:57.977Z
Comments.Text: I just remembered a good example... http://blog.kaggle.com/2012/11/01/deep-learning-how-i-did-it-merck-1st-place-interview/  but that was in 2012 and didn't outperform "by a large margin" and was before the xgboost mania.
Comments.UserDisplayName: user1269942
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-14T20:54:41.490Z
Comments.Text: I'll try out the dataset used in this post: it's great that the columns are uncleaned, since I'd expect the advantage of DNN to be better pronounced when the features are less structured. Thank you @user1269942!
Comments.UserDisplayName: Clara
Tags: <neural-networks><deep-learning><datasets><gradient-boosting><boosting>
Title: Are there tabular datasets where deep neural networks outperform traditional methods?
UserDisplayName: Clara
ThreadText: # Are there tabular datasets where deep neural networks outperform traditional methods?
  
  (Asked by: username_0 on 2019-12-10)
  
  Are there (complex) tabular datasets where deep neural networks (e.g. more than 3 layers) outperform traditional methods such as XGBoost by a large margin?
  
  I'd prefer tabular datasets rather than image datasets, since most image dataset are either too simple that even XGBoost can perform well (e.g. MNIST), or too difficult for XGBoost that its performance is too low (e.g. almost any dataset that is more complex than CIFAR10; please correct me if I'm wrong).
  
  **Tags:** neural-networks, deep-learning, datasets, gradient-boosting, boosting
  
  > **Comments:**
  > **username_1**: I just remembered a good example... http://blog.kaggle.com/2012/11/01/deep-learning-how-i-did-it-merck-1st-place-interview/  but that was in 2012 and didn't outperform "by a large margin" and was before the xgboost mania.  
  > **username_0**: I'll try out the dataset used in this post: it's great that the columns are uncleaned, since I'd expect the advantage of DNN to be better pronounced when the features are less structured. Thank you @username_1!  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-12-11. Score: 0
  
  In my opinion, no.
  Also images could be interpreted as tabular dataset as well, where certain columns represent different rgb codes of pixels. If you seek to use neural nets opt for image datasets, with large sample size.
  Neural networks generally require large sample sizes to perform, and huge dimension inputs to not be outperformed by boosting.
  
  > **Comments:**
  > **username_0**: That makes a lot of sense, thank you Matthew! May I ask if you know any complex / large tabular dataset (e.g. perhaps with low-level features as columns) that could be difficult for boosting? Thank you!  
  > **username_2**: Unfortunately, I don't. But you can always make one for yourself, out of image data sets,  
  > **username_1**: I can confirm this anecdotally.  I have tried many times to apply NNs to tabular and/or non-complex data and I cannot recall a NN ever doing better than more "traditional" models.  Ha ha, of course, we all know that the plural of anecdote is *not* data!  So exceptions may exist.  I do, however, disagree with trying to represent an image as column data.  
  > **username_2**: Disagree on what ground? Possibility or usefulness?  
  > **username_1**: @username_2  on the grounds that it will *likely* make the typical convolution operators ineffective as "columnified" image data will likely lose the benefit that pixels near-by in the actual image are near-by in the new representation(column data) like they are in the standard RGB layered matrix representation. A pedant could assert that the normal matrix representation is constructed from columns...but that is not what is being implied here. One example of image data in column format, used in GIS, is the .csv format with encodes a single channel image into "x,y,value" rows.  
  > **username_1**: @username_2  a quick note to say that current image modeling success is highly dependent on convolution operations.  
  > **username_2**: @username_1 Do we require convolution operator on tabular data set? Since the question originally focused on tabular data, I somehow implied that the "deep neural network" term refers to multilayer perceptrons and not convolutional networks.  
  > **username_1**: @username_2 we're going to get shut down for too many comments!  "are convolution operators required?"  --no.  But in some cases they work(some tabular time series data for example).  
  
  ---
  
  