Help in Understanding num.trees, mtry, and nodsize in Random forest? I am currently creating a random forest quantile regression model in R and I am looking to have a deeper understanding on of num.trees, mtry, and nodsize. I want to understand how increasing or decreasing these values can affect a model. For example, does increasing mtry overfit a model?  I am creating a grid search and looking to leverage dynamic parameters to train the ideal model

num.trees or the Number of Decision Trees
For num.trees, it is referring to the number of trees you want to build in your model. The standard is 500. Changing the number of trees typically won't change the results. However, if you were to plot the build, and it never leveled off, it would be an indicator to increase that number.
This plot is the build process, the x-axis is the trees (first tree to the last tree) and the y-axis reflects the error rate.

This plot does not quite level off, but the error rate is at the hundredths already, so more trees are unlikely to do all that much.
mtry or the Number of Variable Splits
When you build a tree, mtry is how many variables will be included in the first split. I have two dendrograms shown next. One is mtry = 2; the next the next is mtry = 3.
mtry = 2
The first dendrogram reflects a 2-way split or mtry = 2. I colored one blue and one black to try to make this more obvious.

mtry = 3
This next dendrogram, representing a three-way split, has three colors, one for each mtry. You can see the three clusters: blue, black, and red.

Using mtry for Tuning
Using mtry to tune your random forest is best done through tools like the library caret. That library runs many different models through their native packages but adds in automatic resampling. Using caret, resampling with random forest models is automatically done with different mtry values. There are other functions out there, like tuneRF() that indicate some best guess mtry values.
nodesize or Minimum Node Size
This parameter refers to the minimum number of observations to include in a terminal node. That is the bottom of the dendrogram.
I have reused the mtry = 2 dendrogram and marked one path in red. With the argument nodesize = 1, there must be at least one observation that meets the following criteria:




Variable
Requirement




Positive Affect
$\geq$ 0.78


Positive Affect
$\geq$ 0.82




Since it is the same variable in this path it may see obvious, but the path to every terminal node has requirements.
You can see that if you set this nodesize = 9 for example, but you only had 100 observations, you would probably have a really inaccurate model. It all depends on the data, though.

