Answers.Body: <blockquote>
  <p>Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure?</p>
  </blockquote>
  <p>Generally, yes. Most decision tree learners, like the common ID3 and C4.5/C5.0 algorithms, are deterministic. At each step, the learners consider all possible features that have not yet been used to split the data and find the splits that maximize some function (e.g. information gain). There is no randomness (or pseudo-randomness) in this process.</p>
  <p>The exceptions to this would be if you used randomness to break ties (rather than, say, using the index of each feature, as is common), but this would be an unusual modification.
  What about the random forest?</p>
  <p>As the name suggests, random forests <em>do</em> make use of randomness, or at least, pseudo-randomness. If we're only concerned about whether or not the algorithm is deterministic in the usual sense of the word (at least, within computer science), the answer is <strong>no</strong>.</p>
  <p>If you start the same random forest learning algorithm, with the same datasets, at two different times (or, using two different seeds for your pseudorandom number generator), you will get two different forests. This is because the algorithm selects random subsets of the features and/or datapoints to learn on, and, if different seeds are used, the subsets will be different each time.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-15T14:08:01.293Z
Answers.Text: It is also worth noting that early stopping might use a random subset of the data for validation.
Answers.UserDisplayName: NikoNyrh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-09T12:48:40.827Z
Answers.Text: 'At each step, the learners consider all possible features that have not yet been used to split the data and find the splits that maximize some function (e.g. information gain).' I think that decision trees also consider features that have already been used.
Answers.UserDisplayName: kklaw
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-02T03:47:56.720Z
Answers.LastActivityDate: 2022-01-15T05:53:13.960Z
Answers.LastEditDate: 2022-01-15T05:53:13.960Z
Answers.UserDisplayName: John Doucette
Body: <p>Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? </p>
  
  <p>What about the random forest?</p>
  
Tags: <decision-trees><random-forests><id3-algorithm><c4.5-algorithm>
Title: Are decision tree learning algorithms deterministic?
UserDisplayName: mallea
ThreadText: # Are decision tree learning algorithms deterministic?
  
  (Asked by: username_0 on 2019-04-01)
  
  Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure?
  
  What about the random forest?
  
  **Tags:** decision-trees, random-forests, id3-algorithm, c4.5-algorithm
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-04-02. Score: 9 | Accepted answer
  
  > Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure?
  
  Generally, yes. Most decision tree learners, like the common ID3 and C4.5/C5.0 algorithms, are deterministic. At each step, the learners consider all possible features that have not yet been used to split the data and find the splits that maximize some function (e.g. information gain). There is no randomness (or pseudo-randomness) in this process.
  
  The exceptions to this would be if you used randomness to break ties (rather than, say, using the index of each feature, as is common), but this would be an unusual modification.
  What about the random forest?
  
  As the name suggests, random forests *do* make use of randomness, or at least, pseudo-randomness. If we're only concerned about whether or not the algorithm is deterministic in the usual sense of the word (at least, within computer science), the answer is **no**.
  
  If you start the same random forest learning algorithm, with the same datasets, at two different times (or, using two different seeds for your pseudorandom number generator), you will get two different forests. This is because the algorithm selects random subsets of the features and/or datapoints to learn on, and, if different seeds are used, the subsets will be different each time.
  
  > **Comments:**
  > **username_2**: It is also worth noting that early stopping might use a random subset of the data for validation.  
  > **username_3**: 'At each step, the learners consider all possible features that have not yet been used to split the data and find the splits that maximize some function (e.g. information gain).' I think that decision trees also consider features that have already been used.  
  
  ---
  
  