Machine learningLesson 2 of 3 in the course Supervised learning

Decision Trees: Structure, Split Criteria, Ensembles, and Limitations

5 flashcards · 5 quiz questions · 5 match pairs · 5 fill-in sentences · English

Save course
Lesson progress
0 / 5

Your progress is kept in this browser only.Sign up to save it

Summary

Decision trees are supervised learning models that predict a target from input features by following a sequence of understandable tests. This lesson introduces their structure and how a tree is built.

A decision tree predicts a target variable from one or more input variables. Each training example can be represented as (x,Y)(\mathbf{x}, Y), where x=(x1,x2,…,xk)\mathbf{x}=(x_1,x_2,\ldots,x_k) contains its features and YY is the target to predict.

All flashcards

How do classification trees and regression trees differ?
Classification trees predict discrete classes; regression trees predict continuous values.
What is bagging in tree ensembles?
Training models on bootstrap samples drawn with replacement, then combining their predictions.
What is boosting in tree ensembles?
Building learners sequentially, with later learners emphasizing examples earlier learners handled poorly.
What does Gini impurity measure?
Class mixing in a node, calculated as 1−∑i=1Jpi21-\sum_{i=1}^{J}p_i^2.
What is overfitting in a decision tree?
Fitting peculiarities of training data rather than patterns that generalize; pruning can help address it.

Quiz questions

  1. How is information gain for a split calculated?Answer: Parent entropy minus the weighted entropy of the child nodes.
  2. How is the Estimate of Positive Correctness calculated?Answer: TP−FPTP-FP, the true-positive count minus the false-positive count.
  3. Which split criterion is suited to continuous regression targets?Answer: Variance reduction, comparing parent variance with weighted child variances.
  4. What does CART's goodness measure balance?Answer: Differences in child class distributions and how balanced the child sizes are.
  5. How does a decision list typically proceed?Answer: Each condition gives a result or, if it does not apply, continues to the next test.

Based on the Wikipedia article Decision tree learning, licensed CC BY-SA 4.0. Adapted into a summary, flashcards and quizzes.

More in Machine learning

All Machine learning lessons
Machine learningAutoencoder Architectures, Training, Regularization, and Applications5 flashcards · 5 quiz questions · 4 gamesPlay free →Machine learningSupport Vector Machines: Margins, Kernels, Optimization, and Extensions5 flashcards · 5 quiz questions · 4 gamesPlay free →Machine learningNeural-Network Activation Functions: Properties and Families5 flashcards · 5 quiz questions · 4 gamesPlay free →Machine learningHierarchical Clustering: Linkage, Dendrograms, and Scalable Methods5 flashcards · 5 quiz questions · 4 gamesPlay free →Machine learningNeural Networks: Architectures, Training, History, and Applications5 flashcards · 5 quiz questions · 4 gamesPlay free →Machine learningConfusion Matrices: Binary Outcomes, Imbalance, and Extensions5 flashcards · 5 quiz questions · 4 gamesPlay free →