Stay organized with collections
Save and categorize content based on your preferences.
Here is a quick summary of what you learned in the course:
A decision tree is a model
composed of a collection of
conditions organized
hierarchically in the shape of a tree. Conditions fall into various
categories:
Training a decision tree involves searching for the best condition at
each node. The splitter routine
uses metrics such as information
gain or
Gini to determine the best
condition.
A decision forest is
a mode made of multiple decision trees. The prediction of a decision forest
is the aggregation of the predictions of its decision trees.
A random forest is an
ensemble of decision trees in which each decision tree is trained with a
specific random noise.
Bagging is a technique in which
each decision tree in the random forest is trained on a different subset
of examples.
Random forests do not require a validation dataset. Instead, most random
forests use a technique called
out-of-bag-evaluation
to evaluate the quality of the model.
A gradient boosted (decision)
tree
is a type of decision forest trained through iterative adjustments from input
decision trees. A value called
shrinkage controls the rate at
which a gradient boosted (decision) tree learns and the degree to which it
could overfit.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[[["\u003cp\u003eDecision trees utilize conditions organized hierarchically to make predictions, with training focused on finding the optimal condition for each node.\u003c/p\u003e\n"],["\u003cp\u003eDecision forests combine predictions from multiple decision trees, while random forests introduce randomness during training to improve performance.\u003c/p\u003e\n"],["\u003cp\u003eRandom forests employ out-of-bag evaluation for model assessment, eliminating the need for a separate validation dataset.\u003c/p\u003e\n"],["\u003cp\u003eGradient boosted decision trees are iteratively trained with adjustments influenced by shrinkage, balancing learning rate and overfitting potential.\u003c/p\u003e\n"]]],[],null,["# Course Summary\n\n\u003cbr /\u003e\n\nHere is a quick summary of what you learned in the course:\n\n- A [**decision tree**](/machine-learning/glossary#decision-tree) is a model composed of a collection of [**conditions**](/machine-learning/glossary#condition) organized hierarchically in the shape of a tree. Conditions fall into various categories:\n - An [**axis-aligned\n condition**](/machine-learning/glossary#axis-aligned-condition) involves only a single feature. An [**oblique\n condition**](/machine-learning/glossary#oblique-condition) involves multiple features.\n - A [**binary\n condition**](/machine-learning/glossary#binary-condition) has two possible outcomes. A [**non-binary\n condition**](/machine-learning/glossary#non-binary-condition) has more than two possible outcomes.\n- Training a decision tree involves searching for the best condition at each node. The [**splitter**](/machine-learning/glossary#splitter) routine uses metrics such as [**information\n gain**](/machine-learning/glossary#information-gain) or [**Gini**](/machine-learning/glossary#gini-impurity) to determine the best condition.\n- A [**decision forest**](/machine-learning/glossary#decision-forest) is a mode made of multiple decision trees. The prediction of a decision forest is the aggregation of the predictions of its decision trees.\n- A [**random forest**](/machine-learning/glossary#random-forest) is an ensemble of decision trees in which each decision tree is trained with a specific random noise.\n- [**Bagging**](/machine-learning/glossary#bagging) is a technique in which each decision tree in the random forest is trained on a different subset of examples.\n- Random forests do not require a validation dataset. Instead, most random forests use a technique called [**out-of-bag-evaluation**](/machine-learning/glossary#out-of-bag-evaluation) to evaluate the quality of the model.\n- A [**gradient boosted (decision)\n tree**](/machine-learning/glossary#gradient-boosted-decision-trees-gbt) is a type of decision forest trained through iterative adjustments from input decision trees. A value called [**shrinkage**](/machine-learning/glossary#shrinkage) controls the rate at which a gradient boosted (decision) tree learns and the degree to which it could overfit.\n\nReferences\n----------\n\n- [Greedy Function Approximation: A Gradient Boosting\n Machine](https://jerryfriedman.su.domains/ftp/trebst.pdf), J. Friedman.\n- [The Elements of Statistical\n Learning](https://web.stanford.edu/%7Ehastie/Papers/ESLII.pdf), Trevor Hastie. Chapter 10.\n- [Generalized Boosted Models: A guide to the gbm\n package](http://www.saedsayad.com/docs/gbm2.pdf), G. Ridgeway"]]