Original video


information gain
    [data mining]

    The amount of information that's gained by
    knowing the value of the attribute, which
    is the entropy of the distribution before
    the split minus the entropy of the
    distribution after it.

    The largest information gain is equivalent
    to the smallest entropy.

    vim +/"mutual information" "$NOTES/ws/glossaries/information-theory.txt"

information gain ratio
    [#decision tree learning]

    Ratio of information gain to the intrinsic

    It was proposed by Ross Quinlan, to reduce
    a bias towards multi-valued attributes by
    taking the number and size of branches
    into account when choosing an attribute.


A decision tree where logistic regression is used for each decision \({\displaystyle <:}\) ANN.

Instead of using information gain at each node, you use logistic regression.

Logistic regression

vim +/"logistic regression$" "$NOTES/glossary.txt"
Popular Architectures
    [#deep learning]

    - Linear Classifier
      Simple architecture that takes input
      features and combines them with weights
      and biases to predict an output value.
      One of the built in Esimators.