Summary - What is Machine Learning?

A definition

Tom Mitchell describes machine learning as the question of how to construct computer programs that automatically improve with experience. His more formal definition: a program learns from experience E with respect to a class of tasks T and a performance measure P, if its performance at the tasks in T, measured by P, improves with experience E.

For spam filtering:

• E is the emails we have collected.
• T is classifying an email as spam or non-spam.
• P is a performance measure, for example the percentage of emails classified correctly (100% at best, 0% at worst).

Writing rules versus learning from data

The traditional way to filter spam is to find patterns that are common in spam, such as “cheap Rolex”, and write rules: if the email contains “cheap Rolex”, mark it as spam. The rules have to be updated and maintained as new kinds of spam appear. That takes a lot of time, and it is hard to find general patterns that catch all spam without also catching non-spam.

The machine learning approach is different:

1. Collect a dataset of example emails.
2. Label every email manually as spam or non-spam.
3. Let a machine learning algorithm learn from the labelled data, which produces a learned model.
4. Use the model to classify new, unseen emails as they arrive.

Machine learning is the construction of programs that learn from data without being explicitly programmed. Instead of the programmer telling the program how to recognize spam, the program learns it from the data.

When do we need machine learning?

A traditional program works well when you know or control the inputs, can specify how the program reaches its goal, can map out every decision it will make, and can test it with known inputs and be confident that it works.

Some problems don’t have those properties: the range of possible inputs isn’t known in advance, you can only say what the goal is but not how to reach it, and you can’t map out all the decisions needed. Recognizing a cat in a picture is an example. You can’t list every possible cat image, and rules like “it has fur and pointy ears” match other animals too. You can, however, collect pictures with and without cats and label them Cat and No cat. That is where machine learning helps.

The benefits of machine learning are that it is:

• Automatic: the algorithm uses data to build a model that knows how to reach the goal.
• Fast: it finds solutions faster than you can program them by hand.
• Accurate: it can run longer and on more data than you can.
• Scalable: it handles big data, which spreadsheets don’t.

Examples

Machine learning is used to find and recognize faces in pictures, to recommend movies and songs based on what you have consumed before, in medical diagnosis (for example detecting skin cancer), and to recognize handwritten letters and digits, such as addresses on letters.

The lecture demonstrates a model that recognizes handwritten digits. It recognizes digits very reliably, but it has only learned to tell digits apart, not digits from other things. Draw something that isn’t a digit and it still answers with the digit it finds most similar, because that is the only thing it knows.

When machine learning doesn’t work

• Inconsistent data: if one person labels a movie “thriller”, another “action thriller” and a third “action movie”, the same input maps to different outputs. Minor inconsistencies are fine, but severe ones cause problems.
• No mapping between inputs and labels: there must be some relation between the inputs and the labels for the algorithm to find. The colour of a car says nothing about its engine power (unless every brand except Ferrari is forbidden to paint its cars red).
• Data that is hard to collect or label: you need enough data and a way to label it. Labelling images as skin cancer or not, for example, may require lab tests, which is slow and expensive.

Where machine learning fits in

Drew Conway’s Venn diagram places machine learning where hacking skills (programming) meet math and statistics knowledge: you need maths and statistics to understand the algorithms and interpret the results, and programming to use them. Adding substantive expertise in an application area, such as medicine or economics, gives data science. Hacking skills and domain expertise without the maths is the danger zone: you can produce results without knowing how to interpret them.

The Iris dataset

The “hello world” of machine learning is the Iris dataset: classify an iris flower into one of three species (setosa, versicolor and virginica) from four measurements, the length and width of its petals and sepals. It has 150 examples, 50 of each species, so a random guess is right 33% of the time.

The traditional approach would be to study the data, for example in box plots, and write rules. Setosa is easy to separate because its petals are much smaller, but versicolor and virginica overlap, so rules that separate them are hard to write.

A machine learning algorithm can instead learn from the data. In the lecturer’s Web ML Demonstrator, which runs in the browser, one algorithm classifies 95.33% of the training examples correctly and another 98%, far better than the 33% of a random guess. You will implement one such algorithm, Naïve Bayes, in A4.