# introduction to machine learning: linear introduction to machine learning: linear learners lisbon...

Post on 26-Jun-2020

2 views

Embed Size (px)

TRANSCRIPT

Introduction to Machine Learning: Linear Learners

Lisbon Machine Learning School, 2018

Stefan Riezler

Computational Linguistics & IWR Heidelberg University, Germany

riezler@cl.uni-heidelberg.de

Intro: Linear Learners 1(117)

Introduction

Modeling the Frog’s Perceptual System

Intro: Linear Learners 2(117)

Introduction

Modeling the Frog’s Perceptual System

I [Lettvin et al. 1959] show that the frog’s perceptual system constructs reality by four separate operations:

I contrast detection: presence of sharp boundary? I convexity detection: how curved and how big is object? I movement detection: is object moving? I dimming speed: how fast does object obstruct light?

I The frog’s goal: Capture any object of the size of an insect or worm providing it moves like one.

I Can we build a model of this perceptual system and learn to capture the right objects?

Intro: Linear Learners 3(117)

Introduction

Modeling the Frog’s Perceptual System

I [Lettvin et al. 1959] show that the frog’s perceptual system constructs reality by four separate operations:

I contrast detection: presence of sharp boundary? I convexity detection: how curved and how big is object? I movement detection: is object moving? I dimming speed: how fast does object obstruct light?

I The frog’s goal: Capture any object of the size of an insect or worm providing it moves like one.

I Can we build a model of this perceptual system and learn to capture the right objects?

Intro: Linear Learners 3(117)

Introduction

Learning from Data

I Assume training data of edible (+) and inedible (-) objects

convex speed label convex speed label small small - small large + small medium - medium large + small medium - medium large +

medium small - large small + large small - large large + small small - large medium + small large - small medium -

I Learning model parameters from data: I p(+) =

6/14

, p(-) =

8/14

I p(convex = small|-) =

6/8

, p(convex = med|-) =

1/8

, p(convex = large|-) =

1/8

p(speed = small|-) =

4/8

, p(speed = med|-) =

3/8

, p(speed = large|- ) =

1/8

p(convex = small|+) =

1/6

, p(convex = med|+) =

2/6

, p(convex = large|+) =

3/6

p(speed = small|+) =

1/6

, p(speed = med|+) =

1/6

, p(speed = large|+ ) =

4/6

I Predict unseen p(label = ?, convex = med, speed = med) I p(-) · p(convex = med|-) · p(speed = med|-) = 8/14 · 1/8 · 3/8 = 0.027 I p(+) · p(convex = med|+) · p(speed = med|+) = 6/14 · 2/6 · 1/6 = 0.024 I Inedible: p(convex = med, speed = med, label = -) > p(convex = med, speed = med, label = +)!

Intro: Linear Learners 4(117)

Introduction

Learning from Data

I Assume training data of edible (+) and inedible (-) objects

convex speed label convex speed label small small - small large + small medium - medium large + small medium - medium large +

medium small - large small + large small - large large + small small - large medium + small large - small medium -

I Learning model parameters from data: I p(+) =

6/14

, p(-) =

8/14

I p(convex = small|-) =

6/8

, p(convex = med|-) =

1/8

, p(convex = large|-) =

1/8

p(speed = small|-) =

4/8

, p(speed = med|-) =

3/8

, p(speed = large|- ) =

1/8

p(convex = small|+) =

1/6

, p(convex = med|+) =

2/6

, p(convex = large|+) =

3/6

p(speed = small|+) =

1/6

, p(speed = med|+) =

1/6

, p(speed = large|+ ) =

4/6

I Predict unseen p(label = ?, convex = med, speed = med) I p(-) · p(convex = med|-) · p(speed = med|-) = 8/14 · 1/8 · 3/8 = 0.027 I p(+) · p(convex = med|+) · p(speed = med|+) = 6/14 · 2/6 · 1/6 = 0.024 I Inedible: p(convex = med, speed = med, label = -) > p(convex = med, speed = med, label = +)!

Intro: Linear Learners 4(117)

Introduction

Learning from Data

I Assume training data of edible (+) and inedible (-) objects

convex speed label convex speed label small small - small large + small medium - medium large + small medium - medium large +

medium small - large small + large small - large large + small small - large medium + small large - small medium -

I Learning model parameters from data: I p(+) = 6/14, p(-) = 8/14 I p(convex = small|-) =

6/8

, p(convex = med|-) =

1/8

, p(convex = large|-) =

1/8

p(speed = small|-) =

4/8

, p(speed = med|-) =

3/8

, p(speed = large|- ) =

1/8

p(convex = small|+) =

1/6

, p(convex = med|+) =

2/6

, p(convex = large|+) =

3/6

p(speed = small|+) =

1/6

, p(speed = med|+) =

1/6

, p(speed = large|+ ) =

4/6

I Predict unseen p(label = ?, convex = med, speed = med) I p(-) · p(convex = med|-) · p(speed = med|-) = 8/14 · 1/8 · 3/8 = 0.027 I p(+) · p(convex = med|+) · p(speed = med|+) = 6/14 · 2/6 · 1/6 = 0.024 I Inedible: p(convex = med, speed = med, label = -) > p(convex = med, speed = med, label = +)!

Intro: Linear Learners 4(117)

Introduction

Learning from Data

I Assume training data of edible (+) and inedible (-) objects

I Learning model parameters from data: I p(+) = 6/14, p(-) = 8/14 I p(convex = small|-) = 6/8, p(convex = med|-) = 1/8, p(convex = large|-) = 1/8

p(speed = small|-) = 4/8, p(speed = med|-) = 3/8, p(speed = large|- ) = 1/8 p(convex = small|+) = 1/6, p(convex = med|+) = 2/6, p(convex = large|+) = 3/6 p(speed = small|+) = 1/6, p(speed = med|+) = 1/6, p(speed = large|+ ) = 4/6

Intro: Linear Learners 4(117)

Introduction

Learning from Data

I Assume training data of edible (+) and inedible (-) objects

I Learning model parameters from data: I p(+) = 6/14, p(-) = 8/14 I p(convex = small|-) = 6/8, p(convex = med|-) = 1/8, p(convex = large|-) = 1/8

p(speed = small|-) = 4/8, p(speed = med|-) = 3/8, p(speed = large|- ) = 1/8 p(convex = small|+) = 1/6, p(convex = med|+) = 2/6, p(convex = large|+) = 3/6 p(speed = small|+) = 1/6, p(speed = med|+) = 1/6, p(speed = large|+ ) = 4/6

I Predict unseen p(label = ?, convex = med, speed = med) I p(-) · p(convex = med|-) · p(speed = med|-) =

8/14 · 1/8 · 3/8 = 0.027

I p(+) · p(convex = med|+) · p(speed = med|+) =

6/14 · 2/6 · 1/6 = 0.024 I Inedible: p(convex = med, speed = med, label = -) > p(convex = med, speed = med, label = +)!

Intro: Linear Learners 4(117)

Introduction

Learning from Data

I Assume training data of edible (+) and inedible (-) objects

I Learning model parameters from data: I p(+) = 6/14, p(-) = 8/14 I p(convex = small|-) = 6/8, p(convex = med|-) = 1/8, p(convex = large|-) = 1/8

p(speed = small|-) = 4/8, p(speed = med|-) = 3/8, p(speed = large|- ) = 1/8 p(convex = small|+) = 1/6, p(convex = med|+) = 2/6, p(convex = large|+) = 3/6 p(speed = small|+) = 1/6, p(speed = med|+) = 1/6, p(speed = large|+ ) = 4/6

I Predict unseen p(label = ?, convex = med, speed = med) I p(-) · p(convex = med|-) · p(speed = med|-) = 8/14 · 1/8 · 3/8 = 0.027 I p(+) · p(convex = med|+) · p(speed = med|+) =

6/14 · 2/6 · 1/6 = 0.024 I Inedible: p(convex = med, speed = med, label = -) > p(convex = med, speed = med, label = +)!

Intro: Linear Learners 4(117)

Introduction

Learning from Data

I Assume training data of edible (+) and inedible (-) objects

p(speed = small|-) = 4/8, p(speed = med|-) = 3/8, p(speed = large|- ) = 1/8 p(convex = small|+) = 1/6, p(convex = med|+) = 2/6, p(convex = large|+) = 3/6 p(speed = small|+) = 1/6, p(speed = med|+) = 1/6, p(speed = large|+ ) =