Hard-Margin Support Vector Machines

Podobne dokumenty
Linear Classification and Logistic Regression. Pascal Fua IC-CVLab

Previously on CSCI 4622

tum.de/fall2018/ in2357

Wprowadzenie do programu RapidMiner Studio 7.6, część 9 Modele liniowe Michał Bereta

Revenue Maximization. Sept. 25, 2018

Agnostic Learning and VC dimension

Machine Learning for Data Science (CS4786) Lecture11. Random Projections & Canonical Correlation Analysis

Rozpoznawanie twarzy metodą PCA Michał Bereta 1. Testowanie statystycznej istotności różnic między jakością klasyfikatorów

Machine Learning for Data Science (CS4786) Lecture 11. Spectral Embedding + Clustering

Wojewodztwo Koszalinskie: Obiekty i walory krajoznawcze (Inwentaryzacja krajoznawcza Polski) (Polish Edition)

Few-fermion thermometry

Zakopane, plan miasta: Skala ok. 1: = City map (Polish Edition)

Maximum A Posteriori Chris Piech CS109, Stanford University

Analysis of Movie Profitability STAT 469 IN CLASS ANALYSIS #2

Wprowadzenie do programu RapidMiner, część 3 Michał Bereta

y = The Chain Rule Show all work. No calculator unless otherwise stated. If asked to Explain your answer, write in complete sentences.

OpenPoland.net API Documentation

Weronika Mysliwiec, klasa 8W, rok szkolny 2018/2019

Tychy, plan miasta: Skala 1: (Polish Edition)

Helena Boguta, klasa 8W, rok szkolny 2018/2019

deep learning for NLP (5 lectures)

Machine Learning for Data Science (CS4786) Lecture 24. Differential Privacy and Re-useable Holdout

TTIC 31210: Advanced Natural Language Processing. Kevin Gimpel Spring Lecture 9: Inference in Structured Prediction

Gradient Coding using the Stochastic Block Model

EXAMPLES OF CABRI GEOMETRE II APPLICATION IN GEOMETRIC SCIENTIFIC RESEARCH

Klasyfikacja Support Vector Machines

Miedzy legenda a historia: Szlakiem piastowskim z Poznania do Gniezna (Biblioteka Kroniki Wielkopolski) (Polish Edition)

MaPlan Sp. z O.O. Click here if your download doesn"t start automatically

Stargard Szczecinski i okolice (Polish Edition)

Convolution semigroups with linear Jacobi parameters

Wojewodztwo Koszalinskie: Obiekty i walory krajoznawcze (Inwentaryzacja krajoznawcza Polski) (Polish Edition)

Inverse problems - Introduction - Probabilistic approach

Karpacz, plan miasta 1:10 000: Panorama Karkonoszy, mapa szlakow turystycznych (Polish Edition)

Klasyfikacja naiwny Bayes

DODATKOWE ĆWICZENIA EGZAMINACYJNE

Wprowadzenie do programu RapidMiner, część 3 Michał Bereta

New Roads to Cryptopia. Amit Sahai. An NSF Frontier Center

ARNOLD. EDUKACJA KULTURYSTY (POLSKA WERSJA JEZYKOWA) BY DOUGLAS KENT HALL

18. Przydatne zwroty podczas egzaminu ustnego. 19. Mo liwe pytania egzaminatora i przyk³adowe odpowiedzi egzaminowanego

Miedzy legenda a historia: Szlakiem piastowskim z Poznania do Gniezna (Biblioteka Kroniki Wielkopolski) (Polish Edition)

Wojewodztwo Koszalinskie: Obiekty i walory krajoznawcze (Inwentaryzacja krajoznawcza Polski) (Polish Edition)

The Overview of Civilian Applications of Airborne SAR Systems

Lecture 18 Review for Exam 1

The Lorenz System and Chaos in Nonlinear DEs

Katowice, plan miasta: Skala 1: = City map = Stadtplan (Polish Edition)

Camspot 4.4 Camspot 4.5

Wprowadzenie do programu RapidMiner, część 2 Michał Bereta 1. Wykorzystanie wykresu ROC do porównania modeli klasyfikatorów

Jak działa grawitacja?

Proposal of thesis topic for mgr in. (MSE) programme in Telecommunications and Computer Science

Neural Networks (The Machine-Learning Kind) BCS 247 March 2019

Supervised Hierarchical Clustering with Exponential Linkage. Nishant Yadav

DM-ML, DM-FL. Auxiliary Equipment and Accessories. Damper Drives. Dimensions. Descritpion

harmonic functions and the chromatic polynomial

Installation of EuroCert software for qualified electronic signature

Zarządzanie sieciami telekomunikacyjnymi

Wybrzeze Baltyku, mapa turystyczna 1: (Polish Edition)

Boosting. Sewoong Oh. CSE/STAT 416 University of Washington

Blow-Up: Photographs in the Time of Tumult; Black and White Photography Festival Zakopane Warszawa 2002 / Powiekszenie: Fotografie w czasach zgielku

Ankiety Nowe funkcje! Pomoc Twoje konto Wyloguj. BIODIVERSITY OF RIVERS: Survey to students

Dolny Slask 1: , mapa turystycznosamochodowa: Plan Wroclawia (Polish Edition)

MoA-Net: Self-supervised Motion Segmentation. Pia Bideau, Rakesh R Menon, Erik Learned-Miller

Zmiany techniczne wprowadzone w wersji Comarch ERP Altum

Zastosowania obliczeń inteligentnych do wyszukiwania w obrazowych bazach danych

Wprowadzenie do programu RapidMiner, część 5 Michał Bereta

Instrukcja konfiguracji usługi Wirtualnej Sieci Prywatnej w systemie Mac OSX

Extraclass. Football Men. Season 2009/10 - Autumn round


SSW1.1, HFW Fry #20, Zeno #25 Benchmark: Qtr.1. Fry #65, Zeno #67. like

Egzamin maturalny z języka angielskiego na poziomie dwujęzycznym Rozmowa wstępna (wyłącznie dla egzaminującego)

MDK-Plus. Licencja Node-Locked 8260 EUR 5740 EUR 3340 EUR gratis 3300 EUR 2300 EUR 1330 EUR 650 EUR 3970 EUR 2760 EUR 1600 EUR

OSI Network Layer. Network Fundamentals Chapter 5. Version Cisco Systems, Inc. All rights reserved. Cisco Public 1

PROMOTION - Flexible Ducts and Accessories - FLX-REKU P13.1/2013

Wojewodztwo Koszalinskie: Obiekty i walory krajoznawcze (Inwentaryzacja krajoznawcza Polski) (Polish Edition)

USB firmware changing guide. Zmiana oprogramowania za przy użyciu połączenia USB. Changelog / Lista Zmian

Some Linear Algebra. Vectors operations = =(,, ) =( 1 + 2, 1 + 2, ) λ =(λ,λ,λ ) Addition. Scalar multiplication.

Network Services for Spatial Data in European Geo-Portals and their Compliance with ISO and OGC Standards

Probability definition

Karpacz, plan miasta 1:10 000: Panorama Karkonoszy, mapa szlakow turystycznych (Polish Edition)

Pielgrzymka do Ojczyzny: Przemowienia i homilie Ojca Swietego Jana Pawla II (Jan Pawel II-- pierwszy Polak na Stolicy Piotrowej) (Polish Edition)

Financial support for start-uppres. Where to get money? - Equity. - Credit. - Local Labor Office - Six times the national average wage (22000 zł)

Karpacz, plan miasta 1:10 000: Panorama Karkonoszy, mapa szlakow turystycznych (Polish Edition)

EGZAMIN MATURALNY 2012 JĘZYK ANGIELSKI

Jazz EB207S is a slim, compact and outstanding looking SATA to USB 2.0 HDD enclosure. The case is

Towards Stability Analysis of Data Transport Mechanisms: a Fluid Model and an Application

Leba, Rowy, Ustka, Slowinski Park Narodowy, plany miast, mapa turystyczna =: Tourist map = Touristenkarte (Polish Edition)

ABOUT NEW EASTERN EUROPE BESTmQUARTERLYmJOURNAL

TTIC 31210: Advanced Natural Language Processing. Kevin Gimpel Spring Lecture 8: Structured PredicCon 2

EGZAMIN MATURALNY 2012 JĘZYK ANGIELSKI

Wyk lad 8: Leniwe metody klasyfikacji

Please fill in the questionnaire below. Each person who was involved in (parts of) the project can respond.

Formularz dla osób planujących ubiegać się o przyjęcie na studia undergraduate (I stopnia) w USA na rok akademicki

No matter how much you have, it matters how much you need

Pudełko konwersacji. Joel Shaul

Język angielski. Poziom rozszerzony Próbna Matura z OPERONEM i Gazetą Wyborczą CZĘŚĆ I KRYTERIA OCENIANIA ODPOWIEDZI POZIOM ROZSZERZONY CZĘŚĆ I

Wojewodztwo Koszalinskie: Obiekty i walory krajoznawcze (Inwentaryzacja krajoznawcza Polski) (Polish Edition)

OBWIESZCZENIE MINISTRA INFRASTRUKTURY. z dnia 18 kwietnia 2005 r.

Emilka szuka swojej gwiazdy / Emily Climbs (Emily, #2)

Wojewodztwo Koszalinskie: Obiekty i walory krajoznawcze (Inwentaryzacja krajoznawcza Polski) (Polish Edition)

KORELACJA 1. Wykres rozrzutu ocena związku między zmiennymi X i Y. 2. Współczynnik korelacji Pearsona

A Zadanie

Transkrypt:

Hard-Margin Support Vector Machines

<latexit sha1_base64="b4hfbmymyimfofmhtyo9gzgt56w=">aaacaxicbzdlssnafiyn9vbjlepk3ay2gicupasvu4iblxuaw2hjmuwn7ddjjmxm1bkcg1/fjqsvt76fo9/gazqfvn8y+pjpozw5vx8zkpvtfxmlhcwl5zxyqrm2vrg5zw3vxmsoezi4ogkr6phieky5crvvjhriqvdom9l2xxftevuwcekj3lktmhghgniauiyutvrwxtvme34a77kbvg73gtygpjsrfati1+xc8c84bvraowbf+uwnipyehcvmkjrdx46vlykhkgykm3ujjdhcyzqkxy0chur6ax5cbg+1m4bbjptjcubuz4kuhvjoql93hkin5hxtav5x6yyqopnsyuneey5ni4keqrxbar5wqaxbik00icyo/iveiyqqvjo1u4fgzj/8f9x67bzmxnurjzmijtlybwfgcdjgfdtajwgcf2dwaj7ac3g1ho1n4814n7wwjgjmf/ys8fenfycuzq==</latexit> <latexit sha1_base64="b4hfbmymyimfofmhtyo9gzgt56w=">aaacaxicbzdlssnafiyn9vbjlepk3ay2gicupasvu4iblxuaw2hjmuwn7ddjjmxm1bkcg1/fjqsvt76fo9/gazqfvn8y+pjpozw5vx8zkpvtfxmlhcwl5zxyqrm2vrg5zw3vxmsoezi4ogkr6phieky5crvvjhriqvdom9l2xxftevuwcekj3lktmhghgniauiyutvrwxtvme34a77kbvg73gtygpjsrfati1+xc8c84bvraowbf+uwnipyehcvmkjrdx46vlykhkgykm3ujjdhcyzqkxy0chur6ax5cbg+1m4bbjptjcubuz4kuhvjoql93hkin5hxtav5x6yyqopnsyuneey5ni4keqrxbar5wqaxbik00icyo/iveiyqqvjo1u4fgzj/8f9x67bzmxnurjzmijtlybwfgcdjgfdtajwgcf2dwaj7ac3g1ho1n4814n7wwjgjmf/ys8fenfycuzq==</latexit> <latexit sha1_base64="b4hfbmymyimfofmhtyo9gzgt56w=">aaacaxicbzdlssnafiyn9vbjlepk3ay2gicupasvu4iblxuaw2hjmuwn7ddjjmxm1bkcg1/fjqsvt76fo9/gazqfvn8y+pjpozw5vx8zkpvtfxmlhcwl5zxyqrm2vrg5zw3vxmsoezi4ogkr6phieky5crvvjhriqvdom9l2xxftevuwcekj3lktmhghgniauiyutvrwxtvme34a77kbvg73gtygpjsrfati1+xc8c84bvraowbf+uwnipyehcvmkjrdx46vlykhkgykm3ujjdhcyzqkxy0chur6ax5cbg+1m4bbjptjcubuz4kuhvjoql93hkin5hxtav5x6yyqopnsyuneey5ni4keqrxbar5wqaxbik00icyo/iveiyqqvjo1u4fgzj/8f9x67bzmxnurjzmijtlybwfgcdjgfdtajwgcf2dwaj7ac3g1ho1n4814n7wwjgjmf/ys8fenfycuzq==</latexit> <latexit sha1_base64="b4hfbmymyimfofmhtyo9gzgt56w=">aaacaxicbzdlssnafiyn9vbjlepk3ay2gicupasvu4iblxuaw2hjmuwn7ddjjmxm1bkcg1/fjqsvt76fo9/gazqfvn8y+pjpozw5vx8zkpvtfxmlhcwl5zxyqrm2vrg5zw3vxmsoezi4ogkr6phieky5crvvjhriqvdom9l2xxftevuwcekj3lktmhghgniauiyutvrwxtvme34a77kbvg73gtygpjsrfati1+xc8c84bvraowbf+uwnipyehcvmkjrdx46vlykhkgykm3ujjdhcyzqkxy0chur6ax5cbg+1m4bbjptjcubuz4kuhvjoql93hkin5hxtav5x6yyqopnsyuneey5ni4keqrxbar5wqaxbik00icyo/iveiyqqvjo1u4fgzj/8f9x67bzmxnurjzmijtlybwfgcdjgfdtajwgcf2dwaj7ac3g1ho1n4814n7wwjgjmf/ys8fenfycuzq==</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> Linear Classifiers We ve already seen several: o o Logistic Regression Perceptron Def: A linear classifier makes decisions by computing a linear combination of features of the form w T x + b and then classifies based on w T x + b 0 The boundary between the two classes is defined by or the decision boundary w T x + b =0 and is called the separator We estimate the weights and bias using the training data 2

<latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="icmczj6jv+dkorcojigcuvd9d1y=">aaab/nicbvdlssnafl2prxpfucgnm8fweissdofjv3djsokxhbaeyxtsdp08njmojxbhr7hxoelw73dn3zhts9dwaxco59zlvff4cwds2fa3uvhyxfpeka6aa+sbm1vw9s6njfnbqetihoumjyxllkkuyortziiodn1og/7gyuw37qiqli6u1tchnrd3ihywgpwwpguvbn57zopnogpko3ap3iibmwxpktkvewi0t5yclcbh3bo+2t2ypcgnfofyypzjj6qtyaey4xrktlnje0wguedbmky4plktte4fouotdfeqc12rqhp190sgqymhoa87q6z6ctybi/95rvqfz52mrumqaesmi4kuixwjcrioywqlig81wuqwfssifswwutoyu4fgzl48t9xq5bzixfvltzm8jslswwecgqonuinlqimlbb7hgv7hzxgyxox342pawjdymv34a+pzb3nkkq4=</latexit> <latexit sha1_base64="icmczj6jv+dkorcojigcuvd9d1y=">aaab/nicbvdlssnafl2prxpfucgnm8fweissdofjv3djsokxhbaeyxtsdp08njmojxbhr7hxoelw73dn3zhts9dwaxco59zlvff4cwds2fa3uvhyxfpeka6aa+sbm1vw9s6njfnbqetihoumjyxllkkuyortziiodn1og/7gyuw37qiqli6u1tchnrd3ihywgpwwpguvbn57zopnogpko3ap3iibmwxpktkvewi0t5yclcbh3bo+2t2ypcgnfofyypzjj6qtyaey4xrktlnje0wguedbmky4plktte4fouotdfeqc12rqhp190sgqymhoa87q6z6ctybi/95rvqfz52mrumqaesmi4kuixwjcrioywqlig81wuqwfssifswwutoyu4fgzl48t9xq5bzixfvltzm8jslswwecgqonuinlqimlbb7hgv7hzxgyxox342pawjdymv34a+pzb3nkkq4=</latexit> <latexit sha1_base64="icmczj6jv+dkorcojigcuvd9d1y=">aaab/nicbvdlssnafl2prxpfucgnm8fweissdofjv3djsokxhbaeyxtsdp08njmojxbhr7hxoelw73dn3zhts9dwaxco59zlvff4cwds2fa3uvhyxfpeka6aa+sbm1vw9s6njfnbqetihoumjyxllkkuyortziiodn1og/7gyuw37qiqli6u1tchnrd3ihywgpwwpguvbn57zopnogpko3ap3iibmwxpktkvewi0t5yclcbh3bo+2t2ypcgnfofyypzjj6qtyaey4xrktlnje0wguedbmky4plktte4fouotdfeqc12rqhp190sgqymhoa87q6z6ctybi/95rvqfz52mrumqaesmi4kuixwjcrioywqlig81wuqwfssifswwutoyu4fgzl48t9xq5bzixfvltzm8jslswwecgqonuinlqimlbb7hgv7hzxgyxox342pawjdymv34a+pzb3nkkq4=</latexit> <latexit sha1_base64="8hxxgp36artuotcncd/iql/irgi=">aaacaxicbvdlssnafj3uv42vqctxm9gkbixjfz4wqsgnywrwftoqjpnjo3qyctmttytgxl9x40lfrx/hzr9x2mahrqcuhm65l3vv8rngpbltb6o0sli0vfjendfwnza3ro2dwxmnapmwjlksoj6shffowooqrjqjicjygwn7w8ux374jqtky36hrqtwi9tknkuzks561vzufpadewoneeaqemz/p7j0nn6uevbfr9grwnjgfqyactc/66guxtipcfwziyq5jj8rnkfaum5kbvvssboeh6pouphxfrlrz5iuchmolggesdhefj+rviqxfuo4ix3dgsa3krdcw//o6qqrp3izyjfwe4+mimgvqxxccbwyoifixksyic6pvhxiadbjkp2bqejzzl+djq147rznx9urjpeijdpbbatgcdjgfdxafmqafmhgez+avvblpxovxbnxmw0tgmbml/sd4/agemzuu</latexit> <latexit sha1_base64="8hxxgp36artuotcncd/iql/irgi=">aaacaxicbvdlssnafj3uv42vqctxm9gkbixjfz4wqsgnywrwftoqjpnjo3qyctmttytgxl9x40lfrx/hzr9x2mahrqcuhm65l3vv8rngpbltb6o0sli0vfjendfwnza3ro2dwxmnapmwjlksoj6shffowooqrjqjicjygwn7w8ux374jqtky36hrqtwi9tknkuzks561vzufpadewoneeaqemz/p7j0nn6uevbfr9grwnjgfqyactc/66guxtipcfwziyq5jj8rnkfaum5kbvvssboeh6pouphxfrlrz5iuchmolggesdhefj+rviqxfuo4ix3dgsa3krdcw//o6qqrp3izyjfwe4+mimgvqxxccbwyoifixksyic6pvhxiadbjkp2bqejzzl+djq147rznx9urjpeijdpbbatgcdjgfdxafmqafmhgez+avvblpxovxbnxmw0tgmbml/sd4/agemzuu</latexit> <latexit sha1_base64="8hxxgp36artuotcncd/iql/irgi=">aaacaxicbvdlssnafj3uv42vqctxm9gkbixjfz4wqsgnywrwftoqjpnjo3qyctmttytgxl9x40lfrx/hzr9x2mahrqcuhm65l3vv8rngpbltb6o0sli0vfjendfwnza3ro2dwxmnapmwjlksoj6shffowooqrjqjicjygwn7w8ux374jqtky36hrqtwi9tknkuzks561vzufpadewoneeaqemz/p7j0nn6uevbfr9grwnjgfqyactc/66guxtipcfwziyq5jj8rnkfaum5kbvvssboeh6pouphxfrlrz5iuchmolggesdhefj+rviqxfuo4ix3dgsa3krdcw//o6qqrp3izyjfwe4+mimgvqxxccbwyoifixksyic6pvhxiadbjkp2bqejzzl+djq147rznx9urjpeijdpbbatgcdjgfdxafmqafmhgez+avvblpxovxbnxmw0tgmbml/sd4/agemzuu</latexit> <latexit sha1_base64="8hxxgp36artuotcncd/iql/irgi=">aaacaxicbvdlssnafj3uv42vqctxm9gkbixjfz4wqsgnywrwftoqjpnjo3qyctmttytgxl9x40lfrx/hzr9x2mahrqcuhm65l3vv8rngpbltb6o0sli0vfjendfwnza3ro2dwxmnapmwjlksoj6shffowooqrjqjicjygwn7w8ux374jqtky36hrqtwi9tknkuzks561vzufpadewoneeaqemz/p7j0nn6uevbfr9grwnjgfqyactc/66guxtipcfwziyq5jj8rnkfaum5kbvvssboeh6pouphxfrlrz5iuchmolggesdhefj+rviqxfuo4ix3dgsa3krdcw//o6qqrp3izyjfwe4+mimgvqxxccbwyoifixksyic6pvhxiadbjkp2bqejzzl+djq147rznx9urjpeijdpbbatgcdjgfdxafmqafmhgez+avvblpxovxbnxmw0tgmbml/sd4/agemzuu</latexit> <latexit sha1_base64="h5w/ffu3p9clzeunl11+cnonbag=">aaacbhicbvc7tsnaedyhvzavayuujxikgii7by8ueg1lkdcjffvw+xxotjk/udsdkewghl+hoqbey0fq8tdceheqmnjko5ld7e74kancmua3vllyxfpeqa7qa+sbm1vg9s6nsdkoiy0tlvcujwrhnca2pjkrbsojinxgov7wyux37ggxnimv5sglbot6mq0prljjnrff1x88czp9cgupnsdkcod+kd97vqhxpanmnswj4dyxslidjdqe8euecc4iekvmkba9y0ylmymukwak0j1mkbthieqtnqixiohw88kxbtxusgddhkukjzyovydyfakxinzvgse5elpewpzp62uyphnzgqezjdgelgozbmucx5haghkcjrspgjcn6laib0glivvwugrbmn15ntjnxnndumrwwidlglwwbw7aebdakwibs9agnsdgetydv/cmpwkv2rv2mw2taoxmlvgd7fmhos+wqa==</latexit> <latexit sha1_base64="h5w/ffu3p9clzeunl11+cnonbag=">aaacbhicbvc7tsnaedyhvzavayuujxikgii7by8ueg1lkdcjffvw+xxotjk/udsdkewghl+hoqbey0fq8tdceheqmnjko5ld7e74kancmua3vllyxfpeqa7qa+sbm1vg9s6nsdkoiy0tlvcujwrhnca2pjkrbsojinxgov7wyux37ggxnimv5sglbot6mq0prljjnrff1x88czp9cgupnsdkcod+kd97vqhxpanmnswj4dyxslidjdqe8euecc4iekvmkba9y0ylmymukwak0j1mkbthieqtnqixiohw88kxbtxusgddhkukjzyovydyfakxinzvgse5elpewpzp62uyphnzgqezjdgelgozbmucx5haghkcjrspgjcn6laib0glivvwugrbmn15ntjnxnndumrwwidlglwwbw7aebdakwibs9agnsdgetydv/cmpwkv2rv2mw2taoxmlvgd7fmhos+wqa==</latexit> <latexit sha1_base64="h5w/ffu3p9clzeunl11+cnonbag=">aaacbhicbvc7tsnaedyhvzavayuujxikgii7by8ueg1lkdcjffvw+xxotjk/udsdkewghl+hoqbey0fq8tdceheqmnjko5ld7e74kancmua3vllyxfpeqa7qa+sbm1vg9s6nsdkoiy0tlvcujwrhnca2pjkrbsojinxgov7wyux37ggxnimv5sglbot6mq0prljjnrff1x88czp9cgupnsdkcod+kd97vqhxpanmnswj4dyxslidjdqe8euecc4iekvmkba9y0ylmymukwak0j1mkbthieqtnqixiohw88kxbtxusgddhkukjzyovydyfakxinzvgse5elpewpzp62uyphnzgqezjdgelgozbmucx5haghkcjrspgjcn6laib0glivvwugrbmn15ntjnxnndumrwwidlglwwbw7aebdakwibs9agnsdgetydv/cmpwkv2rv2mw2taoxmlvgd7fmhos+wqa==</latexit> <latexit sha1_base64="h5w/ffu3p9clzeunl11+cnonbag=">aaacbhicbvc7tsnaedyhvzavayuujxikgii7by8ueg1lkdcjffvw+xxotjk/udsdkewghl+hoqbey0fq8tdceheqmnjko5ld7e74kancmua3vllyxfpeqa7qa+sbm1vg9s6nsdkoiy0tlvcujwrhnca2pjkrbsojinxgov7wyux37ggxnimv5sglbot6mq0prljjnrff1x88czp9cgupnsdkcod+kd97vqhxpanmnswj4dyxslidjdqe8euecc4iekvmkba9y0ylmymukwak0j1mkbthieqtnqixiohw88kxbtxusgddhkukjzyovydyfakxinzvgse5elpewpzp62uyphnzgqezjdgelgozbmucx5haghkcjrspgjcn6laib0glivvwugrbmn15ntjnxnndumrwwidlglwwbw7aebdakwibs9agnsdgetydv/cmpwkv2rv2mw2taoxmlvgd7fmhos+wqa==</latexit> Linear Classifiers In 1D we just have a single feature, x 1 The decision rule w T b x + b 0 becomes w or, <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> 1 x 1 + b 0 x <latexit sha1_base64="icmczj6jv+dkorcojigcuvd9d1y=">aaab/nicbvdlssnafl2prxpfucgnm8fweissdofjv3djsokxhbaeyxtsdp08njmojxbhr7hxoelw73dn3zhts9dwaxco59zlvff4cwds2fa3uvhyxfpeka6aa+sbm1vw9s6njfnbqetihoumjyxllkkuyortziiodn1og/7gyuw37qiqli6u1tchnrd3ihywgpwwpguvbn57zopnogpko3ap3iibmwxpktkvewi0t5yclcbh3bo+2t2ypcgnfofyypzjj6qtyaey4xrktlnje0wguedbmky4plktte4fouotdfeqc12rqhp190sgqymhoa87q6z6ctybi/95rvqfz52mrumqaesmi4kuixwjcrioywqlig81wuqwfssifswwutoyu4fgzl48t9xq5bzixfvltzm8jslswwecgqonuinlqimlbb7hgv7hzxgyxox342pawjdymv34a+pzb3nkkq4=</latexit> 1 w 1 The decision boundary is just a single point., x 1 = b w 1 mmm DB 3

<latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="angnjlixnssz1htazh+v4zeetfy=">aaacfnicbzc7tsmwfiydrixcaowsfi0sc1wsgcuavimfsuievmqjyhgc1qpzke1aqyhvwckrsdaaykvsva1ugyfoozktx99/juzz+ymjqprml7awuls8slpz09c3nre2jz3dw5fkhbmhjyzhbr8jwmhmhekli+2uext5jlt8wexyb90rlmgs38hrstwi9wiauoykqp5rr+ldz4yx8lgbhbzh/n6zcnxzxdczfqa/rxrnm6pm3zwunbdwkaqgrkznfhadbgcrisvmsiiozabszrgxfdns6n1mkbthaeqrjpixiohw88lebtxujibhwtwjjzzq3xm5ioqyrb7qjjdsi1lvdp/zopkmz9ycxmkmsyynd4uzgzkb45bgqdnbko2uqjht9vei+0glivwuugrbml15xjh2/bxuxdvvxkmzrgxsgwnwbcxwchrgcjsbazb4ae/gbbxqj9qz9qa9t1sxthjmd/wp7embra6dsg==</latexit> <latexit sha1_base64="angnjlixnssz1htazh+v4zeetfy=">aaacfnicbzc7tsmwfiydrixcaowsfi0sc1wsgcuavimfsuievmqjyhgc1qpzke1aqyhvwckrsdaaykvsva1ugyfoozktx99/juzz+ymjqprml7awuls8slpz09c3nre2jz3dw5fkhbmhjyzhbr8jwmhmhekli+2uext5jlt8wexyb90rlmgs38hrstwi9wiauoykqp5rr+ldz4yx8lgbhbzh/n6zcnxzxdczfqa/rxrnm6pm3zwunbdwkaqgrkznfhadbgcrisvmsiiozabszrgxfdns6n1mkbthaeqrjpixiohw88lebtxujibhwtwjjzzq3xm5ioqyrb7qjjdsi1lvdp/zopkmz9ycxmkmsyynd4uzgzkb45bgqdnbko2uqjht9vei+0glivwuugrbml15xjh2/bxuxdvvxkmzrgxsgwnwbcxwchrgcjsbazb4ae/gbbxqj9qz9qa9t1sxthjmd/wp7embra6dsg==</latexit> <latexit sha1_base64="angnjlixnssz1htazh+v4zeetfy=">aaacfnicbzc7tsmwfiydrixcaowsfi0sc1wsgcuavimfsuievmqjyhgc1qpzke1aqyhvwckrsdaaykvsva1ugyfoozktx99/juzz+ymjqprml7awuls8slpz09c3nre2jz3dw5fkhbmhjyzhbr8jwmhmhekli+2uext5jlt8wexyb90rlmgs38hrstwi9wiauoykqp5rr+ldz4yx8lgbhbzh/n6zcnxzxdczfqa/rxrnm6pm3zwunbdwkaqgrkznfhadbgcrisvmsiiozabszrgxfdns6n1mkbthaeqrjpixiohw88lebtxujibhwtwjjzzq3xm5ioqyrb7qjjdsi1lvdp/zopkmz9ycxmkmsyynd4uzgzkb45bgqdnbko2uqjht9vei+0glivwuugrbml15xjh2/bxuxdvvxkmzrgxsgwnwbcxwchrgcjsbazb4ae/gbbxqj9qz9qa9t1sxthjmd/wp7embra6dsg==</latexit> <latexit sha1_base64="angnjlixnssz1htazh+v4zeetfy=">aaacfnicbzc7tsmwfiydrixcaowsfi0sc1wsgcuavimfsuievmqjyhgc1qpzke1aqyhvwckrsdaaykvsva1ugyfoozktx99/juzz+ymjqprml7awuls8slpz09c3nre2jz3dw5fkhbmhjyzhbr8jwmhmhekli+2uext5jlt8wexyb90rlmgs38hrstwi9wiauoykqp5rr+ldz4yx8lgbhbzh/n6zcnxzxdczfqa/rxrnm6pm3zwunbdwkaqgrkznfhadbgcrisvmsiiozabszrgxfdns6n1mkbthaeqrjpixiohw88lebtxujibhwtwjjzzq3xm5ioqyrb7qjjdsi1lvdp/zopkmz9ycxmkmsyynd4uzgzkb45bgqdnbko2uqjht9vei+0glivwuugrbml15xjh2/bxuxdvvxkmzrgxsgwnwbcxwchrgcjsbazb4ae/gbbxqj9qz9qa9t1sxthjmd/wp7embra6dsg==</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="389u0atoa0cw445vps2p6tlojwe=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkgxctkosjeuitfibry7rtfydj9gobvv1zofvwbgasfikblwnbpsbwn7j0qf/nkpj84cpo1lz9rdrwfpewv0rrpsbm1vbo9bu3p1mmogjhxowieaijggue09rxugjfqtfisp1shc1qdcfija04bdqmbi/rh1oi4qr0lzghzbnfuamageewn7gdgjxqwhbhxipbwiwa6tkv+yp4ci4ozrarlpgfbxacc5iwhvmsmqmy6fkhyghkgzkblyysvkee6hdmho5ion0r9ndxvbyo20yjui/rudu/t0xqrguwzjuntfsxtlfm5j/1zqzii78eevppgjhs0vrxqbk4cqv2kacymwgghawvp8v4i4sccudnaldcozpxgtprvxwnbu3vd3l0yica3aetoadzkevxima8aagj+azvii348l4md6nj1lrwchn9sefgz8/yeuvha==</latexit> <latexit sha1_base64="389u0atoa0cw445vps2p6tlojwe=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkgxctkosjeuitfibry7rtfydj9gobvv1zofvwbgasfikblwnbpsbwn7j0qf/nkpj84cpo1lz9rdrwfpewv0rrpsbm1vbo9bu3p1mmogjhxowieaijggue09rxugjfqtfisp1shc1qdcfija04bdqmbi/rh1oi4qr0lzghzbnfuamageewn7gdgjxqwhbhxipbwiwa6tkv+yp4ci4ozrarlpgfbxacc5iwhvmsmqmy6fkhyghkgzkblyysvkee6hdmho5ion0r9ndxvbyo20yjui/rudu/t0xqrguwzjuntfsxtlfm5j/1zqzii78eevppgjhs0vrxqbk4cqv2kacymwgghawvp8v4i4sccudnaldcozpxgtprvxwnbu3vd3l0yica3aetoadzkevxima8aagj+azvii348l4md6nj1lrwchn9sefgz8/yeuvha==</latexit> <latexit sha1_base64="389u0atoa0cw445vps2p6tlojwe=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkgxctkosjeuitfibry7rtfydj9gobvv1zofvwbgasfikblwnbpsbwn7j0qf/nkpj84cpo1lz9rdrwfpewv0rrpsbm1vbo9bu3p1mmogjhxowieaijggue09rxugjfqtfisp1shc1qdcfija04bdqmbi/rh1oi4qr0lzghzbnfuamageewn7gdgjxqwhbhxipbwiwa6tkv+yp4ci4ozrarlpgfbxacc5iwhvmsmqmy6fkhyghkgzkblyysvkee6hdmho5ion0r9ndxvbyo20yjui/rudu/t0xqrguwzjuntfsxtlfm5j/1zqzii78eevppgjhs0vrxqbk4cqv2kacymwgghawvp8v4i4sccudnaldcozpxgtprvxwnbu3vd3l0yica3aetoadzkevxima8aagj+azvii348l4md6nj1lrwchn9sefgz8/yeuvha==</latexit> <latexit sha1_base64="389u0atoa0cw445vps2p6tlojwe=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkgxctkosjeuitfibry7rtfydj9gobvv1zofvwbgasfikblwnbpsbwn7j0qf/nkpj84cpo1lz9rdrwfpewv0rrpsbm1vbo9bu3p1mmogjhxowieaijggue09rxugjfqtfisp1shc1qdcfija04bdqmbi/rh1oi4qr0lzghzbnfuamageewn7gdgjxqwhbhxipbwiwa6tkv+yp4ci4ozrarlpgfbxacc5iwhvmsmqmy6fkhyghkgzkblyysvkee6hdmho5ion0r9ndxvbyo20yjui/rudu/t0xqrguwzjuntfsxtlfm5j/1zqzii78eevppgjhs0vrxqbk4cqv2kacymwgghawvp8v4i4sccudnaldcozpxgtprvxwnbu3vd3l0yica3aetoadzkevxima8aagj+azvii348l4md6nj1lrwchn9sefgz8/yeuvha==</latexit> Linear Classifiers In 2D we just have two features, The decision rule w T x + b 0 x 1 and becomes x 2 w 1 x 1 + w 2 x 2 + b 0 l ye { The decision boundary is the line x 2 = w 1 x 1 w 2 b w 2 Note that the vector is orthogonal to the DB and points in the direction of the positive class : + + 4

<latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="kg8f2gqmefcl9gxch+hg6t9o+70=">aaab7hicbvbntwixej3fl1y/ui9egshee9nvxi8birepmlhaahvslv2odntn2zuswn/w4kgnv3+qn/+nbfag4esmexlvjjpzopqzbtzv2ymsrk6tbxq33a3tnd290v5bq8tmeroqyavqrvhtzgqnddoctljfcrjx2oygn1o/+uivzllcm1fkwwt3byszwczkjyr71d2vdetlr+rngjajn5my5kh3s1+dnirzqouhhgvd9r3uhgosdcocttxopmmkyrd3adtsgroqw/hs2gk6suopxvlzegbn1n8ty5xopuoi25lgm9cl3lt8z2tnjr4kx0ykmagczbffgudgounrqmcujyaplmfemxsrigosmde2ineg4c++veycs+p11b87k9cu8jskcathcao+xeinbqeoarb4ggd4htdhoi/ou/mxby04+cwh/ihz+qnk5y3y</latexit> <latexit sha1_base64="kg8f2gqmefcl9gxch+hg6t9o+70=">aaab7hicbvbntwixej3fl1y/ui9egshee9nvxi8birepmlhaahvslv2odntn2zuswn/w4kgnv3+qn/+nbfag4esmexlvjjpzopqzbtzv2ymsrk6tbxq33a3tnd290v5bq8tmeroqyavqrvhtzgqnddoctljfcrjx2oygn1o/+uivzllcm1fkwwt3byszwczkjyr71d2vdetlr+rngjajn5my5kh3s1+dnirzqouhhgvd9r3uhgosdcocttxopmmkyrd3adtsgroqw/hs2gk6suopxvlzegbn1n8ty5xopuoi25lgm9cl3lt8z2tnjr4kx0ykmagczbffgudgounrqmcujyaplmfemxsrigosmde2ineg4c++veycs+p11b87k9cu8jskcathcao+xeinbqeoarb4ggd4htdhoi/ou/mxby04+cwh/ihz+qnk5y3y</latexit> <latexit sha1_base64="kg8f2gqmefcl9gxch+hg6t9o+70=">aaab7hicbvbntwixej3fl1y/ui9egshee9nvxi8birepmlhaahvslv2odntn2zuswn/w4kgnv3+qn/+nbfag4esmexlvjjpzopqzbtzv2ymsrk6tbxq33a3tnd290v5bq8tmeroqyavqrvhtzgqnddoctljfcrjx2oygn1o/+uivzllcm1fkwwt3byszwczkjyr71d2vdetlr+rngjajn5my5kh3s1+dnirzqouhhgvd9r3uhgosdcocttxopmmkyrd3adtsgroqw/hs2gk6suopxvlzegbn1n8ty5xopuoi25lgm9cl3lt8z2tnjr4kx0ykmagczbffgudgounrqmcujyaplmfemxsrigosmde2ineg4c++veycs+p11b87k9cu8jskcathcao+xeinbqeoarb4ggd4htdhoi/ou/mxby04+cwh/ihz+qnk5y3y</latexit> <latexit sha1_base64="kg8f2gqmefcl9gxch+hg6t9o+70=">aaab7hicbvbntwixej3fl1y/ui9egshee9nvxi8birepmlhaahvslv2odntn2zuswn/w4kgnv3+qn/+nbfag4esmexlvjjpzopqzbtzv2ymsrk6tbxq33a3tnd290v5bq8tmeroqyavqrvhtzgqnddoctljfcrjx2oygn1o/+uivzllcm1fkwwt3byszwczkjyr71d2vdetlr+rngjajn5my5kh3s1+dnirzqouhhgvd9r3uhgosdcocttxopmmkyrd3adtsgroqw/hs2gk6suopxvlzegbn1n8ty5xopuoi25lgm9cl3lt8z2tnjr4kx0ykmagczbffgudgounrqmcujyaplmfemxsrigosmde2ineg4c++veycs+p11b87k9cu8jskcathcao+xeinbqeoarb4ggd4htdhoi/ou/mxby04+cwh/ihz+qnk5y3y</latexit> Linear Classifiers In 3D we just have three features,,, and x 1 x<latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> 2 x 3 The decision rule w T x + b 0 gives the plane w T x + b =0 as the decision boundary X 3 W x, " 5

Linear Classifiers In higher dimensions it becomes harder to visualize We call the decision boundary the separating hyperplane Different types of linear classifiers determine their parameters in different ways o o o Naïve Bayes Model joint probabilities and use Bayes Rule Logistic Regression Model probability that an example belongs to a particular class The Perceptron Use examples to update parameters until we find something that works Support Vector Machines use a more geometric idea called a Margin 6

Support Vector Machines Advantages: o o o o Best off-the-shelf classifier Andrew Ng Nice theoretical bounds Allows for nonlinear classification Optimization problem for learning parameters is convex Disadvantages: o o No probabilistic interpretation Can be prone to overfitting in the nonlinear case 7

SVM Intuition Temporary Assumption: Assume the training data is linearly separable Consider the following training set in 2D 8

SVM Intuition Question: Which decision boundary seems better to you? 2 3. I 9

SVM Intuition Answer: This one feels better 10

SVM Intuition More space between DB and closest training examples 11

<latexit sha1_base64="ee3hmzwjjnf7xcwpctl0mhoqqty=">aaacanicbzdlssnafiyn9vbjlepon4otiagl6cllrudgzyxgftoyjtnjo3qyctmttysag1/fjqsvtz6fo9/gazufvn8y+pjpozw5f5awkpvtfxmlhcwl5zxyqrm2vrg5zw3vxms4fzi4ogax6arieky5crvvjhqsqvaumniorhetevuwcelj3lljhhgrgnaauoyutnxrr2pmvsced/lnawr3uu/hmqzmqm9v7jo9ffwltgevukjpw5+9foztihcfgzky69ij8jikfmwm5gyvlsrbeiqgpkuro4hil5vekmnd7frhgav9uijt9+dehiipx1ggoyokhnk+njh/q3vtfz55gevjqgjhs0vhyqck4sqq2kecymxgghawvp8v4iesccsdm6ldcozp/gtuvxzec67qlczjkuyz7imdcaqccaoa4bi0gqsweabp4aw8go/gs/fmvm9as0yxswt+yfj4bkmclx8=</latexit> <latexit sha1_base64="ee3hmzwjjnf7xcwpctl0mhoqqty=">aaacanicbzdlssnafiyn9vbjlepon4otiagl6cllrudgzyxgftoyjtnjo3qyctmttysag1/fjqsvtz6fo9/gazufvn8y+pjpozw5f5awkpvtfxmlhcwl5zxyqrm2vrg5zw3vxms4fzi4ogax6arieky5crvvjhqsqvaumniorhetevuwcelj3lljhhgrgnaauoyutnxrr2pmvsced/lnawr3uu/hmqzmqm9v7jo9ffwltgevukjpw5+9foztihcfgzky69ij8jikfmwm5gyvlsrbeiqgpkuro4hil5vekmnd7frhgav9uijt9+dehiipx1ggoyokhnk+njh/q3vtfz55gevjqgjhs0vhyqck4sqq2kecymxgghawvp8v4iesccsdm6ldcozp/gtuvxzec67qlczjkuyz7imdcaqccaoa4bi0gqsweabp4aw8go/gs/fmvm9as0yxswt+yfj4bkmclx8=</latexit> <latexit sha1_base64="ee3hmzwjjnf7xcwpctl0mhoqqty=">aaacanicbzdlssnafiyn9vbjlepon4otiagl6cllrudgzyxgftoyjtnjo3qyctmttysag1/fjqsvtz6fo9/gazufvn8y+pjpozw5f5awkpvtfxmlhcwl5zxyqrm2vrg5zw3vxms4fzi4ogax6arieky5crvvjhqsqvaumniorhetevuwcelj3lljhhgrgnaauoyutnxrr2pmvsced/lnawr3uu/hmqzmqm9v7jo9ffwltgevukjpw5+9foztihcfgzky69ij8jikfmwm5gyvlsrbeiqgpkuro4hil5vekmnd7frhgav9uijt9+dehiipx1ggoyokhnk+njh/q3vtfz55gevjqgjhs0vhyqck4sqq2kecymxgghawvp8v4iesccsdm6ldcozp/gtuvxzec67qlczjkuyz7imdcaqccaoa4bi0gqsweabp4aw8go/gs/fmvm9as0yxswt+yfj4bkmclx8=</latexit> <latexit sha1_base64="ee3hmzwjjnf7xcwpctl0mhoqqty=">aaacanicbzdlssnafiyn9vbjlepon4otiagl6cllrudgzyxgftoyjtnjo3qyctmttysag1/fjqsvtz6fo9/gazufvn8y+pjpozw5f5awkpvtfxmlhcwl5zxyqrm2vrg5zw3vxms4fzi4ogax6arieky5crvvjhqsqvaumniorhetevuwcelj3lljhhgrgnaauoyutnxrr2pmvsced/lnawr3uu/hmqzmqm9v7jo9ffwltgevukjpw5+9foztihcfgzky69ij8jikfmwm5gyvlsrbeiqgpkuro4hil5vekmnd7frhgav9uijt9+dehiipx1ggoyokhnk+njh/q3vtfz55gevjqgjhs0vhyqck4sqq2kecymxgghawvp8v4iesccsdm6ldcozp/gtuvxzec67qlczjkuyz7imdcaqccaoa4bi0gqsweabp4aw8go/gs/fmvm9as0yxswt+yfj4bkmclx8=</latexit> <latexit sha1_base64="3hznwolfi//lveukep6mgdowjjg=">aaab83icbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg7bbubomk4xd48adgq3/gm//glfsg4esmexlvjjpzgoqzpr3n2yqtrw9sbpw37z3dvf2dyuhrg4ptsahhyh7ltoav5uxqtzpnaserfecbp+1gfjp77qmvisxixk8t6kd4kfjicnzg8mt21gtc9dtrm7vwr1sdujmhwivuqapqonwvfpugmukjkjthwkmu6ytaz7dujha6s3upogkmyzykxumfjqjys/nrm3rmlaeky2lkadrxf09kofjqggwmm8j6pja9xpzp66y6vpizjpjuu0ewi8kuix2jpae0yjiszaegyckzurwrezayajotbujwl19ejv6jfl137xrv5kwrrhlo4btowyvlamittmadao/wdk/wzk2sf+vd+li0lqxi5hj+wpr8at0bkjo=</latexit> <latexit sha1_base64="3hznwolfi//lveukep6mgdowjjg=">aaab83icbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg7bbubomk4xd48adgq3/gm//glfsg4esmexlvjjpzgoqzpr3n2yqtrw9sbpw37z3dvf2dyuhrg4ptsahhyh7ltoav5uxqtzpnaserfecbp+1gfjp77qmvisxixk8t6kd4kfjicnzg8mt21gtc9dtrm7vwr1sdujmhwivuqapqonwvfpugmukjkjthwkmu6ytaz7dujha6s3upogkmyzykxumfjqjys/nrm3rmlaeky2lkadrxf09kofjqggwmm8j6pja9xpzp66y6vpizjpjuu0ewi8kuix2jpae0yjiszaegyckzurwrezayajotbujwl19ejv6jfl137xrv5kwrrhlo4btowyvlamittmadao/wdk/wzk2sf+vd+li0lqxi5hj+wpr8at0bkjo=</latexit> <latexit sha1_base64="3hznwolfi//lveukep6mgdowjjg=">aaab83icbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg7bbubomk4xd48adgq3/gm//glfsg4esmexlvjjpzgoqzpr3n2yqtrw9sbpw37z3dvf2dyuhrg4ptsahhyh7ltoav5uxqtzpnaserfecbp+1gfjp77qmvisxixk8t6kd4kfjicnzg8mt21gtc9dtrm7vwr1sdujmhwivuqapqonwvfpugmukjkjthwkmu6ytaz7dujha6s3upogkmyzykxumfjqjys/nrm3rmlaeky2lkadrxf09kofjqggwmm8j6pja9xpzp66y6vpizjpjuu0ewi8kuix2jpae0yjiszaegyckzurwrezayajotbujwl19ejv6jfl137xrv5kwrrhlo4btowyvlamittmadao/wdk/wzk2sf+vd+li0lqxi5hj+wpr8at0bkjo=</latexit> <latexit sha1_base64="3hznwolfi//lveukep6mgdowjjg=">aaab83icbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg7bbubomk4xd48adgq3/gm//glfsg4esmexlvjjpzgoqzpr3n2yqtrw9sbpw37z3dvf2dyuhrg4ptsahhyh7ltoav5uxqtzpnaserfecbp+1gfjp77qmvisxixk8t6kd4kfjicnzg8mt21gtc9dtrm7vwr1sdujmhwivuqapqonwvfpugmukjkjthwkmu6ytaz7dujha6s3upogkmyzykxumfjqjys/nrm3rmlaeky2lkadrxf09kofjqggwmm8j6pja9xpzp66y6vpizjpjuu0ewi8kuix2jpae0yjiszaegyckzurwrezayajotbujwl19ejv6jfl137xrv5kwrrhlo4btowyvlamittmadao/wdk/wzk2sf+vd+li0lqxi5hj+wpr8at0bkjo=</latexit> SVM Intuition Think about the size of w T x i + b as a measure of confidence in classification of example x i = 12

<latexit sha1_base64="zdqmwzpientmhc2e4j1smnltydk=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkpkem6qvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f30syfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy9zwkckmrsswvjpqstldomdl1xpfteveocehjqcmncrmekiiotzgsyninwnczx/phfx7bkeahdyk8gr50ggcaet01ambdlaqxwsqhbkq1xeplgcy4dukkmunc9c0zkymmcukxi7nupiikci9rqpokixqsmciks3j4pjwh9gouxirh4f6eyfaoxct0vgei5ejm16bmf7v+kv3zquajjjukwrnffsqgjoe0fjikngdjjgoq5lt9feir4ghlfz6uqrdmt14eu9m4afjxzvrrteyjcg7aitggfjgdlxaf2saggdycz/ak3rqn7uv71z5mrrwtnnkhf6r9/gde4pdv</latexit> <latexit sha1_base64="zdqmwzpientmhc2e4j1smnltydk=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkpkem6qvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f30syfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy9zwkckmrsswvjpqstldomdl1xpfteveocehjqcmncrmekiiotzgsyninwnczx/phfx7bkeahdyk8gr50ggcaet01ambdlaqxwsqhbkq1xeplgcy4dukkmunc9c0zkymmcukxi7nupiikci9rqpokixqsmciks3j4pjwh9gouxirh4f6eyfaoxct0vgei5ejm16bmf7v+kv3zquajjjukwrnffsqgjoe0fjikngdjjgoq5lt9feir4ghlfz6uqrdmt14eu9m4afjxzvrrteyjcg7aitggfjgdlxaf2saggdycz/ak3rqn7uv71z5mrrwtnnkhf6r9/gde4pdv</latexit> <latexit sha1_base64="zdqmwzpientmhc2e4j1smnltydk=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkpkem6qvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f30syfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy9zwkckmrsswvjpqstldomdl1xpfteveocehjqcmncrmekiiotzgsyninwnczx/phfx7bkeahdyk8gr50ggcaet01ambdlaqxwsqhbkq1xeplgcy4dukkmunc9c0zkymmcukxi7nupiikci9rqpokixqsmciks3j4pjwh9gouxirh4f6eyfaoxct0vgei5ejm16bmf7v+kv3zquajjjukwrnffsqgjoe0fjikngdjjgoq5lt9feir4ghlfz6uqrdmt14eu9m4afjxzvrrteyjcg7aitggfjgdlxaf2saggdycz/ak3rqn7uv71z5mrrwtnnkhf6r9/gde4pdv</latexit> <latexit sha1_base64="zdqmwzpientmhc2e4j1smnltydk=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkpkem6qvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f30syfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy9zwkckmrsswvjpqstldomdl1xpfteveocehjqcmncrmekiiotzgsyninwnczx/phfx7bkeahdyk8gr50ggcaet01ambdlaqxwsqhbkq1xeplgcy4dukkmunc9c0zkymmcukxi7nupiikci9rqpokixqsmciks3j4pjwh9gouxirh4f6eyfaoxct0vgei5ejm16bmf7v+kv3zquajjjukwrnffsqgjoe0fjikngdjjgoq5lt9feir4ghlfz6uqrdmt14eu9m4afjxzvrrteyjcg7aitggfjgdlxaf2saggdycz/ak3rqn7uv71z5mrrwtnnkhf6r9/gde4pdv</latexit> Very confident y if w T i =1 x i + b 0 <latexit sha1_base64="szliz79nkodwecuwebl+0qltz8y=">aaab8xicbvbns8naej3urxq/qh69llacp5l04mdbkhjxwmhyqhrkzrtpl242yxcjhncf4cwdilf/jtf/jds2b219mpb4b4azewhkmdko821v1ty3nreq2/bo7t7+qe3w6felmstuiwlpzc/einimqkez5rsxsorjknnuolmd+d0nkhvlxipouxreecryxajwrvibdj5g6aa5dmnqqztnzw60stys1kfez1d76g8tksvuamkxur7rpdoosnsmcdq1+5miksytpkk+oqlhvaxf/oqpojpkeewjncu0mqu/jwock5xhoemmsr6rzw8m/uf5my6ugokjnnnukmwikonij2j2pxoysynmusgysgzurwsmjsbapgsbenzll1ej12pen937vr19uazrhrm4hxnw4rlacacd8ibaas/wcm+wtl6sd+tj0vqxyplj+apr8weqyy75</latexit> SVM Intuition 13

<latexit sha1_base64="jt2yhavzrqe2kymjlibgp6ikgim=">aaab8nicbvbns8naej3urxq/qh69llacf0vsq9wdupdisykxhtauzxbtlt1swu5gckf/w4shfa/+gm/+g7dtdtr6yodx3gwz84kem6ud59sqra1vbg6vt+2d3b39g8rh0aoku0mor2iey26afevmue8zzwk3krrhaaedyhi78ztpvcowiwedjdsp8eiwkbgsjdsv2dmaort04dq1qaxq1j050cpxc1kfau1b5as/jekauaejx0r1xcfrfo6lzottqd1pfu0wmear7rkqcesvn89vnqizowxrgettqqo5+nsix5fswrsyzgjrsvr2zuj/xi/v4zwfm5gkmgqywbsmhokyzqjaqyyp0twzbbpjzk2ijlherjuybbocu/zykvea9eu6e9+otppfgmu4gvm4bxcuoqv30aypcctwdk/wzqxwi/vufsxas1yxcwx/yh3+apw6jza=</latexit> <latexit sha1_base64="jt2yhavzrqe2kymjlibgp6ikgim=">aaab8nicbvbns8naej3urxq/qh69llacf0vsq9wdupdisykxhtauzxbtlt1swu5gckf/w4shfa/+gm/+g7dtdtr6yodx3gwz84kem6ud59sqra1vbg6vt+2d3b39g8rh0aoku0mor2iey26afevmue8zzwk3krrhaaedyhi78ztpvcowiwedjdsp8eiwkbgsjdsv2dmaort04dq1qaxq1j050cpxc1kfau1b5as/jekauaejx0r1xcfrfo6lzottqd1pfu0wmear7rkqcesvn89vnqizowxrgettqqo5+nsix5fswrsyzgjrsvr2zuj/xi/v4zwfm5gkmgqywbsmhokyzqjaqyyp0twzbbpjzk2ijlherjuybbocu/zykvea9eu6e9+otppfgmu4gvm4bxcuoqv30aypcctwdk/wzqxwi/vufsxas1yxcwx/yh3+apw6jza=</latexit> <latexit sha1_base64="jt2yhavzrqe2kymjlibgp6ikgim=">aaab8nicbvbns8naej3urxq/qh69llacf0vsq9wdupdisykxhtauzxbtlt1swu5gckf/w4shfa/+gm/+g7dtdtr6yodx3gwz84kem6ud59sqra1vbg6vt+2d3b39g8rh0aoku0mor2iey26afevmue8zzwk3krrhaaedyhi78ztpvcowiwedjdsp8eiwkbgsjdsv2dmaort04dq1qaxq1j050cpxc1kfau1b5as/jekauaejx0r1xcfrfo6lzottqd1pfu0wmear7rkqcesvn89vnqizowxrgettqqo5+nsix5fswrsyzgjrsvr2zuj/xi/v4zwfm5gkmgqywbsmhokyzqjaqyyp0twzbbpjzk2ijlherjuybbocu/zykvea9eu6e9+otppfgmu4gvm4bxcuoqv30aypcctwdk/wzqxwi/vufsxas1yxcwx/yh3+apw6jza=</latexit> <latexit sha1_base64="+ncglnlpq1rhvrhmyb7naacxhnc=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkjkem6rvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f308yfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy93wkckmrsswvjpqstldom9l1x5fteveocehjqcmncxfdnixoqdgsyvimwnczxw/gfx7bkeah9yg8gt50giomxvemmtkwc8ffseqogvjtz/hybjfoqxjjzjaqfctmpjshlilmjnedvjae4teakr7cciveuflxsq6pldoaqczviyqs3n8tgqqfmis+6gyrhin52tt8r9zpzxduzjrkukkipfsupazkge5jgqpkczzsogbhttvfir4hjrbu4ekqbgv+5ewwm42lhnxdrlvoyzsq4aacgmnggtpqalegdwyawsn4bq/gtxvsxrr37wpwwthkmx3wr9rnd9qtl18=</latexit> <latexit sha1_base64="+ncglnlpq1rhvrhmyb7naacxhnc=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkjkem6rvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f308yfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy93wkckmrsswvjpqstldom9l1x5fteveocehjqcmncxfdnixoqdgsyvimwnczxw/gfx7bkeah9yg8gt50giomxvemmtkwc8ffseqogvjtz/hybjfoqxjjzjaqfctmpjshlilmjnedvjae4teakr7cciveuflxsq6pldoaqczviyqs3n8tgqqfmis+6gyrhin52tt8r9zpzxduzjrkukkipfsupazkge5jgqpkczzsogbhttvfir4hjrbu4ekqbgv+5ewwm42lhnxdrlvoyzsq4aacgmnggtpqalegdwyawsn4bq/gtxvsxrr37wpwwthkmx3wr9rnd9qtl18=</latexit> <latexit sha1_base64="+ncglnlpq1rhvrhmyb7naacxhnc=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkjkem6rvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f308yfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy93wkckmrsswvjpqstldom9l1x5fteveocehjqcmncxfdnixoqdgsyvimwnczxw/gfx7bkeah9yg8gt50giomxvemmtkwc8ffseqogvjtz/hybjfoqxjjzjaqfctmpjshlilmjnedvjae4teakr7cciveuflxsq6pldoaqczviyqs3n8tgqqfmis+6gyrhin52tt8r9zpzxduzjrkukkipfsupazkge5jgqpkczzsogbhttvfir4hjrbu4ekqbgv+5ewwm42lhnxdrlvoyzsq4aacgmnggtpqalegdwyawsn4bq/gtxvsxrr37wpwwthkmx3wr9rnd9qtl18=</latexit> <latexit sha1_base64="+ncglnlpq1rhvrhmyb7naacxhnc=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkjkem6rvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f308yfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy93wkckmrsswvjpqstldom9l1x5fteveocehjqcmncxfdnixoqdgsyvimwnczxw/gfx7bkeah9yg8gt50giomxvemmtkwc8ffseqogvjtz/hybjfoqxjjzjaqfctmpjshlilmjnedvjae4teakr7cciveuflxsq6pldoaqczviyqs3n8tgqqfmis+6gyrhin52tt8r9zpzxduzjrkukkipfsupazkge5jgqpkczzsogbhttvfir4hjrbu4ekqbgv+5ewwm42lhnxdrlvoyzsq4aacgmnggtpqalegdwyawsn4bq/gtxvsxrr37wpwwthkmx3wr9rnd9qtl18=</latexit> SVM Intuition Very confident y i = 1 if w T x <latexit sha1_base64="jt2yhavzrqe2kymjlibgp6ikgim=">aaab8nicbvbns8naej3urxq/qh69llacf0vsq9wdupdisykxhtauzxbtlt1swu5gckf/w4shfa/+gm/+g7dtdtr6yodx3gwz84kem6ud59sqra1vbg6vt+2d3b39g8rh0aoku0mor2iey26afevmue8zzwk3krrhaaedyhi78ztpvcowiwedjdsp8eiwkbgsjdsv2dmaort04dq1qaxq1j050cpxc1kfau1b5as/jekauaejx0r1xcfrfo6lzottqd1pfu0wmear7rkqcesvn89vnqizowxrgettqqo5+nsix5fswrsyzgjrsvr2zuj/xi/v4zwfm5gkmgqywbsmhokyzqjaqyyp0twzbbpjzk2ijlherjuybbocu/zykvea9eu6e9+otppfgmu4gvm4bxcuoqv30aypcctwdk/wzqxwi/vufsxas1yxcwx/yh3+apw6jza=</latexit> i + b 0 14

SVM Intuition Want a DB that makes us very confident in the classification of each training example 15

SVM Intuition Do this by maximizing distance from DB to closest training examples 16

SVM Intuition Do this by maximizing distance from DB to closest training examples 17

SVM Intuition This distance from the DB to closest training examples is called the Margin M M M 18

SVM Intuition The points closest to the DB are called the support vectors We support vectors to nig support vector. 19

<latexit sha1_base64="n1hi3t6avtccm+j8q9yiz1vl41k=">aaacbhicbvc7tsnaedyhvzavayuujxikqshowaolremzjewijvf0vpytu85n624nrjybgn6fhgiqlr9bx99wersqmnjko5ld7e4eieaaxpfbkiwtr6yufdftjc2t7r1nd+9wx6mizkexifuzijojlpkpharrjoqrkbcseqwvx37jjinny3kdo4r1itkxposugjg6zmhzztpbio9z3ab2acrkmje9nopalnedkltxj8clxjurepqh3nw+2r2yphgtqaxruuw5cxqyoobtwxk7nwqwedokfdyyvjki6u42+slhx0bp4tbwpitgifp7iior1qmomj0rgyge98bif14rhfc8k3gzpmakns4ku4ehxunici8rrkgmdcfucxmrpgoicautng1c8ozfxir+txjr8a6rpdrpli0iokbh6ar56azv0bwqix9r9iie0st6s56sf+vd+pi2fqzzzd76a+vzbz+blqu=</latexit> <latexit sha1_base64="n1hi3t6avtccm+j8q9yiz1vl41k=">aaacbhicbvc7tsnaedyhvzavayuujxikqshowaolremzjewijvf0vpytu85n624nrjybgn6fhgiqlr9bx99wersqmnjko5ld7e4eieaaxpfbkiwtr6yufdftjc2t7r1nd+9wx6mizkexifuzijojlpkpharrjoqrkbcseqwvx37jjinny3kdo4r1itkxposugjg6zmhzztpbio9z3ab2acrkmje9nopalnedkltxj8clxjurepqh3nw+2r2yphgtqaxruuw5cxqyoobtwxk7nwqwedokfdyyvjki6u42+slhx0bp4tbwpitgifp7iior1qmomj0rgyge98bif14rhfc8k3gzpmakns4ku4ehxunici8rrkgmdcfucxmrpgoicautng1c8ozfxir+txjr8a6rpdrpli0iokbh6ar56azv0bwqix9r9iie0st6s56sf+vd+pi2fqzzzd76a+vzbz+blqu=</latexit> <latexit sha1_base64="n1hi3t6avtccm+j8q9yiz1vl41k=">aaacbhicbvc7tsnaedyhvzavayuujxikqshowaolremzjewijvf0vpytu85n624nrjybgn6fhgiqlr9bx99wersqmnjko5ld7e4eieaaxpfbkiwtr6yufdftjc2t7r1nd+9wx6mizkexifuzijojlpkpharrjoqrkbcseqwvx37jjinny3kdo4r1itkxposugjg6zmhzztpbio9z3ab2acrkmje9nopalnedkltxj8clxjurepqh3nw+2r2yphgtqaxruuw5cxqyoobtwxk7nwqwedokfdyyvjki6u42+slhx0bp4tbwpitgifp7iior1qmomj0rgyge98bif14rhfc8k3gzpmakns4ku4ehxunici8rrkgmdcfucxmrpgoicautng1c8ozfxir+txjr8a6rpdrpli0iokbh6ar56azv0bwqix9r9iie0st6s56sf+vd+pi2fqzzzd76a+vzbz+blqu=</latexit> <latexit sha1_base64="n1hi3t6avtccm+j8q9yiz1vl41k=">aaacbhicbvc7tsnaedyhvzavayuujxikqshowaolremzjewijvf0vpytu85n624nrjybgn6fhgiqlr9bx99wersqmnjko5ld7e4eieaaxpfbkiwtr6yufdftjc2t7r1nd+9wx6mizkexifuzijojlpkpharrjoqrkbcseqwvx37jjinny3kdo4r1itkxposugjg6zmhzztpbio9z3ab2acrkmje9nopalnedkltxj8clxjurepqh3nw+2r2yphgtqaxruuw5cxqyoobtwxk7nwqwedokfdyyvjki6u42+slhx0bp4tbwpitgifp7iior1qmomj0rgyge98bif14rhfc8k3gzpmakns4ku4ehxunici8rrkgmdcfucxmrpgoicautng1c8ozfxir+txjr8a6rpdrpli0iokbh6ar56azv0bwqix9r9iie0st6s56sf+vd+pi2fqzzzd76a+vzbz+blqu=</latexit> <latexit sha1_base64="gmv1gq3syo6vn4lfcnewxkhwoco=">aaacdhicbvfli9raeo7e1xpfo14eedqdhfgwhgqvpk4rxrwikzjuwmqykp3kblpdndbd0qkh/gj/ntf/hhevdmyj6k4fdv991fx6kq+vdjqkp4lwytvr12/s3ixu3b5z997k/onprmqswlmovgvpcncopme5svj4ulsenss8zs/edvhjz2idrmxhamtcalgbwuob5knv5ntelow4lqyda6hto6h6kcukqxw0jlqfmybnquuyvorf+v2vec+f8fc8y6kmcenwdy6mecd/+g+u4mrbgmnwhdega4woi8pafkralhq450fawndcu4ydo71onzkmcbi1fhmki5iy0y5wk+9zuylgo6ft1uwa1lt0bukkhb5847agcqzrxhhoqknbdlvvej7ztmhlyvpnig/zvzm60m612i8800cn7mjsip8xwzruvlx20tqnorhnjcrgi1lx4qs8kkmshqrhpz+vi1owimgfahahvbjyzta/if/f6yed6ehruy0d9pjtsucszs/yixvhjticcfyzebq8dxadx+gtcbrozr+gwzjzkp1jyfwbfjo/ag==</latexit> <latexit sha1_base64="gmv1gq3syo6vn4lfcnewxkhwoco=">aaacdhicbvfli9raeo7e1xpfo14eedqdhfgwhgqvpk4rxrwikzjuwmqykp3kblpdndbd0qkh/gj/ntf/hhevdmyj6k4fdv991fx6kq+vdjqkp4lwytvr12/s3ixu3b5z997k/onprmqswlmovgvpcncopme5svj4ulsenss8zs/edvhjz2idrmxhamtcalgbwuob5knv5ntelow4lqyda6hto6h6kcukqxw0jlqfmybnquuyvorf+v2vec+f8fc8y6kmcenwdy6mecd/+g+u4mrbgmnwhdega4woi8pafkralhq450fawndcu4ydo71onzkmcbi1fhmki5iy0y5wk+9zuylgo6ft1uwa1lt0bukkhb5847agcqzrxhhoqknbdlvvej7ztmhlyvpnig/zvzm60m612i8800cn7mjsip8xwzruvlx20tqnorhnjcrgi1lx4qs8kkmshqrhpz+vi1owimgfahahvbjyzta/if/f6yed6ehruy0d9pjtsucszs/yixvhjticcfyzebq8dxadx+gtcbrozr+gwzjzkp1jyfwbfjo/ag==</latexit> <latexit sha1_base64="gmv1gq3syo6vn4lfcnewxkhwoco=">aaacdhicbvfli9raeo7e1xpfo14eedqdhfgwhgqvpk4rxrwikzjuwmqykp3kblpdndbd0qkh/gj/ntf/hhevdmyj6k4fdv991fx6kq+vdjqkp4lwytvr12/s3ixu3b5z997k/onprmqswlmovgvpcncopme5svj4ulsenss8zs/edvhjz2idrmxhamtcalgbwuob5knv5ntelow4lqyda6hto6h6kcukqxw0jlqfmybnquuyvorf+v2vec+f8fc8y6kmcenwdy6mecd/+g+u4mrbgmnwhdega4woi8pafkralhq450fawndcu4ydo71onzkmcbi1fhmki5iy0y5wk+9zuylgo6ft1uwa1lt0bukkhb5847agcqzrxhhoqknbdlvvej7ztmhlyvpnig/zvzm60m612i8800cn7mjsip8xwzruvlx20tqnorhnjcrgi1lx4qs8kkmshqrhpz+vi1owimgfahahvbjyzta/if/f6yed6ehruy0d9pjtsucszs/yixvhjticcfyzebq8dxadx+gtcbrozr+gwzjzkp1jyfwbfjo/ag==</latexit> <latexit sha1_base64="gmv1gq3syo6vn4lfcnewxkhwoco=">aaacdhicbvfli9raeo7e1xpfo14eedqdhfgwhgqvpk4rxrwikzjuwmqykp3kblpdndbd0qkh/gj/ntf/hhevdmyj6k4fdv991fx6kq+vdjqkp4lwytvr12/s3ixu3b5z997k/onprmqswlmovgvpcncopme5svj4ulsenss8zs/edvhjz2idrmxhamtcalgbwuob5knv5ntelow4lqyda6hto6h6kcukqxw0jlqfmybnquuyvorf+v2vec+f8fc8y6kmcenwdy6mecd/+g+u4mrbgmnwhdega4woi8pafkralhq450fawndcu4ydo71onzkmcbi1fhmki5iy0y5wk+9zuylgo6ft1uwa1lt0bukkhb5847agcqzrxhhoqknbdlvvej7ztmhlyvpnig/zvzm60m612i8800cn7mjsip8xwzruvlx20tqnorhnjcrgi1lx4qs8kkmshqrhpz+vi1owimgfahahvbjyzta/if/f6yed6ehruy0d9pjtsucszs/yixvhjticcfyzebq8dxadx+gtcbrozr+gwzjzkp1jyfwbfjo/ag==</latexit> Maximum Margin Classifier How do we do this mathematically? Let the margin be represented by M Goal: Determine w and b that maximize M while classifying all training points correctly This gives the following (informal) optimization problem max w, b s.t. M All training examples correctly classified 20