Hard-Margin Support Vector Machines

Transkrypt

1 Hard-Margin Support Vector Machines

2 <latexit sha1_base64="b4hfbmymyimfofmhtyo9gzgt56w=">aaacaxicbzdlssnafiyn9vbjlepk3ay2gicupasvu4iblxuaw2hjmuwn7ddjjmxm1bkcg1/fjqsvt76fo9/gazqfvn8y+pjpozw5vx8zkpvtfxmlhcwl5zxyqrm2vrg5zw3vxmsoezi4ogkr6phieky5crvvjhriqvdom9l2xxftevuwcekj3lktmhghgniauiyutvrwxtvme34a77kbvg73gtygpjsrfati1+xc8c84bvraowbf+uwnipyehcvmkjrdx46vlykhkgykm3ujjdhcyzqkxy0chur6ax5cbg+1m4bbjptjcubuz4kuhvjoql93hkin5hxtav5x6yyqopnsyuneey5ni4keqrxbar5wqaxbik00icyo/iveiyqqvjo1u4fgzj/8f9x67bzmxnurjzmijtlybwfgcdjgfdtajwgcf2dwaj7ac3g1ho1n4814n7wwjgjmf/ys8fenfycuzq==</latexit> <latexit sha1_base64="b4hfbmymyimfofmhtyo9gzgt56w=">aaacaxicbzdlssnafiyn9vbjlepk3ay2gicupasvu4iblxuaw2hjmuwn7ddjjmxm1bkcg1/fjqsvt76fo9/gazqfvn8y+pjpozw5vx8zkpvtfxmlhcwl5zxyqrm2vrg5zw3vxmsoezi4ogkr6phieky5crvvjhriqvdom9l2xxftevuwcekj3lktmhghgniauiyutvrwxtvme34a77kbvg73gtygpjsrfati1+xc8c84bvraowbf+uwnipyehcvmkjrdx46vlykhkgykm3ujjdhcyzqkxy0chur6ax5cbg+1m4bbjptjcubuz4kuhvjoql93hkin5hxtav5x6yyqopnsyuneey5ni4keqrxbar5wqaxbik00icyo/iveiyqqvjo1u4fgzj/8f9x67bzmxnurjzmijtlybwfgcdjgfdtajwgcf2dwaj7ac3g1ho1n4814n7wwjgjmf/ys8fenfycuzq==</latexit> <latexit sha1_base64="b4hfbmymyimfofmhtyo9gzgt56w=">aaacaxicbzdlssnafiyn9vbjlepk3ay2gicupasvu4iblxuaw2hjmuwn7ddjjmxm1bkcg1/fjqsvt76fo9/gazqfvn8y+pjpozw5vx8zkpvtfxmlhcwl5zxyqrm2vrg5zw3vxmsoezi4ogkr6phieky5crvvjhriqvdom9l2xxftevuwcekj3lktmhghgniauiyutvrwxtvme34a77kbvg73gtygpjsrfati1+xc8c84bvraowbf+uwnipyehcvmkjrdx46vlykhkgykm3ujjdhcyzqkxy0chur6ax5cbg+1m4bbjptjcubuz4kuhvjoql93hkin5hxtav5x6yyqopnsyuneey5ni4keqrxbar5wqaxbik00icyo/iveiyqqvjo1u4fgzj/8f9x67bzmxnurjzmijtlybwfgcdjgfdtajwgcf2dwaj7ac3g1ho1n4814n7wwjgjmf/ys8fenfycuzq==</latexit> <latexit sha1_base64="b4hfbmymyimfofmhtyo9gzgt56w=">aaacaxicbzdlssnafiyn9vbjlepk3ay2gicupasvu4iblxuaw2hjmuwn7ddjjmxm1bkcg1/fjqsvt76fo9/gazqfvn8y+pjpozw5vx8zkpvtfxmlhcwl5zxyqrm2vrg5zw3vxmsoezi4ogkr6phieky5crvvjhriqvdom9l2xxftevuwcekj3lktmhghgniauiyutvrwxtvme34a77kbvg73gtygpjsrfati1+xc8c84bvraowbf+uwnipyehcvmkjrdx46vlykhkgykm3ujjdhcyzqkxy0chur6ax5cbg+1m4bbjptjcubuz4kuhvjoql93hkin5hxtav5x6yyqopnsyuneey5ni4keqrxbar5wqaxbik00icyo/iveiyqqvjo1u4fgzj/8f9x67bzmxnurjzmijtlybwfgcdjgfdtajwgcf2dwaj7ac3g1ho1n4814n7wwjgjmf/ys8fenfycuzq==</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> Linear Classifiers We ve already seen several: o o Logistic Regression Perceptron Def: A linear classifier makes decisions by computing a linear combination of features of the form w T x + b and then classifies based on w T x + b 0 The boundary between the two classes is defined by or the decision boundary w T x + b =0 and is called the separator We estimate the weights and bias using the training data 2

3 <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="icmczj6jv+dkorcojigcuvd9d1y=">aaab/nicbvdlssnafl2prxpfucgnm8fweissdofjv3djsokxhbaeyxtsdp08njmojxbhr7hxoelw73dn3zhts9dwaxco59zlvff4cwds2fa3uvhyxfpeka6aa+sbm1vw9s6njfnbqetihoumjyxllkkuyortziiodn1og/7gyuw37qiqli6u1tchnrd3ihywgpwwpguvbn57zopnogpko3ap3iibmwxpktkvewi0t5yclcbh3bo+2t2ypcgnfofyypzjj6qtyaey4xrktlnje0wguedbmky4plktte4fouotdfeqc12rqhp190sgqymhoa87q6z6ctybi/95rvqfz52mrumqaesmi4kuixwjcrioywqlig81wuqwfssifswwutoyu4fgzl48t9xq5bzixfvltzm8jslswwecgqonuinlqimlbb7hgv7hzxgyxox342pawjdymv34a+pzb3nkkq4=</latexit> <latexit sha1_base64="icmczj6jv+dkorcojigcuvd9d1y=">aaab/nicbvdlssnafl2prxpfucgnm8fweissdofjv3djsokxhbaeyxtsdp08njmojxbhr7hxoelw73dn3zhts9dwaxco59zlvff4cwds2fa3uvhyxfpeka6aa+sbm1vw9s6njfnbqetihoumjyxllkkuyortziiodn1og/7gyuw37qiqli6u1tchnrd3ihywgpwwpguvbn57zopnogpko3ap3iibmwxpktkvewi0t5yclcbh3bo+2t2ypcgnfofyypzjj6qtyaey4xrktlnje0wguedbmky4plktte4fouotdfeqc12rqhp190sgqymhoa87q6z6ctybi/95rvqfz52mrumqaesmi4kuixwjcrioywqlig81wuqwfssifswwutoyu4fgzl48t9xq5bzixfvltzm8jslswwecgqonuinlqimlbb7hgv7hzxgyxox342pawjdymv34a+pzb3nkkq4=</latexit> <latexit sha1_base64="icmczj6jv+dkorcojigcuvd9d1y=">aaab/nicbvdlssnafl2prxpfucgnm8fweissdofjv3djsokxhbaeyxtsdp08njmojxbhr7hxoelw73dn3zhts9dwaxco59zlvff4cwds2fa3uvhyxfpeka6aa+sbm1vw9s6njfnbqetihoumjyxllkkuyortziiodn1og/7gyuw37qiqli6u1tchnrd3ihywgpwwpguvbn57zopnogpko3ap3iibmwxpktkvewi0t5yclcbh3bo+2t2ypcgnfofyypzjj6qtyaey4xrktlnje0wguedbmky4plktte4fouotdfeqc12rqhp190sgqymhoa87q6z6ctybi/95rvqfz52mrumqaesmi4kuixwjcrioywqlig81wuqwfssifswwutoyu4fgzl48t9xq5bzixfvltzm8jslswwecgqonuinlqimlbb7hgv7hzxgyxox342pawjdymv34a+pzb3nkkq4=</latexit> <latexit sha1_base64="8hxxgp36artuotcncd/iql/irgi=">aaacaxicbvdlssnafj3uv42vqctxm9gkbixjfz4wqsgnywrwftoqjpnjo3qyctmttytgxl9x40lfrx/hzr9x2mahrqcuhm65l3vv8rngpbltb6o0sli0vfjendfwnza3ro2dwxmnapmwjlksoj6shffowooqrjqjicjygwn7w8ux374jqtky36hrqtwi9tknkuzks561vzufpadewoneeaqemz/p7j0nn6uevbfr9grwnjgfqyactc/66guxtipcfwziyq5jj8rnkfaum5kbvvssboeh6pouphxfrlrz5iuchmolggesdhefj+rviqxfuo4ix3dgsa3krdcw//o6qqrp3izyjfwe4+mimgvqxxccbwyoifixksyic6pvhxiadbjkp2bqejzzl+djq147rznx9urjpeijdpbbatgcdjgfdxafmqafmhgez+avvblpxovxbnxmw0tgmbml/sd4/agemzuu</latexit> <latexit sha1_base64="8hxxgp36artuotcncd/iql/irgi=">aaacaxicbvdlssnafj3uv42vqctxm9gkbixjfz4wqsgnywrwftoqjpnjo3qyctmttytgxl9x40lfrx/hzr9x2mahrqcuhm65l3vv8rngpbltb6o0sli0vfjendfwnza3ro2dwxmnapmwjlksoj6shffowooqrjqjicjygwn7w8ux374jqtky36hrqtwi9tknkuzks561vzufpadewoneeaqemz/p7j0nn6uevbfr9grwnjgfqyactc/66guxtipcfwziyq5jj8rnkfaum5kbvvssboeh6pouphxfrlrz5iuchmolggesdhefj+rviqxfuo4ix3dgsa3krdcw//o6qqrp3izyjfwe4+mimgvqxxccbwyoifixksyic6pvhxiadbjkp2bqejzzl+djq147rznx9urjpeijdpbbatgcdjgfdxafmqafmhgez+avvblpxovxbnxmw0tgmbml/sd4/agemzuu</latexit> <latexit sha1_base64="8hxxgp36artuotcncd/iql/irgi=">aaacaxicbvdlssnafj3uv42vqctxm9gkbixjfz4wqsgnywrwftoqjpnjo3qyctmttytgxl9x40lfrx/hzr9x2mahrqcuhm65l3vv8rngpbltb6o0sli0vfjendfwnza3ro2dwxmnapmwjlksoj6shffowooqrjqjicjygwn7w8ux374jqtky36hrqtwi9tknkuzks561vzufpadewoneeaqemz/p7j0nn6uevbfr9grwnjgfqyactc/66guxtipcfwziyq5jj8rnkfaum5kbvvssboeh6pouphxfrlrz5iuchmolggesdhefj+rviqxfuo4ix3dgsa3krdcw//o6qqrp3izyjfwe4+mimgvqxxccbwyoifixksyic6pvhxiadbjkp2bqejzzl+djq147rznx9urjpeijdpbbatgcdjgfdxafmqafmhgez+avvblpxovxbnxmw0tgmbml/sd4/agemzuu</latexit> <latexit sha1_base64="8hxxgp36artuotcncd/iql/irgi=">aaacaxicbvdlssnafj3uv42vqctxm9gkbixjfz4wqsgnywrwftoqjpnjo3qyctmttytgxl9x40lfrx/hzr9x2mahrqcuhm65l3vv8rngpbltb6o0sli0vfjendfwnza3ro2dwxmnapmwjlksoj6shffowooqrjqjicjygwn7w8ux374jqtky36hrqtwi9tknkuzks561vzufpadewoneeaqemz/p7j0nn6uevbfr9grwnjgfqyactc/66guxtipcfwziyq5jj8rnkfaum5kbvvssboeh6pouphxfrlrz5iuchmolggesdhefj+rviqxfuo4ix3dgsa3krdcw//o6qqrp3izyjfwe4+mimgvqxxccbwyoifixksyic6pvhxiadbjkp2bqejzzl+djq147rznx9urjpeijdpbbatgcdjgfdxafmqafmhgez+avvblpxovxbnxmw0tgmbml/sd4/agemzuu</latexit> <latexit sha1_base64="h5w/ffu3p9clzeunl11+cnonbag=">aaacbhicbvc7tsnaedyhvzavayuujxikgii7by8ueg1lkdcjffvw+xxotjk/udsdkewghl+hoqbey0fq8tdceheqmnjko5ld7e74kancmua3vllyxfpeqa7qa+sbm1vg9s6nsdkoiy0tlvcujwrhnca2pjkrbsojinxgov7wyux37ggxnimv5sglbot6mq0prljjnrff1x88czp9cgupnsdkcod+kd97vqhxpanmnswj4dyxslidjdqe8euecc4iekvmkba9y0ylmymukwak0j1mkbthieqtnqixiohw88kxbtxusgddhkukjzyovydyfakxinzvgse5elpewpzp62uyphnzgqezjdgelgozbmucx5haghkcjrspgjcn6laib0glivvwugrbmn15ntjnxnndumrwwidlglwwbw7aebdakwibs9agnsdgetydv/cmpwkv2rv2mw2taoxmlvgd7fmhos+wqa==</latexit> <latexit sha1_base64="h5w/ffu3p9clzeunl11+cnonbag=">aaacbhicbvc7tsnaedyhvzavayuujxikgii7by8ueg1lkdcjffvw+xxotjk/udsdkewghl+hoqbey0fq8tdceheqmnjko5ld7e74kancmua3vllyxfpeqa7qa+sbm1vg9s6nsdkoiy0tlvcujwrhnca2pjkrbsojinxgov7wyux37ggxnimv5sglbot6mq0prljjnrff1x88czp9cgupnsdkcod+kd97vqhxpanmnswj4dyxslidjdqe8euecc4iekvmkba9y0ylmymukwak0j1mkbthieqtnqixiohw88kxbtxusgddhkukjzyovydyfakxinzvgse5elpewpzp62uyphnzgqezjdgelgozbmucx5haghkcjrspgjcn6laib0glivvwugrbmn15ntjnxnndumrwwidlglwwbw7aebdakwibs9agnsdgetydv/cmpwkv2rv2mw2taoxmlvgd7fmhos+wqa==</latexit> <latexit sha1_base64="h5w/ffu3p9clzeunl11+cnonbag=">aaacbhicbvc7tsnaedyhvzavayuujxikgii7by8ueg1lkdcjffvw+xxotjk/udsdkewghl+hoqbey0fq8tdceheqmnjko5ld7e74kancmua3vllyxfpeqa7qa+sbm1vg9s6nsdkoiy0tlvcujwrhnca2pjkrbsojinxgov7wyux37ggxnimv5sglbot6mq0prljjnrff1x88czp9cgupnsdkcod+kd97vqhxpanmnswj4dyxslidjdqe8euecc4iekvmkba9y0ylmymukwak0j1mkbthieqtnqixiohw88kxbtxusgddhkukjzyovydyfakxinzvgse5elpewpzp62uyphnzgqezjdgelgozbmucx5haghkcjrspgjcn6laib0glivvwugrbmn15ntjnxnndumrwwidlglwwbw7aebdakwibs9agnsdgetydv/cmpwkv2rv2mw2taoxmlvgd7fmhos+wqa==</latexit> <latexit sha1_base64="h5w/ffu3p9clzeunl11+cnonbag=">aaacbhicbvc7tsnaedyhvzavayuujxikgii7by8ueg1lkdcjffvw+xxotjk/udsdkewghl+hoqbey0fq8tdceheqmnjko5ld7e74kancmua3vllyxfpeqa7qa+sbm1vg9s6nsdkoiy0tlvcujwrhnca2pjkrbsojinxgov7wyux37ggxnimv5sglbot6mq0prljjnrff1x88czp9cgupnsdkcod+kd97vqhxpanmnswj4dyxslidjdqe8euecc4iekvmkba9y0ylmymukwak0j1mkbthieqtnqixiohw88kxbtxusgddhkukjzyovydyfakxinzvgse5elpewpzp62uyphnzgqezjdgelgozbmucx5haghkcjrspgjcn6laib0glivvwugrbmn15ntjnxnndumrwwidlglwwbw7aebdakwibs9agnsdgetydv/cmpwkv2rv2mw2taoxmlvgd7fmhos+wqa==</latexit> Linear Classifiers In 1D we just have a single feature, x 1 The decision rule w T b x + b 0 becomes w or, <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> 1 x 1 + b 0 x <latexit sha1_base64="icmczj6jv+dkorcojigcuvd9d1y=">aaab/nicbvdlssnafl2prxpfucgnm8fweissdofjv3djsokxhbaeyxtsdp08njmojxbhr7hxoelw73dn3zhts9dwaxco59zlvff4cwds2fa3uvhyxfpeka6aa+sbm1vw9s6njfnbqetihoumjyxllkkuyortziiodn1og/7gyuw37qiqli6u1tchnrd3ihywgpwwpguvbn57zopnogpko3ap3iibmwxpktkvewi0t5yclcbh3bo+2t2ypcgnfofyypzjj6qtyaey4xrktlnje0wguedbmky4plktte4fouotdfeqc12rqhp190sgqymhoa87q6z6ctybi/95rvqfz52mrumqaesmi4kuixwjcrioywqlig81wuqwfssifswwutoyu4fgzl48t9xq5bzixfvltzm8jslswwecgqonuinlqimlbb7hgv7hzxgyxox342pawjdymv34a+pzb3nkkq4=</latexit> 1 w 1 The decision boundary is just a single point., x 1 = b w 1 mmm DB 3

4 <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="angnjlixnssz1htazh+v4zeetfy=">aaacfnicbzc7tsmwfiydrixcaowsfi0sc1wsgcuavimfsuievmqjyhgc1qpzke1aqyhvwckrsdaaykvsva1ugyfoozktx99/juzz+ymjqprml7awuls8slpz09c3nre2jz3dw5fkhbmhjyzhbr8jwmhmhekli+2uext5jlt8wexyb90rlmgs38hrstwi9wiauoykqp5rr+ldz4yx8lgbhbzh/n6zcnxzxdczfqa/rxrnm6pm3zwunbdwkaqgrkznfhadbgcrisvmsiiozabszrgxfdns6n1mkbthaeqrjpixiohw88lebtxujibhwtwjjzzq3xm5ioqyrb7qjjdsi1lvdp/zopkmz9ycxmkmsyynd4uzgzkb45bgqdnbko2uqjht9vei+0glivwuugrbml15xjh2/bxuxdvvxkmzrgxsgwnwbcxwchrgcjsbazb4ae/gbbxqj9qz9qa9t1sxthjmd/wp7embra6dsg==</latexit> <latexit sha1_base64="angnjlixnssz1htazh+v4zeetfy=">aaacfnicbzc7tsmwfiydrixcaowsfi0sc1wsgcuavimfsuievmqjyhgc1qpzke1aqyhvwckrsdaaykvsva1ugyfoozktx99/juzz+ymjqprml7awuls8slpz09c3nre2jz3dw5fkhbmhjyzhbr8jwmhmhekli+2uext5jlt8wexyb90rlmgs38hrstwi9wiauoykqp5rr+ldz4yx8lgbhbzh/n6zcnxzxdczfqa/rxrnm6pm3zwunbdwkaqgrkznfhadbgcrisvmsiiozabszrgxfdns6n1mkbthaeqrjpixiohw88lebtxujibhwtwjjzzq3xm5ioqyrb7qjjdsi1lvdp/zopkmz9ycxmkmsyynd4uzgzkb45bgqdnbko2uqjht9vei+0glivwuugrbml15xjh2/bxuxdvvxkmzrgxsgwnwbcxwchrgcjsbazb4ae/gbbxqj9qz9qa9t1sxthjmd/wp7embra6dsg==</latexit> <latexit sha1_base64="angnjlixnssz1htazh+v4zeetfy=">aaacfnicbzc7tsmwfiydrixcaowsfi0sc1wsgcuavimfsuievmqjyhgc1qpzke1aqyhvwckrsdaaykvsva1ugyfoozktx99/juzz+ymjqprml7awuls8slpz09c3nre2jz3dw5fkhbmhjyzhbr8jwmhmhekli+2uext5jlt8wexyb90rlmgs38hrstwi9wiauoykqp5rr+ldz4yx8lgbhbzh/n6zcnxzxdczfqa/rxrnm6pm3zwunbdwkaqgrkznfhadbgcrisvmsiiozabszrgxfdns6n1mkbthaeqrjpixiohw88lebtxujibhwtwjjzzq3xm5ioqyrb7qjjdsi1lvdp/zopkmz9ycxmkmsyynd4uzgzkb45bgqdnbko2uqjht9vei+0glivwuugrbml15xjh2/bxuxdvvxkmzrgxsgwnwbcxwchrgcjsbazb4ae/gbbxqj9qz9qa9t1sxthjmd/wp7embra6dsg==</latexit> <latexit sha1_base64="angnjlixnssz1htazh+v4zeetfy=">aaacfnicbzc7tsmwfiydrixcaowsfi0sc1wsgcuavimfsuievmqjyhgc1qpzke1aqyhvwckrsdaaykvsva1ugyfoozktx99/juzz+ymjqprml7awuls8slpz09c3nre2jz3dw5fkhbmhjyzhbr8jwmhmhekli+2uext5jlt8wexyb90rlmgs38hrstwi9wiauoykqp5rr+ldz4yx8lgbhbzh/n6zcnxzxdczfqa/rxrnm6pm3zwunbdwkaqgrkznfhadbgcrisvmsiiozabszrgxfdns6n1mkbthaeqrjpixiohw88lebtxujibhwtwjjzzq3xm5ioqyrb7qjjdsi1lvdp/zopkmz9ycxmkmsyynd4uzgzkb45bgqdnbko2uqjht9vei+0glivwuugrbml15xjh2/bxuxdvvxkmzrgxsgwnwbcxwchrgcjsbazb4ae/gbbxqj9qz9qa9t1sxthjmd/wp7embra6dsg==</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="389u0atoa0cw445vps2p6tlojwe=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkgxctkosjeuitfibry7rtfydj9gobvv1zofvwbgasfikblwnbpsbwn7j0qf/nkpj84cpo1lz9rdrwfpewv0rrpsbm1vbo9bu3p1mmogjhxowieaijggue09rxugjfqtfisp1shc1qdcfija04bdqmbi/rh1oi4qr0lzghzbnfuamageewn7gdgjxqwhbhxipbwiwa6tkv+yp4ci4ozrarlpgfbxacc5iwhvmsmqmy6fkhyghkgzkblyysvkee6hdmho5ion0r9ndxvbyo20yjui/rudu/t0xqrguwzjuntfsxtlfm5j/1zqzii78eevppgjhs0vrxqbk4cqv2kacymwgghawvp8v4i4sccudnaldcozpxgtprvxwnbu3vd3l0yica3aetoadzkevxima8aagj+azvii348l4md6nj1lrwchn9sefgz8/yeuvha==</latexit> <latexit sha1_base64="389u0atoa0cw445vps2p6tlojwe=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkgxctkosjeuitfibry7rtfydj9gobvv1zofvwbgasfikblwnbpsbwn7j0qf/nkpj84cpo1lz9rdrwfpewv0rrpsbm1vbo9bu3p1mmogjhxowieaijggue09rxugjfqtfisp1shc1qdcfija04bdqmbi/rh1oi4qr0lzghzbnfuamageewn7gdgjxqwhbhxipbwiwa6tkv+yp4ci4ozrarlpgfbxacc5iwhvmsmqmy6fkhyghkgzkblyysvkee6hdmho5ion0r9ndxvbyo20yjui/rudu/t0xqrguwzjuntfsxtlfm5j/1zqzii78eevppgjhs0vrxqbk4cqv2kacymwgghawvp8v4i4sccudnaldcozpxgtprvxwnbu3vd3l0yica3aetoadzkevxima8aagj+azvii348l4md6nj1lrwchn9sefgz8/yeuvha==</latexit> <latexit sha1_base64="389u0atoa0cw445vps2p6tlojwe=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkgxctkosjeuitfibry7rtfydj9gobvv1zofvwbgasfikblwnbpsbwn7j0qf/nkpj84cpo1lz9rdrwfpewv0rrpsbm1vbo9bu3p1mmogjhxowieaijggue09rxugjfqtfisp1shc1qdcfija04bdqmbi/rh1oi4qr0lzghzbnfuamageewn7gdgjxqwhbhxipbwiwa6tkv+yp4ci4ozrarlpgfbxacc5iwhvmsmqmy6fkhyghkgzkblyysvkee6hdmho5ion0r9ndxvbyo20yjui/rudu/t0xqrguwzjuntfsxtlfm5j/1zqzii78eevppgjhs0vrxqbk4cqv2kacymwgghawvp8v4i4sccudnaldcozpxgtprvxwnbu3vd3l0yica3aetoadzkevxima8aagj+azvii348l4md6nj1lrwchn9sefgz8/yeuvha==</latexit> <latexit sha1_base64="389u0atoa0cw445vps2p6tlojwe=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkgxctkosjeuitfibry7rtfydj9gobvv1zofvwbgasfikblwnbpsbwn7j0qf/nkpj84cpo1lz9rdrwfpewv0rrpsbm1vbo9bu3p1mmogjhxowieaijggue09rxugjfqtfisp1shc1qdcfija04bdqmbi/rh1oi4qr0lzghzbnfuamageewn7gdgjxqwhbhxipbwiwa6tkv+yp4ci4ozrarlpgfbxacc5iwhvmsmqmy6fkhyghkgzkblyysvkee6hdmho5ion0r9ndxvbyo20yjui/rudu/t0xqrguwzjuntfsxtlfm5j/1zqzii78eevppgjhs0vrxqbk4cqv2kacymwgghawvp8v4i4sccudnaldcozpxgtprvxwnbu3vd3l0yica3aetoadzkevxima8aagj+azvii348l4md6nj1lrwchn9sefgz8/yeuvha==</latexit> Linear Classifiers In 2D we just have two features, The decision rule w T x + b 0 x 1 and becomes x 2 w 1 x 1 + w 2 x 2 + b 0 l ye { The decision boundary is the line x 2 = w 1 x 1 w 2 b w 2 Note that the vector is orthogonal to the DB and points in the direction of the positive class : + + 4

5 <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="wmptcbrvegnoxzssodo3874tau8=">aaacb3icbzc7tsmwfiadcivhfmbkwkjfqkkqkg5ctkosjevqakumvi7rtfadc7ydvffgfl6fhqeqk6/axtvgphmg5zcsffrpoto+vxczkqrpfmulhcwl5zxyqr62vrg5zwzvxiso4zjyogir73hieezdyksqgenenkday6ttjs4m9fyd4yjgyuuoy+igabbsn2iklduz9qt66ng+vm9uwjk8zpayetazkfto6tweutfrzi44d1ybfvco2to+nh6ek4ceejmkrncyy+mmieukgcl0jxekrniebqsrmeqbew6ah5lbq+x0or9x9uijc/f3riocicabpzodjiditjyx/6t1e+mfuskn40ssee8x+qmdmoktvgcfcoilgytamfp1v4ihicmsvxa6csgapxke7hrtvgzd1sunkyknmtgdb+aiwoaunmalaaibypainserenoetbftxfuytpa0ymyx/jh2+qmm5jby</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="4tlvctcueii1ge6ko3/g9hkbl+c=">aaab7hicbva9twjbej3dlzy/ueubjwbire4o/ohibcwx8yaelmrv2yovvd3l7p6rep6djyuaw3+qnf/gba5q8cwtvlw3k5l5ucqznp737rtw1jc2t4rb7s7u3v5b6fcoqwwmca2i5fk1i6wpz4ighhlo26mioik4buwjm5nfeqrkmynuztilyyihgswmygolzsv96vmvxqnsvb050crxc1kghi1e6avblyrlqdcey607vpeacikvyyttqdvnne0xgeeb7vgqcej1ojlfo0vnvumjwcpbwqc5+ntighotx0lkoxnshnrzm4n/ez3mxffhhik0m1sqxai448hinhsd9zmixpcxjzgozm9fzigvjsyg5noq/owxv0lqq15x/btaux6rp1geezifc/dheupwcw0igmadpmmrvdnsexheny9fa8hjz47hd5zph2hdjdy=</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> <latexit sha1_base64="kg8f2gqmefcl9gxch+hg6t9o+70=">aaab7hicbvbntwixej3fl1y/ui9egshee9nvxi8birepmlhaahvslv2odntn2zuswn/w4kgnv3+qn/+nbfag4esmexlvjjpzopqzbtzv2ymsrk6tbxq33a3tnd290v5bq8tmeroqyavqrvhtzgqnddoctljfcrjx2oygn1o/+uivzllcm1fkwwt3byszwczkjyr71d2vdetlr+rngjajn5my5kh3s1+dnirzqouhhgvd9r3uhgosdcocttxopmmkyrd3adtsgroqw/hs2gk6suopxvlzegbn1n8ty5xopuoi25lgm9cl3lt8z2tnjr4kx0ykmagczbffgudgounrqmcujyaplmfemxsrigosmde2ineg4c++veycs+p11b87k9cu8jskcathcao+xeinbqeoarb4ggd4htdhoi/ou/mxby04+cwh/ihz+qnk5y3y</latexit> <latexit sha1_base64="kg8f2gqmefcl9gxch+hg6t9o+70=">aaab7hicbvbntwixej3fl1y/ui9egshee9nvxi8birepmlhaahvslv2odntn2zuswn/w4kgnv3+qn/+nbfag4esmexlvjjpzopqzbtzv2ymsrk6tbxq33a3tnd290v5bq8tmeroqyavqrvhtzgqnddoctljfcrjx2oygn1o/+uivzllcm1fkwwt3byszwczkjyr71d2vdetlr+rngjajn5my5kh3s1+dnirzqouhhgvd9r3uhgosdcocttxopmmkyrd3adtsgroqw/hs2gk6suopxvlzegbn1n8ty5xopuoi25lgm9cl3lt8z2tnjr4kx0ykmagczbffgudgounrqmcujyaplmfemxsrigosmde2ineg4c++veycs+p11b87k9cu8jskcathcao+xeinbqeoarb4ggd4htdhoi/ou/mxby04+cwh/ihz+qnk5y3y</latexit> <latexit sha1_base64="kg8f2gqmefcl9gxch+hg6t9o+70=">aaab7hicbvbntwixej3fl1y/ui9egshee9nvxi8birepmlhaahvslv2odntn2zuswn/w4kgnv3+qn/+nbfag4esmexlvjjpzopqzbtzv2ymsrk6tbxq33a3tnd290v5bq8tmeroqyavqrvhtzgqnddoctljfcrjx2oygn1o/+uivzllcm1fkwwt3byszwczkjyr71d2vdetlr+rngjajn5my5kh3s1+dnirzqouhhgvd9r3uhgosdcocttxopmmkyrd3adtsgroqw/hs2gk6suopxvlzegbn1n8ty5xopuoi25lgm9cl3lt8z2tnjr4kx0ykmagczbffgudgounrqmcujyaplmfemxsrigosmde2ineg4c++veycs+p11b87k9cu8jskcathcao+xeinbqeoarb4ggd4htdhoi/ou/mxby04+cwh/ihz+qnk5y3y</latexit> <latexit sha1_base64="kg8f2gqmefcl9gxch+hg6t9o+70=">aaab7hicbvbntwixej3fl1y/ui9egshee9nvxi8birepmlhaahvslv2odntn2zuswn/w4kgnv3+qn/+nbfag4esmexlvjjpzopqzbtzv2ymsrk6tbxq33a3tnd290v5bq8tmeroqyavqrvhtzgqnddoctljfcrjx2oygn1o/+uivzllcm1fkwwt3byszwczkjyr71d2vdetlr+rngjajn5my5kh3s1+dnirzqouhhgvd9r3uhgosdcocttxopmmkyrd3adtsgroqw/hs2gk6suopxvlzegbn1n8ty5xopuoi25lgm9cl3lt8z2tnjr4kx0ykmagczbffgudgounrqmcujyaplmfemxsrigosmde2ineg4c++veycs+p11b87k9cu8jskcathcao+xeinbqeoarb4ggd4htdhoi/ou/mxby04+cwh/ihz+qnk5y3y</latexit> Linear Classifiers In 3D we just have three features,,, and x 1 x<latexit sha1_base64="5lpcrwqeloshdqhvq1ycsz2btby=">aaab7xicbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg3w2zuzwshh/hxymar/4fb/4bf+hbwzdm8vletgbmrsln2njet1naw9/y3cpvuzu7e/shlcojb51kitcajdxrnqhrypmkgwgg006qkbyrp+1ofdpz249uazbieznjasjwulkyewys1k65t/2gw+txql7dmwoter8gvsjq6le+eooezijkqzjwuut7qqlzrawjne7dxqzpiskyd2nxuokf1we+p3ekzqwyqhgibemd5urvirwlrscisp0cm5fe9mbif143m/fvmdozzozkslguzxyzbm1+rwomkdf8ygkmitlberlhhymxcbk2bh/55vusnorxdf+uuw1efgmu4qro4rx8uiqm3eilaiawhmd4htcndv6cd+dj0vpyiplj+apn8weyry3r</latexit> 2 x 3 The decision rule w T x + b 0 gives the plane w T x + b =0 as the decision boundary X 3 W x, " 5

6 Linear Classifiers In higher dimensions it becomes harder to visualize We call the decision boundary the separating hyperplane Different types of linear classifiers determine their parameters in different ways o o o Naïve Bayes Model joint probabilities and use Bayes Rule Logistic Regression Model probability that an example belongs to a particular class The Perceptron Use examples to update parameters until we find something that works Support Vector Machines use a more geometric idea called a Margin 6

7 Support Vector Machines Advantages: o o o o Best off-the-shelf classifier Andrew Ng Nice theoretical bounds Allows for nonlinear classification Optimization problem for learning parameters is convex Disadvantages: o o No probabilistic interpretation Can be prone to overfitting in the nonlinear case 7

8 SVM Intuition Temporary Assumption: Assume the training data is linearly separable Consider the following training set in 2D 8

9 SVM Intuition Question: Which decision boundary seems better to you? 2 3. I 9

10 SVM Intuition Answer: This one feels better 10

11 SVM Intuition More space between DB and closest training examples 11

12 <latexit sha1_base64="ee3hmzwjjnf7xcwpctl0mhoqqty=">aaacanicbzdlssnafiyn9vbjlepon4otiagl6cllrudgzyxgftoyjtnjo3qyctmttysag1/fjqsvtz6fo9/gazufvn8y+pjpozw5f5awkpvtfxmlhcwl5zxyqrm2vrg5zw3vxms4fzi4ogax6arieky5crvvjhqsqvaumniorhetevuwcelj3lljhhgrgnaauoyutnxrr2pmvsced/lnawr3uu/hmqzmqm9v7jo9ffwltgevukjpw5+9foztihcfgzky69ij8jikfmwm5gyvlsrbeiqgpkuro4hil5vekmnd7frhgav9uijt9+dehiipx1ggoyokhnk+njh/q3vtfz55gevjqgjhs0vhyqck4sqq2kecymxgghawvp8v4iesccsdm6ldcozp/gtuvxzec67qlczjkuyz7imdcaqccaoa4bi0gqsweabp4aw8go/gs/fmvm9as0yxswt+yfj4bkmclx8=</latexit> <latexit sha1_base64="ee3hmzwjjnf7xcwpctl0mhoqqty=">aaacanicbzdlssnafiyn9vbjlepon4otiagl6cllrudgzyxgftoyjtnjo3qyctmttysag1/fjqsvtz6fo9/gazufvn8y+pjpozw5f5awkpvtfxmlhcwl5zxyqrm2vrg5zw3vxms4fzi4ogax6arieky5crvvjhqsqvaumniorhetevuwcelj3lljhhgrgnaauoyutnxrr2pmvsced/lnawr3uu/hmqzmqm9v7jo9ffwltgevukjpw5+9foztihcfgzky69ij8jikfmwm5gyvlsrbeiqgpkuro4hil5vekmnd7frhgav9uijt9+dehiipx1ggoyokhnk+njh/q3vtfz55gevjqgjhs0vhyqck4sqq2kecymxgghawvp8v4iesccsdm6ldcozp/gtuvxzec67qlczjkuyz7imdcaqccaoa4bi0gqsweabp4aw8go/gs/fmvm9as0yxswt+yfj4bkmclx8=</latexit> <latexit sha1_base64="ee3hmzwjjnf7xcwpctl0mhoqqty=">aaacanicbzdlssnafiyn9vbjlepon4otiagl6cllrudgzyxgftoyjtnjo3qyctmttysag1/fjqsvtz6fo9/gazufvn8y+pjpozw5f5awkpvtfxmlhcwl5zxyqrm2vrg5zw3vxms4fzi4ogax6arieky5crvvjhqsqvaumniorhetevuwcelj3lljhhgrgnaauoyutnxrr2pmvsced/lnawr3uu/hmqzmqm9v7jo9ffwltgevukjpw5+9foztihcfgzky69ij8jikfmwm5gyvlsrbeiqgpkuro4hil5vekmnd7frhgav9uijt9+dehiipx1ggoyokhnk+njh/q3vtfz55gevjqgjhs0vhyqck4sqq2kecymxgghawvp8v4iesccsdm6ldcozp/gtuvxzec67qlczjkuyz7imdcaqccaoa4bi0gqsweabp4aw8go/gs/fmvm9as0yxswt+yfj4bkmclx8=</latexit> <latexit sha1_base64="ee3hmzwjjnf7xcwpctl0mhoqqty=">aaacanicbzdlssnafiyn9vbjlepon4otiagl6cllrudgzyxgftoyjtnjo3qyctmttysag1/fjqsvtz6fo9/gazufvn8y+pjpozw5f5awkpvtfxmlhcwl5zxyqrm2vrg5zw3vxms4fzi4ogax6arieky5crvvjhqsqvaumniorhetevuwcelj3lljhhgrgnaauoyutnxrr2pmvsced/lnawr3uu/hmqzmqm9v7jo9ffwltgevukjpw5+9foztihcfgzky69ij8jikfmwm5gyvlsrbeiqgpkuro4hil5vekmnd7frhgav9uijt9+dehiipx1ggoyokhnk+njh/q3vtfz55gevjqgjhs0vhyqck4sqq2kecymxgghawvp8v4iesccsdm6ldcozp/gtuvxzec67qlczjkuyz7imdcaqccaoa4bi0gqsweabp4aw8go/gs/fmvm9as0yxswt+yfj4bkmclx8=</latexit> <latexit sha1_base64="3hznwolfi//lveukep6mgdowjjg=">aaab83icbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg7bbubomk4xd48adgq3/gm//glfsg4esmexlvjjpzgoqzpr3n2yqtrw9sbpw37z3dvf2dyuhrg4ptsahhyh7ltoav5uxqtzpnaserfecbp+1gfjp77qmvisxixk8t6kd4kfjicnzg8mt21gtc9dtrm7vwr1sdujmhwivuqapqonwvfpugmukjkjthwkmu6ytaz7dujha6s3upogkmyzykxumfjqjys/nrm3rmlaeky2lkadrxf09kofjqggwmm8j6pja9xpzp66y6vpizjpjuu0ewi8kuix2jpae0yjiszaegyckzurwrezayajotbujwl19ejv6jfl137xrv5kwrrhlo4btowyvlamittmadao/wdk/wzk2sf+vd+li0lqxi5hj+wpr8at0bkjo=</latexit> <latexit sha1_base64="3hznwolfi//lveukep6mgdowjjg=">aaab83icbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg7bbubomk4xd48adgq3/gm//glfsg4esmexlvjjpzgoqzpr3n2yqtrw9sbpw37z3dvf2dyuhrg4ptsahhyh7ltoav5uxqtzpnaserfecbp+1gfjp77qmvisxixk8t6kd4kfjicnzg8mt21gtc9dtrm7vwr1sdujmhwivuqapqonwvfpugmukjkjthwkmu6ytaz7dujha6s3upogkmyzykxumfjqjys/nrm3rmlaeky2lkadrxf09kofjqggwmm8j6pja9xpzp66y6vpizjpjuu0ewi8kuix2jpae0yjiszaegyckzurwrezayajotbujwl19ejv6jfl137xrv5kwrrhlo4btowyvlamittmadao/wdk/wzk2sf+vd+li0lqxi5hj+wpr8at0bkjo=</latexit> <latexit sha1_base64="3hznwolfi//lveukep6mgdowjjg=">aaab83icbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg7bbubomk4xd48adgq3/gm//glfsg4esmexlvjjpzgoqzpr3n2yqtrw9sbpw37z3dvf2dyuhrg4ptsahhyh7ltoav5uxqtzpnaserfecbp+1gfjp77qmvisxixk8t6kd4kfjicnzg8mt21gtc9dtrm7vwr1sdujmhwivuqapqonwvfpugmukjkjthwkmu6ytaz7dujha6s3upogkmyzykxumfjqjys/nrm3rmlaeky2lkadrxf09kofjqggwmm8j6pja9xpzp66y6vpizjpjuu0ewi8kuix2jpae0yjiszaegyckzurwrezayajotbujwl19ejv6jfl137xrv5kwrrhlo4btowyvlamittmadao/wdk/wzk2sf+vd+li0lqxi5hj+wpr8at0bkjo=</latexit> <latexit sha1_base64="3hznwolfi//lveukep6mgdowjjg=">aaab83icbvbnt8jaej3if9yv1koxjwdiibqc/lirepgiirusamh22ckg7bbubomk4xd48adgq3/gm//glfsg4esmexlvjjpzgoqzpr3n2yqtrw9sbpw37z3dvf2dyuhrg4ptsahhyh7ltoav5uxqtzpnaserfecbp+1gfjp77qmvisxixk8t6kd4kfjicnzg8mt21gtc9dtrm7vwr1sdujmhwivuqapqonwvfpugmukjkjthwkmu6ytaz7dujha6s3upogkmyzykxumfjqjys/nrm3rmlaeky2lkadrxf09kofjqggwmm8j6pja9xpzp66y6vpizjpjuu0ewi8kuix2jpae0yjiszaegyckzurwrezayajotbujwl19ejv6jfl137xrv5kwrrhlo4btowyvlamittmadao/wdk/wzk2sf+vd+li0lqxi5hj+wpr8at0bkjo=</latexit> SVM Intuition Think about the size of w T x i + b as a measure of confidence in classification of example x i = 12

13 <latexit sha1_base64="zdqmwzpientmhc2e4j1smnltydk=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkpkem6qvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f30syfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy9zwkckmrsswvjpqstldomdl1xpfteveocehjqcmncrmekiiotzgsyninwnczx/phfx7bkeahdyk8gr50ggcaet01ambdlaqxwsqhbkq1xeplgcy4dukkmunc9c0zkymmcukxi7nupiikci9rqpokixqsmciks3j4pjwh9gouxirh4f6eyfaoxct0vgei5ejm16bmf7v+kv3zquajjjukwrnffsqgjoe0fjikngdjjgoq5lt9feir4ghlfz6uqrdmt14eu9m4afjxzvrrteyjcg7aitggfjgdlxaf2saggdycz/ak3rqn7uv71z5mrrwtnnkhf6r9/gde4pdv</latexit> <latexit sha1_base64="zdqmwzpientmhc2e4j1smnltydk=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkpkem6qvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f30syfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy9zwkckmrsswvjpqstldomdl1xpfteveocehjqcmncrmekiiotzgsyninwnczx/phfx7bkeahdyk8gr50ggcaet01ambdlaqxwsqhbkq1xeplgcy4dukkmunc9c0zkymmcukxi7nupiikci9rqpokixqsmciks3j4pjwh9gouxirh4f6eyfaoxct0vgei5ejm16bmf7v+kv3zquajjjukwrnffsqgjoe0fjikngdjjgoq5lt9feir4ghlfz6uqrdmt14eu9m4afjxzvrrteyjcg7aitggfjgdlxaf2saggdycz/ak3rqn7uv71z5mrrwtnnkhf6r9/gde4pdv</latexit> <latexit sha1_base64="zdqmwzpientmhc2e4j1smnltydk=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkpkem6qvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f30syfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy9zwkckmrsswvjpqstldomdl1xpfteveocehjqcmncrmekiiotzgsyninwnczx/phfx7bkeahdyk8gr50ggcaet01ambdlaqxwsqhbkq1xeplgcy4dukkmunc9c0zkymmcukxi7nupiikci9rqpokixqsmciks3j4pjwh9gouxirh4f6eyfaoxct0vgei5ejm16bmf7v+kv3zquajjjukwrnffsqgjoe0fjikngdjjgoq5lt9feir4ghlfz6uqrdmt14eu9m4afjxzvrrteyjcg7aitggfjgdlxaf2saggdycz/ak3rqn7uv71z5mrrwtnnkhf6r9/gde4pdv</latexit> <latexit sha1_base64="zdqmwzpientmhc2e4j1smnltydk=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkpkem6qvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f30syfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy9zwkckmrsswvjpqstldomdl1xpfteveocehjqcmncrmekiiotzgsyninwnczx/phfx7bkeahdyk8gr50ggcaet01ambdlaqxwsqhbkq1xeplgcy4dukkmunc9c0zkymmcukxi7nupiikci9rqpokixqsmciks3j4pjwh9gouxirh4f6eyfaoxct0vgei5ejm16bmf7v+kv3zquajjjukwrnffsqgjoe0fjikngdjjgoq5lt9feir4ghlfz6uqrdmt14eu9m4afjxzvrrteyjcg7aitggfjgdlxaf2saggdycz/ak3rqn7uv71z5mrrwtnnkhf6r9/gde4pdv</latexit> Very confident y if w T i =1 x i + b 0 <latexit sha1_base64="szliz79nkodwecuwebl+0qltz8y=">aaab8xicbvbns8naej3urxq/qh69llacp5l04mdbkhjxwmhyqhrkzrtpl242yxcjhncf4cwdilf/jtf/jds2b219mpb4b4azewhkmdko821v1ty3nreq2/bo7t7+qe3w6felmstuiwlpzc/einimqkez5rsxsorjknnuolmd+d0nkhvlxipouxreecryxajwrvibdj5g6aa5dmnqqztnzw60stys1kfez1d76g8tksvuamkxur7rpdoosnsmcdq1+5miksytpkk+oqlhvaxf/oqpojpkeewjncu0mqu/jwock5xhoemmsr6rzw8m/uf5my6ugokjnnnukmwikonij2j2pxoysynmusgysgzurwsmjsbapgsbenzll1ej12pen937vr19uazrhrm4hxnw4rlacacd8ibaas/wcm+wtl6sd+tj0vqxyplj+apr8weqyy75</latexit> SVM Intuition 13

14 <latexit sha1_base64="jt2yhavzrqe2kymjlibgp6ikgim=">aaab8nicbvbns8naej3urxq/qh69llacf0vsq9wdupdisykxhtauzxbtlt1swu5gckf/w4shfa/+gm/+g7dtdtr6yodx3gwz84kem6ud59sqra1vbg6vt+2d3b39g8rh0aoku0mor2iey26afevmue8zzwk3krrhaaedyhi78ztpvcowiwedjdsp8eiwkbgsjdsv2dmaort04dq1qaxq1j050cpxc1kfau1b5as/jekauaejx0r1xcfrfo6lzottqd1pfu0wmear7rkqcesvn89vnqizowxrgettqqo5+nsix5fswrsyzgjrsvr2zuj/xi/v4zwfm5gkmgqywbsmhokyzqjaqyyp0twzbbpjzk2ijlherjuybbocu/zykvea9eu6e9+otppfgmu4gvm4bxcuoqv30aypcctwdk/wzqxwi/vufsxas1yxcwx/yh3+apw6jza=</latexit> <latexit sha1_base64="jt2yhavzrqe2kymjlibgp6ikgim=">aaab8nicbvbns8naej3urxq/qh69llacf0vsq9wdupdisykxhtauzxbtlt1swu5gckf/w4shfa/+gm/+g7dtdtr6yodx3gwz84kem6ud59sqra1vbg6vt+2d3b39g8rh0aoku0mor2iey26afevmue8zzwk3krrhaaedyhi78ztpvcowiwedjdsp8eiwkbgsjdsv2dmaort04dq1qaxq1j050cpxc1kfau1b5as/jekauaejx0r1xcfrfo6lzottqd1pfu0wmear7rkqcesvn89vnqizowxrgettqqo5+nsix5fswrsyzgjrsvr2zuj/xi/v4zwfm5gkmgqywbsmhokyzqjaqyyp0twzbbpjzk2ijlherjuybbocu/zykvea9eu6e9+otppfgmu4gvm4bxcuoqv30aypcctwdk/wzqxwi/vufsxas1yxcwx/yh3+apw6jza=</latexit> <latexit sha1_base64="jt2yhavzrqe2kymjlibgp6ikgim=">aaab8nicbvbns8naej3urxq/qh69llacf0vsq9wdupdisykxhtauzxbtlt1swu5gckf/w4shfa/+gm/+g7dtdtr6yodx3gwz84kem6ud59sqra1vbg6vt+2d3b39g8rh0aoku0mor2iey26afevmue8zzwk3krrhaaedyhi78ztpvcowiwedjdsp8eiwkbgsjdsv2dmaort04dq1qaxq1j050cpxc1kfau1b5as/jekauaejx0r1xcfrfo6lzottqd1pfu0wmear7rkqcesvn89vnqizowxrgettqqo5+nsix5fswrsyzgjrsvr2zuj/xi/v4zwfm5gkmgqywbsmhokyzqjaqyyp0twzbbpjzk2ijlherjuybbocu/zykvea9eu6e9+otppfgmu4gvm4bxcuoqv30aypcctwdk/wzqxwi/vufsxas1yxcwx/yh3+apw6jza=</latexit> <latexit sha1_base64="+ncglnlpq1rhvrhmyb7naacxhnc=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkjkem6rvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f308yfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy93wkckmrsswvjpqstldom9l1x5fteveocehjqcmncxfdnixoqdgsyvimwnczxw/gfx7bkeah9yg8gt50giomxvemmtkwc8ffseqogvjtz/hybjfoqxjjzjaqfctmpjshlilmjnedvjae4teakr7cciveuflxsq6pldoaqczviyqs3n8tgqqfmis+6gyrhin52tt8r9zpzxduzjrkukkipfsupazkge5jgqpkczzsogbhttvfir4hjrbu4ekqbgv+5ewwm42lhnxdrlvoyzsq4aacgmnggtpqalegdwyawsn4bq/gtxvsxrr37wpwwthkmx3wr9rnd9qtl18=</latexit> <latexit sha1_base64="+ncglnlpq1rhvrhmyb7naacxhnc=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkjkem6rvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f308yfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy93wkckmrsswvjpqstldom9l1x5fteveocehjqcmncxfdnixoqdgsyvimwnczxw/gfx7bkeah9yg8gt50giomxvemmtkwc8ffseqogvjtz/hybjfoqxjjzjaqfctmpjshlilmjnedvjae4teakr7cciveuflxsq6pldoaqczviyqs3n8tgqqfmis+6gyrhin52tt8r9zpzxduzjrkukkipfsupazkge5jgqpkczzsogbhttvfir4hjrbu4ekqbgv+5ewwm42lhnxdrlvoyzsq4aacgmnggtpqalegdwyawsn4bq/gtxvsxrr37wpwwthkmx3wr9rnd9qtl18=</latexit> <latexit sha1_base64="+ncglnlpq1rhvrhmyb7naacxhnc=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkjkem6rvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f308yfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy93wkckmrsswvjpqstldom9l1x5fteveocehjqcmncxfdnixoqdgsyvimwnczxw/gfx7bkeah9yg8gt50giomxvemmtkwc8ffseqogvjtz/hybjfoqxjjzjaqfctmpjshlilmjnedvjae4teakr7cciveuflxsq6pldoaqczviyqs3n8tgqqfmis+6gyrhin52tt8r9zpzxduzjrkukkipfsupazkge5jgqpkczzsogbhttvfir4hjrbu4ekqbgv+5ewwm42lhnxdrlvoyzsq4aacgmnggtpqalegdwyawsn4bq/gtxvsxrr37wpwwthkmx3wr9rnd9qtl18=</latexit> <latexit sha1_base64="+ncglnlpq1rhvrhmyb7naacxhnc=">aaacchicbzc7tsmwfiadcivhfmbkswirkjcqpaoxrrily5eawqkjkem6rvxnitsbqigrc6/cwgciludg421w0wzq8kuwpv3nhb2f308yfdi0v7xk0vlk6lp1xd/y3nremxb3bkscckxshloy93wkckmrsswvjpqstldom9l1x5fteveocehjqcmncxfdnixoqdgsyvimwnczxw/gfx7bkeah9yg8gt50giomxvemmtkwc8ffseqogvjtz/hybjfoqxjjzjaqfctmpjshlilmjnedvjae4teakr7cciveuflxsq6pldoaqczviyqs3n8tgqqfmis+6gyrhin52tt8r9zpzxduzjrkukkipfsupazkge5jgqpkczzsogbhttvfir4hjrbu4ekqbgv+5ewwm42lhnxdrlvoyzsq4aacgmnggtpqalegdwyawsn4bq/gtxvsxrr37wpwwthkmx3wr9rnd9qtl18=</latexit> SVM Intuition Very confident y i = 1 if w T x <latexit sha1_base64="jt2yhavzrqe2kymjlibgp6ikgim=">aaab8nicbvbns8naej3urxq/qh69llacf0vsq9wdupdisykxhtauzxbtlt1swu5gckf/w4shfa/+gm/+g7dtdtr6yodx3gwz84kem6ud59sqra1vbg6vt+2d3b39g8rh0aoku0mor2iey26afevmue8zzwk3krrhaaedyhi78ztpvcowiwedjdsp8eiwkbgsjdsv2dmaort04dq1qaxq1j050cpxc1kfau1b5as/jekauaejx0r1xcfrfo6lzottqd1pfu0wmear7rkqcesvn89vnqizowxrgettqqo5+nsix5fswrsyzgjrsvr2zuj/xi/v4zwfm5gkmgqywbsmhokyzqjaqyyp0twzbbpjzk2ijlherjuybbocu/zykvea9eu6e9+otppfgmu4gvm4bxcuoqv30aypcctwdk/wzqxwi/vufsxas1yxcwx/yh3+apw6jza=</latexit> i + b 0 14

15 SVM Intuition Want a DB that makes us very confident in the classification of each training example 15

16 SVM Intuition Do this by maximizing distance from DB to closest training examples 16

17 SVM Intuition Do this by maximizing distance from DB to closest training examples 17

18 SVM Intuition This distance from the DB to closest training examples is called the Margin M M M 18

19 SVM Intuition The points closest to the DB are called the support vectors We support vectors to nig support vector. 19

20 <latexit sha1_base64="n1hi3t6avtccm+j8q9yiz1vl41k=">aaacbhicbvc7tsnaedyhvzavayuujxikqshowaolremzjewijvf0vpytu85n624nrjybgn6fhgiqlr9bx99wersqmnjko5ld7e4eieaaxpfbkiwtr6yufdftjc2t7r1nd+9wx6mizkexifuzijojlpkpharrjoqrkbcseqwvx37jjinny3kdo4r1itkxposugjg6zmhzztpbio9z3ab2acrkmje9nopalnedkltxj8clxjurepqh3nw+2r2yphgtqaxruuw5cxqyoobtwxk7nwqwedokfdyyvjki6u42+slhx0bp4tbwpitgifp7iior1qmomj0rgyge98bif14rhfc8k3gzpmakns4ku4ehxunici8rrkgmdcfucxmrpgoicautng1c8ozfxir+txjr8a6rpdrpli0iokbh6ar56azv0bwqix9r9iie0st6s56sf+vd+pi2fqzzzd76a+vzbz+blqu=</latexit> <latexit sha1_base64="n1hi3t6avtccm+j8q9yiz1vl41k=">aaacbhicbvc7tsnaedyhvzavayuujxikqshowaolremzjewijvf0vpytu85n624nrjybgn6fhgiqlr9bx99wersqmnjko5ld7e4eieaaxpfbkiwtr6yufdftjc2t7r1nd+9wx6mizkexifuzijojlpkpharrjoqrkbcseqwvx37jjinny3kdo4r1itkxposugjg6zmhzztpbio9z3ab2acrkmje9nopalnedkltxj8clxjurepqh3nw+2r2yphgtqaxruuw5cxqyoobtwxk7nwqwedokfdyyvjki6u42+slhx0bp4tbwpitgifp7iior1qmomj0rgyge98bif14rhfc8k3gzpmakns4ku4ehxunici8rrkgmdcfucxmrpgoicautng1c8ozfxir+txjr8a6rpdrpli0iokbh6ar56azv0bwqix9r9iie0st6s56sf+vd+pi2fqzzzd76a+vzbz+blqu=</latexit> <latexit sha1_base64="n1hi3t6avtccm+j8q9yiz1vl41k=">aaacbhicbvc7tsnaedyhvzavayuujxikqshowaolremzjewijvf0vpytu85n624nrjybgn6fhgiqlr9bx99wersqmnjko5ld7e4eieaaxpfbkiwtr6yufdftjc2t7r1nd+9wx6mizkexifuzijojlpkpharrjoqrkbcseqwvx37jjinny3kdo4r1itkxposugjg6zmhzztpbio9z3ab2acrkmje9nopalnedkltxj8clxjurepqh3nw+2r2yphgtqaxruuw5cxqyoobtwxk7nwqwedokfdyyvjki6u42+slhx0bp4tbwpitgifp7iior1qmomj0rgyge98bif14rhfc8k3gzpmakns4ku4ehxunici8rrkgmdcfucxmrpgoicautng1c8ozfxir+txjr8a6rpdrpli0iokbh6ar56azv0bwqix9r9iie0st6s56sf+vd+pi2fqzzzd76a+vzbz+blqu=</latexit> <latexit sha1_base64="n1hi3t6avtccm+j8q9yiz1vl41k=">aaacbhicbvc7tsnaedyhvzavayuujxikqshowaolremzjewijvf0vpytu85n624nrjybgn6fhgiqlr9bx99wersqmnjko5ld7e4eieaaxpfbkiwtr6yufdftjc2t7r1nd+9wx6mizkexifuzijojlpkpharrjoqrkbcseqwvx37jjinny3kdo4r1itkxposugjg6zmhzztpbio9z3ab2acrkmje9nopalnedkltxj8clxjurepqh3nw+2r2yphgtqaxruuw5cxqyoobtwxk7nwqwedokfdyyvjki6u42+slhx0bp4tbwpitgifp7iior1qmomj0rgyge98bif14rhfc8k3gzpmakns4ku4ehxunici8rrkgmdcfucxmrpgoicautng1c8ozfxir+txjr8a6rpdrpli0iokbh6ar56azv0bwqix9r9iie0st6s56sf+vd+pi2fqzzzd76a+vzbz+blqu=</latexit> <latexit sha1_base64="gmv1gq3syo6vn4lfcnewxkhwoco=">aaacdhicbvfli9raeo7e1xpfo14eedqdhfgwhgqvpk4rxrwikzjuwmqykp3kblpdndbd0qkh/gj/ntf/hhevdmyj6k4fdv991fx6kq+vdjqkp4lwytvr12/s3ixu3b5z997k/onprmqswlmovgvpcncopme5svj4ulsenss8zs/edvhjz2idrmxhamtcalgbwuob5knv5ntelow4lqyda6hto6h6kcukqxw0jlqfmybnquuyvorf+v2vec+f8fc8y6kmcenwdy6mecd/+g+u4mrbgmnwhdega4woi8pafkralhq450fawndcu4ydo71onzkmcbi1fhmki5iy0y5wk+9zuylgo6ft1uwa1lt0bukkhb5847agcqzrxhhoqknbdlvvej7ztmhlyvpnig/zvzm60m612i8800cn7mjsip8xwzruvlx20tqnorhnjcrgi1lx4qs8kkmshqrhpz+vi1owimgfahahvbjyzta/if/f6yed6ehruy0d9pjtsucszs/yixvhjticcfyzebq8dxadx+gtcbrozr+gwzjzkp1jyfwbfjo/ag==</latexit> <latexit sha1_base64="gmv1gq3syo6vn4lfcnewxkhwoco=">aaacdhicbvfli9raeo7e1xpfo14eedqdhfgwhgqvpk4rxrwikzjuwmqykp3kblpdndbd0qkh/gj/ntf/hhevdmyj6k4fdv991fx6kq+vdjqkp4lwytvr12/s3ixu3b5z997k/onprmqswlmovgvpcncopme5svj4ulsenss8zs/edvhjz2idrmxhamtcalgbwuob5knv5ntelow4lqyda6hto6h6kcukqxw0jlqfmybnquuyvorf+v2vec+f8fc8y6kmcenwdy6mecd/+g+u4mrbgmnwhdega4woi8pafkralhq450fawndcu4ydo71onzkmcbi1fhmki5iy0y5wk+9zuylgo6ft1uwa1lt0bukkhb5847agcqzrxhhoqknbdlvvej7ztmhlyvpnig/zvzm60m612i8800cn7mjsip8xwzruvlx20tqnorhnjcrgi1lx4qs8kkmshqrhpz+vi1owimgfahahvbjyzta/if/f6yed6ehruy0d9pjtsucszs/yixvhjticcfyzebq8dxadx+gtcbrozr+gwzjzkp1jyfwbfjo/ag==</latexit> <latexit sha1_base64="gmv1gq3syo6vn4lfcnewxkhwoco=">aaacdhicbvfli9raeo7e1xpfo14eedqdhfgwhgqvpk4rxrwikzjuwmqykp3kblpdndbd0qkh/gj/ntf/hhevdmyj6k4fdv991fx6kq+vdjqkp4lwytvr12/s3ixu3b5z997k/onprmqswlmovgvpcncopme5svj4ulsenss8zs/edvhjz2idrmxhamtcalgbwuob5knv5ntelow4lqyda6hto6h6kcukqxw0jlqfmybnquuyvorf+v2vec+f8fc8y6kmcenwdy6mecd/+g+u4mrbgmnwhdega4woi8pafkralhq450fawndcu4ydo71onzkmcbi1fhmki5iy0y5wk+9zuylgo6ft1uwa1lt0bukkhb5847agcqzrxhhoqknbdlvvej7ztmhlyvpnig/zvzm60m612i8800cn7mjsip8xwzruvlx20tqnorhnjcrgi1lx4qs8kkmshqrhpz+vi1owimgfahahvbjyzta/if/f6yed6ehruy0d9pjtsucszs/yixvhjticcfyzebq8dxadx+gtcbrozr+gwzjzkp1jyfwbfjo/ag==</latexit> <latexit sha1_base64="gmv1gq3syo6vn4lfcnewxkhwoco=">aaacdhicbvfli9raeo7e1xpfo14eedqdhfgwhgqvpk4rxrwikzjuwmqykp3kblpdndbd0qkh/gj/ntf/hhevdmyj6k4fdv991fx6kq+vdjqkp4lwytvr12/s3ixu3b5z997k/onprmqswlmovgvpcncopme5svj4ulsenss8zs/edvhjz2idrmxhamtcalgbwuob5knv5ntelow4lqyda6hto6h6kcukqxw0jlqfmybnquuyvorf+v2vec+f8fc8y6kmcenwdy6mecd/+g+u4mrbgmnwhdega4woi8pafkralhq450fawndcu4ydo71onzkmcbi1fhmki5iy0y5wk+9zuylgo6ft1uwa1lt0bukkhb5847agcqzrxhhoqknbdlvvej7ztmhlyvpnig/zvzm60m612i8800cn7mjsip8xwzruvlx20tqnorhnjcrgi1lx4qs8kkmshqrhpz+vi1owimgfahahvbjyzta/if/f6yed6ehruy0d9pjtsucszs/yixvhjticcfyzebq8dxadx+gtcbrozr+gwzjzkp1jyfwbfjo/ag==</latexit> Maximum Margin Classifier How do we do this mathematically? Let the margin be represented by M Goal: Determine w and b that maximize M while classifying all training points correctly This gives the following (informal) optimization problem max w, b s.t. M All training examples correctly classified 20