tum.de/fall2018/ in2357

Transkrypt

1 tum.de/fall2018/ in2357

2 Prof. Daniel Cremers From to Classification

3 Categories of Learning (Rep.) Learning Unsupervised Learning clustering, density estimation Supervised Learning learning from a training data set, inference on the test data Reinforcement Learning no supervision, but a reward function Classification target set is continuous, e.g. Y = R target set is discrete, e.g. Y =[1,...,C] Computer Vision!3

4 Visualisation Linear Principle: Minimise loss function during training Use the found parameters for prediction!4

5 <latexit sha1_base64="d5mi7+ixmi6um1wcsmnlep5ahzo=">aaacqhiczvhbbtnaen2ywwmxpvdiy4ooukglsimkekzghzdkqsjnur2i9xqsrloxa3emhfn+cr6gv/gk/ozxakgbjrsrs2duo3oyqquacfynf925e+/+g72h/uepnzzdhxw8owuu9bkm0mnnzzmrqcslu1so4bzwieymyzzdfmz9s+/gg3l2c24kmbuxsmqppecifom3xsf+s0vanhqv82qlj+ghcj0t66r5j7f5trgm4+n4a/w2sdowzj1nfgc9mezolgyssi1cuejiaue18kikhqaflgekis/fci4iwmegzovtxa0fezpzpfn0lpitez2jfiaejckosv1j2pw15d/f6lotq7zi2ompy/fzwtmirldyqv2y1bwdb1fhc+vbot4qenirmodltfbcii34rqvwjfzil5pu6uw7fbclycbdtsprlap0trldw/mi01poqyf3p+zdfa5sqkgprj8ngnkvlhrvp6i6fehvrekgph7htz9ksxzfua3oxscj4c9vhycfooh22av2kh2yhl1jj+wtm7apk+wh+8l+sd/r62gszakvv6frr8t5zm5ylp0fjypueg==</latexit> <latexit sha1_base64="d5mi7+ixmi6um1wcsmnlep5ahzo=">aaacqhiczvhbbtnaen2ywwmxpvdiy4ooukglsimkekzghzdkqsjnur2i9xqsrloxa3emhfn+cr6gv/gk/ozxakgbjrsrs2duo3oyqquacfynf925e+/+g72h/uepnzzdhxw8owuu9bkm0mnnzzmrqcslu1so4bzwieymyzzdfmz9s+/gg3l2c24kmbuxsmqppecifom3xsf+s0vanhqv82qlj+ghcj0t66r5j7f5trgm4+n4a/w2sdowzj1nfgc9mezolgyssi1cuejiaue18kikhqaflgekis/fci4iwmegzovtxa0fezpzpfn0lpitez2jfiaejckosv1j2pw15d/f6lotq7zi2ompy/fzwtmirldyqv2y1bwdb1fhc+vbot4qenirmodltfbcii34rqvwjfzil5pu6uw7fbclycbdtsprlap0trldw/mi01poqyf3p+zdfa5sqkgprj8ngnkvlhrvp6i6fehvrekgph7htz9ksxzfua3oxscj4c9vhycfooh22av2kh2yhl1jj+wtm7apk+wh+8l+sd/r62gszakvv6frr8t5zm5ylp0fjypueg==</latexit> <latexit sha1_base64="d5mi7+ixmi6um1wcsmnlep5ahzo=">aaacqhiczvhbbtnaen2ywwmxpvdiy4ooukglsimkekzghzdkqsjnur2i9xqsrloxa3emhfn+cr6gv/gk/ozxakgbjrsrs2duo3oyqquacfynf925e+/+g72h/uepnzzdhxw8owuu9bkm0mnnzzmrqcslu1so4bzwieymyzzdfmz9s+/gg3l2c24kmbuxsmqppecifom3xsf+s0vanhqv82qlj+ghcj0t66r5j7f5trgm4+n4a/w2sdowzj1nfgc9mezolgyssi1cuejiaue18kikhqaflgekis/fci4iwmegzovtxa0fezpzpfn0lpitez2jfiaejckosv1j2pw15d/f6lotq7zi2ompy/fzwtmirldyqv2y1bwdb1fhc+vbot4qenirmodltfbcii34rqvwjfzil5pu6uw7fbclycbdtsprlap0trldw/mi01poqyf3p+zdfa5sqkgprj8ngnkvlhrvp6i6fehvrekgph7htz9ksxzfua3oxscj4c9vhycfooh22av2kh2yhl1jj+wtm7apk+wh+8l+sd/r62gszakvv6frr8t5zm5ylp0fjypueg==</latexit> <latexit sha1_base64="d5mi7+ixmi6um1wcsmnlep5ahzo=">aaacqhiczvhbbtnaen2ywwmxpvdiy4ooukglsimkekzghzdkqsjnur2i9xqsrloxa3emhfn+cr6gv/gk/ozxakgbjrsrs2duo3oyqquacfynf925e+/+g72h/uepnzzdhxw8owuu9bkm0mnnzzmrqcslu1so4bzwieymyzzdfmz9s+/gg3l2c24kmbuxsmqppecifom3xsf+s0vanhqv82qlj+ghcj0t66r5j7f5trgm4+n4a/w2sdowzj1nfgc9mezolgyssi1cuejiaue18kikhqaflgekis/fci4iwmegzovtxa0fezpzpfn0lpitez2jfiaejckosv1j2pw15d/f6lotq7zi2ompy/fzwtmirldyqv2y1bwdb1fhc+vbot4qenirmodltfbcii34rqvwjfzil5pu6uw7fbclycbdtsprlap0trldw/mi01poqyf3p+zdfa5sqkgprj8ngnkvlhrvp6i6fehvrekgph7htz9ksxzfua3oxscj4c9vhycfooh22av2kh2yhl1jj+wtm7apk+wh+8l+sd/r62gszakvv6frr8t5zm5ylp0fjypueg==</latexit> <latexit sha1_base64="y+un4abymsibp4d5znhp5y0fsuy=">aaacphiczvhbbtnaen2ywwm3fb55wreftvjv2resfazghqcqfuhasnwi1utxsupern1xswt5g/gaxue7+bvgqanadkrdnt1z25mtlvofjoo/veje/qcph2097j95+uz5i8h2y5pgki9hip12/iwtabsymegfgs5kd8jkgk6zi4+t//qsffdofsnvcvmj5lyvsgokajbylxdsi3crffwphqdg5fz/e9nsxwnsdmedybwfr43fbukhhqyz49l2t6a5k5ubi1klem6tumrpltwqqahpp1wausglmydzglyycnn6pvpdr8tkvhcejkw+zm9m1mkesdizrbz/dju+lrz2jw46cdlwdbv9stiy1sqwfykvv+2lsnn0vf0bz5uhixpfqeivaaiuf8ilibtcw5vajfi9x2m6ptpt9mfwjgmpoq2v547sf2ymdecjtmuhbqxe/zh3vbiyaywmgv00aepxwwpuh4nlkucvlirsaopx3prjlmrthlvgzlyfep7ybnj4ornoi71mb9gos9h7dsg+swm2yzl9zl/yb/yneht9jr5gk6vqqnflvgk3lpr+d0nb01m=</latexit> <latexit sha1_base64="y+un4abymsibp4d5znhp5y0fsuy=">aaacphiczvhbbtnaen2ywwm3fb55wreftvjv2resfazghqcqfuhasnwi1utxsupern1xswt5g/gaxue7+bvgqanadkrdnt1z25mtlvofjoo/veje/qcph2097j95+uz5i8h2y5pgki9hip12/iwtabsymegfgs5kd8jkgk6zi4+t//qsffdofsnvcvmj5lyvsgokajbylxdsi3crffwphqdg5fz/e9nsxwnsdmedybwfr43fbukhhqyz49l2t6a5k5ubi1klem6tumrpltwqqahpp1wausglmydzglyycnn6pvpdr8tkvhcejkw+zm9m1mkesdizrbz/dju+lrz2jw46cdlwdbv9stiy1sqwfykvv+2lsnn0vf0bz5uhixpfqeivaaiuf8ilibtcw5vajfi9x2m6ptpt9mfwjgmpoq2v547sf2ymdecjtmuhbqxe/zh3vbiyaywmgv00aepxwwpuh4nlkucvlirsaopx3prjlmrthlvgzlyfep7ybnj4ornoi71mb9gos9h7dsg+swm2yzl9zl/yb/yneht9jr5gk6vqqnflvgk3lpr+d0nb01m=</latexit> <latexit sha1_base64="y+un4abymsibp4d5znhp5y0fsuy=">aaacphiczvhbbtnaen2ywwm3fb55wreftvjv2resfazghqcqfuhasnwi1utxsupern1xswt5g/gaxue7+bvgqanadkrdnt1z25mtlvofjoo/veje/qcph2097j95+uz5i8h2y5pgki9hip12/iwtabsymegfgs5kd8jkgk6zi4+t//qsffdofsnvcvmj5lyvsgokajbylxdsi3crffwphqdg5fz/e9nsxwnsdmedybwfr43fbukhhqyz49l2t6a5k5ubi1klem6tumrpltwqqahpp1wausglmydzglyycnn6pvpdr8tkvhcejkw+zm9m1mkesdizrbz/dju+lrz2jw46cdlwdbv9stiy1sqwfykvv+2lsnn0vf0bz5uhixpfqeivaaiuf8ilibtcw5vajfi9x2m6ptpt9mfwjgmpoq2v547sf2ymdecjtmuhbqxe/zh3vbiyaywmgv00aepxwwpuh4nlkucvlirsaopx3prjlmrthlvgzlyfep7ybnj4ornoi71mb9gos9h7dsg+swm2yzl9zl/yb/yneht9jr5gk6vqqnflvgk3lpr+d0nb01m=</latexit> <latexit sha1_base64="y+un4abymsibp4d5znhp5y0fsuy=">aaacphiczvhbbtnaen2ywwm3fb55wreftvjv2resfazghqcqfuhasnwi1utxsupern1xswt5g/gaxue7+bvgqanadkrdnt1z25mtlvofjoo/veje/qcph2097j95+uz5i8h2y5pgki9hip12/iwtabsymegfgs5kd8jkgk6zi4+t//qsffdofsnvcvmj5lyvsgokajbylxdsi3crffwphqdg5fz/e9nsxwnsdmedybwfr43fbukhhqyz49l2t6a5k5ubi1klem6tumrpltwqqahpp1wausglmydzglyycnn6pvpdr8tkvhcejkw+zm9m1mkesdizrbz/dju+lrz2jw46cdlwdbv9stiy1sqwfykvv+2lsnn0vf0bz5uhixpfqeivaaiuf8ilibtcw5vajfi9x2m6ptpt9mfwjgmpoq2v547sf2ymdecjtmuhbqxe/zh3vbiyaywmgv00aepxwwpuh4nlkucvlirsaopx3prjlmrthlvgzlyfep7ybnj4ornoi71mb9gos9h7dsg+swm2yzl9zl/yb/yneht9jr5gk6vqqnflvgk3lpr+d0nb01m=</latexit> Visualisation Linear Principle: Minimise loss function during training Use the found parameters for prediction Bayesian Linear Principle: Compute parameter posterior p(w x, t) from training data During inference, compute the predictive distribution p(t x, x, t) Advantages: Less tendency of overfitting Probabilistic interpretation, uncertainty estimation!5

6 Visualisation Linear Logistic Bayesian Linear probabilistic reasoning from regression to classification!6

7 <latexit sha1_base64="fasvtz9uyaydbxh/4tnroxwumik=">aaacm3iczvhbahsxejw3t9s9xgkfs0hugbzaml1tsf8koxkppyeu6isqnwzwo7zfdfmk2wcz7gu/pq/tv/rvqnwwkjgdeoczzrnnovmhpkc4/tuj7t1/8pdrzupuk6fpnu/29l6cels6grnhlxxngxhu0ucejck8lxyczhsezzdhtf3scp2x1vygdyftdqsj51iahdssx1mvfxqgsm8/8xtuqfrjxsvvu1wvw/ewx896/xgub4lfbukl+qynk9ler6s5favgq0kb9xdjxnc0akdskky7aemxahejc7wi0ibgp602q9r8edi5n1sxjig+yd7sqeb7v9zzekmbln671it/1wy3i7rqgp3wfjp/nfbsfcwhedfj56xizhmjfs+lq0fqhqaij8mgxcwhqerb01tmjqh5o1eqcaurm+29kxwgdvowvfry0l7uy6w5h/agsxdi8/bhvgxh0ngcfti6qucstjrhuhumq2mgj1fbt19x47jublusbrpugtpxkilhyfcp/cpprue77bv7w4ysyqfskh1hj2zcbpvjfrhf7e/0ojqkvkbfrp9gnbbnjbsv0eqf97dn+g==</latexit> <latexit sha1_base64="fasvtz9uyaydbxh/4tnroxwumik=">aaacm3iczvhbahsxejw3t9s9xgkfs0hugbzaml1tsf8koxkppyeu6isqnwzwo7zfdfmk2wcz7gu/pq/tv/rvqnwwkjgdeoczzrnnovmhpkc4/tuj7t1/8pdrzupuk6fpnu/29l6cels6grnhlxxngxhu0ucejck8lxyczhsezzdhtf3scp2x1vygdyftdqsj51iahdssx1mvfxqgsm8/8xtuqfrjxsvvu1wvw/ewx896/xgub4lfbukl+qynk9ler6s5favgq0kb9xdjxnc0akdskky7aemxahejc7wi0ibgp602q9r8edi5n1sxjig+yd7sqeb7v9zzekmbln671it/1wy3i7rqgp3wfjp/nfbsfcwhedfj56xizhmjfs+lq0fqhqaij8mgxcwhqerb01tmjqh5o1eqcaurm+29kxwgdvowvfry0l7uy6w5h/agsxdi8/bhvgxh0ngcfti6qucstjrhuhumq2mgj1fbt19x47jublusbrpugtpxkilhyfcp/cpprue77bv7w4ysyqfskh1hj2zcbpvjfrhf7e/0ojqkvkbfrp9gnbbnjbsv0eqf97dn+g==</latexit> <latexit sha1_base64="fasvtz9uyaydbxh/4tnroxwumik=">aaacm3iczvhbahsxejw3t9s9xgkfs0hugbzaml1tsf8koxkppyeu6isqnwzwo7zfdfmk2wcz7gu/pq/tv/rvqnwwkjgdeoczzrnnovmhpkc4/tuj7t1/8pdrzupuk6fpnu/29l6cels6grnhlxxngxhu0ucejck8lxyczhsezzdhtf3scp2x1vygdyftdqsj51iahdssx1mvfxqgsm8/8xtuqfrjxsvvu1wvw/ewx896/xgub4lfbukl+qynk9ler6s5favgq0kb9xdjxnc0akdskky7aemxahejc7wi0ibgp602q9r8edi5n1sxjig+yd7sqeb7v9zzekmbln671it/1wy3i7rqgp3wfjp/nfbsfcwhedfj56xizhmjfs+lq0fqhqaij8mgxcwhqerb01tmjqh5o1eqcaurm+29kxwgdvowvfry0l7uy6w5h/agsxdi8/bhvgxh0ngcfti6qucstjrhuhumq2mgj1fbt19x47jublusbrpugtpxkilhyfcp/cpprue77bv7w4ysyqfskh1hj2zcbpvjfrhf7e/0ojqkvkbfrp9gnbbnjbsv0eqf97dn+g==</latexit> <latexit sha1_base64="fasvtz9uyaydbxh/4tnroxwumik=">aaacm3iczvhbahsxejw3t9s9xgkfs0hugbzaml1tsf8koxkppyeu6isqnwzwo7zfdfmk2wcz7gu/pq/tv/rvqnwwkjgdeoczzrnnovmhpkc4/tuj7t1/8pdrzupuk6fpnu/29l6cels6grnhlxxngxhu0ucejck8lxyczhsezzdhtf3scp2x1vygdyftdqsj51iahdssx1mvfxqgsm8/8xtuqfrjxsvvu1wvw/ewx896/xgub4lfbukl+qynk9ler6s5favgq0kb9xdjxnc0akdskky7aemxahejc7wi0ibgp602q9r8edi5n1sxjig+yd7sqeb7v9zzekmbln671it/1wy3i7rqgp3wfjp/nfbsfcwhedfj56xizhmjfs+lq0fqhqaij8mgxcwhqerb01tmjqh5o1eqcaurm+29kxwgdvowvfry0l7uy6w5h/agsxdi8/bhvgxh0ngcfti6qucstjrhuhumq2mgj1fbt19x47jublusbrpugtpxkilhyfcp/cpprue77bv7w4ysyqfskh1hj2zcbpvjfrhf7e/0ojqkvkbfrp9gnbbnjbsv0eqf97dn+g==</latexit> Logistic To convert the regression problem into a classification problem, we use a sigmoid function σ(), e.g.: (a) = exp( a) This can be interpreted as a classification probability!7

8 <latexit sha1_base64="mwcjp0wao1t17m3bymctiqqwdyy=">aaacyhiczvhbattaef2rt9s9oe1jx5yaqwimscbqvgrc+1iklinessbyxwq1spfsrdkdnrzcl/2lfk1e2x/pybehcqakpczzmdkzjy2u9bcg173g0emnt59tpe+/epnq9zvb9tstb0vhxyxbzd1zyrxq0ogzsfdirhcc6vsj0/tis8uf/hlos2uoosrexlofkbnkddcvdkzviukbjb1calytawblnk+vmp/hne6tynyl8ajjyimbw3rvjhj3l8axlyxlqdyixnfmwy+/j4nhubeugz4euqegpiujzlvhszsxwhjginl/houfzgvmqhilmn5celewfsew4hyhyvr4eb3w3darzjkaw4efabro3q2omfatalzzvtxvcm3ylhvdjwhvdvqb8yh/nk+lkuoqht+mz0tfwdj2ttsttnbqfqlgnuqflc+zyxzqghudwreysssv/rnvrxpvsp0kjziurxywy5d6ihpkrxtxggvythsx7bpqatwwht36srfabwlwud1lqykdj1dovm/qsdj00zzo04sh4gsyfyh+st88/nwztexekw9kh0tkizkkx8krmrfo/pbr8pf8c74frxavvddxg15x847ci+d3f0wj4xg=</latexit> <latexit sha1_base64="mwcjp0wao1t17m3bymctiqqwdyy=">aaacyhiczvhbattaef2rt9s9oe1jx5yaqwimscbqvgrc+1iklinessbyxwq1spfsrdkdnrzcl/2lfk1e2x/pybehcqakpczzmdkzjy2u9bcg173g0emnt59tpe+/epnq9zvb9tstb0vhxyxbzd1zyrxq0ogzsfdirhcc6vsj0/tis8uf/hlos2uoosrexlofkbnkddcvdkzviukbjb1calytawblnk+vmp/hne6tynyl8ajjyimbw3rvjhj3l8axlyxlqdyixnfmwy+/j4nhubeugz4euqegpiujzlvhszsxwhjginl/houfzgvmqhilmn5celewfsew4hyhyvr4eb3w3darzjkaw4efabro3q2omfatalzzvtxvcm3ylhvdjwhvdvqb8yh/nk+lkuoqht+mz0tfwdj2ttsttnbqfqlgnuqflc+zyxzqghudwreysssv/rnvrxpvsp0kjziurxywy5d6ihpkrxtxggvythsx7bpqatwwht36srfabwlwud1lqykdj1dovm/qsdj00zzo04sh4gsyfyh+st88/nwztexekw9kh0tkizkkx8krmrfo/pbr8pf8c74frxavvddxg15x847ci+d3f0wj4xg=</latexit> <latexit sha1_base64="mwcjp0wao1t17m3bymctiqqwdyy=">aaacyhiczvhbattaef2rt9s9oe1jx5yaqwimscbqvgrc+1iklinessbyxwq1spfsrdkdnrzcl/2lfk1e2x/pybehcqakpczzmdkzjy2u9bcg173g0emnt59tpe+/epnq9zvb9tstb0vhxyxbzd1zyrxq0ogzsfdirhcc6vsj0/tis8uf/hlos2uoosrexlofkbnkddcvdkzviukbjb1calytawblnk+vmp/hne6tynyl8ajjyimbw3rvjhj3l8axlyxlqdyixnfmwy+/j4nhubeugz4euqegpiujzlvhszsxwhjginl/houfzgvmqhilmn5celewfsew4hyhyvr4eb3w3darzjkaw4efabro3q2omfatalzzvtxvcm3ylhvdjwhvdvqb8yh/nk+lkuoqht+mz0tfwdj2ttsttnbqfqlgnuqflc+zyxzqghudwreysssv/rnvrxpvsp0kjziurxywy5d6ihpkrxtxggvythsx7bpqatwwht36srfabwlwud1lqykdj1dovm/qsdj00zzo04sh4gsyfyh+st88/nwztexekw9kh0tkizkkx8krmrfo/pbr8pf8c74frxavvddxg15x847ci+d3f0wj4xg=</latexit> <latexit sha1_base64="mwcjp0wao1t17m3bymctiqqwdyy=">aaacyhiczvhbattaef2rt9s9oe1jx5yaqwimscbqvgrc+1iklinessbyxwq1spfsrdkdnrzcl/2lfk1e2x/pybehcqakpczzmdkzjy2u9bcg173g0emnt59tpe+/epnq9zvb9tstb0vhxyxbzd1zyrxq0ogzsfdirhcc6vsj0/tis8uf/hlos2uoosrexlofkbnkddcvdkzviukbjb1calytawblnk+vmp/hne6tynyl8ajjyimbw3rvjhj3l8axlyxlqdyixnfmwy+/j4nhubeugz4euqegpiujzlvhszsxwhjginl/houfzgvmqhilmn5celewfsew4hyhyvr4eb3w3darzjkaw4efabro3q2omfatalzzvtxvcm3ylhvdjwhvdvqb8yh/nk+lkuoqht+mz0tfwdj2ttsttnbqfqlgnuqflc+zyxzqghudwreysssv/rnvrxpvsp0kjziurxywy5d6ihpkrxtxggvythsx7bpqatwwht36srfabwlwud1lqykdj1dovm/qsdj00zzo04sh4gsyfyh+st88/nwztexekw9kh0tkizkkx8krmrfo/pbr8pf8c74frxavvddxg15x847ci+d3f0wj4xg=</latexit> <latexit sha1_base64="3auk+0ftdbq8bgrhhjsq0kfqonq=">aaacrniczvfba9swffa8w5fd0u1xl2ihkeipdjryxgzle9lloymmldrekoxjwfqxix1vdsy/zl9mr9tp6l+pnjqttq9i+ntu53xpkyxdmlwaba8epnr8zofp8nnzfy9fjxzfl5yplic5n9ly85q5keldhavkoc8tmjvkoesvv3t2sx9gntd6fncljiqttmgfz+hvy9fhpq33y8wwsppmz7thp9h/v++nne6nznxa+aejy0k003pvorqhb+fg6h0q9wbmejlz7g54nblekddijxpuigpltbpmuxaj7tcuhjsmx7ivxhiomqkxnjvpwjrxmozmxvqjkw60tymaplzxoffsonfbtk75zza5bcs6y+i26mp+mwmelisezw/k55wkagi3qjojcxzl2gpgrfatuf4wyzj6nd/j1hgs7dtk+psb1u3vdkvssjd54exk+pbczacldel9wvycho07pn0wkrrdjn2oyewauam0l9qcs/qyorw1p9i1zsxsh56wajue+2axo4g8/vz+fps5j2ihvcxvyjre5am5il/jczkttn6r3+qp+ruewsjiguwnazdoy96qoxiu16jr1zq=</latexit> <latexit sha1_base64="3auk+0ftdbq8bgrhhjsq0kfqonq=">aaacrniczvfba9swffa8w5fd0u1xl2ihkeipdjryxgzle9lloymmldrekoxjwfqxix1vdsy/zl9mr9tp6l+pnjqttq9i+ntu53xpkyxdmlwaba8epnr8zofp8nnzfy9fjxzfl5yplic5n9ly85q5keldhavkoc8tmjvkoesvv3t2sx9gntd6fncljiqttmgfz+hvy9fhpq33y8wwsppmz7thp9h/v++nne6nznxa+aejy0k003pvorqhb+fg6h0q9wbmejlz7g54nblekddijxpuigpltbpmuxaj7tcuhjsmx7ivxhiomqkxnjvpwjrxmozmxvqjkw60tymaplzxoffsonfbtk75zza5bcs6y+i26mp+mwmelisezw/k55wkagi3qjojcxzl2gpgrfatuf4wyzj6nd/j1hgs7dtk+psb1u3vdkvssjd54exk+pbczacldel9wvycho07pn0wkrrdjn2oyewauam0l9qcs/qyorw1p9i1zsxsh56wajue+2axo4g8/vz+fps5j2ihvcxvyjre5am5il/jczkttn6r3+qp+ruewsjiguwnazdoy96qoxiu16jr1zq=</latexit> <latexit sha1_base64="3auk+0ftdbq8bgrhhjsq0kfqonq=">aaacrniczvfba9swffa8w5fd0u1xl2ihkeipdjryxgzle9lloymmldrekoxjwfqxix1vdsy/zl9mr9tp6l+pnjqttq9i+ntu53xpkyxdmlwaba8epnr8zofp8nnzfy9fjxzfl5yplic5n9ly85q5keldhavkoc8tmjvkoesvv3t2sx9gntd6fncljiqttmgfz+hvy9fhpq33y8wwsppmz7thp9h/v++nne6nznxa+aejy0k003pvorqhb+fg6h0q9wbmejlz7g54nblekddijxpuigpltbpmuxaj7tcuhjsmx7ivxhiomqkxnjvpwjrxmozmxvqjkw60tymaplzxoffsonfbtk75zza5bcs6y+i26mp+mwmelisezw/k55wkagi3qjojcxzl2gpgrfatuf4wyzj6nd/j1hgs7dtk+psb1u3vdkvssjd54exk+pbczacldel9wvycho07pn0wkrrdjn2oyewauam0l9qcs/qyorw1p9i1zsxsh56wajue+2axo4g8/vz+fps5j2ihvcxvyjre5am5il/jczkttn6r3+qp+ruewsjiguwnazdoy96qoxiu16jr1zq=</latexit> <latexit sha1_base64="3auk+0ftdbq8bgrhhjsq0kfqonq=">aaacrniczvfba9swffa8w5fd0u1xl2ihkeipdjryxgzle9lloymmldrekoxjwfqxix1vdsy/zl9mr9tp6l+pnjqttq9i+ntu53xpkyxdmlwaba8epnr8zofp8nnzfy9fjxzfl5yplic5n9ly85q5keldhavkoc8tmjvkoesvv3t2sx9gntd6fncljiqttmgfz+hvy9fhpq33y8wwsppmz7thp9h/v++nne6nznxa+aejy0k003pvorqhb+fg6h0q9wbmejlz7g54nblekddijxpuigpltbpmuxaj7tcuhjsmx7ivxhiomqkxnjvpwjrxmozmxvqjkw60tymaplzxoffsonfbtk75zza5bcs6y+i26mp+mwmelisezw/k55wkagi3qjojcxzl2gpgrfatuf4wyzj6nd/j1hgs7dtk+psb1u3vdkvssjd54exk+pbczacldel9wvycho07pn0wkrrdjn2oyewauam0l9qcs/qyorw1p9i1zsxsh56wajue+2axo4g8/vz+fps5j2ihvcxvyjre5am5il/jczkttn6r3+qp+ruewsjiguwnazdoy96qoxiu16jr1zq=</latexit> Logistic To convert the regression problem into a classification problem, we use a sigmoid function σ() We still use our linear prediction model f(x, w) =w T (x) but now we use the sigmoid function to model a foreground class probability y i = (w T (x i )) i =1,...,N Thus, we consider a two-class problem (binary classification).!8

9 <latexit sha1_base64="ajw1t1ccv3j9oye5zypdkuciifq=">aaacs3iczvfna9taef2rx6n75bthxpyagw1pkexlcyme9tjliiu6cviowk1w9pl9eluj1mbrx+mv6bw999905iisoananubtzoybl1dkeojjv73o3v0hdx/tpe4/efrs+yvb/sszb2vhxyxbzd1fzrxq0ogzsfdionkc6vyj8/zqc8uffxfos2u+waysc82wrpasm8bunjiqxpbjmmpz0humd1lnyjwx4uczor/pjpoxafmgjpo3+do5dagwmwmywta+jldb74kka0psxwm23+npyxmthqgumpfzjk5gezgdyzvo+mntrcx4fvukouldtpclsjxy0bfmclpah58bus3erahme7/rod5sffhdrk3+50y3svi3hf3ofcipfkgaqgzh+px4slyulg23savpbae1qcc4k6ia8hvzjapu+lan1pjiwnukt251q96bwufciqlfq6xf8pweiobsecw14ii87v5muy5ugg9myy9+6gvwwxscfe7y+osbk2v02tdhgjd9tcxzneeuojsejoi/vhsef+om2iovyrsyjgn5qi7jf3jkzostn+qx+u3+ro+jezrhxfxvqnfvvck3itl/al3w124=</latexit> <latexit sha1_base64="ajw1t1ccv3j9oye5zypdkuciifq=">aaacs3iczvfna9taef2rx6n75bthxpyagw1pkexlcyme9tjliiu6cviowk1w9pl9eluj1mbrx+mv6bw999905iisoananubtzoybl1dkeojjv73o3v0hdx/tpe4/efrs+yvb/sszb2vhxyxbzd1fzrxq0ogzsfdionkc6vyj8/zqc8uffxfos2u+waysc82wrpasm8bunjiqxpbjmmpz0humd1lnyjwx4uczor/pjpoxafmgjpo3+do5dagwmwmywta+jldb74kka0psxwm23+npyxmthqgumpfzjk5gezgdyzvo+mntrcx4fvukouldtpclsjxy0bfmclpah58bus3erahme7/rod5sffhdrk3+50y3svi3hf3ofcipfkgaqgzh+px4slyulg23savpbae1qcc4k6ia8hvzjapu+lan1pjiwnukt251q96bwufciqlfq6xf8pweiobsecw14ii87v5muy5ugg9myy9+6gvwwxscfe7y+osbk2v02tdhgjd9tcxzneeuojsejoi/vhsef+om2iovyrsyjgn5qi7jf3jkzostn+qx+u3+ro+jezrhxfxvqnfvvck3itl/al3w124=</latexit> <latexit sha1_base64="ajw1t1ccv3j9oye5zypdkuciifq=">aaacs3iczvfna9taef2rx6n75bthxpyagw1pkexlcyme9tjliiu6cviowk1w9pl9eluj1mbrx+mv6bw999905iisoananubtzoybl1dkeojjv73o3v0hdx/tpe4/efrs+yvb/sszb2vhxyxbzd1fzrxq0ogzsfdionkc6vyj8/zqc8uffxfos2u+waysc82wrpasm8bunjiqxpbjmmpz0humd1lnyjwx4uczor/pjpoxafmgjpo3+do5dagwmwmywta+jldb74kka0psxwm23+npyxmthqgumpfzjk5gezgdyzvo+mntrcx4fvukouldtpclsjxy0bfmclpah58bus3erahme7/rod5sffhdrk3+50y3svi3hf3ofcipfkgaqgzh+px4slyulg23savpbae1qcc4k6ia8hvzjapu+lan1pjiwnukt251q96bwufciqlfq6xf8pweiobsecw14ii87v5muy5ugg9myy9+6gvwwxscfe7y+osbk2v02tdhgjd9tcxzneeuojsejoi/vhsef+om2iovyrsyjgn5qi7jf3jkzostn+qx+u3+ro+jezrhxfxvqnfvvck3itl/al3w124=</latexit> <latexit sha1_base64="ajw1t1ccv3j9oye5zypdkuciifq=">aaacs3iczvfna9taef2rx6n75bthxpyagw1pkexlcyme9tjliiu6cviowk1w9pl9eluj1mbrx+mv6bw999905iisoananubtzoybl1dkeojjv73o3v0hdx/tpe4/efrs+yvb/sszb2vhxyxbzd1fzrxq0ogzsfdionkc6vyj8/zqc8uffxfos2u+waysc82wrpasm8bunjiqxpbjmmpz0humd1lnyjwx4uczor/pjpoxafmgjpo3+do5dagwmwmywta+jldb74kka0psxwm23+npyxmthqgumpfzjk5gezgdyzvo+mntrcx4fvukouldtpclsjxy0bfmclpah58bus3erahme7/rod5sffhdrk3+50y3svi3hf3ofcipfkgaqgzh+px4slyulg23savpbae1qcc4k6ia8hvzjapu+lan1pjiwnukt251q96bwufciqlfq6xf8pweiobsecw14ii87v5muy5ugg9myy9+6gvwwxscfe7y+osbk2v02tdhgjd9tcxzneeuojsejoi/vhsef+om2iovyrsyjgn5qi7jf3jkzostn+qx+u3+ro+jezrhxfxvqnfvvck3itl/al3w124=</latexit> <latexit sha1_base64="jov2hse3yko07lmwbnupurvduri=">aaaczniczvjba9swffa8w5fd0u1xl2ihkeepdihsl4xsveyljwnlw6gzi8tyiqqlky5bbyp2ur+0n9knvnb/ynlijjq9iphp3m/5lbacwwjd607w6pgtp8+2nndfvhz1+k1v++2j1awhbek10oysjzyjrtgeoah2vhhgzcryaxrxpbgfxjjjuvy/yfgwqsqzxxnocxhv0vtedgnjyj7mntgcs57hu3fldu7glfui93fcgj0lnd+p3m9jxawh4auikuhrrkmvh+6gs8epqdscpmplngx3ajxpwkqmgapi7xkufjctiqfobxpdulssipsczni5h4pizqf1cnqhb16t4vwbfxtgpxy9oibs2ovmvwfto920ncr/tsg6eaomo92od/nnac1vuqjtdfu+lwugjzsf44wbrkespcducd8bpnnicavpw71mdwfzjimfv6mwzfrwltjlhmv+edhtpnwur8xhpmb+lx5bfrcd4zyl5socet5hn7ymqc6vl1qfkeqigogv/wlw1apqdt0t0syjd8hjadfy+nte/+cwjwglvucf0bbf6bm6qf/rge0qrx/qdbpff4nxcbm44nfknei0me/qpql+/wnugupb</latexit> <latexit sha1_base64="jov2hse3yko07lmwbnupurvduri=">aaaczniczvjba9swffa8w5fd0u1xl2ihkeepdihsl4xsveyljwnlw6gzi8tyiqqlky5bbyp2ur+0n9knvnb/ynlijjq9iphp3m/5lbacwwjd607w6pgtp8+2nndfvhz1+k1v++2j1awhbek10oysjzyjrtgeoah2vhhgzcryaxrxpbgfxjjjuvy/yfgwqsqzxxnocxhv0vtedgnjyj7mntgcs57hu3fldu7glfui93fcgj0lnd+p3m9jxawh4auikuhrrkmvh+6gs8epqdscpmplngx3ajxpwkqmgapi7xkufjctiqfobxpdulssipsczni5h4pizqf1cnqhb16t4vwbfxtgpxy9oibs2ovmvwfto920ncr/tsg6eaomo92od/nnac1vuqjtdfu+lwugjzsf44wbrkespcducd8bpnnicavpw71mdwfzjimfv6mwzfrwltjlhmv+edhtpnwur8xhpmb+lx5bfrcd4zyl5socet5hn7ymqc6vl1qfkeqigogv/wlw1apqdt0t0syjd8hjadfy+nte/+cwjwglvucf0bbf6bm6qf/rge0qrx/qdbpff4nxcbm44nfknei0me/qpql+/wnugupb</latexit> <latexit sha1_base64="jov2hse3yko07lmwbnupurvduri=">aaaczniczvjba9swffa8w5fd0u1xl2ihkeepdihsl4xsveyljwnlw6gzi8tyiqqlky5bbyp2ur+0n9knvnb/ynlijjq9iphp3m/5lbacwwjd607w6pgtp8+2nndfvhz1+k1v++2j1awhbek10oysjzyjrtgeoah2vhhgzcryaxrxpbgfxjjjuvy/yfgwqsqzxxnocxhv0vtedgnjyj7mntgcs57hu3fldu7glfui93fcgj0lnd+p3m9jxawh4auikuhrrkmvh+6gs8epqdscpmplngx3ajxpwkqmgapi7xkufjctiqfobxpdulssipsczni5h4pizqf1cnqhb16t4vwbfxtgpxy9oibs2ovmvwfto920ncr/tsg6eaomo92od/nnac1vuqjtdfu+lwugjzsf44wbrkespcducd8bpnnicavpw71mdwfzjimfv6mwzfrwltjlhmv+edhtpnwur8xhpmb+lx5bfrcd4zyl5socet5hn7ymqc6vl1qfkeqigogv/wlw1apqdt0t0syjd8hjadfy+nte/+cwjwglvucf0bbf6bm6qf/rge0qrx/qdbpff4nxcbm44nfknei0me/qpql+/wnugupb</latexit> <latexit sha1_base64="jov2hse3yko07lmwbnupurvduri=">aaaczniczvjba9swffa8w5fd0u1xl2ihkeepdihsl4xsveyljwnlw6gzi8tyiqqlky5bbyp2ur+0n9knvnb/ynlijjq9iphp3m/5lbacwwjd607w6pgtp8+2nndfvhz1+k1v++2j1awhbek10oysjzyjrtgeoah2vhhgzcryaxrxpbgfxjjjuvy/yfgwqsqzxxnocxhv0vtedgnjyj7mntgcs57hu3fldu7glfui93fcgj0lnd+p3m9jxawh4auikuhrrkmvh+6gs8epqdscpmplngx3ajxpwkqmgapi7xkufjctiqfobxpdulssipsczni5h4pizqf1cnqhb16t4vwbfxtgpxy9oibs2ovmvwfto920ncr/tsg6eaomo92od/nnac1vuqjtdfu+lwugjzsf44wbrkespcducd8bpnnicavpw71mdwfzjimfv6mwzfrwltjlhmv+edhtpnwur8xhpmb+lx5bfrcd4zyl5socet5hn7ymqc6vl1qfkeqigogv/wlw1apqdt0t0syjd8hjadfy+nte/+cwjwglvucf0bbf6bm6qf/rge0qrx/qdbpff4nxcbm44nfknei0me/qpql+/wnugupb</latexit> <latexit sha1_base64="76zxil2sjvynseuxnwohhromc9s=">aaacuxiczvhbattaef2rt9s9oe1jx5yaqwohycbqqqge9quvgrtqjbazm1qt7g32inzhryzq3/rr+tq+5g8ycpq2cqz2ottnljtz0kkrghf80yvu3x/w8nhw4/6tp8+evxhsvzworvrctoxtzp+mekrwvk5rozanhzdgui1p0vpplx/yq/qgnp2gq0lodcysypuajnd88debv+cjgwpofy7tvp7z8gln+oenksrj1++q2f0f95bpb8n4l14bvwvghriyzo7m2z2rze6urlougki4g8cfzmrwqiswtt8pgyxanmncnhg0ygsy1etbgz4it8zz5+ly5gvvzywatagrk1jk+8mwybxof9zojolvwzfs9mf8/axwtihrwnhvpi81r8fbxfjmeslqrwia8iom4gijhgtsxm9vauxjdn2p6rbotnmhw5puepnr8hrhkh1pjrlhfmrplbsgwlsf864kvzygakrrt4je4upljepdqa4bvapi7dduk7jpkyzjtrhugupj3pjw1/3hwadooc32mr1ho2zm3red9oudsskt7bf7zf6wv9ghckjl9p0qnop1oa/ylyvcjd+w3dk=</latexit> <latexit sha1_base64="76zxil2sjvynseuxnwohhromc9s=">aaacuxiczvhbattaef2rt9s9oe1jx5yaqwohycbqqqge9quvgrtqjbazm1qt7g32inzhryzq3/rr+tq+5g8ycpq2cqz2ottnljtz0kkrghf80yvu3x/w8nhw4/6tp8+evxhsvzworvrctoxtzp+mekrwvk5rozanhzdgui1p0vpplx/yq/qgnp2gq0lodcysypuajnd88debv+cjgwpofy7tvp7z8gln+oenksrj1++q2f0f95bpb8n4l14bvwvghriyzo7m2z2rze6urlougki4g8cfzmrwqiswtt8pgyxanmncnhg0ygsy1etbgz4it8zz5+ly5gvvzywatagrk1jk+8mwybxof9zojolvwzfs9mf8/axwtihrwnhvpi81r8fbxfjmeslqrwia8iom4gijhgtsxm9vauxjdn2p6rbotnmhw5puepnr8hrhkh1pjrlhfmrplbsgwlsf864kvzygakrrt4je4upljepdqa4bvapi7dduk7jpkyzjtrhugupj3pjw1/3hwadooc32mr1ho2zm3red9oudsskt7bf7zf6wv9ghckjl9p0qnop1oa/ylyvcjd+w3dk=</latexit> <latexit sha1_base64="76zxil2sjvynseuxnwohhromc9s=">aaacuxiczvhbattaef2rt9s9oe1jx5yaqwohycbqqqge9quvgrtqjbazm1qt7g32inzhryzq3/rr+tq+5g8ycpq2cqz2ottnljtz0kkrghf80yvu3x/w8nhw4/6tp8+evxhsvzworvrctoxtzp+mekrwvk5rozanhzdgui1p0vpplx/yq/qgnp2gq0lodcysypuajnd88debv+cjgwpofy7tvp7z8gln+oenksrj1++q2f0f95bpb8n4l14bvwvghriyzo7m2z2rze6urlougki4g8cfzmrwqiswtt8pgyxanmncnhg0ygsy1etbgz4it8zz5+ly5gvvzywatagrk1jk+8mwybxof9zojolvwzfs9mf8/axwtihrwnhvpi81r8fbxfjmeslqrwia8iom4gijhgtsxm9vauxjdn2p6rbotnmhw5puepnr8hrhkh1pjrlhfmrplbsgwlsf864kvzygakrrt4je4upljepdqa4bvapi7dduk7jpkyzjtrhugupj3pjw1/3hwadooc32mr1ho2zm3red9oudsskt7bf7zf6wv9ghckjl9p0qnop1oa/ylyvcjd+w3dk=</latexit> <latexit sha1_base64="76zxil2sjvynseuxnwohhromc9s=">aaacuxiczvhbattaef2rt9s9oe1jx5yaqwohycbqqqge9quvgrtqjbazm1qt7g32inzhryzq3/rr+tq+5g8ycpq2cqz2ottnljtz0kkrghf80yvu3x/w8nhw4/6tp8+evxhsvzworvrctoxtzp+mekrwvk5rozanhzdgui1p0vpplx/yq/qgnp2gq0lodcysypuajnd88debv+cjgwpofy7tvp7z8gln+oenksrj1++q2f0f95bpb8n4l14bvwvghriyzo7m2z2rze6urlougki4g8cfzmrwqiswtt8pgyxanmncnhg0ygsy1etbgz4it8zz5+ly5gvvzywatagrk1jk+8mwybxof9zojolvwzfs9mf8/axwtihrwnhvpi81r8fbxfjmeslqrwia8iom4gijhgtsxm9vauxjdn2p6rbotnmhw5puepnr8hrhkh1pjrlhfmrplbsgwlsf864kvzygakrrt4je4upljepdqa4bvapi7dduk7jpkyzjtrhugupj3pjw1/3hwadooc32mr1ho2zm3red9oudsskt7bf7zf6wv9ghckjl9p0qnop1oa/ylyvcjd+w3dk=</latexit> <latexit sha1_base64="plawgixtbafu5ove+cmu9iwgici=">aaaciniczvfla9taef4rfaru2jjtszelxtbdcjiptkwx0obqsycfoglexqxwy3vjpstubler+iw9nj8q/6yjr5tegdjlyx7fzhxtvfoftnpbxrlz5omz57sv+i/3xr3ehxy8oq8uegkt6btzl4uiojwfcsrucfl5ekbqcffcf2/jf7/bb+xsl1xxmdviydvcsyhkmg32caz4rizp6/qwy5vzyjgeprvjj0hwgshr7gx20jn56wq0yffqecjvlly4ryvhjtu0/twgqis8fgu4imifgtctn5m3feseks+dp2erb7z3k2phqlibgjknwgxyjrxo/7hr/scuwsaw1r/nn6e1slvespku/txqjo634vbsezco1wse9io24hipvjbiej5gavuud33u9etn2u2djayadyutrxeoypdmda3ni06ykecbdxpzjourg1bo4ujnavc6aklrfspwpwk9wth5qlop06zpz8m2j/ayni+pmsi/pw6pv3uh2mxv2hv2gwxseztmp9gzmzdjivvd/rkbzc8zj1+sr3epsa+recsewhlydxvvx6m=</latexit> <latexit sha1_base64="plawgixtbafu5ove+cmu9iwgici=">aaaciniczvfla9taef4rfaru2jjtszelxtbdcjiptkwx0obqsycfoglexqxwy3vjpstubler+iw9nj8q/6yjr5tegdjlyx7fzhxtvfoftnpbxrlz5omz57sv+i/3xr3ehxy8oq8uegkt6btzl4uiojwfcsrucfl5ekbqcffcf2/jf7/bb+xsl1xxmdviydvcsyhkmg32caz4rizp6/qwy5vzyjgeprvjj0hwgshr7gx20jn56wq0yffqecjvlly4ryvhjtu0/twgqis8fgu4imifgtctn5m3feseks+dp2erb7z3k2phqlibgjknwgxyjrxo/7hr/scuwsaw1r/nn6e1slvespku/txqjo634vbsezco1wse9io24hipvjbiej5gavuud33u9etn2u2djayadyutrxeoypdmda3ni06ykecbdxpzjourg1bo4ujnavc6aklrfspwpwk9wth5qlop06zpz8m2j/ayni+pmsi/pw6pv3uh2mxv2hv2gwxseztmp9gzmzdjivvd/rkbzc8zj1+sr3epsa+recsewhlydxvvx6m=</latexit> <latexit sha1_base64="plawgixtbafu5ove+cmu9iwgici=">aaaciniczvfla9taef4rfaru2jjtszelxtbdcjiptkwx0obqsycfoglexqxwy3vjpstubler+iw9nj8q/6yjr5tegdjlyx7fzhxtvfoftnpbxrlz5omz57sv+i/3xr3ehxy8oq8uegkt6btzl4uiojwfcsrucfl5ekbqcffcf2/jf7/bb+xsl1xxmdviydvcsyhkmg32caz4rizp6/qwy5vzyjgeprvjj0hwgshr7gx20jn56wq0yffqecjvlly4ryvhjtu0/twgqis8fgu4imifgtctn5m3feseks+dp2erb7z3k2phqlibgjknwgxyjrxo/7hr/scuwsaw1r/nn6e1slvespku/txqjo634vbsezco1wse9io24hipvjbiej5gavuud33u9etn2u2djayadyutrxeoypdmda3ni06ykecbdxpzjourg1bo4ujnavc6aklrfspwpwk9wth5qlop06zpz8m2j/ayni+pmsi/pw6pv3uh2mxv2hv2gwxseztmp9gzmzdjivvd/rkbzc8zj1+sr3epsa+recsewhlydxvvx6m=</latexit> <latexit sha1_base64="plawgixtbafu5ove+cmu9iwgici=">aaaciniczvfla9taef4rfaru2jjtszelxtbdcjiptkwx0obqsycfoglexqxwy3vjpstubler+iw9nj8q/6yjr5tegdjlyx7fzhxtvfoftnpbxrlz5omz57sv+i/3xr3ehxy8oq8uegkt6btzl4uiojwfcsrucfl5ekbqcffcf2/jf7/bb+xsl1xxmdviydvcsyhkmg32caz4rizp6/qwy5vzyjgeprvjj0hwgshr7gx20jn56wq0yffqecjvlly4ryvhjtu0/twgqis8fgu4imifgtctn5m3feseks+dp2erb7z3k2phqlibgjknwgxyjrxo/7hr/scuwsaw1r/nn6e1slvespku/txqjo634vbsezco1wse9io24hipvjbiej5gavuud33u9etn2u2djayadyutrxeoypdmda3ni06ykecbdxpzjourg1bo4ujnavc6aklrfspwpwk9wth5qlop06zpz8m2j/ayni+pmsi/pw6pv3uh2mxv2hv2gwxseztmp9gzmzdjivvd/rkbzc8zj1+sr3epsa+recsewhlydxvvx6m=</latexit> Logistic Again, we formulate a model of the likelihood of the training data: p(t x, w) = NY i=1 p(t i x i, w) t i 2 {0, 1} But now, we use the Bernoulli distribution: p(t i x i, w) =y t i i (1 y i) (1 t i) And again, we aim to maximise the (log)-likelihood arg max w p(t x, w)!9

10 Logistic We minimise the negative log-likelihood: E(w) = log p(t 1,...,t N x, w)!10

11 <latexit sha1_base64="1kp1wnnwb0qgtjlaidnkwutcdrq=">aaacr3iczvhbahsxejw3t9s9oeljx0snwazn2dwb9iuq2pe+pkrqx4gss9vqzvtel0uabztlf6rf09f2d/o3nxwwkjgdeoczmjoaoxmppic4/tujhjx89pjjztpus+cvxr7q7e6derscfxnulxxnofncssmmiegj89ijpnmlpvnv5yy/vrbos2u+w7oum80wrs4lz4bu1js8ovupdzqr5ffsx36lq8gktzvd0dwcd3sy7cmz2lcikrynsl4/pog3qe+dpav90szpttvhawf50miav8z7iyquyvyxb5iruxft4exj+bvbiauehmnhz9vmvjookcno3do8buigvv1rme39wuf4ujny+u1cq/7pdw4nyduo+q3+mp84q6qpawjdb9rpg6jgabnbwkgnokg1asadxakoxzlhooce7yg1phtvxvb4c6ub6b0johdofdi8wlgsx+qxqckdufwllsjt9se0vahsegaknbqpf8btmnioomgrewzortbjx1fjuo6ilcm2cffb2fggqfztsh/8qtvoh7whb8mqjoqdoszfycmzee5+kl/kn/ktjde0uox+3dynom3na3inivkp91pudg==</latexit> <latexit sha1_base64="1kp1wnnwb0qgtjlaidnkwutcdrq=">aaacr3iczvhbahsxejw3t9s9oeljx0snwazn2dwb9iuq2pe+pkrqx4gss9vqzvtel0uabztlf6rf09f2d/o3nxwwkjgdeoczmjoaoxmppic4/tujhjx89pjjztpus+cvxr7q7e6derscfxnulxxnofncssmmiegj89ijpnmlpvnv5yy/vrbos2u+w7oum80wrs4lz4bu1js8ovupdzqr5ffsx36lq8gktzvd0dwcd3sy7cmz2lcikrynsl4/pog3qe+dpav90szpttvhawf50miav8z7iyquyvyxb5iruxft4exj+bvbiauehmnhz9vmvjookcno3do8buigvv1rme39wuf4ujny+u1cq/7pdw4nyduo+q3+mp84q6qpawjdb9rpg6jgabnbwkgnokg1asadxakoxzlhooce7yg1phtvxvb4c6ub6b0johdofdi8wlgsx+qxqckdufwllsjt9se0vahsegaknbqpf8btmnioomgrewzortbjx1fjuo6ilcm2cffb2fggqfztsh/8qtvoh7whb8mqjoqdoszfycmzee5+kl/kn/ktjde0uox+3dynom3na3inivkp91pudg==</latexit> <latexit sha1_base64="1kp1wnnwb0qgtjlaidnkwutcdrq=">aaacr3iczvhbahsxejw3t9s9oeljx0snwazn2dwb9iuq2pe+pkrqx4gss9vqzvtel0uabztlf6rf09f2d/o3nxwwkjgdeoczmjoaoxmppic4/tujhjx89pjjztpus+cvxr7q7e6derscfxnulxxnofncssmmiegj89ijpnmlpvnv5yy/vrbos2u+w7oum80wrs4lz4bu1js8ovupdzqr5ffsx36lq8gktzvd0dwcd3sy7cmz2lcikrynsl4/pog3qe+dpav90szpttvhawf50miav8z7iyquyvyxb5iruxft4exj+bvbiauehmnhz9vmvjookcno3do8buigvv1rme39wuf4ujny+u1cq/7pdw4nyduo+q3+mp84q6qpawjdb9rpg6jgabnbwkgnokg1asadxakoxzlhooce7yg1phtvxvb4c6ub6b0johdofdi8wlgsx+qxqckdufwllsjt9se0vahsegaknbqpf8btmnioomgrewzortbjx1fjuo6ilcm2cffb2fggqfztsh/8qtvoh7whb8mqjoqdoszfycmzee5+kl/kn/ktjde0uox+3dynom3na3inivkp91pudg==</latexit> <latexit sha1_base64="1kp1wnnwb0qgtjlaidnkwutcdrq=">aaacr3iczvhbahsxejw3t9s9oeljx0snwazn2dwb9iuq2pe+pkrqx4gss9vqzvtel0uabztlf6rf09f2d/o3nxwwkjgdeoczmjoaoxmppic4/tujhjx89pjjztpus+cvxr7q7e6derscfxnulxxnofncssmmiegj89ijpnmlpvnv5yy/vrbos2u+w7oum80wrs4lz4bu1js8ovupdzqr5ffsx36lq8gktzvd0dwcd3sy7cmz2lcikrynsl4/pog3qe+dpav90szpttvhawf50miav8z7iyquyvyxb5iruxft4exj+bvbiauehmnhz9vmvjookcno3do8buigvv1rme39wuf4ujny+u1cq/7pdw4nyduo+q3+mp84q6qpawjdb9rpg6jgabnbwkgnokg1asadxakoxzlhooce7yg1phtvxvb4c6ub6b0johdofdi8wlgsx+qxqckdufwllsjt9se0vahsegaknbqpf8btmnioomgrewzortbjx1fjuo6ilcm2cffb2fggqfztsh/8qtvoh7whb8mqjoqdoszfycmzee5+kl/kn/ktjde0uox+3dynom3na3inivkp91pudg==</latexit> Logistic We minimise the negative log-likelihood: E(w) = log p(t 1,...,t N x, w) NX = i=1 (t i log y i +(1 t i ) log(1 y i )) Cross entropy!11

12 <latexit sha1_base64="+b9lcm3futa7enkmabxgszwx9ty=">aaacyniczvfbaxnbfj6stxovtfxrl4mhkikw3sdos6eogg9wk5i20i3l7oxsmnquy8ysjgz75l/yx4iv+j88my6ltq/mzmf55ly/vjlc+tj+3ytu3b5z997w/f6dh48ebw92nhw7u1vgp8xiy09z6rgumk+98jkfvpztlut+kp+/a/mt79w6yfrxv6r4tng5fqvg1kmrg3xonc0lhffjvfg/ymvwo9mffuhdrbig9ppm2ycyrzibl8fnyhfs3mjcrrq+ia0workmxdbiz4nhvbevdw6cpand0tlrttnjawfyrbj2tflnzpk48rnarrdm8qaf1o5xlj3tot9dqknibhbwkzcwqk8bpbf4tie192peomq1zelptku3ybxos250lftlnqpbqo/ln7mgdfv7rtlf+bkw4a20y4vcwm68xcggzaqcanicwso8snatu6tx8clwem9mvdu907xkueufdi/nbsmxasibgbhgwnbbdrqoocscqjtpjebop457zmqnhcihxr5sb8us5xdnmmrnh2vjnkw4cy4newnil6+gb287gbbim/kcjelcxpmd8oeckslh5bf5q/6sf9hhyearkfx8jxpdzfnyzakf/wfwaegg</latexit> <latexit sha1_base64="+b9lcm3futa7enkmabxgszwx9ty=">aaacyniczvfbaxnbfj6stxovtfxrl4mhkikw3sdos6eogg9wk5i20i3l7oxsmnquy8ysjgz75l/yx4iv+j88my6ltq/mzmf55ly/vjlc+tj+3ytu3b5z997w/f6dh48ebw92nhw7u1vgp8xiy09z6rgumk+98jkfvpztlut+kp+/a/mt79w6yfrxv6r4tng5fqvg1kmrg3xonc0lhffjvfg/ymvwo9mffuhdrbig9ppm2ycyrzibl8fnyhfs3mjcrrq+ia0workmxdbiz4nhvbevdw6cpand0tlrttnjawfyrbj2tflnzpk48rnarrdm8qaf1o5xlj3tot9dqknibhbwkzcwqk8bpbf4tie192peomq1zelptku3ybxos250lftlnqpbqo/ln7mgdfv7rtlf+bkw4a20y4vcwm68xcggzaqcanicwso8snatu6tx8clwem9mvdu907xkueufdi/nbsmxasibgbhgwnbbdrqoocscqjtpjebop457zmqnhcihxr5sb8us5xdnmmrnh2vjnkw4cy4newnil6+gb287gbbim/kcjelcxpmd8oeckslh5bf5q/6sf9hhyearkfx8jxpdzfnyzakf/wfwaegg</latexit> <latexit sha1_base64="+b9lcm3futa7enkmabxgszwx9ty=">aaacyniczvfbaxnbfj6stxovtfxrl4mhkikw3sdos6eogg9wk5i20i3l7oxsmnquy8ysjgz75l/yx4iv+j88my6ltq/mzmf55ly/vjlc+tj+3ytu3b5z997w/f6dh48ebw92nhw7u1vgp8xiy09z6rgumk+98jkfvpztlut+kp+/a/mt79w6yfrxv6r4tng5fqvg1kmrg3xonc0lhffjvfg/ymvwo9mffuhdrbig9ppm2ycyrzibl8fnyhfs3mjcrrq+ia0workmxdbiz4nhvbevdw6cpand0tlrttnjawfyrbj2tflnzpk48rnarrdm8qaf1o5xlj3tot9dqknibhbwkzcwqk8bpbf4tie192peomq1zelptku3ybxos250lftlnqpbqo/ln7mgdfv7rtlf+bkw4a20y4vcwm68xcggzaqcanicwso8snatu6tx8clwem9mvdu907xkueufdi/nbsmxasibgbhgwnbbdrqoocscqjtpjebop457zmqnhcihxr5sb8us5xdnmmrnh2vjnkw4cy4newnil6+gb287gbbim/kcjelcxpmd8oeckslh5bf5q/6sf9hhyearkfx8jxpdzfnyzakf/wfwaegg</latexit> <latexit sha1_base64="+b9lcm3futa7enkmabxgszwx9ty=">aaacyniczvfbaxnbfj6stxovtfxrl4mhkikw3sdos6eogg9wk5i20i3l7oxsmnquy8ysjgz75l/yx4iv+j88my6ltq/mzmf55ly/vjlc+tj+3ytu3b5z997w/f6dh48ebw92nhw7u1vgp8xiy09z6rgumk+98jkfvpztlut+kp+/a/mt79w6yfrxv6r4tng5fqvg1kmrg3xonc0lhffjvfg/ymvwo9mffuhdrbig9ppm2ycyrzibl8fnyhfs3mjcrrq+ia0workmxdbiz4nhvbevdw6cpand0tlrttnjawfyrbj2tflnzpk48rnarrdm8qaf1o5xlj3tot9dqknibhbwkzcwqk8bpbf4tie192peomq1zelptku3ybxos250lftlnqpbqo/ln7mgdfv7rtlf+bkw4a20y4vcwm68xcggzaqcanicwso8snatu6tx8clwem9mvdu907xkueufdi/nbsmxasibgbhgwnbbdrqoocscqjtpjebop457zmqnhcihxr5sb8us5xdnmmrnh2vjnkw4cy4newnil6+gb287gbbim/kcjelcxpmd8oeckslh5bf5q/6sf9hhyearkfx8jxpdzfnyzakf/wfwaegg</latexit> <latexit sha1_base64="1kp1wnnwb0qgtjlaidnkwutcdrq=">aaacr3iczvhbahsxejw3t9s9oeljx0snwazn2dwb9iuq2pe+pkrqx4gss9vqzvtel0uabztlf6rf09f2d/o3nxwwkjgdeoczmjoaoxmppic4/tujhjx89pjjztpus+cvxr7q7e6derscfxnulxxnofncssmmiegj89ijpnmlpvnv5yy/vrbos2u+w7oum80wrs4lz4bu1js8ovupdzqr5ffsx36lq8gktzvd0dwcd3sy7cmz2lcikrynsl4/pog3qe+dpav90szpttvhawf50miav8z7iyquyvyxb5iruxft4exj+bvbiauehmnhz9vmvjookcno3do8buigvv1rme39wuf4ujny+u1cq/7pdw4nyduo+q3+mp84q6qpawjdb9rpg6jgabnbwkgnokg1asadxakoxzlhooce7yg1phtvxvb4c6ub6b0johdofdi8wlgsx+qxqckdufwllsjt9se0vahsegaknbqpf8btmnioomgrewzortbjx1fjuo6ilcm2cffb2fggqfztsh/8qtvoh7whb8mqjoqdoszfycmzee5+kl/kn/ktjde0uox+3dynom3na3inivkp91pudg==</latexit> <latexit sha1_base64="1kp1wnnwb0qgtjlaidnkwutcdrq=">aaacr3iczvhbahsxejw3t9s9oeljx0snwazn2dwb9iuq2pe+pkrqx4gss9vqzvtel0uabztlf6rf09f2d/o3nxwwkjgdeoczmjoaoxmppic4/tujhjx89pjjztpus+cvxr7q7e6derscfxnulxxnofncssmmiegj89ijpnmlpvnv5yy/vrbos2u+w7oum80wrs4lz4bu1js8ovupdzqr5ffsx36lq8gktzvd0dwcd3sy7cmz2lcikrynsl4/pog3qe+dpav90szpttvhawf50miav8z7iyquyvyxb5iruxft4exj+bvbiauehmnhz9vmvjookcno3do8buigvv1rme39wuf4ujny+u1cq/7pdw4nyduo+q3+mp84q6qpawjdb9rpg6jgabnbwkgnokg1asadxakoxzlhooce7yg1phtvxvb4c6ub6b0johdofdi8wlgsx+qxqckdufwllsjt9se0vahsegaknbqpf8btmnioomgrewzortbjx1fjuo6ilcm2cffb2fggqfztsh/8qtvoh7whb8mqjoqdoszfycmzee5+kl/kn/ktjde0uox+3dynom3na3inivkp91pudg==</latexit> <latexit sha1_base64="1kp1wnnwb0qgtjlaidnkwutcdrq=">aaacr3iczvhbahsxejw3t9s9oeljx0snwazn2dwb9iuq2pe+pkrqx4gss9vqzvtel0uabztlf6rf09f2d/o3nxwwkjgdeoczmjoaoxmppic4/tujhjx89pjjztpus+cvxr7q7e6derscfxnulxxnofncssmmiegj89ijpnmlpvnv5yy/vrbos2u+w7oum80wrs4lz4bu1js8ovupdzqr5ffsx36lq8gktzvd0dwcd3sy7cmz2lcikrynsl4/pog3qe+dpav90szpttvhawf50miav8z7iyquyvyxb5iruxft4exj+bvbiauehmnhz9vmvjookcno3do8buigvv1rme39wuf4ujny+u1cq/7pdw4nyduo+q3+mp84q6qpawjdb9rpg6jgabnbwkgnokg1asadxakoxzlhooce7yg1phtvxvb4c6ub6b0johdofdi8wlgsx+qxqckdufwllsjt9se0vahsegaknbqpf8btmnioomgrewzortbjx1fjuo6ilcm2cffb2fggqfztsh/8qtvoh7whb8mqjoqdoszfycmzee5+kl/kn/ktjde0uox+3dynom3na3inivkp91pudg==</latexit> <latexit sha1_base64="1kp1wnnwb0qgtjlaidnkwutcdrq=">aaacr3iczvhbahsxejw3t9s9oeljx0snwazn2dwb9iuq2pe+pkrqx4gss9vqzvtel0uabztlf6rf09f2d/o3nxwwkjgdeoczmjoaoxmppic4/tujhjx89pjjztpus+cvxr7q7e6derscfxnulxxnofncssmmiegj89ijpnmlpvnv5yy/vrbos2u+w7oum80wrs4lz4bu1js8ovupdzqr5ffsx36lq8gktzvd0dwcd3sy7cmz2lcikrynsl4/pog3qe+dpav90szpttvhawf50miav8z7iyquyvyxb5iruxft4exj+bvbiauehmnhz9vmvjookcno3do8buigvv1rme39wuf4ujny+u1cq/7pdw4nyduo+q3+mp84q6qpawjdb9rpg6jgabnbwkgnokg1asadxakoxzlhooce7yg1phtvxvb4c6ub6b0johdofdi8wlgsx+qxqckdufwllsjt9se0vahsegaknbqpf8btmnioomgrewzortbjx1fjuo6ilcm2cffb2fggqfztsh/8qtvoh7whb8mqjoqdoszfycmzee5+kl/kn/ktjde0uox+3dynom3na3inivkp91pudg==</latexit> Logistic We minimise the negative log-likelihood: E(w) = log p(t 1,...,t N x, w) NX = i=1 NX (t i log y i +(1 t i ) log(1 y i )) Cross entropy re(w) = (y i t i ) (x i ) i=1!12

13 <latexit sha1_base64="qrhjo1lbk3bahxtzpnw9a2wzh8a=">aaacj3iczvfbaxnbfj6stxovtdq3xwzdoiku3sdujwmkoa+fcqytdem4o3s2gtqxzeasji77x3zvf+s/ctzdpe0pzpbxlt855ztzqasnop7bi+7cvxf/wd7d/qpht57ud4bptr2tnmczsmq68ww8kmlwrpiunpcoqwckz7llj2387ds6l635rpss5xqwrhzsaaxxyvainzap4j8oug20yor6r/n6mrjfh/hw+g2qdgdeojtzdhsiza2onboscry/soks5ju4kkjh008rjywis1jiryagnpp5vr2/4epgyxlhxxig+nz7vaig7f1gzygzndhvxlrn/9j4epdwlapf6u/fu3kttvkrgnhvvqguj8tbhxguhqpsmwbaobk24gifdgqfhw8wtalnb1ylwi+sbrf3ptizoszd8mppq/lkt7dhfmydleegz7ujecfcpfeeknd0u48kbgvco/oy1sdatq7ddx1tt+kmh86s7b7hnjidhcybf307mn7odrthxrjx7ial7ihn2wd2wmzmsj/sf/vn/ktd6ch6h02vuqnev/oc3bdoyz/vjmo2</latexit> <latexit sha1_base64="qrhjo1lbk3bahxtzpnw9a2wzh8a=">aaacj3iczvfbaxnbfj6stxovtdq3xwzdoiku3sdujwmkoa+fcqytdem4o3s2gtqxzeasji77x3zvf+s/ctzdpe0pzpbxlt855ztzqasnop7bi+7cvxf/wd7d/qpht57ud4bptr2tnmczsmq68ww8kmlwrpiunpcoqwckz7llj2387ds6l635rpss5xqwrhzsaaxxyvainzap4j8oug20yor6r/n6mrjfh/hw+g2qdgdeojtzdhsiza2onboscry/soks5ju4kkjh008rjywis1jiryagnpp5vr2/4epgyxlhxxig+nz7vaig7f1gzygzndhvxlrn/9j4epdwlapf6u/fu3kttvkrgnhvvqguj8tbhxguhqpsmwbaobk24gifdgqfhw8wtalnb1ylwi+sbrf3ptizoszd8mppq/lkt7dhfmydleegz7ujecfcpfeeknd0u48kbgvco/oy1sdatq7ddx1tt+kmh86s7b7hnjidhcybf307mn7odrthxrjx7ial7ihn2wd2wmzmsj/sf/vn/ktd6ch6h02vuqnev/oc3bdoyz/vjmo2</latexit> <latexit sha1_base64="qrhjo1lbk3bahxtzpnw9a2wzh8a=">aaacj3iczvfbaxnbfj6stxovtdq3xwzdoiku3sdujwmkoa+fcqytdem4o3s2gtqxzeasji77x3zvf+s/ctzdpe0pzpbxlt855ztzqasnop7bi+7cvxf/wd7d/qpht57ud4bptr2tnmczsmq68ww8kmlwrpiunpcoqwckz7llj2387ds6l635rpss5xqwrhzsaaxxyvainzap4j8oug20yor6r/n6mrjfh/hw+g2qdgdeojtzdhsiza2onboscry/soks5ju4kkjh008rjywis1jiryagnpp5vr2/4epgyxlhxxig+nz7vaig7f1gzygzndhvxlrn/9j4epdwlapf6u/fu3kttvkrgnhvvqguj8tbhxguhqpsmwbaobk24gifdgqfhw8wtalnb1ylwi+sbrf3ptizoszd8mppq/lkt7dhfmydleegz7ujecfcpfeeknd0u48kbgvco/oy1sdatq7ddx1tt+kmh86s7b7hnjidhcybf307mn7odrthxrjx7ial7ihn2wd2wmzmsj/sf/vn/ktd6ch6h02vuqnev/oc3bdoyz/vjmo2</latexit> <latexit sha1_base64="qrhjo1lbk3bahxtzpnw9a2wzh8a=">aaacj3iczvfbaxnbfj6stxovtdq3xwzdoiku3sdujwmkoa+fcqytdem4o3s2gtqxzeasji77x3zvf+s/ctzdpe0pzpbxlt855ztzqasnop7bi+7cvxf/wd7d/qpht57ud4bptr2tnmczsmq68ww8kmlwrpiunpcoqwckz7llj2387ds6l635rpss5xqwrhzsaaxxyvainzap4j8oug20yor6r/n6mrjfh/hw+g2qdgdeojtzdhsiza2onboscry/soks5ju4kkjh008rjywis1jiryagnpp5vr2/4epgyxlhxxig+nz7vaig7f1gzygzndhvxlrn/9j4epdwlapf6u/fu3kttvkrgnhvvqguj8tbhxguhqpsmwbaobk24gifdgqfhw8wtalnb1ylwi+sbrf3ptizoszd8mppq/lkt7dhfmydleegz7ujecfcpfeeknd0u48kbgvco/oy1sdatq7ddx1tt+kmh86s7b7hnjidhcybf307mn7odrthxrjx7ial7ihn2wd2wmzmsj/sf/vn/ktd6ch6h02vuqnev/oc3bdoyz/vjmo2</latexit> <latexit sha1_base64="av05c4tnhwfjdz+p+pg7nobgagy=">aaac+nichvjbaxnbfj5dbzvvm3p58uvgccsgztci7uuhkiivlqpnw+imyxz2kgydyzjzvhpwxf/im/jqn/eh+d+ctzespg8emjmpcz/nmzsxwmeu/qnco3fv3x+w8bc1+ejxk6329tmtzwrl+jazaexzsh2xqvmhcpt8llecqlty0/tyfw0//ckte0yf4ylni0wnwkweo+hv4/b3rnnuuvjqsxtfwtopv1z92ifefwpciv24uvgevcsjqairlhfhswpk5hbkp2wsz0r1bz1xy9hvwxta/8j//mbttrqtlqvug7gbhdli0xg7yelmwkg4riapc+dxlooopbyfk7xqjyxjowwxdmrppdructcql5uqoos1guym9ucjllwresvvrm7we9zdunvbrby2dvenok8zurx6onkblulnbxlnrsppcgloocydmme5q7nwgdir/ataztrshp6yg5lqfrpxtpd+zkbv0ztdqjrbnvnh5dt48jka8aqgc34tfkeomo6hyqjco6ts52gljimzhfafykm6p6roxdx/f1evg6hqevridrjug5pbtuzx57edg3cnqrvkjxlfeiqmu+safcrhzegy+rtsbs+df+g38ef4m/x15rogtcwzckpc3/8a1y/zva==</latexit> <latexit sha1_base64="av05c4tnhwfjdz+p+pg7nobgagy=">aaac+nichvjbaxnbfj5dbzvvm3p58uvgccsgztci7uuhkiivlqpnw+imyxz2kgydyzjzvhpwxf/im/jqn/eh+d+ctzespg8emjmpcz/nmzsxwmeu/qnco3fv3x+w8bc1+ejxk6329tmtzwrl+jazaexzsh2xqvmhcpt8llecqlty0/tyfw0//ckte0yf4ylni0wnwkweo+hv4/b3rnnuuvjqsxtfwtopv1z92ifefwpciv24uvgevcsjqairlhfhswpk5hbkp2wsz0r1bz1xy9hvwxta/8j//mbttrqtlqvug7gbhdli0xg7yelmwkg4riapc+dxlooopbyfk7xqjyxjowwxdmrppdructcql5uqoos1guym9ucjllwresvvrm7we9zdunvbrby2dvenok8zurx6onkblulnbxlnrsppcgloocydmme5q7nwgdir/ataztrshp6yg5lqfrpxtpd+zkbv0ztdqjrbnvnh5dt48jka8aqgc34tfkeomo6hyqjco6ts52gljimzhfafykm6p6roxdx/f1evg6hqevridrjug5pbtuzx57edg3cnqrvkjxlfeiqmu+safcrhzegy+rtsbs+df+g38ef4m/x15rogtcwzckpc3/8a1y/zva==</latexit> <latexit sha1_base64="av05c4tnhwfjdz+p+pg7nobgagy=">aaac+nichvjbaxnbfj5dbzvvm3p58uvgccsgztci7uuhkiivlqpnw+imyxz2kgydyzjzvhpwxf/im/jqn/eh+d+ctzespg8emjmpcz/nmzsxwmeu/qnco3fv3x+w8bc1+ejxk6329tmtzwrl+jazaexzsh2xqvmhcpt8llecqlty0/tyfw0//ckte0yf4ylni0wnwkweo+hv4/b3rnnuuvjqsxtfwtopv1z92ifefwpciv24uvgevcsjqairlhfhswpk5hbkp2wsz0r1bz1xy9hvwxta/8j//mbttrqtlqvug7gbhdli0xg7yelmwkg4riapc+dxlooopbyfk7xqjyxjowwxdmrppdructcql5uqoos1guym9ucjllwresvvrm7we9zdunvbrby2dvenok8zurx6onkblulnbxlnrsppcgloocydmme5q7nwgdir/ataztrshp6yg5lqfrpxtpd+zkbv0ztdqjrbnvnh5dt48jka8aqgc34tfkeomo6hyqjco6ts52gljimzhfafykm6p6roxdx/f1evg6hqevridrjug5pbtuzx57edg3cnqrvkjxlfeiqmu+safcrhzegy+rtsbs+df+g38ef4m/x15rogtcwzckpc3/8a1y/zva==</latexit> <latexit sha1_base64="av05c4tnhwfjdz+p+pg7nobgagy=">aaac+nichvjbaxnbfj5dbzvvm3p58uvgccsgztci7uuhkiivlqpnw+imyxz2kgydyzjzvhpwxf/im/jqn/eh+d+ctzespg8emjmpcz/nmzsxwmeu/qnco3fv3x+w8bc1+ejxk6329tmtzwrl+jazaexzsh2xqvmhcpt8llecqlty0/tyfw0//ckte0yf4ylni0wnwkweo+hv4/b3rnnuuvjqsxtfwtopv1z92ifefwpciv24uvgevcsjqairlhfhswpk5hbkp2wsz0r1bz1xy9hvwxta/8j//mbttrqtlqvug7gbhdli0xg7yelmwkg4riapc+dxlooopbyfk7xqjyxjowwxdmrppdructcql5uqoos1guym9ucjllwresvvrm7we9zdunvbrby2dvenok8zurx6onkblulnbxlnrsppcgloocydmme5q7nwgdir/ataztrshp6yg5lqfrpxtpd+zkbv0ztdqjrbnvnh5dt48jka8aqgc34tfkeomo6hyqjco6ts52gljimzhfafykm6p6roxdx/f1evg6hqevridrjug5pbtuzx57edg3cnqrvkjxlfeiqmu+safcrhzegy+rtsbs+df+g38ef4m/x15rogtcwzckpc3/8a1y/zva==</latexit> <latexit sha1_base64="vlpk1jvyaj1p3m3gydp3xbfodk4=">aaacjhiczvhbattaef0rvsrum9rty16wgkmfspbmiivscc2evgrsqjnabmxqnbkx7exsjmobou/pa/jn/zuohfesz2cxw1zozjxjc60cxvhftrtz7pmll7t73vev3+y/7fxfxqrxeglj6btzv6kiojwfmsrucfv4ecbvcjnefg/il7/bb+xsl1wxmdviblwupebyzxr9irg4spnqwc8qc8uaz3qd+ddegh8kkhymwgvns35htjinswmwpryhxcdxgdnkefrsq92dlaekiw/ehk4jwmegtkvn7dufkifjufp0lpkn92ffjuwia5nszjnp2i41zv+x4cmgrhrgsnuf88/tstmirldyvn1eao6on/lwthmqqncehpsknubyibyqsci+ymouzz75utmvnwm2d7y0kxjiahk9d1s+mcooor9ykouecrydmlcsxnmaqhnhdxiapsstnaroxopmofcromcoq1fcd+ksyfyrnokl0wfc+ofr4orbe6bddsa+si8sycfshp1g52zmjfuyp+yw3ux70vh0jfp6nxp12pr37jffp/8azjdjyq==</latexit> <latexit sha1_base64="vlpk1jvyaj1p3m3gydp3xbfodk4=">aaacjhiczvhbattaef0rvsrum9rty16wgkmfspbmiivscc2evgrsqjnabmxqnbkx7exsjmobou/pa/jn/zuohfesz2cxw1zozjxjc60cxvhftrtz7pmll7t73vev3+y/7fxfxqrxeglj6btzv6kiojwfmsrucfv4ecbvcjnefg/il7/bb+xsl1wxmdviblwupebyzxr9irg4spnqwc8qc8uaz3qd+ddegh8kkhymwgvns35htjinswmwpryhxcdxgdnkefrsq92dlaekiw/ehk4jwmegtkvn7dufkifjufp0lpkn92ffjuwia5nszjnp2i41zv+x4cmgrhrgsnuf88/tstmirldyvn1eao6on/lwthmqqncehpsknubyibyqsci+ymouzz75utmvnwm2d7y0kxjiahk9d1s+mcooor9ykouecrydmlcsxnmaqhnhdxiapsstnaroxopmofcromcoq1fcd+ksyfyrnokl0wfc+ofr4orbe6bddsa+si8sycfshp1g52zmjfuyp+yw3ux70vh0jfp6nxp12pr37jffp/8azjdjyq==</latexit> <latexit sha1_base64="vlpk1jvyaj1p3m3gydp3xbfodk4=">aaacjhiczvhbattaef0rvsrum9rty16wgkmfspbmiivscc2evgrsqjnabmxqnbkx7exsjmobou/pa/jn/zuohfesz2cxw1zozjxjc60cxvhftrtz7pmll7t73vev3+y/7fxfxqrxeglj6btzv6kiojwfmsrucfv4ecbvcjnefg/il7/bb+xsl1wxmdviblwupebyzxr9irg4spnqwc8qc8uaz3qd+ddegh8kkhymwgvns35htjinswmwpryhxcdxgdnkefrsq92dlaekiw/ehk4jwmegtkvn7dufkifjufp0lpkn92ffjuwia5nszjnp2i41zv+x4cmgrhrgsnuf88/tstmirldyvn1eao6on/lwthmqqncehpsknubyibyqsci+ymouzz75utmvnwm2d7y0kxjiahk9d1s+mcooor9ykouecrydmlcsxnmaqhnhdxiapsstnaroxopmofcromcoq1fcd+ksyfyrnokl0wfc+ofr4orbe6bddsa+si8sycfshp1g52zmjfuyp+yw3ux70vh0jfp6nxp12pr37jffp/8azjdjyq==</latexit> <latexit sha1_base64="vlpk1jvyaj1p3m3gydp3xbfodk4=">aaacjhiczvhbattaef0rvsrum9rty16wgkmfspbmiivscc2evgrsqjnabmxqnbkx7exsjmobou/pa/jn/zuohfesz2cxw1zozjxjc60cxvhftrtz7pmll7t73vev3+y/7fxfxqrxeglj6btzv6kiojwfmsrucfv4ecbvcjnefg/il7/bb+xsl1wxmdviblwupebyzxr9irg4spnqwc8qc8uaz3qd+ddegh8kkhymwgvns35htjinswmwpryhxcdxgdnkefrsq92dlaekiw/ehk4jwmegtkvn7dufkifjufp0lpkn92ffjuwia5nszjnp2i41zv+x4cmgrhrgsnuf88/tstmirldyvn1eao6on/lwthmqqncehpsknubyibyqsci+ymouzz75utmvnwm2d7y0kxjiahk9d1s+mcooor9ykouecrydmlcsxnmaqhnhdxiapsstnaroxopmofcromcoq1fcd+ksyfyrnokl0wfc+ofr4orbe6bddsa+si8sycfshp1g52zmjfuyp+yw3ux70vh0jfp6nxp12pr37jffp/8azjdjyq==</latexit> <latexit sha1_base64="tx2xng/oixf3mkx4ac5stw97xx8=">aaacjhiczvftsxtben5ctbxratj+7jelidapre6cofak0klpf8fco4ijyw9vllncl2n3riyc91p6tf1n/hvn4le0duzymc/pzdytflofjoo7tvria/vlq53x3d29n/shvf7bi+bkl2esnxb+khubtliwrouargopwqqaltobb0388jf4ojz9hascpkbmrcqvfeiuwa8/mqixav7d1rpk6azms94gpozxxp+dpaud1tr5rn+rk8zj0obfquui10lc4lqshpxuuhcnzybcybsxh2ucvhgi02o9e82h5ml47jw9i3ztfvxrcrpcyqsu2uwanmon839s+diiy4yxbpth/grakvuucfy+tm9lzdhxrh6ekq8s9yqakf7rblwuhbcsscqnti3i2sdfavqlm832wzymbq8zla/njsoxzgq150nospbagbct85afkxtqaolotgkgdkwlrtwzwj4j9gpjbwx1nyrrlp0l2tzcc3axokwi/zwanh5td7td3rmp7cnl2de7zt/yorszyw7zh/ax/yv2o6poc/tlitxqtdxv2bolvt8dtzhjvg==</latexit> <latexit sha1_base64="tx2xng/oixf3mkx4ac5stw97xx8=">aaacjhiczvftsxtben5ctbxratj+7jelidapre6cofak0klpf8fco4ijyw9vllncl2n3riyc91p6tf1n/hvn4le0duzymc/pzdytflofjoo7tvria/vlq53x3d29n/shvf7bi+bkl2esnxb+khubtliwrouargopwqqaltobb0388jf4ojz9hascpkbmrcqvfeiuwa8/mqixav7d1rpk6azms94gpozxxp+dpaud1tr5rn+rk8zj0obfquui10lc4lqshpxuuhcnzybcybsxh2ucvhgi02o9e82h5ml47jw9i3ztfvxrcrpcyqsu2uwanmon839s+diiy4yxbpth/grakvuucfy+tm9lzdhxrh6ekq8s9yqakf7rblwuhbcsscqnti3i2sdfavqlm832wzymbq8zla/njsoxzgq150nospbagbct85afkxtqaolotgkgdkwlrtwzwj4j9gpjbwx1nyrrlp0l2tzcc3axokwi/zwanh5td7td3rmp7cnl2de7zt/yorszyw7zh/ax/yv2o6poc/tlitxqtdxv2bolvt8dtzhjvg==</latexit> <latexit sha1_base64="tx2xng/oixf3mkx4ac5stw97xx8=">aaacjhiczvftsxtben5ctbxratj+7jelidapre6cofak0klpf8fco4ijyw9vllncl2n3riyc91p6tf1n/hvn4le0duzymc/pzdytflofjoo7tvria/vlq53x3d29n/shvf7bi+bkl2esnxb+khubtliwrouargopwqqaltobb0388jf4ojz9hascpkbmrcqvfeiuwa8/mqixav7d1rpk6azms94gpozxxp+dpaud1tr5rn+rk8zj0obfquui10lc4lqshpxuuhcnzybcybsxh2ucvhgi02o9e82h5ml47jw9i3ztfvxrcrpcyqsu2uwanmon839s+diiy4yxbpth/grakvuucfy+tm9lzdhxrh6ekq8s9yqakf7rblwuhbcsscqnti3i2sdfavqlm832wzymbq8zla/njsoxzgq150nospbagbct85afkxtqaolotgkgdkwlrtwzwj4j9gpjbwx1nyrrlp0l2tzcc3axokwi/zwanh5td7td3rmp7cnl2de7zt/yorszyw7zh/ax/yv2o6poc/tlitxqtdxv2bolvt8dtzhjvg==</latexit> <latexit sha1_base64="tx2xng/oixf3mkx4ac5stw97xx8=">aaacjhiczvftsxtben5ctbxratj+7jelidapre6cofak0klpf8fco4ijyw9vllncl2n3riyc91p6tf1n/hvn4le0duzymc/pzdytflofjoo7tvria/vlq53x3d29n/shvf7bi+bkl2esnxb+khubtliwrouargopwqqaltobb0388jf4ojz9hascpkbmrcqvfeiuwa8/mqixav7d1rpk6azms94gpozxxp+dpaud1tr5rn+rk8zj0obfquui10lc4lqshpxuuhcnzybcybsxh2ucvhgi02o9e82h5ml47jw9i3ztfvxrcrpcyqsu2uwanmon839s+diiy4yxbpth/grakvuucfy+tm9lzdhxrh6ekq8s9yqakf7rblwuhbcsscqnti3i2sdfavqlm832wzymbq8zla/njsoxzgq150nospbagbct85afkxtqaolotgkgdkwlrtwzwj4j9gpjbwx1nyrrlp0l2tzcc3axokwi/zwanh5td7td3rmp7cnl2de7zt/yorszyw7zh/ax/yv2o6poc/tlitxqtdxv2bolvt8dtzhjvg==</latexit> <latexit sha1_base64="t99u/9q28x6pdjefcaeowqdsga8=">aaacvhiczvfbaxnbfj6stxovtfxrl4mhumgw3sdos0tvhl4ukpi20i1hdvzsmmquy8xsm7dm7/hx+kr4b5xnf03tazn8fod+vqwu3lo4/toj7t1/8pdr1upuk6fpnm/3dl6cwv0zhiomhtyxgbuoumkr407grwmqykzgetb/3pjpr9byrtu3tyxxlolu8yiz6gi16x1mjxwzrkiv/arweo3hanyplxipe3d8vd5lpkskzolcl93/iw8mvx68h68m7okkbx3s2ulkp8psxlnkonjmugsvk7h045oax5la300riyvlczrfywavlwjh9wpxd4pa5fboe55ysglxm2oqrv3kleq2m9pnx0p+8w3wnw7rvlqb/v3xyvxzvvyofbtpx1qcnibmnjbzg8yjzqcugr42adajhjixjn6ruqnq/tzuivxmy2z7qyqzoce8lc+moqtp5ba9wadcwckbllqtq1sfulkoilcjm1p0tfcqnkpp6okeosmxqxdr62hsu0gwzfoeu+bsuj8e/pvd/+htk9aweuvek12skpfkibytuziijpwgp8kv8js6jpjohsmb0kjt5rwktyy6+gtohdyv</latexit> <latexit sha1_base64="t99u/9q28x6pdjefcaeowqdsga8=">aaacvhiczvfbaxnbfj6stxovtfxrl4mhumgw3sdos0tvhl4ukpi20i1hdvzsmmquy8xsm7dm7/hx+kr4b5xnf03tazn8fod+vqwu3lo4/toj7t1/8pdr1upuk6fpnm/3dl6cwv0zhiomhtyxgbuoumkr407grwmqykzgetb/3pjpr9byrtu3tyxxlolu8yiz6gi16x1mjxwzrkiv/arweo3hanyplxipe3d8vd5lpkskzolcl93/iw8mvx68h68m7okkbx3s2ulkp8psxlnkonjmugsvk7h045oax5la300riyvlczrfywavlwjh9wpxd4pa5fboe55ysglxm2oqrv3kleq2m9pnx0p+8w3wnw7rvlqb/v3xyvxzvvyofbtpx1qcnibmnjbzg8yjzqcugr42adajhjixjn6ruqnq/tzuivxmy2z7qyqzoce8lc+moqtp5ba9wadcwckbllqtq1sfulkoilcjm1p0tfcqnkpp6okeosmxqxdr62hsu0gwzfoeu+bsuj8e/pvd/+htk9aweuvek12skpfkibytuziijpwgp8kv8js6jpjohsmb0kjt5rwktyy6+gtohdyv</latexit> <latexit sha1_base64="t99u/9q28x6pdjefcaeowqdsga8=">aaacvhiczvfbaxnbfj6stxovtfxrl4mhumgw3sdos0tvhl4ukpi20i1hdvzsmmquy8xsm7dm7/hx+kr4b5xnf03tazn8fod+vqwu3lo4/toj7t1/8pdr1upuk6fpnm/3dl6cwv0zhiomhtyxgbuoumkr407grwmqykzgetb/3pjpr9byrtu3tyxxlolu8yiz6gi16x1mjxwzrkiv/arweo3hanyplxipe3d8vd5lpkskzolcl93/iw8mvx68h68m7okkbx3s2ulkp8psxlnkonjmugsvk7h045oax5la300riyvlczrfywavlwjh9wpxd4pa5fboe55ysglxm2oqrv3kleq2m9pnx0p+8w3wnw7rvlqb/v3xyvxzvvyofbtpx1qcnibmnjbzg8yjzqcugr42adajhjixjn6ruqnq/tzuivxmy2z7qyqzoce8lc+moqtp5ba9wadcwckbllqtq1sfulkoilcjm1p0tfcqnkpp6okeosmxqxdr62hsu0gwzfoeu+bsuj8e/pvd/+htk9aweuvek12skpfkibytuziijpwgp8kv8js6jpjohsmb0kjt5rwktyy6+gtohdyv</latexit> <latexit sha1_base64="t99u/9q28x6pdjefcaeowqdsga8=">aaacvhiczvfbaxnbfj6stxovtfxrl4mhumgw3sdos0tvhl4ukpi20i1hdvzsmmquy8xsm7dm7/hx+kr4b5xnf03tazn8fod+vqwu3lo4/toj7t1/8pdr1upuk6fpnm/3dl6cwv0zhiomhtyxgbuoumkr407grwmqykzgetb/3pjpr9byrtu3tyxxlolu8yiz6gi16x1mjxwzrkiv/arweo3hanyplxipe3d8vd5lpkskzolcl93/iw8mvx68h68m7okkbx3s2ulkp8psxlnkonjmugsvk7h045oax5la300riyvlczrfywavlwjh9wpxd4pa5fboe55ysglxm2oqrv3kleq2m9pnx0p+8w3wnw7rvlqb/v3xyvxzvvyofbtpx1qcnibmnjbzg8yjzqcugr42adajhjixjn6ruqnq/tzuivxmy2z7qyqzoce8lc+moqtp5ba9wadcwckbllqtq1sfulkoilcjm1p0tfcqnkpp6okeosmxqxdr62hsu0gwzfoeu+bsuj8e/pvd/+htk9aweuvek12skpfkibytuziijpwgp8kv8js6jpjohsmb0kjt5rwktyy6+gtohdyv</latexit> Minimisation Problem: The error equation can not be solved in closed form re(w) = NX ( (w T (x i )) t i ) (x i ) i=1 Instead, we need to apply an iterative approach, e.g. Newton-Raphson re(w) w new = w old H 1 re(w) 10 Hessian Matrix w new w old !13

14 <latexit sha1_base64="ltbdgj9ialz46h4/tyrjpfx4toy=">aaac03iczvllattafb2rr9r9oe2ym6hg4eatjfnonoxqbrojucvodjfjrqmre8g8xmwosrlmu7rtl/u/uu+2/yaobku4zgvjr+e+5t4zwcmzsxh8qxxdu//g4aodx+0nt589f9hzfxlqvkupjkjiso8zyoazcspllidxqygijmnzdvmp9p9dgtzmyro7lgeiyeyyglfiazxtjfnb7dwr3lwfognxhn/am5tiucf7uj8o5+zi5gv92btw+4nha6z/e7ysw25+rn+bdrrxqbwyfbckdeiixobt3rznc0uradjstow5t+lsthzrlleovp1wbkpcl8kmzgouriczunukpo4fjsef0ugrfq/yzqxhhdflkyxi+oxm21et/329tadd1bxnvn9bhe4ck2vlqdj1+6li2cpcbxnntao1fbkaozqfctcde02odvrcqlqll7/vfq9vqkq9vzgvyebdhobnmxxs52iahumedmsjczk4otfuqmamjsu81ginbixvlqyn3dfzhbor2slca+pdipbtieuylcjdcdo4sal+8q579leraae9rm9qhyxoptpcn9eqjrbfp9fv9af9juari75f39ehuavjeyvuwftjhysi5bu=</latexit> <latexit sha1_base64="ltbdgj9ialz46h4/tyrjpfx4toy=">aaac03iczvllattafb2rr9r9oe2ym6hg4eatjfnonoxqbrojucvodjfjrqmre8g8xmwosrlmu7rtl/u/uu+2/yaobku4zgvjr+e+5t4zwcmzsxh8qxxdu//g4aodx+0nt589f9hzfxlqvkupjkjiso8zyoazcspllidxqygijmnzdvmp9p9dgtzmyro7lgeiyeyyglfiazxtjfnb7dwr3lwfognxhn/am5tiucf7uj8o5+zi5gv92btw+4nha6z/e7ysw25+rn+bdrrxqbwyfbckdeiixobt3rznc0uradjstow5t+lsthzrlleovp1wbkpcl8kmzgouriczunukpo4fjsef0ugrfq/yzqxhhdflkyxi+oxm21et/329tadd1bxnvn9bhe4ck2vlqdj1+6li2cpcbxnntao1fbkaozqfctcde02odvrcqlqll7/vfq9vqkq9vzgvyebdhobnmxxs52iahumedmsjczk4otfuqmamjsu81ginbixvlqyn3dfzhbor2slca+pdipbtieuylcjdcdo4sal+8q579leraae9rm9qhyxoptpcn9eqjrbfp9fv9af9juari75f39ehuavjeyvuwftjhysi5bu=</latexit> <latexit sha1_base64="ltbdgj9ialz46h4/tyrjpfx4toy=">aaac03iczvllattafb2rr9r9oe2ym6hg4eatjfnonoxqbrojucvodjfjrqmre8g8xmwosrlmu7rtl/u/uu+2/yaobku4zgvjr+e+5t4zwcmzsxh8qxxdu//g4aodx+0nt589f9hzfxlqvkupjkjiso8zyoazcspllidxqygijmnzdvmp9p9dgtzmyro7lgeiyeyyglfiazxtjfnb7dwr3lwfognxhn/am5tiucf7uj8o5+zi5gv92btw+4nha6z/e7ysw25+rn+bdrrxqbwyfbckdeiixobt3rznc0uradjstow5t+lsthzrlleovp1wbkpcl8kmzgouriczunukpo4fjsef0ugrfq/yzqxhhdflkyxi+oxm21et/329tadd1bxnvn9bhe4ck2vlqdj1+6li2cpcbxnntao1fbkaozqfctcde02odvrcqlqll7/vfq9vqkq9vzgvyebdhobnmxxs52iahumedmsjczk4otfuqmamjsu81ginbixvlqyn3dfzhbor2slca+pdipbtieuylcjdcdo4sal+8q579leraae9rm9qhyxoptpcn9eqjrbfp9fv9af9juari75f39ehuavjeyvuwftjhysi5bu=</latexit> <latexit sha1_base64="ltbdgj9ialz46h4/tyrjpfx4toy=">aaac03iczvllattafb2rr9r9oe2ym6hg4eatjfnonoxqbrojucvodjfjrqmre8g8xmwosrlmu7rtl/u/uu+2/yaobku4zgvjr+e+5t4zwcmzsxh8qxxdu//g4aodx+0nt589f9hzfxlqvkupjkjiso8zyoazcspllidxqygijmnzdvmp9p9dgtzmyro7lgeiyeyyglfiazxtjfnb7dwr3lwfognxhn/am5tiucf7uj8o5+zi5gv92btw+4nha6z/e7ysw25+rn+bdrrxqbwyfbckdeiixobt3rznc0uradjstow5t+lsthzrlleovp1wbkpcl8kmzgouriczunukpo4fjsef0ugrfq/yzqxhhdflkyxi+oxm21et/329tadd1bxnvn9bhe4ck2vlqdj1+6li2cpcbxnntao1fbkaozqfctcde02odvrcqlqll7/vfq9vqkq9vzgvyebdhobnmxxs52iahumedmsjczk4otfuqmamjsu81ginbixvlqyn3dfzhbor2slca+pdipbtieuylcjdcdo4sal+8q579leraae9rm9qhyxoptpcn9eqjrbfp9fv9af9juari75f39ehuavjeyvuwftjhysi5bu=</latexit> Iterative Weighted Least Squares The update rule for the logistic regression methods is then: w new = w old ( T R ) 1 T (y t) Where the weighting metric R depends also on the weights w!14