Maximum A Posteriori Chris Piech CS109, Stanford University

Transkrypt

1 Maximum A Posteriori Chris Piech CS109, Stanford University

2 Previously in CS109

3 Game of Estimators Estimators Maximum Likelihood Non spoiler alert: this didn t happen in game of thrones

4 <latexit sha1_base64="oebugeluaj17t+qkzaz7yakqleq=">aaab7nicbva9swnbej2lxzf+rs1tfomqm3anghywarvlcoydkjpsbfasjxt7x+6cei78cbslrwz9pxb+gzfjfzr4yodx3gwz84jecoou++0u1ty3nrek26wd3b39g/lhucveqwa8ywiz605adzdc8sykllytae6jqpj2ml6d+e0nro2i1qnoeu5hdkhekbhfk7u7j1lvne/75ypbc+cgq8tlsqvynprlr94gzmneftjjjel6boj+rjukjvm01esntygb0yhvwqpoxi2fzc+dkjordegya1skyvz9pzhryjhjfnjoiolilhsz8t+vm2j47wdcjslyxralwlqsjmnsdziqmjoue0so08lestiiasrqjlsyixjll6+s1kxnc2ve/wwlfpphuyqtoiuqehafdbidbjsbwrie4rxenmr5cd6dj0vrwclnjuepnm8fuskpig==</latexit> <latexit sha1_base64="oebugeluaj17t+qkzaz7yakqleq=">aaab7nicbva9swnbej2lxzf+rs1tfomqm3anghywarvlcoydkjpsbfasjxt7x+6cei78cbslrwz9pxb+gzfjfzr4yodx3gwz84jecoou++0u1ty3nrek26wd3b39g/lhucveqwa8ywiz605adzdc8sykllytae6jqpj2ml6d+e0nro2i1qnoeu5hdkhekbhfk7u7j1lvne/75ypbc+cgq8tlsqvynprlr94gzmneftjjjel6boj+rjukjvm01esntygb0yhvwqpoxi2fzc+dkjordegya1skyvz9pzhryjhjfnjoiolilhsz8t+vm2j47wdcjslyxralwlqsjmnsdziqmjoue0so08lestiiasrqjlsyixjll6+s1kxnc2ve/wwlfpphuyqtoiuqehafdbidbjsbwrie4rxenmr5cd6dj0vrwclnjuepnm8fuskpig==</latexit> <latexit sha1_base64="oebugeluaj17t+qkzaz7yakqleq=">aaab7nicbva9swnbej2lxzf+rs1tfomqm3anghywarvlcoydkjpsbfasjxt7x+6cei78cbslrwz9pxb+gzfjfzr4yodx3gwz84jecoou++0u1ty3nrek26wd3b39g/lhucveqwa8ywiz605adzdc8sykllytae6jqpj2ml6d+e0nro2i1qnoeu5hdkhekbhfk7u7j1lvne/75ypbc+cgq8tlsqvynprlr94gzmneftjjjel6boj+rjukjvm01esntygb0yhvwqpoxi2fzc+dkjordegya1skyvz9pzhryjhjfnjoiolilhsz8t+vm2j47wdcjslyxralwlqsjmnsdziqmjoue0so08lestiiasrqjlsyixjll6+s1kxnc2ve/wwlfpphuyqtoiuqehafdbidbjsbwrie4rxenmr5cd6dj0vrwclnjuepnm8fuskpig==</latexit> <latexit sha1_base64="oebugeluaj17t+qkzaz7yakqleq=">aaab7nicbva9swnbej2lxzf+rs1tfomqm3anghywarvlcoydkjpsbfasjxt7x+6cei78cbslrwz9pxb+gzfjfzr4yodx3gwz84jecoou++0u1ty3nrek26wd3b39g/lhucveqwa8ywiz605adzdc8sykllytae6jqpj2ml6d+e0nro2i1qnoeu5hdkhekbhfk7u7j1lvne/75ypbc+cgq8tlsqvynprlr94gzmneftjjjel6boj+rjukjvm01esntygb0yhvwqpoxi2fzc+dkjordegya1skyvz9pzhryjhjfnjoiolilhsz8t+vm2j47wdcjslyxralwlqsjmnsdziqmjoue0so08lestiiasrqjlsyixjll6+s1kxnc2ve/wwlfpphuyqtoiuqehafdbidbjsbwrie4rxenmr5cd6dj0vrwclnjuepnm8fuskpig==</latexit> <latexit sha1_base64="hdn6guza9yu12fk9okkzvwvw+fy=">aaach3icbvc7sgnbfj31gemramkzgirne3zfni0qslfiece8ibux2clsmmt2wcxdmaz5ext/xczcebhl3zhjttdeawohc87lzj1ellgcy5oyk6tr6xubua389s7u3n7h4lcpokrs1qcrigtbi4ojhrigcbcshutgak+wlje8nvqtbyyvj8i7gmwsg5b+yh1ocwjjlvzuaqydawakhk+wo5la5dgrur/7zvs+nxlprpxhjd3hloswilbzmgevezsjrzsh7ha+nv5ek4cfqavrqmnbmxrtioftwcz5j1esjnri+qyjaugcprrp7l4xptvkd/ur1c8epfn/t6qkugouedozebiorw8q/ud1evar3zshcqisppnffiiwrhhafu5xysiiksaesq7/iumasejbv5rxjdiljy+t5lnztsr27xmxwsnqykfjdijmzknlveu3qi4aikjn9ire0yfxyrwzn8bxplpizdnh6a+myq+u7aa+</latexit> <latexit sha1_base64="hdn6guza9yu12fk9okkzvwvw+fy=">aaach3icbvc7sgnbfj31gemramkzgirne3zfni0qslfiece8ibux2clsmmt2wcxdmaz5ext/xczcebhl3zhjttdeawohc87lzj1ellgcy5oyk6tr6xubua389s7u3n7h4lcpokrs1qcrigtbi4ojhrigcbcshutgak+wlje8nvqtbyyvj8i7gmwsg5b+yh1ocwjjlvzuaqydawakhk+wo5la5dgrur/7zvs+nxlprpxhjd3hloswilbzmgevezsjrzsh7ha+nv5ek4cfqavrqmnbmxrtioftwcz5j1esjnri+qyjaugcprrp7l4xptvkd/ur1c8epfn/t6qkugouedozebiorw8q/ud1evar3zshcqisppnffiiwrhhafu5xysiiksaesq7/iumasejbv5rxjdiljy+t5lnztsr27xmxwsnqykfjdijmzknlveu3qi4aikjn9ire0yfxyrwzn8bxplpizdnh6a+myq+u7aa+</latexit> <latexit sha1_base64="hdn6guza9yu12fk9okkzvwvw+fy=">aaach3icbvc7sgnbfj31gemramkzgirne3zfni0qslfiece8ibux2clsmmt2wcxdmaz5ext/xczcebhl3zhjttdeawohc87lzj1ellgcy5oyk6tr6xubua389s7u3n7h4lcpokrs1qcrigtbi4ojhrigcbcshutgak+wlje8nvqtbyyvj8i7gmwsg5b+yh1ocwjjlvzuaqydawakhk+wo5la5dgrur/7zvs+nxlprpxhjd3hloswilbzmgevezsjrzsh7ha+nv5ek4cfqavrqmnbmxrtioftwcz5j1esjnri+qyjaugcprrp7l4xptvkd/ur1c8epfn/t6qkugouedozebiorw8q/ud1evar3zshcqisppnffiiwrhhafu5xysiiksaesq7/iumasejbv5rxjdiljy+t5lnztsr27xmxwsnqykfjdijmzknlveu3qi4aikjn9ire0yfxyrwzn8bxplpizdnh6a+myq+u7aa+</latexit> <latexit sha1_base64="hdn6guza9yu12fk9okkzvwvw+fy=">aaach3icbvc7sgnbfj31gemramkzgirne3zfni0qslfiece8ibux2clsmmt2wcxdmaz5ext/xczcebhl3zhjttdeawohc87lzj1ellgcy5oyk6tr6xubua389s7u3n7h4lcpokrs1qcrigtbi4ojhrigcbcshutgak+wlje8nvqtbyyvj8i7gmwsg5b+yh1ocwjjlvzuaqydawakhk+wo5la5dgrur/7zvs+nxlprpxhjd3hloswilbzmgevezsjrzsh7ha+nv5ek4cfqavrqmnbmxrtioftwcz5j1esjnri+qyjaugcprrp7l4xptvkd/ur1c8epfn/t6qkugouedozebiorw8q/ud1evar3zshcqisppnffiiwrhhafu5xysiiksaesq7/iumasejbv5rxjdiljy+t5lnztsr27xmxwsnqykfjdijmzknlveu3qi4aikjn9ire0yfxyrwzn8bxplpizdnh6a+myq+u7aa+</latexit> <latexit sha1_base64="ptbfrb57zghu2wz6s4+ybggkedm=">aaacrxicbvdpsxtbfj5vazvwte3ry9agxevyfuevbwkrpkrgovehg8lbyusyoduzzlwthmx/os/evfkf9njdi3htz5nq/pvg4hvf9z7emy/jlhquhnfbwulsm+w3k6u1txfrg5v191tnzurwyecyzexfag6v1nghsqovmouqjgrpk8svlx7+a62trn+nsya9fezadqua8ls/hsdjockmmrku/ejr+7jkn3ic64e34x+lie2gfshydskwyecpxpgy5dhhfrxcg89y1gy3mzpdbtxu9uunsbvoi78e0rw02lxo+/xbegbenqimocc5bhrm1pprsaqfzs3ohwyglmgexq+rw1yvmkzq8h3pdpjqwp808sn72ffa6twktfxkcjr2z7wkfe3r5jq87bvszzmhfrnfw1xxmrykla+krufq4geik/2txizbgicfyc2hed3/8ktwttekwlb0bb9xddipy4vts4+sysj2wi7ycttlhsbynfvjfrm/wu3wk7gphmajc8hc84e9qedvpzfasxg=</latexit> <latexit sha1_base64="ptbfrb57zghu2wz6s4+ybggkedm=">aaacrxicbvdpsxtbfj5vazvwte3ry9agxevyfuevbwkrpkrgovehg8lbyusyoduzzlwthmx/os/evfkf9njdi3htz5nq/pvg4hvf9z7emy/jlhquhnfbwulsm+w3k6u1txfrg5v191tnzurwyecyzexfag6v1nghsqovmouqjgrpk8svlx7+a62trn+nsya9fezadqua8ls/hsdjockmmrku/ejr+7jkn3ic64e34x+lie2gfshydskwyecpxpgy5dhhfrxcg89y1gy3mzpdbtxu9uunsbvoi78e0rw02lxo+/xbegbenqimocc5bhrm1pprsaqfzs3ohwyglmgexq+rw1yvmkzq8h3pdpjqwp808sn72ffa6twktfxkcjr2z7wkfe3r5jq87bvszzmhfrnfw1xxmrykla+krufq4geik/2txizbgicfyc2hed3/8ktwttekwlb0bb9xddipy4vts4+sysj2wi7ycttlhsbynfvjfrm/wu3wk7gphmajc8hc84e9qedvpzfasxg=</latexit> <latexit sha1_base64="ptbfrb57zghu2wz6s4+ybggkedm=">aaacrxicbvdpsxtbfj5vazvwte3ry9agxevyfuevbwkrpkrgovehg8lbyusyoduzzlwthmx/os/evfkf9njdi3htz5nq/pvg4hvf9z7emy/jlhquhnfbwulsm+w3k6u1txfrg5v191tnzurwyecyzexfag6v1nghsqovmouqjgrpk8svlx7+a62trn+nsya9fezadqua8ls/hsdjockmmrku/ejr+7jkn3ic64e34x+lie2gfshydskwyecpxpgy5dhhfrxcg89y1gy3mzpdbtxu9uunsbvoi78e0rw02lxo+/xbegbenqimocc5bhrm1pprsaqfzs3ohwyglmgexq+rw1yvmkzq8h3pdpjqwp808sn72ffa6twktfxkcjr2z7wkfe3r5jq87bvszzmhfrnfw1xxmrykla+krufq4geik/2txizbgicfyc2hed3/8ktwttekwlb0bb9xddipy4vts4+sysj2wi7ycttlhsbynfvjfrm/wu3wk7gphmajc8hc84e9qedvpzfasxg=</latexit> <latexit sha1_base64="ptbfrb57zghu2wz6s4+ybggkedm=">aaacrxicbvdpsxtbfj5vazvwte3ry9agxevyfuevbwkrpkrgovehg8lbyusyoduzzlwthmx/os/evfkf9njdi3htz5nq/pvg4hvf9z7emy/jlhquhnfbwulsm+w3k6u1txfrg5v191tnzurwyecyzexfag6v1nghsqovmouqjgrpk8svlx7+a62trn+nsya9fezadqua8ls/hsdjockmmrku/ejr+7jkn3ic64e34x+lie2gfshydskwyecpxpgy5dhhfrxcg89y1gy3mzpdbtxu9uunsbvoi78e0rw02lxo+/xbegbenqimocc5bhrm1pprsaqfzs3ohwyglmgexq+rw1yvmkzq8h3pdpjqwp808sn72ffa6twktfxkcjr2z7wkfe3r5jq87bvszzmhfrnfw1xxmrykla+krufq4geik/2txizbgicfyc2hed3/8ktwttekwlb0bb9xddipy4vts4+sysj2wi7ycttlhsbynfvjfrm/wu3wk7gphmajc8hc84e9qedvpzfasxg=</latexit> <latexit sha1_base64="xueh2vomh7cx8tkkop/dsmc0u7m=">aaacgnicbvc7sgnbfj31genr1djmmahje3zfmi0qslgwigaekf2x2clsmmt2wcxdmaz5dht/xczcetux8w+cjfto4ogbwznncucepxfcgwv9g0vlk6tr64wn4ubw9s6uubffuneqkwvswmsy4xpfbi9yezgi1kkki6evwnsfxkz89h2tisfrdyws5oakh/gauwja8kz7quzagagp4hpsjdluerwh5c5tvuavsdbuc/aa85xnlqyqnqvejhzosihhwzm/nv5m05bfqavrqmtbcbgzkccpyooikyqwedokfdbvncihu242pw2mj7xsw0es9ysat9xfexkjlrqfvk6gbazq3pui/3ndfikam/eosyffdlyosawgge96wj0ugqux0orqyfvfmr0qssjonou6bhv+5exsoqnavtw+pi3va3kdbxsijlaz2egm1delaqamougrpanx9gy8gs/gu/exiy4z+cwb+gpj6wcauz5i</latexit> <latexit sha1_base64="xueh2vomh7cx8tkkop/dsmc0u7m=">aaacgnicbvc7sgnbfj31genr1djmmahje3zfmi0qslgwigaekf2x2clsmmt2wcxdmaz5dht/xczcetux8w+cjfto4ogbwznncucepxfcgwv9g0vlk6tr64wn4ubw9s6uubffuneqkwvswmsy4xpfbi9yezgi1kkki6evwnsfxkz89h2tisfrdyws5oakh/gauwja8kz7quzagagp4hpsjdluerwh5c5tvuavsdbuc/aa85xnlqyqnqvejhzosihhwzm/nv5m05bfqavrqmtbcbgzkccpyooikyqwedokfdbvncihu242pw2mj7xsw0es9ysat9xfexkjlrqfvk6gbazq3pui/3ndfikam/eosyffdlyosawgge96wj0ugqux0orqyfvfmr0qssjonou6bhv+5exsoqnavtw+pi3va3kdbxsijlaz2egm1delaqamougrpanx9gy8gs/gu/exiy4z+cwb+gpj6wcauz5i</latexit> <latexit sha1_base64="xueh2vomh7cx8tkkop/dsmc0u7m=">aaacgnicbvc7sgnbfj31genr1djmmahje3zfmi0qslgwigaekf2x2clsmmt2wcxdmaz5dht/xczcetux8w+cjfto4ogbwznncucepxfcgwv9g0vlk6tr64wn4ubw9s6uubffuneqkwvswmsy4xpfbi9yezgi1kkki6evwnsfxkz89h2tisfrdyws5oakh/gauwja8kz7quzagagp4hpsjdluerwh5c5tvuavsdbuc/aa85xnlqyqnqvejhzosihhwzm/nv5m05bfqavrqmtbcbgzkccpyooikyqwedokfdbvncihu242pw2mj7xsw0es9ysat9xfexkjlrqfvk6gbazq3pui/3ndfikam/eosyffdlyosawgge96wj0ugqux0orqyfvfmr0qssjonou6bhv+5exsoqnavtw+pi3va3kdbxsijlaz2egm1delaqamougrpanx9gy8gs/gu/exiy4z+cwb+gpj6wcauz5i</latexit> <latexit sha1_base64="xueh2vomh7cx8tkkop/dsmc0u7m=">aaacgnicbvc7sgnbfj31genr1djmmahje3zfmi0qslgwigaekf2x2clsmmt2wcxdmaz5dht/xczcetux8w+cjfto4ogbwznncucepxfcgwv9g0vlk6tr64wn4ubw9s6uubffuneqkwvswmsy4xpfbi9yezgi1kkki6evwnsfxkz89h2tisfrdyws5oakh/gauwja8kz7quzagagp4hpsjdluerwh5c5tvuavsdbuc/aa85xnlqyqnqvejhzosihhwzm/nv5m05bfqavrqmtbcbgzkccpyooikyqwedokfdbvncihu242pw2mj7xsw0es9ysat9xfexkjlrqfvk6gbazq3pui/3ndfikam/eosyffdlyosawgge96wj0ugqux0orqyfvfmr0qssjonou6bhv+5exsoqnavtw+pi3va3kdbxsijlaz2egm1delaqamougrpanx9gy8gs/gu/exiy4z+cwb+gpj6wcauz5i</latexit> <latexit sha1_base64="jv06+ctdcbpbq8syhnkvx0bw2uo=">aaacbnicbzdlssnafiynxmu9rv2kmfiedlmsexshuhdjsok9qbvlzdpph04myezellern76kgxekupuz3pk2ttsstpwhgy//nmoz8/ux4boc59tawfxaxlnnrexxnza3tu2d3bqoekvzjuyiuk2faca4zdxgifgzvoyevmanf3a5rjfumni8kjcwjjkxkp7kaacejnwxd4ji8zyt8tiix+d7jb5wg/omskljf5yymxgebzedaspu7dhf7w5ek5bjoijo3xkdglyukobusfg+nwgwezogpdyykenitjdozhjhi+n0crap8ytgift7iiwh1spqn50hgb6ery3n/2qtbiizl+uytobjol0ujajdhmez4c5xjiiygibucfnxtpteeqomubwjwz09er7qx2xxkbvxj4xkerzhdu2jq1reljpffxsfqqigkhpez+gvvvlp1ov1bn1mwxesbgyp/zh1+qnug5cz</latexit> <latexit sha1_base64="jv06+ctdcbpbq8syhnkvx0bw2uo=">aaacbnicbzdlssnafiynxmu9rv2kmfiedlmsexshuhdjsok9qbvlzdpph04myezellern76kgxekupuz3pk2ttsstpwhgy//nmoz8/ux4boc59tawfxaxlnnrexxnza3tu2d3bqoekvzjuyiuk2faca4zdxgifgzvoyevmanf3a5rjfumni8kjcwjjkxkp7kaacejnwxd4ji8zyt8tiix+d7jb5wg/omskljf5yymxgebzedaspu7dhf7w5ek5bjoijo3xkdglyukobusfg+nwgwezogpdyykenitjdozhjhi+n0crap8ytgift7iiwh1spqn50hgb6ery3n/2qtbiizl+uytobjol0ujajdhmez4c5xjiiygibucfnxtpteeqomubwjwz09er7qx2xxkbvxj4xkerzhdu2jq1reljpffxsfqqigkhpez+gvvvlp1ov1bn1mwxesbgyp/zh1+qnug5cz</latexit> <latexit sha1_base64="jv06+ctdcbpbq8syhnkvx0bw2uo=">aaacbnicbzdlssnafiynxmu9rv2kmfiedlmsexshuhdjsok9qbvlzdpph04myezellern76kgxekupuz3pk2ttsstpwhgy//nmoz8/ux4boc59tawfxaxlnnrexxnza3tu2d3bqoekvzjuyiuk2faca4zdxgifgzvoyevmanf3a5rjfumni8kjcwjjkxkp7kaacejnwxd4ji8zyt8tiix+d7jb5wg/omskljf5yymxgebzedaspu7dhf7w5ek5bjoijo3xkdglyukobusfg+nwgwezogpdyykenitjdozhjhi+n0crap8ytgift7iiwh1spqn50hgb6ery3n/2qtbiizl+uytobjol0ujajdhmez4c5xjiiygibucfnxtpteeqomubwjwz09er7qx2xxkbvxj4xkerzhdu2jq1reljpffxsfqqigkhpez+gvvvlp1ov1bn1mwxesbgyp/zh1+qnug5cz</latexit> <latexit sha1_base64="jv06+ctdcbpbq8syhnkvx0bw2uo=">aaacbnicbzdlssnafiynxmu9rv2kmfiedlmsexshuhdjsok9qbvlzdpph04myezellern76kgxekupuz3pk2ttsstpwhgy//nmoz8/ux4boc59tawfxaxlnnrexxnza3tu2d3bqoekvzjuyiuk2faca4zdxgifgzvoyevmanf3a5rjfumni8kjcwjjkxkp7kaacejnwxd4ji8zyt8tiix+d7jb5wg/omskljf5yymxgebzedaspu7dhf7w5ek5bjoijo3xkdglyukobusfg+nwgwezogpdyykenitjdozhjhi+n0crap8ytgift7iiwh1spqn50hgb6ery3n/2qtbiizl+uytobjol0ujajdhmez4c5xjiiygibucfnxtpteeqomubwjwz09er7qx2xxkbvxj4xkerzhdu2jq1reljpffxsfqqigkhpez+gvvvlp1ov1bn1mwxesbgyp/zh1+qnug5cz</latexit> Maximum Likelihood Estimator You observe n datapoints: X (i) x (1),...,x (n) <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> Think: observations of n IID random variables: Where: has likelihood (PDF) function: Likelihood of data L( ) = Y i X (1),...,X (n) <latexit sha1_base64="8ecrizo7nk5izr4/gabdstbkcjw=">aaacahicbvdlsgmxfl3js9bxqasxbojfakgugrf04algxmuf+4b2ljk004zmmkosecowg3/fjqtf3poz7vwb03yw2nogchlovte5x485u9pxvq2v1bx1jc3cvnf7z3dv3z44bkkokyq2scqj2fgxopwj2trmc9qjjcwhz2nbh99m/fyjlypf4l5pyuqfechywajwrurbx52htoxwsirqdsktqmh6f5wsb5ecmjmdwizutkqqo9g3v8wakoruamkxul3xibwxyqkz4tqr9hjfy0zgeei7hgocuuwlswuydgauaqoiay7qakb+7khxqnqk9e1lipvilxpt8t+vm+jgykuzibnnbzk/fcqc6qhn00adjinrfgiijpkzvyiywhitbtirmhdcxzwxseu85jo19+6ivl/o4yjaczxcgvy4hdrcqgoaqccdz3ifn+vjerhery956yqv9xzbh1ifpzgxlny=</latexit> <latexit sha1_base64="8ecrizo7nk5izr4/gabdstbkcjw=">aaacahicbvdlsgmxfl3js9bxqasxbojfakgugrf04algxmuf+4b2ljk004zmmkosecowg3/fjqtf3poz7vwb03yw2nogchlovte5x485u9pxvq2v1bx1jc3cvnf7z3dv3z44bkkokyq2scqj2fgxopwj2trmc9qjjcwhz2nbh99m/fyjlypf4l5pyuqfechywajwrurbx52htoxwsirqdsktqmh6f5wsb5ecmjmdwizutkqqo9g3v8wakoruamkxul3xibwxyqkz4tqr9hjfy0zgeei7hgocuuwlswuydgauaqoiay7qakb+7khxqnqk9e1lipvilxpt8t+vm+jgykuzibnnbzk/fcqc6qhn00adjinrfgiijpkzvyiywhitbtirmhdcxzwxseu85jo19+6ivl/o4yjaczxcgvy4hdrcqgoaqccdz3ifn+vjerhery956yqv9xzbh1ifpzgxlny=</latexit> <latexit sha1_base64="8ecrizo7nk5izr4/gabdstbkcjw=">aaacahicbvdlsgmxfl3js9bxqasxbojfakgugrf04algxmuf+4b2ljk004zmmkosecowg3/fjqtf3poz7vwb03yw2nogchlovte5x485u9pxvq2v1bx1jc3cvnf7z3dv3z44bkkokyq2scqj2fgxopwj2trmc9qjjcwhz2nbh99m/fyjlypf4l5pyuqfechywajwrurbx52htoxwsirqdsktqmh6f5wsb5ecmjmdwizutkqqo9g3v8wakoruamkxul3xibwxyqkz4tqr9hjfy0zgeei7hgocuuwlswuydgauaqoiay7qakb+7khxqnqk9e1lipvilxpt8t+vm+jgykuzibnnbzk/fcqc6qhn00adjinrfgiijpkzvyiywhitbtirmhdcxzwxseu85jo19+6ivl/o4yjaczxcgvy4hdrcqgoaqccdz3ifn+vjerhery956yqv9xzbh1ifpzgxlny=</latexit> <latexit sha1_base64="8ecrizo7nk5izr4/gabdstbkcjw=">aaacahicbvdlsgmxfl3js9bxqasxbojfakgugrf04algxmuf+4b2ljk004zmmkosecowg3/fjqtf3poz7vwb03yw2nogchlovte5x485u9pxvq2v1bx1jc3cvnf7z3dv3z44bkkokyq2scqj2fgxopwj2trmc9qjjcwhz2nbh99m/fyjlypf4l5pyuqfechywajwrurbx52htoxwsirqdsktqmh6f5wsb5ecmjmdwizutkqqo9g3v8wakoruamkxul3xibwxyqkz4tqr9hjfy0zgeei7hgocuuwlswuydgauaqoiay7qakb+7khxqnqk9e1lipvilxpt8t+vm+jgykuzibnnbzk/fcqc6qhn00adjinrfgiijpkzvyiywhitbtirmhdcxzwxseu85jo19+6ivl/o4yjaczxcgvy4hdrcqgoaqccdz3ifn+vjerhery956yqv9xzbh1ifpzgxlny=</latexit> f(x (i) = x (i) ) f(x (i) = x (i) ) Log Likelihood LL( ) = X i log f(x (i) = x (i) ) ˆ MLE = argmax LL( ) You have now estimated parameters Max Likelihood

5 Side Plot argmax argmax of log Gradient Ascent Mother of optimizations?

6 Linear Regression (simple) X = CO 2 level Y = Average Global Temperature N training datapoints (x (1),y (1) ), (x (2),y (2) ),...(x (n),y (n) ) Linear Regression Lite Model Y = X + Z Z N(0, 2 ) Y X N( X, 2 )

7 <latexit sha1_base64="w1jf0sjduy9hrz8xxchawj45ysm=">aaactxicbzhpsxtbfmdny6sxtjbqszdhg7a5noykuc+c6kwhhcw0jpbnwuxknhmcnv1m3oph2x/qi+dn/6ixd4qik2rl648ha18+3/eyn98juykmet6tu1n58hf1rbpe2/j0efnlfwv7zcszzrzdepnoxkgnl0lxdgquvjdqtunq8m54fjl3uxdcg5go3zhl+scmeyuiwshankqpg0htlgcp1siohhbbdxdkktah+eexqibdgmbk8sgxh34xvab74m6gusuabxwvu+byczrg/lwjesnreyuct8ivryoudtqq3wtjhguxv8gknabveyko8vk2tpkifmsgp5sd0wnvw6lozm0gx6rrwk4ly4gsby9cwnd/j3iagzolq9szu5ya194cvuf1m4wobrlqayzcsevfusybe5hhc2ohoum5s4iyleyuwkbuxov2a2o2bp/1k9+ks72w77x8x/uno+myjir5sr4rl/jkbzkip8kp6rbgrsgfck8engvnznl0npatfaec2sevqrl2dexksw8=</latexit> <latexit sha1_base64="w1jf0sjduy9hrz8xxchawj45ysm=">aaactxicbzhpsxtbfmdny6sxtjbqszdhg7a5noykuc+c6kwhhcw0jpbnwuxknhmcnv1m3oph2x/qi+dn/6ixd4qik2rl648ha18+3/eyn98juykmet6tu1n58hf1rbpe2/j0efnlfwv7zcszzrzdepnoxkgnl0lxdgquvjdqtunq8m54fjl3uxdcg5go3zhl+scmeyuiwshankqpg0htlgcp1siohhbbdxdkktah+eexqibdgmbk8sgxh34xvab74m6gusuabxwvu+byczrg/lwjesnreyuct8ivryoudtqq3wtjhguxv8gknabveyko8vk2tpkifmsgp5sd0wnvw6lozm0gx6rrwk4ly4gsby9cwnd/j3iagzolq9szu5ya194cvuf1m4wobrlqayzcsevfusybe5hhc2ohoum5s4iyleyuwkbuxov2a2o2bp/1k9+ks72w77x8x/uno+myjir5sr4rl/jkbzkip8kp6rbgrsgfck8engvnznl0npatfaec2sevqrl2dexksw8=</latexit> <latexit sha1_base64="w1jf0sjduy9hrz8xxchawj45ysm=">aaactxicbzhpsxtbfmdny6sxtjbqszdhg7a5noykuc+c6kwhhcw0jpbnwuxknhmcnv1m3oph2x/qi+dn/6ixd4qik2rl648ha18+3/eyn98juykmet6tu1n58hf1rbpe2/j0efnlfwv7zcszzrzdepnoxkgnl0lxdgquvjdqtunq8m54fjl3uxdcg5go3zhl+scmeyuiwshankqpg0htlgcp1siohhbbdxdkktah+eexqibdgmbk8sgxh34xvab74m6gusuabxwvu+byczrg/lwjesnreyuct8ivryoudtqq3wtjhguxv8gknabveyko8vk2tpkifmsgp5sd0wnvw6lozm0gx6rrwk4ly4gsby9cwnd/j3iagzolq9szu5ya194cvuf1m4wobrlqayzcsevfusybe5hhc2ohoum5s4iyleyuwkbuxov2a2o2bp/1k9+ks72w77x8x/uno+myjir5sr4rl/jkbzkip8kp6rbgrsgfck8engvnznl0npatfaec2sevqrl2dexksw8=</latexit> <latexit sha1_base64="w1jf0sjduy9hrz8xxchawj45ysm=">aaactxicbzhpsxtbfmdny6sxtjbqszdhg7a5noykuc+c6kwhhcw0jpbnwuxknhmcnv1m3oph2x/qi+dn/6ixd4qik2rl648ha18+3/eyn98juykmet6tu1n58hf1rbpe2/j0efnlfwv7zcszzrzdepnoxkgnl0lxdgquvjdqtunq8m54fjl3uxdcg5go3zhl+scmeyuiwshankqpg0htlgcp1siohhbbdxdkktah+eexqibdgmbk8sgxh34xvab74m6gusuabxwvu+byczrg/lwjesnreyuct8ivryoudtqq3wtjhguxv8gknabveyko8vk2tpkifmsgp5sd0wnvw6lozm0gx6rrwk4ly4gsby9cwnd/j3iagzolq9szu5ya194cvuf1m4wobrlqayzcsevfusybe5hhc2ohoum5s4iyleyuwkbuxov2a2o2bp/1k9+ks72w77x8x/uno+myjir5sr4rl/jkbzkip8kp6rbgrsgfck8engvnznl0npatfaec2sevqrl2dexksw8=</latexit> Linear Regression (simple) N training datapoints: (x (1),y (1) ), (x (2),y (2) ),...(x (n),y (n) ) ˆ = = nx i=1 nx (y (i) x (i) ) 2 i=1 2(y (i) x (i) )(x (i) )

8 Linear Regression (simple) Initialize: θ = 0 Repeat many times: gradient = 0 For each training example (x, y): gradient += 2(y - θ x)(x)! += h * gradient

9 Linear Regression (regular) X 1 = Temperature X 2 = Elevation X 3 = CO 2 level yesterday X 4 = GDP of region X 5 = Acres of forest growth Y = CO 2 levels

10 <latexit sha1_base64="op4au8hhvbhdfsncb+88r15unku=">aaacqnicbvbns8naen34wetx1aoxxaiiqkle0ioc4mvjbavvppbndmixd5owoxfl6g/z4i/w5g/w4kerrx7cpd34ntdm2zdvmnkxjfiydn0nz2x8ynjqujrtnp2bx1islc2fmtjvhbo8lrfubsyafbe0ukcezqkbqudcexbzlpfpb0ebeuen2e+grdh1jelbgvqqu7m4oowna+pjd5b1pnq0uux9bowmr0nawvrz5yx1/uidxljr+sqwoeiqox1ymdxhegbnqshtvkpuzs2c/gxecftjkoqdyqndy1mfexljjgl5boltjgkuxmkg7kcgeszv2dw0liyyatpocgsgdn0yxrrg2maetgc/t2rmmfxiq1qme+z3lyf/67vsdpfamyisfchiw0vhkingnpetdougjrjvaena2fsp7zhnofrxy9ye7/ex/4kz7zrn1rytnerh/sioelkla2stegsxhjjjuicnwsk9esav5m15cf6cd+djkb1zrjmr5ec4n193r6y0</latexit> <latexit sha1_base64="op4au8hhvbhdfsncb+88r15unku=">aaacqnicbvbns8naen34wetx1aoxxaiiqkle0ioc4mvjbavvppbndmixd5owoxfl6g/z4i/w5g/w4kerrx7cpd34ntdm2zdvmnkxjfiydn0nz2x8ynjqujrtnp2bx1islc2fmtjvhbo8lrfubsyafbe0ukcezqkbqudcexbzlpfpb0ebeuen2e+grdh1jelbgvqqu7m4oowna+pjd5b1pnq0uux9bowmr0nawvrz5yx1/uidxljr+sqwoeiqox1ymdxhegbnqshtvkpuzs2c/gxecftjkoqdyqndy1mfexljjgl5boltjgkuxmkg7kcgeszv2dw0liyyatpocgsgdn0yxrrg2maetgc/t2rmmfxiq1qme+z3lyf/67vsdpfamyisfchiw0vhkingnpetdougjrjvaena2fsp7zhnofrxy9ye7/ex/4kz7zrn1rytnerh/sioelkla2stegsxhjjjuicnwsk9esav5m15cf6cd+djkb1zrjmr5ec4n193r6y0</latexit> <latexit sha1_base64="op4au8hhvbhdfsncb+88r15unku=">aaacqnicbvbns8naen34wetx1aoxxaiiqkle0ioc4mvjbavvppbndmixd5owoxfl6g/z4i/w5g/w4kerrx7cpd34ntdm2zdvmnkxjfiydn0nz2x8ynjqujrtnp2bx1islc2fmtjvhbo8lrfubsyafbe0ukcezqkbqudcexbzlpfpb0ebeuen2e+grdh1jelbgvqqu7m4oowna+pjd5b1pnq0uux9bowmr0nawvrz5yx1/uidxljr+sqwoeiqox1ymdxhegbnqshtvkpuzs2c/gxecftjkoqdyqndy1mfexljjgl5boltjgkuxmkg7kcgeszv2dw0liyyatpocgsgdn0yxrrg2maetgc/t2rmmfxiq1qme+z3lyf/67vsdpfamyisfchiw0vhkingnpetdougjrjvaena2fsp7zhnofrxy9ye7/ex/4kz7zrn1rytnerh/sioelkla2stegsxhjjjuicnwsk9esav5m15cf6cd+djkb1zrjmr5ec4n193r6y0</latexit> <latexit sha1_base64="op4au8hhvbhdfsncb+88r15unku=">aaacqnicbvbns8naen34wetx1aoxxaiiqkle0ioc4mvjbavvppbndmixd5owoxfl6g/z4i/w5g/w4kerrx7cpd34ntdm2zdvmnkxjfiydn0nz2x8ynjqujrtnp2bx1islc2fmtjvhbo8lrfubsyafbe0ukcezqkbqudcexbzlpfpb0ebeuen2e+grdh1jelbgvqqu7m4oowna+pjd5b1pnq0uux9bowmr0nawvrz5yx1/uidxljr+sqwoeiqox1ymdxhegbnqshtvkpuzs2c/gxecftjkoqdyqndy1mfexljjgl5boltjgkuxmkg7kcgeszv2dw0liyyatpocgsgdn0yxrrg2maetgc/t2rmmfxiq1qme+z3lyf/67vsdpfamyisfchiw0vhkingnpetdougjrjvaena2fsp7zhnofrxy9ye7/ex/4kz7zrn1rytnerh/sioelkla2stegsxhjjjuicnwsk9esav5m15cf6cd+djkb1zrjmr5ec4n193r6y0</latexit> Linear Regression (regular) Problem: Predict real value Y based on observing variable X Model: Linear weight every feature Y = 1 X m X m + Z = T X + Z Training: Gradient ascent to chose the best thetas to describe your data ˆ MLE = argmax n X i=1 (y (i) x (i) ) 2

11 Linear Regression (regular) Initialize: θ j = 0 for all 0 j m Repeat many times: gradient[j] = 0 for all 0 j m For each training example (x, y): For each parameter j: gradient[j] += (y θ T x)(-x[j])! j += h * gradient[j] for all 0 j m

12 <latexit sha1_base64="op4au8hhvbhdfsncb+88r15unku=">aaacqnicbvbns8naen34wetx1aoxxaiiqkle0ioc4mvjbavvppbndmixd5owoxfl6g/z4i/w5g/w4kerrx7cpd34ntdm2zdvmnkxjfiydn0nz2x8ynjqujrtnp2bx1islc2fmtjvhbo8lrfubsyafbe0ukcezqkbqudcexbzlpfpb0ebeuen2e+grdh1jelbgvqqu7m4oowna+pjd5b1pnq0uux9bowmr0nawvrz5yx1/uidxljr+sqwoeiqox1ymdxhegbnqshtvkpuzs2c/gxecftjkoqdyqndy1mfexljjgl5boltjgkuxmkg7kcgeszv2dw0liyyatpocgsgdn0yxrrg2maetgc/t2rmmfxiq1qme+z3lyf/67vsdpfamyisfchiw0vhkingnpetdougjrjvaena2fsp7zhnofrxy9ye7/ex/4kz7zrn1rytnerh/sioelkla2stegsxhjjjuicnwsk9esav5m15cf6cd+djkb1zrjmr5ec4n193r6y0</latexit> <latexit sha1_base64="op4au8hhvbhdfsncb+88r15unku=">aaacqnicbvbns8naen34wetx1aoxxaiiqkle0ioc4mvjbavvppbndmixd5owoxfl6g/z4i/w5g/w4kerrx7cpd34ntdm2zdvmnkxjfiydn0nz2x8ynjqujrtnp2bx1islc2fmtjvhbo8lrfubsyafbe0ukcezqkbqudcexbzlpfpb0ebeuen2e+grdh1jelbgvqqu7m4oowna+pjd5b1pnq0uux9bowmr0nawvrz5yx1/uidxljr+sqwoeiqox1ymdxhegbnqshtvkpuzs2c/gxecftjkoqdyqndy1mfexljjgl5boltjgkuxmkg7kcgeszv2dw0liyyatpocgsgdn0yxrrg2maetgc/t2rmmfxiq1qme+z3lyf/67vsdpfamyisfchiw0vhkingnpetdougjrjvaena2fsp7zhnofrxy9ye7/ex/4kz7zrn1rytnerh/sioelkla2stegsxhjjjuicnwsk9esav5m15cf6cd+djkb1zrjmr5ec4n193r6y0</latexit> <latexit sha1_base64="op4au8hhvbhdfsncb+88r15unku=">aaacqnicbvbns8naen34wetx1aoxxaiiqkle0ioc4mvjbavvppbndmixd5owoxfl6g/z4i/w5g/w4kerrx7cpd34ntdm2zdvmnkxjfiydn0nz2x8ynjqujrtnp2bx1islc2fmtjvhbo8lrfubsyafbe0ukcezqkbqudcexbzlpfpb0ebeuen2e+grdh1jelbgvqqu7m4oowna+pjd5b1pnq0uux9bowmr0nawvrz5yx1/uidxljr+sqwoeiqox1ymdxhegbnqshtvkpuzs2c/gxecftjkoqdyqndy1mfexljjgl5boltjgkuxmkg7kcgeszv2dw0liyyatpocgsgdn0yxrrg2maetgc/t2rmmfxiq1qme+z3lyf/67vsdpfamyisfchiw0vhkingnpetdougjrjvaena2fsp7zhnofrxy9ye7/ex/4kz7zrn1rytnerh/sioelkla2stegsxhjjjuicnwsk9esav5m15cf6cd+djkb1zrjmr5ec4n193r6y0</latexit> <latexit sha1_base64="op4au8hhvbhdfsncb+88r15unku=">aaacqnicbvbns8naen34wetx1aoxxaiiqkle0ioc4mvjbavvppbndmixd5owoxfl6g/z4i/w5g/w4kerrx7cpd34ntdm2zdvmnkxjfiydn0nz2x8ynjqujrtnp2bx1islc2fmtjvhbo8lrfubsyafbe0ukcezqkbqudcexbzlpfpb0ebeuen2e+grdh1jelbgvqqu7m4oowna+pjd5b1pnq0uux9bowmr0nawvrz5yx1/uidxljr+sqwoeiqox1ymdxhegbnqshtvkpuzs2c/gxecftjkoqdyqndy1mfexljjgl5boltjgkuxmkg7kcgeszv2dw0liyyatpocgsgdn0yxrrg2maetgc/t2rmmfxiq1qme+z3lyf/67vsdpfamyisfchiw0vhkingnpetdougjrjvaena2fsp7zhnofrxy9ye7/ex/4kz7zrn1rytnerh/sioelkla2stegsxhjjjuicnwsk9esav5m15cf6cd+djkb1zrjmr5ec4n193r6y0</latexit> Predicting Warriors Y = Warriors points Y = 1 X m X m + Z = T X + Z X 1 = Opposing team ELO X 2 = Points in last game X 3 = Curry playing? X 4 = Playing at home? X 5 = 1! 1 = -2.3! 2 = +1.2! 3 = +10.2! 4 = +3.3! 5 = +95.4

13 Supervised Learning Real World Problem Model the problem Formal Model! Training Data Learning Algorithm New Data Prediction Function! * Predictions

14 Modelling Real World Problem Model the problem Formal Model! Training Data Learning Algorithm New Data Prediction Function! * Predictions

15 Training Real World Problem Model the problem Formal Model! Training Data Learning Algorithm New Data Prediction Function! * Predictions

16 Make Predictions Real World Problem Model the problem Formal Model! Training Data Learning Algorithm New Data Prediction Function! * Predictions

17 Our Path Neural Networks Linear Regression Naïve Bayes Logistic Regression Unbiased estimators Maximizing likelihood Bayesian estimation

18 Episode 2 The Song of The Last Estimator

19 Something rotten in the world of MLE

20 Foreshadowing..

21 Need a Volunteer So good to see you again!

22 I have two envelopes, will allow you to have one One contains $X, the other contains $2X Select an envelope o Open it! Now, would you like to switch for other envelope? To help you decide, compute E[$ in other envelope] o Let Y = $ in envelope you selected 1 Y 1 E $in other envelope] = Before opening envelope, think either equally good So, what happened by opening envelope? o Two Envelopes [ Y = And does it really make sense to switch? 5 4 Y

23 Thinking Deeper About Two Envelopes The two envelopes problem set-up Two envelopes: one contains $X, other contains $2X You select an envelope and open it o o Let Y = $ in envelope you selected Let Z = $ in other envelope 1 Y 1 5 E[ Z Y] = Y = 4 Y E[Z Y] above assumes all values X (where 0 < X < ) are equally likely o Note: there are infinitely many values of X o So, not true probability distribution over X (doesn t integrate to 1)

24 All Values are Equally Likely? p(x) Infinite powers of two X

25 Subjectivity of Probability Belief about contents of envelopes Since implied distribution over X is not a true probability distribution, what is our distribution over X? o o Frequentist: play game infinitely many times and see how often different values come up. Problem: I only allow you to play the game once Bayesian probability o o o o Have prior belief of distribution for X (or anything for that matter) Prior belief is a subjective probability By extension, all probabilities are subjective Allows us to answer question when we have no/limited data E.g., probability a coin you ve never flipped lands on heads As we get more data, prior belief is swamped by data

26 Subjectivity of Probability p(x) X

27 The Envelope, Please Bayesian: have prior distribution over X, P(X) Let Y = $ in envelope you selected Let Z = $ in other envelope Open your envelope to determine Y If Y > E[Z Y], keep your envelope, otherwise switch o No inconsistency! Opening envelope provides data to compute P(X Y) and thereby compute E[Z Y] Of course, there s the issue of how you determined your prior distribution over X o Bayesian: Doesn t matter how you determined prior, but you must have one (whatever it is) o Imagine if envelope you opened contained $20.01

28 Envelope Summary: Probabilities are beliefs Incorporating prior beliefs is useful

29 Priors for Parameter Estimation?

30 Flash Back: Bayes Theorem Bayes Theorem (q = model parameters, D = data): Posterior Likelihood Prior Likelihood: you ve seen this before (in context of MLE) o Probability of data given probability model (parameter q) Prior: before seeing any data, what is belief about model o I.e., what is distribution over parameters q Posterior: after seeing data, what is belief about model o P(q D) = P(D q) P(q) P(D) After data D observed, have posterior distribution p(q D) over parameters q conditioned on data. Use this to predict new data.

31 <latexit sha1_base64="cthwmp611gyf340xxnllj+lhflg=">aaaccnicbvbdi9nafj3er7v+vcuxbr0tshdksurwx4qvx3wrkmx3c00tn9obdtjjtji5kzayh+df881f4ys/wek2llrrgyhdoffm3dlpoasjkpozhfeuxrt+y+9m79bto3fv9e8/ohamtaknwihjzyk4vfljlcqpnbuwiu8vnqsnhxr/5ctaj40+ol2bixzwwmzsahlp2f+ebicqhdziuc+rt+8nnx/1jielxvkunltvygq0qmzqylecu85hw9ecj4rbqniddzojzpl7cbvh3/jsszwm92svbc/yiccrq27uovrcadj+sj+ixlelfpnehrmwdpnl/4e/tjq5ahiknjvhuuelvxpjobdujaxdasqprhhuabo3w1rtztv/6zuvz4z1rxnv1b8tfeto7flut+zag3fra8t/efossonfjxvrempx9lbwkk6gn/3zlbqoso08awgl35wldvgq5pvu+rlii1++ti5fj+nohh9+mzg86oryy0/yczzkmxvldtlhnmftjtiv4fhwnhgw/a4fh8/drrsw6dip2t8ir38albo7vq==</latexit> <latexit sha1_base64="cthwmp611gyf340xxnllj+lhflg=">aaaccnicbvbdi9nafj3er7v+vcuxbr0tshdksurwx4qvx3wrkmx3c00tn9obdtjjtji5kzayh+df881f4ys/wek2llrrgyhdoffm3dlpoasjkpozhfeuxrt+y+9m79bto3fv9e8/ohamtaknwihjzyk4vfljlcqpnbuwiu8vnqsnhxr/5ctaj40+ol2bixzwwmzsahlp2f+ebicqhdziuc+rt+8nnx/1jielxvkunltvygq0qmzqylecu85hw9ecj4rbqniddzojzpl7cbvh3/jsszwm92svbc/yiccrq27uovrcadj+sj+ixlelfpnehrmwdpnl/4e/tjq5ahiknjvhuuelvxpjobdujaxdasqprhhuabo3w1rtztv/6zuvz4z1rxnv1b8tfeto7flut+zag3fra8t/efossonfjxvrempx9lbwkk6gn/3zlbqoso08awgl35wldvgq5pvu+rlii1++ti5fj+nohh9+mzg86oryy0/yczzkmxvldtlhnmftjtiv4fhwnhgw/a4fh8/drrsw6dip2t8ir38albo7vq==</latexit> <latexit sha1_base64="cthwmp611gyf340xxnllj+lhflg=">aaaccnicbvbdi9nafj3er7v+vcuxbr0tshdksurwx4qvx3wrkmx3c00tn9obdtjjtji5kzayh+df881f4ys/wek2llrrgyhdoffm3dlpoasjkpozhfeuxrt+y+9m79bto3fv9e8/ohamtaknwihjzyk4vfljlcqpnbuwiu8vnqsnhxr/5ctaj40+ol2bixzwwmzsahlp2f+ebicqhdziuc+rt+8nnx/1jielxvkunltvygq0qmzqylecu85hw9ecj4rbqniddzojzpl7cbvh3/jsszwm92svbc/yiccrq27uovrcadj+sj+ixlelfpnehrmwdpnl/4e/tjq5ahiknjvhuuelvxpjobdujaxdasqprhhuabo3w1rtztv/6zuvz4z1rxnv1b8tfeto7flut+zag3fra8t/efossonfjxvrempx9lbwkk6gn/3zlbqoso08awgl35wldvgq5pvu+rlii1++ti5fj+nohh9+mzg86oryy0/yczzkmxvldtlhnmftjtiv4fhwnhgw/a4fh8/drrsw6dip2t8ir38albo7vq==</latexit> <latexit sha1_base64="cthwmp611gyf340xxnllj+lhflg=">aaaccnicbvbdi9nafj3er7v+vcuxbr0tshdksurwx4qvx3wrkmx3c00tn9obdtjjtji5kzayh+df881f4ys/wek2llrrgyhdoffm3dlpoasjkpozhfeuxrt+y+9m79bto3fv9e8/ohamtaknwihjzyk4vfljlcqpnbuwiu8vnqsnhxr/5ctaj40+ol2bixzwwmzsahlp2f+ebicqhdziuc+rt+8nnx/1jielxvkunltvygq0qmzqylecu85hw9ecj4rbqniddzojzpl7cbvh3/jsszwm92svbc/yiccrq27uovrcadj+sj+ixlelfpnehrmwdpnl/4e/tjq5ahiknjvhuuelvxpjobdujaxdasqprhhuabo3w1rtztv/6zuvz4z1rxnv1b8tfeto7flut+zag3fra8t/efossonfjxvrempx9lbwkk6gn/3zlbqoso08awgl35wldvgq5pvu+rlii1++ti5fj+nohh9+mzg86oryy0/yczzkmxvldtlhnmftjtiv4fhwnhgw/a4fh8/drrsw6dip2t8ir38albo7vq==</latexit> <latexit sha1_base64="ausk+mvq/05/+ztpcugmo3jxx+k=">aaac1niclvhlihnbfk1ux2n8rv26uriubibqpqjozmbqbbcki5qzqco21zvkp5jqqqbqtis07uirt36boz/cf7a6hcm448ylbydz7q1zh2mhpmmo+hmely5fuxpt53rnxs1bt dqa0xiy4ucaou+aeklqmukis48iklqdknksnzxv95iowthr9dlefmoys03iuounpjd1fdmgworgqyoqkev3qrq2pd4cweuarxfaqqcmezwiszrmomm1ytqxraipiirxu8/74fdwpbzucwljfu0bnbt0unirekg36co2/mkc08z9+f+yappnzh6gr80qcvsadtgo59zfnfabr/ehs7uxdab1wecqb0coboeq6p/wivmyfrq6yc5m4kndq20hjlag7thsiypyuzwliydorm1brs9twydmzmbvrn0zys2crkpy7t8ptn5kzxljzwkp+s5uuon+fvlixjqrnw6n5qqanndegmbsco1p5wlivvlfgc2yzr7/jjl9cfh7ki+b4bxhhw/jnk97h/mydo+qbeuj6jczpysf5sy7iipdgbbakpgdfwnh4kfwafmttw2btc5/8feh33x8v4u0=</latexit> <latexit sha1_base64="ausk+mvq/05/+ztpcugmo3jxx+k=">aaac1niclvhlihnbfk1ux2n8rv26uriubibqpqjozmbqbbcki5qzqco21zvkp5jqqqbqtis07uirt36boz/cf7a6hcm448ylbydz7q1zh2mhpmmo+hmely5fuxpt53rnxs1bt dqa0xiy4ucaou+aeklqmukis48iklqdknksnzxv95iowthr9dlefmoys03iuounpjd1fdmgworgqyoqkev3qrq2pd4cweuarxfaqqcmezwiszrmomm1ytqxraipiirxu8/74fdwpbzucwljfu0bnbt0unirekg36co2/mkc08z9+f+yappnzh6gr80qcvsadtgo59zfnfabr/ehs7uxdab1wecqb0coboeq6p/wivmyfrq6yc5m4kndq20hjlag7thsiypyuzwliydorm1brs9twydmzmbvrn0zys2crkpy7t8ptn5kzxljzwkp+s5uuon+fvlixjqrnw6n5qqanndegmbsco1p5wlivvlfgc2yzr7/jjl9cfh7ki+b4bxhhw/jnk97h/mydo+qbeuj6jczpysf5sy7iipdgbbakpgdfwnh4kfwafmttw2btc5/8feh33x8v4u0=</latexit> <latexit sha1_base64="ausk+mvq/05/+ztpcugmo3jxx+k=">aaac1niclvhlihnbfk1ux2n8rv26uriubibqpqjozmbqbbcki5qzqco21zvkp5jqqqbqtis07uirt36boz/cf7a6hcm448ylbydz7q1zh2mhpmmo+hmely5fuxpt53rnxs1bt dqa0xiy4ucaou+aeklqmukis48iklqdknksnzxv95iowthr9dlefmoys03iuounpjd1fdmgworgqyoqkev3qrq2pd4cweuarxfaqqcmezwiszrmomm1ytqxraipiirxu8/74fdwpbzucwljfu0bnbt0unirekg36co2/mkc08z9+f+yappnzh6gr80qcvsadtgo59zfnfabr/ehs7uxdab1wecqb0coboeq6p/wivmyfrq6yc5m4kndq20hjlag7thsiypyuzwliydorm1brs9twydmzmbvrn0zys2crkpy7t8ptn5kzxljzwkp+s5uuon+fvlixjqrnw6n5qqanndegmbsco1p5wlivvlfgc2yzr7/jjl9cfh7ki+b4bxhhw/jnk97h/mydo+qbeuj6jczpysf5sy7iipdgbbakpgdfwnh4kfwafmttw2btc5/8feh33x8v4u0=</latexit> <latexit sha1_base64="ausk+mvq/05/+ztpcugmo3jxx+k=">aaac1niclvhlihnbfk1ux2n8rv26uriubibqpqjozmbqbbcki5qzqco21zvkp5jqqqbqtis07uirt36boz/cf7a6hcm448ylbydz7q1zh2mhpmmo+hmely5fuxpt53rnxs1bt dqa0xiy4ucaou+aeklqmukis48iklqdknksnzxv95iowthr9dlefmoys03iuounpjd1fdmgworgqyoqkev3qrq2pd4cweuarxfaqqcmezwiszrmomm1ytqxraipiirxu8/74fdwpbzucwljfu0bnbt0unirekg36co2/mkc08z9+f+yappnzh6gr80qcvsadtgo59zfnfabr/ehs7uxdab1wecqb0coboeq6p/wivmyfrq6yc5m4kndq20hjlag7thsiypyuzwliydorm1brs9twydmzmbvrn0zys2crkpy7t8ptn5kzxljzwkp+s5uuon+fvlixjqrnw6n5qqanndegmbsco1p5wlivvlfgc2yzr7/jjl9cfh7ki+b4bxhhw/jnk97h/mydo+qbeuj6jczpysf5sy7iipdgbbakpgdfwnh4kfwafmttw2btc5/8feh33x8v4u0=</latexit> MLE vs MAP Data: x (1),...,x (n) <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> Maximum Likelihood Estimation ˆ MLE = argmax = argmax f(x (1) = x (1),...,X (n) = x (n) ) X log f(x (i) = x (i) ) i Maximum A Posteriori ˆ MAP = argmax f( = X (1) = x (1),...,X (n) = x (n) )

32 <latexit sha1_base64="cthwmp611gyf340xxnllj+lhflg=">aaaccnicbvbdi9nafj3er7v+vcuxbr0tshdksurwx4qvx3wrkmx3c00tn9obdtjjtji5kzayh+df881f4ys/wek2llrrgyhdoffm3dlpoasjkpozhfeuxrt+y+9m79bto3fv9e8/ohamtaknwihjzyk4vfljlcqpnbuwiu8vnqsnhxr/5ctaj40+ol2bixzwwmzsahlp2f+ebicqhdziuc+rt+8nnx/1jielxvkunltvygq0qmzqylecu85hw9ecj4rbqniddzojzpl7cbvh3/jsszwm92svbc/yiccrq27uovrcadj+sj+ixlelfpnehrmwdpnl/4e/tjq5ahiknjvhuuelvxpjobdujaxdasqprhhuabo3w1rtztv/6zuvz4z1rxnv1b8tfeto7flut+zag3fra8t/efossonfjxvrempx9lbwkk6gn/3zlbqoso08awgl35wldvgq5pvu+rlii1++ti5fj+nohh9+mzg86oryy0/yczzkmxvldtlhnmftjtiv4fhwnhgw/a4fh8/drrsw6dip2t8ir38albo7vq==</latexit> <latexit sha1_base64="cthwmp611gyf340xxnllj+lhflg=">aaaccnicbvbdi9nafj3er7v+vcuxbr0tshdksurwx4qvx3wrkmx3c00tn9obdtjjtji5kzayh+df881f4ys/wek2llrrgyhdoffm3dlpoasjkpozhfeuxrt+y+9m79bto3fv9e8/ohamtaknwihjzyk4vfljlcqpnbuwiu8vnqsnhxr/5ctaj40+ol2bixzwwmzsahlp2f+ebicqhdziuc+rt+8nnx/1jielxvkunltvygq0qmzqylecu85hw9ecj4rbqniddzojzpl7cbvh3/jsszwm92svbc/yiccrq27uovrcadj+sj+ixlelfpnehrmwdpnl/4e/tjq5ahiknjvhuuelvxpjobdujaxdasqprhhuabo3w1rtztv/6zuvz4z1rxnv1b8tfeto7flut+zag3fra8t/efossonfjxvrempx9lbwkk6gn/3zlbqoso08awgl35wldvgq5pvu+rlii1++ti5fj+nohh9+mzg86oryy0/yczzkmxvldtlhnmftjtiv4fhwnhgw/a4fh8/drrsw6dip2t8ir38albo7vq==</latexit> <latexit sha1_base64="cthwmp611gyf340xxnllj+lhflg=">aaaccnicbvbdi9nafj3er7v+vcuxbr0tshdksurwx4qvx3wrkmx3c00tn9obdtjjtji5kzayh+df881f4ys/wek2llrrgyhdoffm3dlpoasjkpozhfeuxrt+y+9m79bto3fv9e8/ohamtaknwihjzyk4vfljlcqpnbuwiu8vnqsnhxr/5ctaj40+ol2bixzwwmzsahlp2f+ebicqhdziuc+rt+8nnx/1jielxvkunltvygq0qmzqylecu85hw9ecj4rbqniddzojzpl7cbvh3/jsszwm92svbc/yiccrq27uovrcadj+sj+ixlelfpnehrmwdpnl/4e/tjq5ahiknjvhuuelvxpjobdujaxdasqprhhuabo3w1rtztv/6zuvz4z1rxnv1b8tfeto7flut+zag3fra8t/efossonfjxvrempx9lbwkk6gn/3zlbqoso08awgl35wldvgq5pvu+rlii1++ti5fj+nohh9+mzg86oryy0/yczzkmxvldtlhnmftjtiv4fhwnhgw/a4fh8/drrsw6dip2t8ir38albo7vq==</latexit> <latexit sha1_base64="cthwmp611gyf340xxnllj+lhflg=">aaaccnicbvbdi9nafj3er7v+vcuxbr0tshdksurwx4qvx3wrkmx3c00tn9obdtjjtji5kzayh+df881f4ys/wek2llrrgyhdoffm3dlpoasjkpozhfeuxrt+y+9m79bto3fv9e8/ohamtaknwihjzyk4vfljlcqpnbuwiu8vnqsnhxr/5ctaj40+ol2bixzwwmzsahlp2f+ebicqhdziuc+rt+8nnx/1jielxvkunltvygq0qmzqylecu85hw9ecj4rbqniddzojzpl7cbvh3/jsszwm92svbc/yiccrq27uovrcadj+sj+ixlelfpnehrmwdpnl/4e/tjq5ahiknjvhuuelvxpjobdujaxdasqprhhuabo3w1rtztv/6zuvz4z1rxnv1b8tfeto7flut+zag3fra8t/efossonfjxvrempx9lbwkk6gn/3zlbqoso08awgl35wldvgq5pvu+rlii1++ti5fj+nohh9+mzg86oryy0/yczzkmxvldtlhnmftjtiv4fhwnhgw/a4fh8/drrsw6dip2t8ir38albo7vq==</latexit> <latexit sha1_base64="lc8ldtvond31m6wsqdhic9ir3ow=">aaacj3icbvdlsgmxfm34tr6qlt0ei6cbmiociiicg5ck9gftlzn0jhpmjenyryxd/8anv+jgubfd+iem7sx83dxhpjj7t5hkydh3p7yjyanpmdm5+dlc4tlysnl1rw51zjjuujbanenmqqofnrqoozkayekoorhena71xi0yk7s6xh4knyrdkxejztbr3flx3vw+lxygti1whzkvltpyg4yz6tfig4oxulgfhyd0qjuf+ygwsw654lf90dc/ichahrrz3i0/t3uaz4l7ketmbsvwu+zkzkdgegaldmyhzfygxuplqcussj18doeabjlmvfwkfdir+z2rs8tafhi6z8iwtr+1ifmf1sowoujkqquzguljj6jmutr0wbrtcqmczd8bxo1wu1iem8m4umplrotg98l/qx23gvjv4gkvcnjq1dfhnsgm2syb2scn5iyckxrh5j48khfy6j14t96b9z62tnhfzp38go/zc+s+pky=</latexit> <latexit sha1_base64="lc8ldtvond31m6wsqdhic9ir3ow=">aaacj3icbvdlsgmxfm34tr6qlt0ei6cbmiociiicg5ck9gftlzn0jhpmjenyryxd/8anv+jgubfd+iem7sx83dxhpjj7t5hkydh3p7yjyanpmdm5+dlc4tlysnl1rw51zjjuujbanenmqqofnrqoozkayekoorhena71xi0yk7s6xh4knyrdkxejztbr3flx3vw+lxygti1whzkvltpyg4yz6tfig4oxulgfhyd0qjuf+ygwsw654lf90dc/ichahrrz3i0/t3uaz4l7ketmbsvwu+zkzkdgegaldmyhzfygxuplqcussj18doeabjlmvfwkfdir+z2rs8tafhi6z8iwtr+1ifmf1sowoujkqquzguljj6jmutr0wbrtcqmczd8bxo1wu1iem8m4umplrotg98l/qx23gvjv4gkvcnjq1dfhnsgm2syb2scn5iyckxrh5j48khfy6j14t96b9z62tnhfzp38go/zc+s+pky=</latexit> <latexit sha1_base64="lc8ldtvond31m6wsqdhic9ir3ow=">aaacj3icbvdlsgmxfm34tr6qlt0ei6cbmiociiicg5ck9gftlzn0jhpmjenyryxd/8anv+jgubfd+iem7sx83dxhpjj7t5hkydh3p7yjyanpmdm5+dlc4tlysnl1rw51zjjuujbanenmqqofnrqoozkayekoorhena71xi0yk7s6xh4knyrdkxejztbr3flx3vw+lxygti1whzkvltpyg4yz6tfig4oxulgfhyd0qjuf+ygwsw654lf90dc/ichahrrz3i0/t3uaz4l7ketmbsvwu+zkzkdgegaldmyhzfygxuplqcussj18doeabjlmvfwkfdir+z2rs8tafhi6z8iwtr+1ifmf1sowoujkqquzguljj6jmutr0wbrtcqmczd8bxo1wu1iem8m4umplrotg98l/qx23gvjv4gkvcnjq1dfhnsgm2syb2scn5iyckxrh5j48khfy6j14t96b9z62tnhfzp38go/zc+s+pky=</latexit> <latexit sha1_base64="lc8ldtvond31m6wsqdhic9ir3ow=">aaacj3icbvdlsgmxfm34tr6qlt0ei6cbmiociiicg5ck9gftlzn0jhpmjenyryxd/8anv+jgubfd+iem7sx83dxhpjj7t5hkydh3p7yjyanpmdm5+dlc4tlysnl1rw51zjjuujbanenmqqofnrqoozkayekoorhena71xi0yk7s6xh4knyrdkxejztbr3flx3vw+lxygti1whzkvltpyg4yz6tfig4oxulgfhyd0qjuf+ygwsw654lf90dc/ichahrrz3i0/t3uaz4l7ketmbsvwu+zkzkdgegaldmyhzfygxuplqcussj18doeabjlmvfwkfdir+z2rs8tafhi6z8iwtr+1ifmf1sowoujkqquzguljj6jmutr0wbrtcqmczd8bxo1wu1iem8m4umplrotg98l/qx23gvjv4gkvcnjq1dfhnsgm2syb2scn5iyckxrh5j48khfy6j14t96b9z62tnhfzp38go/zc+s+pky=</latexit> <latexit sha1_base64="ocuu1hhk/az3jg80lccu+mgitt4=">aaacj3icbvbnaxsxenwmaeu4h3hbyy5dtkens1skcyguqc49uma7adsyrtzrfdzkizqbyhb/m176v3ojncekx/ytypyeejvvosd7m6ozf+dkogrd+2dr1fbrn29ro/v37z983g18+txzpracu8ioy89j7lbjjv2sppa8t8izwofzpd1d+gcxaj00ukozhiczn2izsmhjs6pgt4abpujcp3hjjughljwwuq7hkbgl3gw8qe1hahmydjbfx1xxqnemw+essemiijrzhfao8x8wnqli/dyhuhp9kmxpwhjluiic1wefw5ylkz9g31pnm3tdcnnnhl56zbvuyjtbun3auflmuvkw+8qmu+rwvyx4ktcvkdkclllnbaewq4+sqgezwiqgy2lrkjp5wowvflcqkbdcki+27koi1k/ejl3vrshsrb9/ne8oqzhqbi/ts28sygfshp1ibdzlgv1h/9g1uwn+blfbbxc3kt0kqp4v7bmch0clvkte</latexit> <latexit sha1_base64="ocuu1hhk/az3jg80lccu+mgitt4=">aaacj3icbvbnaxsxenwmaeu4h3hbyy5dtkens1skcyguqc49uma7adsyrtzrfdzkizqbyhb/m176v3ojncekx/ytypyeejvvosd7m6ozf+dkogrd+2dr1fbrn29ro/v37z983g18+txzpracu8ioy89j7lbjjv2sppa8t8izwofzpd1d+gcxaj00ukozhiczn2izsmhjs6pgt4abpujcp3hjjughljwwuq7hkbgl3gw8qe1hahmydjbfx1xxqnemw+essemiijrzhfao8x8wnqli/dyhuhp9kmxpwhjluiic1wefw5ylkz9g31pnm3tdcnnnhl56zbvuyjtbun3auflmuvkw+8qmu+rwvyx4ktcvkdkclllnbaewq4+sqgezwiqgy2lrkjp5wowvflcqkbdcki+27koi1k/ejl3vrshsrb9/ne8oqzhqbi/ts28sygfshp1ibdzlgv1h/9g1uwn+blfbbxc3kt0kqp4v7bmch0clvkte</latexit> <latexit sha1_base64="ocuu1hhk/az3jg80lccu+mgitt4=">aaacj3icbvbnaxsxenwmaeu4h3hbyy5dtkens1skcyguqc49uma7adsyrtzrfdzkizqbyhb/m176v3ojncekx/ytypyeejvvosd7m6ozf+dkogrd+2dr1fbrn29ro/v37z983g18+txzpracu8ioy89j7lbjjv2sppa8t8izwofzpd1d+gcxaj00ukozhiczn2izsmhjs6pgt4abpujcp3hjjughljwwuq7hkbgl3gw8qe1hahmydjbfx1xxqnemw+essemiijrzhfao8x8wnqli/dyhuhp9kmxpwhjluiic1wefw5ylkz9g31pnm3tdcnnnhl56zbvuyjtbun3auflmuvkw+8qmu+rwvyx4ktcvkdkclllnbaewq4+sqgezwiqgy2lrkjp5wowvflcqkbdcki+27koi1k/ejl3vrshsrb9/ne8oqzhqbi/ts28sygfshp1ibdzlgv1h/9g1uwn+blfbbxc3kt0kqp4v7bmch0clvkte</latexit> <latexit sha1_base64="ocuu1hhk/az3jg80lccu+mgitt4=">aaacj3icbvbnaxsxenwmaeu4h3hbyy5dtkens1skcyguqc49uma7adsyrtzrfdzkizqbyhb/m176v3ojncekx/ytypyeejvvosd7m6ozf+dkogrd+2dr1fbrn29ro/v37z983g18+txzpracu8ioy89j7lbjjv2sppa8t8izwofzpd1d+gcxaj00ukozhiczn2izsmhjs6pgt4abpujcp3hjjughljwwuq7hkbgl3gw8qe1hahmydjbfx1xxqnemw+essemiijrzhfao8x8wnqli/dyhuhp9kmxpwhjluiic1wefw5ylkz9g31pnm3tdcnnnhl56zbvuyjtbun3auflmuvkw+8qmu+rwvyx4ktcvkdkclllnbaewq4+sqgezwiqgy2lrkjp5wowvflcqkbdcki+27koi1k/ejl3vrshsrb9/ne8oqzhqbi/ts28sygfshp1ibdzlgv1h/9g1uwn+blfbbxc3kt0kqp4v7bmch0clvkte</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> ˆ MAP = argmax Notation Shorthand MAP, without shorthand f( = X (1) = x (1),...,X (n) = x (n) ) Our shorthand notation is shorthand for the event: = x (i) is shorthand for the event: X (i) = x (i) MAP, now with shorthand ˆ MAP = argmax f( x (1),...,x (n) )

33 <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="hwtjai6zj9txhdv1epw014cbypq=">aaacuhiclvfdi9qwfe3r1zp+zeqjlxchzqrl0c7klokwkiipcis4uwutsazp2gmbjiw5lrlqf6pgm//gdgauddcxlwrozrkfjzdzratdop4zhneu37h5a+f24m7de/cfdhcfnjjtwc6m3chjzzlmhjjatfgieme1fazkldjnzt/0+ulxyz00+houajgvwkllitldt6xd73tbskw4emi6tp3w/m0hz14bbxtuq8qfqawmfpahszpvomw2rniy6waoiiw20bxj5ed2netdhtdconud/q6jdr7bpgdela7+p/fv1kbfy3im1dvvkoequ8jmmrzqpvpnrelwfe/idcbvkgzbigzjob3+8hz5uwmnxdhnzklc49ybqmmv6aa0cajm/jyvyuzh79dn2/xio3jqmrwky/3rcgv2ykxlkudwveyzk4yld1nryx9pswalw3krdd2g0hwzqgguoih+fygxvnbukw8yt9j7bb5glnh0mx34jssxn3wvnoxpknisfhw+ojrcrmohpczpyjgk5iackxfkmewjd14esyaprpgy/bkwodykhsg25hh5k0l7cx4816u=</latexit> <latexit sha1_base64="hwtjai6zj9txhdv1epw014cbypq=">aaacuhiclvfdi9qwfe3r1zp+zeqjlxchzqrl0c7klokwkiipcis4uwutsazp2gmbjiw5lrlqf6pgm//gdgauddcxlwrozrkfjzdzratdop4zhneu37h5a+f24m7de/cfdhcfnjjtwc6m3chjzzlmhjjatfgieme1fazkldjnzt/0+ulxyz00+houajgvwkllitldt6xd73tbskw4emi6tp3w/m0hz14bbxtuq8qfqawmfpahszpvomw2rniy6waoiiw20bxj5ed2netdhtdconud/q6jdr7bpgdela7+p/fv1kbfy3im1dvvkoequ8jmmrzqpvpnrelwfe/idcbvkgzbigzjob3+8hz5uwmnxdhnzklc49ybqmmv6aa0cajm/jyvyuzh79dn2/xio3jqmrwky/3rcgv2ykxlkudwveyzk4yld1nryx9pswalw3krdd2g0hwzqgguoih+fygxvnbukw8yt9j7bb5glnh0mx34jssxn3wvnoxpknisfhw+ojrcrmohpczpyjgk5iackxfkmewjd14esyaprpgy/bkwodykhsg25hh5k0l7cx4816u=</latexit> <latexit sha1_base64="hwtjai6zj9txhdv1epw014cbypq=">aaacuhiclvfdi9qwfe3r1zp+zeqjlxchzqrl0c7klokwkiipcis4uwutsazp2gmbjiw5lrlqf6pgm//gdgauddcxlwrozrkfjzdzratdop4zhneu37h5a+f24m7de/cfdhcfnjjtwc6m3chjzzlmhjjatfgieme1fazkldjnzt/0+ulxyz00+houajgvwkllitldt6xd73tbskw4emi6tp3w/m0hz14bbxtuq8qfqawmfpahszpvomw2rniy6waoiiw20bxj5ed2netdhtdconud/q6jdr7bpgdela7+p/fv1kbfy3im1dvvkoequ8jmmrzqpvpnrelwfe/idcbvkgzbigzjob3+8hz5uwmnxdhnzklc49ybqmmv6aa0cajm/jyvyuzh79dn2/xio3jqmrwky/3rcgv2ykxlkudwveyzk4yld1nryx9pswalw3krdd2g0hwzqgguoih+fygxvnbukw8yt9j7bb5glnh0mx34jssxn3wvnoxpknisfhw+ojrcrmohpczpyjgk5iackxfkmewjd14esyaprpgy/bkwodykhsg25hh5k0l7cx4816u=</latexit> <latexit sha1_base64="hwtjai6zj9txhdv1epw014cbypq=">aaacuhiclvfdi9qwfe3r1zp+zeqjlxchzqrl0c7klokwkiipcis4uwutsazp2gmbjiw5lrlqf6pgm//gdgauddcxlwrozrkfjzdzratdop4zhneu37h5a+f24m7de/cfdhcfnjjtwc6m3chjzzlmhjjatfgieme1fazkldjnzt/0+ulxyz00+houajgvwkllitldt6xd73tbskw4emi6tp3w/m0hz14bbxtuq8qfqawmfpahszpvomw2rniy6waoiiw20bxj5ed2netdhtdconud/q6jdr7bpgdela7+p/fv1kbfy3im1dvvkoequ8jmmrzqpvpnrelwfe/idcbvkgzbigzjob3+8hz5uwmnxdhnzklc49ybqmmv6aa0cajm/jyvyuzh79dn2/xio3jqmrwky/3rcgv2ykxlkudwveyzk4yld1nryx9pswalw3krdd2g0hwzqgguoih+fygxvnbukw8yt9j7bb5glnh0mx34jssxn3wvnoxpknisfhw+ojrcrmohpczpyjgk5iackxfkmewjd14esyaprpgy/bkwodykhsg25hh5k0l7cx4816u=</latexit> MLE vs MAP Data: x (1),...,x (n) <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> Maximum Likelihood Estimation ˆ MLE = argmax = argmax f(x (1),...,x (n) ) X log f(x (i) ) Maximum A Posteriori i ˆ MAP = argmax f( x (1),...,x (n) )

34 Most important slide of today

35 <latexit sha1_base64="cb7kxzj54tosqxtnih+zrbq7was=">aaacoxicbvfdb9mwfhuyykn8rmajlxyrqjwmkili2wvsec/wgfqq3sbfoxlcmyza4kt2dbqy/in8mh4e/wgnjrcsxmnw8bn3xf8fp01zaazdn55/cov2ncoju4n79x88pb4+enyh61yjmiu6rnvvyjwuhyq5fljcvaoav2kjl+n12y5/+rwulmr5gtcnjbvfysirbedhlyy/biznabjmgnwuzic3m0tfv6cslusngz8pw+ogfmdasv6b4wpv8tw1ljkenrrqqka4mnlozsa6ow6fnlc2rffvd0za+p3umo3patqja/j9qv8/tothee7cbdb9epugih3mfsnfrofok5aosq51hiunjm5aleqjdsbadq0x13wfsypdu3ritjza+twxs5rvyi2jdmv+rtc80nptpa6y4pjrm7mo/f8ubje7s0whmxzbit1fwvtsrgn3j3rzkbbybhzgqhvuvipy7txe9wudjugbqkuky6vhyomomv/mkijgtvmnuungpriytqjwen18fzyf9u4dkafkgrmrijysc/kozmicco/ag3tt76uf+o/9mf9pv+p7veyj+sf8+dd4q82p</latexit> <latexit sha1_base64="cb7kxzj54tosqxtnih+zrbq7was=">aaacoxicbvfdb9mwfhuyykn8rmajlxyrqjwmkili2wvsec/wgfqq3sbfoxlcmyza4kt2dbqy/in8mh4e/wgnjrcsxmnw8bn3xf8fp01zaazdn55/cov2ncoju4n79x88pb4+enyh61yjmiu6rnvvyjwuhyq5fljcvaoav2kjl+n12y5/+rwulmr5gtcnjbvfysirbedhlyy/biznabjmgnwuzic3m0tfv6cslusngz8pw+ogfmdasv6b4wpv8tw1ljkenrrqqka4mnlozsa6ow6fnlc2rffvd0za+p3umo3patqja/j9qv8/tothee7cbdb9epugih3mfsnfrofok5aosq51hiunjm5aleqjdsbadq0x13wfsypdu3ritjza+twxs5rvyi2jdmv+rtc80nptpa6y4pjrm7mo/f8ubje7s0whmxzbit1fwvtsrgn3j3rzkbbybhzgqhvuvipy7txe9wudjugbqkuky6vhyomomv/mkijgtvmnuungpriytqjwen18fzyf9u4dkafkgrmrijysc/kozmicco/ag3tt76uf+o/9mf9pv+p7veyj+sf8+dd4q82p</latexit> <latexit sha1_base64="cb7kxzj54tosqxtnih+zrbq7was=">aaacoxicbvfdb9mwfhuyykn8rmajlxyrqjwmkili2wvsec/wgfqq3sbfoxlcmyza4kt2dbqy/in8mh4e/wgnjrcsxmnw8bn3xf8fp01zaazdn55/cov2ncoju4n79x88pb4+enyh61yjmiu6rnvvyjwuhyq5fljcvaoav2kjl+n12y5/+rwulmr5gtcnjbvfysirbedhlyy/biznabjmgnwuzic3m0tfv6cslusngz8pw+ogfmdasv6b4wpv8tw1ljkenrrqqka4mnlozsa6ow6fnlc2rffvd0za+p3umo3patqja/j9qv8/tothee7cbdb9epugih3mfsnfrofok5aosq51hiunjm5aleqjdsbadq0x13wfsypdu3ritjza+twxs5rvyi2jdmv+rtc80nptpa6y4pjrm7mo/f8ubje7s0whmxzbit1fwvtsrgn3j3rzkbbybhzgqhvuvipy7txe9wudjugbqkuky6vhyomomv/mkijgtvmnuungpriytqjwen18fzyf9u4dkafkgrmrijysc/kozmicco/ag3tt76uf+o/9mf9pv+p7veyj+sf8+dd4q82p</latexit> <latexit sha1_base64="cb7kxzj54tosqxtnih+zrbq7was=">aaacoxicbvfdb9mwfhuyykn8rmajlxyrqjwmkili2wvsec/wgfqq3sbfoxlcmyza4kt2dbqy/in8mh4e/wgnjrcsxmnw8bn3xf8fp01zaazdn55/cov2ncoju4n79x88pb4+enyh61yjmiu6rnvvyjwuhyq5fljcvaoav2kjl+n12y5/+rwulmr5gtcnjbvfysirbedhlyy/biznabjmgnwuzic3m0tfv6cslusngz8pw+ogfmdasv6b4wpv8tw1ljkenrrqqka4mnlozsa6ow6fnlc2rffvd0za+p3umo3patqja/j9qv8/tothee7cbdb9epugih3mfsnfrofok5aosq51hiunjm5aleqjdsbadq0x13wfsypdu3ritjza+twxs5rvyi2jdmv+rtc80nptpa6y4pjrm7mo/f8ubje7s0whmxzbit1fwvtsrgn3j3rzkbbybhzgqhvuvipy7txe9wudjugbqkuky6vhyomomv/mkijgtvmnuungpriytqjwen18fzyf9u4dkafkgrmrijysc/kozmicco/ag3tt76uf+o/9mf9pv+p7veyj+sf8+dd4q82p</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> Maximum A Posteriori data: x (1),...,x (n) <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> ˆ MAP = argmax f( x (1),...,x (n) ) ˆ MAP =argmax likelihood f(x (1),x (2),...,x (n) )g( ) h(x (1),x (2),...x (n) ) posterior prior

36 <latexit sha1_base64="4+ghiltfrjgdhdnj7wgptwaw+pc=">aaaconicbvfdb9mwfhxcgfg+cjzyylfalzqqzexil0hdvcakpe7qbvidkse9aawltmtfqcvjv8l/4kfwh3daggdlsrao7z3hvj43rqtpmip+bog1ves3bu7f6t2+c/fe/f6dh2emarsaqaiksl+k3eahfuxrygextqzepgwcp5dv2/r5f9bgvuotrmtisr5umpoco0/n+997lodogeaa3m3thzctr18/p6xrcy+dpyxlqho0x0orxollelnyfxwomoqvwupbprmwy+fwmklzcmugpqb+pzygbfgh2ryscvqb/c1ynt/ldbsrm/ch0tjabn0fcqcgpivjvp/txycaehskghszi6mae98uslga67hgqm3fjv/czmp2lyaxgx8dfeyzc5pv2i+fdjp9w2f5acy6td2z5jibq7u2+b/armhsolfs1q2cetuhsqagwnf2khqhnqgs1h5woaxvlyqcezvrz6dhfhwvvvlytbam3cxo7gc7hmqj1/otx3vjf5wdjunohj8edu6oo6f2ywpyhaxjtf6re/kotmiuigavebg8di7cp+h78dt8ukwgqad5rp6jkp0cbv3nuq==</latexit> <latexit sha1_base64="4+ghiltfrjgdhdnj7wgptwaw+pc=">aaaconicbvfdb9mwfhxcgfg+cjzyylfalzqqzexil0hdvcakpe7qbvidkse9aawltmtfqcvjv8l/4kfwh3daggdlsrao7z3hvj43rqtpmip+bog1ves3bu7f6t2+c/fe/f6dh2emarsaqaiksl+k3eahfuxrygextqzepgwcp5dv2/r5f9bgvuotrmtisr5umpoco0/n+997lodogeaa3m3thzctr18/p6xrcy+dpyxlqho0x0orxollelnyfxwomoqvwupbprmwy+fwmklzcmugpqb+pzygbfgh2ryscvqb/c1ynt/ldbsrm/ch0tjabn0fcqcgpivjvp/txycaehskghszi6mae98uslga67hgqm3fjv/czmp2lyaxgx8dfeyzc5pv2i+fdjp9w2f5acy6td2z5jibq7u2+b/armhsolfs1q2cetuhsqagwnf2khqhnqgs1h5woaxvlyqcezvrz6dhfhwvvvlytbam3cxo7gc7hmqj1/otx3vjf5wdjunohj8edu6oo6f2ywpyhaxjtf6re/kotmiuigavebg8di7cp+h78dt8ukwgqad5rp6jkp0cbv3nuq==</latexit> <latexit sha1_base64="4+ghiltfrjgdhdnj7wgptwaw+pc=">aaaconicbvfdb9mwfhxcgfg+cjzyylfalzqqzexil0hdvcakpe7qbvidkse9aawltmtfqcvjv8l/4kfwh3daggdlsrao7z3hvj43rqtpmip+bog1ves3bu7f6t2+c/fe/f6dh2emarsaqaiksl+k3eahfuxrygextqzepgwcp5dv2/r5f9bgvuotrmtisr5umpoco0/n+997lodogeaa3m3thzctr18/p6xrcy+dpyxlqho0x0orxollelnyfxwomoqvwupbprmwy+fwmklzcmugpqb+pzygbfgh2ryscvqb/c1ynt/ldbsrm/ch0tjabn0fcqcgpivjvp/txycaehskghszi6mae98uslga67hgqm3fjv/czmp2lyaxgx8dfeyzc5pv2i+fdjp9w2f5acy6td2z5jibq7u2+b/armhsolfs1q2cetuhsqagwnf2khqhnqgs1h5woaxvlyqcezvrz6dhfhwvvvlytbam3cxo7gc7hmqj1/otx3vjf5wdjunohj8edu6oo6f2ywpyhaxjtf6re/kotmiuigavebg8di7cp+h78dt8ukwgqad5rp6jkp0cbv3nuq==</latexit> <latexit sha1_base64="4+ghiltfrjgdhdnj7wgptwaw+pc=">aaaconicbvfdb9mwfhxcgfg+cjzyylfalzqqzexil0hdvcakpe7qbvidkse9aawltmtfqcvjv8l/4kfwh3daggdlsrao7z3hvj43rqtpmip+bog1ves3bu7f6t2+c/fe/f6dh2emarsaqaiksl+k3eahfuxrygextqzepgwcp5dv2/r5f9bgvuotrmtisr5umpoco0/n+997lodogeaa3m3thzctr18/p6xrcy+dpyxlqho0x0orxollelnyfxwomoqvwupbprmwy+fwmklzcmugpqb+pzygbfgh2ryscvqb/c1ynt/ldbsrm/ch0tjabn0fcqcgpivjvp/txycaehskghszi6mae98uslga67hgqm3fjv/czmp2lyaxgx8dfeyzc5pv2i+fdjp9w2f5acy6td2z5jibq7u2+b/armhsolfs1q2cetuhsqagwnf2khqhnqgs1h5woaxvlyqcezvrz6dhfhwvvvlytbam3cxo7gc7hmqj1/otx3vjf5wdjunohj8edu6oo6f2ywpyhaxjtf6re/kotmiuigavebg8di7cp+h78dt8ukwgqad5rp6jkp0cbv3nuq==</latexit> <latexit sha1_base64="5+zrdjvibpglvta+3xk5c+xnami=">aaacgxicbvbbaxqxfm5ovbtrpwv76etwuwyqlpkiwjbcwrcfk7htybndmtkzs6gtzejo6c4xp9rnn/0pzi+ith4i+fgunmnxto10moc/esneg4ephu8f9j88ffb8cpdi6nkzzgoyc9mye11yb43umeajdvy3frgqg7gqbz+u9auvyj00+guuwpgqxmtzscexurnbohtdwafn0qlogs4aefdmtga5gqu5as9trfishkazwhi9jacbctp4mzqmdbrnzrl9s5nr1mzlsylc6i2evumsgzebfqd/fmzkeohzbddmr/lm6h1q7mcq7ozinvjf5kz0cjskhjs3kfiwp/fclkkb0gedg5alw17djml1+w7qnzuf+joyc1ozg59gumh/tniunfupmjovx4w7q63j/2mtdqvtqze67rc02c6quoaioevo6vxaenisiudcyngrfqtuucbye59p+camulzppymwkab+xqfdigsh9lpcbem+udwzffmo+pxueh66a2qfvcsvseok8p6ck0/kgoyjid97b72j3ngyl2rjnpxsrulvlzkm/0zy4tf7n8fb</latexit> <latexit sha1_base64="5+zrdjvibpglvta+3xk5c+xnami=">aaacgxicbvbbaxqxfm5ovbtrpwv76etwuwyqlpkiwjbcwrcfk7htybndmtkzs6gtzejo6c4xp9rnn/0pzi+ith4i+fgunmnxto10moc/esneg4ephu8f9j88ffb8cpdi6nkzzgoyc9mye11yb43umeajdvy3frgqg7gqbz+u9auvyj00+guuwpgqxmtzscexurnbohtdwafn0qlogs4aefdmtga5gqu5as9trfishkazwhi9jacbctp4mzqmdbrnzrl9s5nr1mzlsylc6i2evumsgzebfqd/fmzkeohzbddmr/lm6h1q7mcq7ozinvjf5kz0cjskhjs3kfiwp/fclkkb0gedg5alw17djml1+w7qnzuf+joyc1ozg59gumh/tniunfupmjovx4w7q63j/2mtdqvtqze67rc02c6quoaioevo6vxaenisiudcyngrfqtuucbye59p+camulzppymwkab+xqfdigsh9lpcbem+udwzffmo+pxueh66a2qfvcsvseok8p6ck0/kgoyjid97b72j3ngyl2rjnpxsrulvlzkm/0zy4tf7n8fb</latexit> <latexit sha1_base64="5+zrdjvibpglvta+3xk5c+xnami=">aaacgxicbvbbaxqxfm5ovbtrpwv76etwuwyqlpkiwjbcwrcfk7htybndmtkzs6gtzejo6c4xp9rnn/0pzi+ith4i+fgunmnxto10moc/esneg4ephu8f9j88ffb8cpdi6nkzzgoyc9mye11yb43umeajdvy3frgqg7gqbz+u9auvyj00+guuwpgqxmtzscexurnbohtdwafn0qlogs4aefdmtga5gqu5as9trfishkazwhi9jacbctp4mzqmdbrnzrl9s5nr1mzlsylc6i2evumsgzebfqd/fmzkeohzbddmr/lm6h1q7mcq7ozinvjf5kz0cjskhjs3kfiwp/fclkkb0gedg5alw17djml1+w7qnzuf+joyc1ozg59gumh/tniunfupmjovx4w7q63j/2mtdqvtqze67rc02c6quoaioevo6vxaenisiudcyngrfqtuucbye59p+camulzppymwkab+xqfdigsh9lpcbem+udwzffmo+pxueh66a2qfvcsvseok8p6ck0/kgoyjid97b72j3ngyl2rjnpxsrulvlzkm/0zy4tf7n8fb</latexit> <latexit sha1_base64="5+zrdjvibpglvta+3xk5c+xnami=">aaacgxicbvbbaxqxfm5ovbtrpwv76etwuwyqlpkiwjbcwrcfk7htybndmtkzs6gtzejo6c4xp9rnn/0pzi+ith4i+fgunmnxto10moc/esneg4ephu8f9j88ffb8cpdi6nkzzgoyc9mye11yb43umeajdvy3frgqg7gqbz+u9auvyj00+guuwpgqxmtzscexurnbohtdwafn0qlogs4aefdmtga5gqu5as9trfishkazwhi9jacbctp4mzqmdbrnzrl9s5nr1mzlsylc6i2evumsgzebfqd/fmzkeohzbddmr/lm6h1q7mcq7ozinvjf5kz0cjskhjs3kfiwp/fclkkb0gedg5alw17djml1+w7qnzuf+joyc1ozg59gumh/tniunfupmjovx4w7q63j/2mtdqvtqze67rc02c6quoaioevo6vxaenisiudcyngrfqtuucbye59p+camulzppymwkab+xqfdigsh9lpcbem+udwzffmo+pxueh66a2qfvcsvseok8p6ck0/kgoyjid97b72j3ngyl2rjnpxsrulvlzkm/0zy4tf7n8fb</latexit> <latexit sha1_base64="maqkcy/6+1ki4wpufveijtgkwf4=">aaacixicbvbda9swfjw9ry7d1nr7w1/ewkycjvhlsdaofpqyxw6wthc7qzgve1fbmtl1lqdpct9yv2hsp0xo/lc1uybxdm650tvz1kw0mcq/o/jbw0epn+w97e0/e/7ioh/48tlqxgiycl1qc73gfkqpyioss7iudfbquclv4va81a++grfsqy+4qsgr+fljqgqogzr3f/ro39g0uxnwalouv4dcu1txydhqo3gfjptlxdfue5oirnhraardhvsodw2jtdy6nzt5yvynosvwnyktnn6nne7dquxymq37ytfnc412e3dkj/y8p0jgybbofca6mcbdxcz7v8mnoqlaosi5ttow1jifqvgkenwvbszuxnzyjcwcbh9hm7eny9o3gclpou1ycumw/bvd8cratbuizorjyt7vwvj/2qzbypi5qeogqyndq0vtuts0zz7m0odachmaf0agwaly8rakhvr7qyjvqlcvv7llwc9y5m7ccmbgvs2h3u3jprg8gbnkzd6/h5xnuqt2ybf5q4aekq/kjhwif2rkbpkvhuavo6n4p2bxjp64s8zr1/ok/fpx+r9lzcwp</latexit> <latexit sha1_base64="maqkcy/6+1ki4wpufveijtgkwf4=">aaacixicbvbda9swfjw9ry7d1nr7w1/ewkycjvhlsdaofpqyxw6wthc7qzgve1fbmtl1lqdpct9yv2hsp0xo/lc1uybxdm650tvz1kw0mcq/o/jbw0epn+w97e0/e/7ioh/48tlqxgiycl1qc73gfkqpyioss7iudfbquclv4va81a++grfsqy+4qsgr+fljqgqogzr3f/ro39g0uxnwalouv4dcu1txydhqo3gfjptlxdfue5oirnhraardhvsodw2jtdy6nzt5yvynosvwnyktnn6nne7dquxymq37ytfnc412e3dkj/y8p0jgybbofca6mcbdxcz7v8mnoqlaosi5ttow1jifqvgkenwvbszuxnzyjcwcbh9hm7eny9o3gclpou1ycumw/bvd8cratbuizorjyt7vwvj/2qzbypi5qeogqyndq0vtuts0zz7m0odachmaf0agwaly8rakhvr7qyjvqlcvv7llwc9y5m7ccmbgvs2h3u3jprg8gbnkzd6/h5xnuqt2ybf5q4aekq/kjhwif2rkbpkvhuavo6n4p2bxjp64s8zr1/ok/fpx+r9lzcwp</latexit> <latexit sha1_base64="maqkcy/6+1ki4wpufveijtgkwf4=">aaacixicbvbda9swfjw9ry7d1nr7w1/ewkycjvhlsdaofpqyxw6wthc7qzgve1fbmtl1lqdpct9yv2hsp0xo/lc1uybxdm650tvz1kw0mcq/o/jbw0epn+w97e0/e/7ioh/48tlqxgiycl1qc73gfkqpyioss7iudfbquclv4va81a++grfsqy+4qsgr+fljqgqogzr3f/ro39g0uxnwalouv4dcu1txydhqo3gfjptlxdfue5oirnhraardhvsodw2jtdy6nzt5yvynosvwnyktnn6nne7dquxymq37ytfnc412e3dkj/y8p0jgybbofca6mcbdxcz7v8mnoqlaosi5ttow1jifqvgkenwvbszuxnzyjcwcbh9hm7eny9o3gclpou1ycumw/bvd8cratbuizorjyt7vwvj/2qzbypi5qeogqyndq0vtuts0zz7m0odachmaf0agwaly8rakhvr7qyjvqlcvv7llwc9y5m7ccmbgvs2h3u3jprg8gbnkzd6/h5xnuqt2ybf5q4aekq/kjhwif2rkbpkvhuavo6n4p2bxjp64s8zr1/ok/fpx+r9lzcwp</latexit> <latexit sha1_base64="maqkcy/6+1ki4wpufveijtgkwf4=">aaacixicbvbda9swfjw9ry7d1nr7w1/ewkycjvhlsdaofpqyxw6wthc7qzgve1fbmtl1lqdpct9yv2hsp0xo/lc1uybxdm650tvz1kw0mcq/o/jbw0epn+w97e0/e/7ioh/48tlqxgiycl1qc73gfkqpyioss7iudfbquclv4va81a++grfsqy+4qsgr+fljqgqogzr3f/ro39g0uxnwalouv4dcu1txydhqo3gfjptlxdfue5oirnhraardhvsodw2jtdy6nzt5yvynosvwnyktnn6nne7dquxymq37ytfnc412e3dkj/y8p0jgybbofca6mcbdxcz7v8mnoqlaosi5ttow1jifqvgkenwvbszuxnzyjcwcbh9hm7eny9o3gclpou1ycumw/bvd8cratbuizorjyt7vwvj/2qzbypi5qeogqyndq0vtuts0zz7m0odachmaf0agwaly8rakhvr7qyjvqlcvv7llwc9y5m7ccmbgvs2h3u3jprg8gbnkzd6/h5xnuqt2ybf5q4aekq/kjhwif2rkbpkvhuavo6n4p2bxjp64s8zr1/ok/fpx+r9lzcwp</latexit> <latexit sha1_base64="zcva1vkadzs39azyjq2c7bkkz/w=">aaacz3icbvblaxsxgjs3r9r9xg2hfhpra1qci9ktgeqscptsywp1era2rqv91hzzsyv0bwkj6gf2j/qn9fb6tw6vh4u26ybgmjkpdvm0txsypt87yz279+4/2hryfft4ydpt3rpnx860vsbimnry04i7qkwgeuqs4bsxwfvrw0lx/nhpn1yaddlol7hoifd8qmulbccotxri4d1lrs5jbnaznahy4jlpwhi0vnmfntup4nmaamuic/q00olghd2hrlgmnhhjd2gwzjstbnmgxgb6lf6jthr9djiuqg+tbep6ziojse8nk41ofwgunxdunkun5rehslfd6llwqcpfoz/conjlszf71ribvotksstj49niv+rff54r5xaqienfcezuekvxf964xwo/91i3lyiw64+qtqzo6hjzwkolautfjfxygbtsmeowc4zjdpmgs2gu4rr0dmi4y/2zh/sznrdiptnnnw6t4w/dlb1mn3f7h/ubpbbia/kwdehg9sgh+usoyigi8o38ilfkuvmj2u5ejq/w0aszuxlb/khy5jff8ru4</latexit> <latexit sha1_base64="zcva1vkadzs39azyjq2c7bkkz/w=">aaacz3icbvblaxsxgjs3r9r9xg2hfhpra1qci9ktgeqscptsywp1era2rqv91hzzsyv0bwkj6gf2j/qn9fb6tw6vh4u26ybgmjkpdvm0txsypt87yz279+4/2hryfft4ydpt3rpnx860vsbimnry04i7qkwgeuqs4bsxwfvrw0lx/nhpn1yaddlol7hoifd8qmulbccotxri4d1lrs5jbnaznahy4jlpwhi0vnmfntup4nmaamuic/q00olghd2hrlgmnhhjd2gwzjstbnmgxgb6lf6jthr9djiuqg+tbep6ziojse8nk41ofwgunxdunkun5rehslfd6llwqcpfoz/conjlszf71ribvotksstj49niv+rff54r5xaqienfcezuekvxf964xwo/91i3lyiw64+qtqzo6hjzwkolautfjfxygbtsmeowc4zjdpmgs2gu4rr0dmi4y/2zh/sznrdiptnnnw6t4w/dlb1mn3f7h/ubpbbia/kwdehg9sgh+usoyigi8o38ilfkuvmj2u5ejq/w0aszuxlb/khy5jff8ru4</latexit> <latexit sha1_base64="zcva1vkadzs39azyjq2c7bkkz/w=">aaacz3icbvblaxsxgjs3r9r9xg2hfhpra1qci9ktgeqscptsywp1era2rqv91hzzsyv0bwkj6gf2j/qn9fb6tw6vh4u26ybgmjkpdvm0txsypt87yz279+4/2hryfft4ydpt3rpnx860vsbimnry04i7qkwgeuqs4bsxwfvrw0lx/nhpn1yaddlol7hoifd8qmulbccotxri4d1lrs5jbnaznahy4jlpwhi0vnmfntup4nmaamuic/q00olghd2hrlgmnhhjd2gwzjstbnmgxgb6lf6jthr9djiuqg+tbep6ziojse8nk41ofwgunxdunkun5rehslfd6llwqcpfoz/conjlszf71ribvotksstj49niv+rff54r5xaqienfcezuekvxf964xwo/91i3lyiw64+qtqzo6hjzwkolautfjfxygbtsmeowc4zjdpmgs2gu4rr0dmi4y/2zh/sznrdiptnnnw6t4w/dlb1mn3f7h/ubpbbia/kwdehg9sgh+usoyigi8o38ilfkuvmj2u5ejq/w0aszuxlb/khy5jff8ru4</latexit> <latexit sha1_base64="zcva1vkadzs39azyjq2c7bkkz/w=">aaacz3icbvblaxsxgjs3r9r9xg2hfhpra1qci9ktgeqscptsywp1era2rqv91hzzsyv0bwkj6gf2j/qn9fb6tw6vh4u26ybgmjkpdvm0txsypt87yz279+4/2hryfft4ydpt3rpnx860vsbimnry04i7qkwgeuqs4bsxwfvrw0lx/nhpn1yaddlol7hoifd8qmulbccotxri4d1lrs5jbnaznahy4jlpwhi0vnmfntup4nmaamuic/q00olghd2hrlgmnhhjd2gwzjstbnmgxgb6lf6jthr9djiuqg+tbep6ziojse8nk41ofwgunxdunkun5rehslfd6llwqcpfoz/conjlszf71ribvotksstj49niv+rff54r5xaqienfcezuekvxf964xwo/91i3lyiw64+qtqzo6hjzwkolautfjfxygbtsmeowc4zjdpmgs2gu4rr0dmi4y/2zh/sznrdiptnnnw6t4w/dlb1mn3f7h/ubpbbia/kwdehg9sgh+usoyigi8o38ilfkuvmj2u5ejq/w0aszuxlb/khy5jff8ru4</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> Maximum A Posteriori data: x (1),...,x (n) <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> ˆ MAP = argmax f( x (1),...,x (n) ) ˆ MAP =argmax =argmax =argmax =argmax g( )f(x (1),x (2),...,x (n) ) h(x (1),x (2),...x (n) ) g( ) Q n i=1 f(x(i) ) h(x (1),x (2),...,x (n) ) g( ) ny f(x (i) ) i=1 log(g( )) +! nx log(f(x (i) )) i=1 Fury K monotonic

37 <latexit sha1_base64="rh5tzuddphfcxqeqfm6m0yqtyci=">aaaclhicbvfdixmxfe3hr7v+bfxwrr+czzcpqpkrwx1wywurffeq2n2fplvs9m407cqzkjvaevmf/hv+cf99nv1g0v0vhbzouycctqz1jr1m2c9wcupmrdt3du62791/8hc38+jxitonftaupjl2bmodvfldecvwcfzb4gpawen04niln34f66trx3bzw1jxustcco6rmnr+sdlhz3aoympef3w3cprwn7jgz6illixpta2wo7gak/dcloovaaiuiszq0wgqkdcnlylpwqybx69hx1lmgjxx8jap53qtp0w6yoblon/p5r6zspxjb9lpzv1spfq6ylegs7yzmhr+szkrjqknoulojfksxnfmifjuenqscvbzccflgew4iu/gft1cohurmdhc2hg00jx7t8nz5dxsteom4jh3v7uv+t9t1gbxmpzs1w2cfpuhiqaiaojqg+hmwhbylspgwsqylyo5t1xgrl3nnhwtrimuz55bgovjf+7tbt4lifatx23jojh51c+zfv75dffoynvudnlgxpcu5oqnosifyiamisc/w89be6395gnynjlo3m9wk9bw84t8m8mnp5skyxe=</latexit> <latexit sha1_base64="rh5tzuddphfcxqeqfm6m0yqtyci=">aaaclhicbvfdixmxfe3hr7v+bfxwrr+czzcpqpkrwx1wywurffeq2n2fplvs9m407cqzkjvaevmf/hv+cf99nv1g0v0vhbzouycctqz1jr1m2c9wcupmrdt3du62791/8hc38+jxitonftaupjl2bmodvfldecvwcfzb4gpawen04niln34f66trx3bzw1jxustcco6rmnr+sdlhz3aoympef3w3cprwn7jgz6illixpta2wo7gak/dcloovaaiuiszq0wgqkdcnlylpwqybx69hx1lmgjxx8jap53qtp0w6yoblon/p5r6zspxjb9lpzv1spfq6ylegs7yzmhr+szkrjqknoulojfksxnfmifjuenqscvbzccflgew4iu/gft1cohurmdhc2hg00jx7t8nz5dxsteom4jh3v7uv+t9t1gbxmpzs1w2cfpuhiqaiaojqg+hmwhbylspgwsqylyo5t1xgrl3nnhwtrimuz55bgovjf+7tbt4lifatx23jojh51c+zfv75dffoynvudnlgxpcu5oqnosifyiamisc/w89be6395gnynjlo3m9wk9bw84t8m8mnp5skyxe=</latexit> <latexit sha1_base64="rh5tzuddphfcxqeqfm6m0yqtyci=">aaaclhicbvfdixmxfe3hr7v+bfxwrr+czzcpqpkrwx1wywurffeq2n2fplvs9m407cqzkjvaevmf/hv+cf99nv1g0v0vhbzouycctqz1jr1m2c9wcupmrdt3du62791/8hc38+jxitonftaupjl2bmodvfldecvwcfzb4gpawen04niln34f66trx3bzw1jxustcco6rmnr+sdlhz3aoympef3w3cprwn7jgz6illixpta2wo7gak/dcloovaaiuiszq0wgqkdcnlylpwqybx69hx1lmgjxx8jap53qtp0w6yoblon/p5r6zspxjb9lpzv1spfq6ylegs7yzmhr+szkrjqknoulojfksxnfmifjuenqscvbzccflgew4iu/gft1cohurmdhc2hg00jx7t8nz5dxsteom4jh3v7uv+t9t1gbxmpzs1w2cfpuhiqaiaojqg+hmwhbylspgwsqylyo5t1xgrl3nnhwtrimuz55bgovjf+7tbt4lifatx23jojh51c+zfv75dffoynvudnlgxpcu5oqnosifyiamisc/w89be6395gnynjlo3m9wk9bw84t8m8mnp5skyxe=</latexit> <latexit sha1_base64="rh5tzuddphfcxqeqfm6m0yqtyci=">aaaclhicbvfdixmxfe3hr7v+bfxwrr+czzcpqpkrwx1wywurffeq2n2fplvs9m407cqzkjvaevmf/hv+cf99nv1g0v0vhbzouycctqz1jr1m2c9wcupmrdt3du62791/8hc38+jxitonftaupjl2bmodvfldecvwcfzb4gpawen04niln34f66trx3bzw1jxustcco6rmnr+sdlhz3aoympef3w3cprwn7jgz6illixpta2wo7gak/dcloovaaiuiszq0wgqkdcnlylpwqybx69hx1lmgjxx8jap53qtp0w6yoblon/p5r6zspxjb9lpzv1spfq6ylegs7yzmhr+szkrjqknoulojfksxnfmifjuenqscvbzccflgew4iu/gft1cohurmdhc2hg00jx7t8nz5dxsteom4jh3v7uv+t9t1gbxmpzs1w2cfpuhiqaiaojqg+hmwhbylspgwsqylyo5t1xgrl3nnhwtrimuz55bgovjf+7tbt4lifatx23jojh51c+zfv75dffoynvudnlgxpcu5oqnosifyiamisc/w89be6395gnynjlo3m9wk9bw84t8m8mnp5skyxe=</latexit> Maximum A Posteriori Estimated parameter Log prior ˆ MAP =argmax log(g( )) + nx i=1 log(f(x (i) ))! Chose the value of theta that maximizes: Sum of log likelihood

38 <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="5typk9xpuuq9e5+lwmrcj7zgilk=">aaacvxicbvbnaxsxenvu0ir1p+kmx15etysnweygqnmpjptss8gfogl4xtmrz9oiwmmrzoonsn8yl9j/0kuh8gelttogehpvnmb08kpjr0nyi4p3dh/t7r88bj15+uz5yfvf0yuztru4feyze5wdqyu1dkmswqvkips5wsv8+unkv7xb66trx2lz4biemzaffecbmrrvngfygc2rojn4z+edhr/9wlnat4ml/0g+mxvaigm1lojbzkpyna3ngegcpg+k7qat3/lfn99ne80xz6ag3ph6rnsn703anasfris/bokwdni2bpp2xxhc1cvqegqcg6vjremwnarq2lsy2mef4hpmoapwtzob+3uqdx8tmckvja1he1+zfzs8lm4tyzx0lkbzd19bkf/trjuvp2mvdvutarezvnskk+gripluwhsklggasdlsysuclagkkbzccon9lz8efyf9nomnx951zk63crywv+w167kuvwdn7bmbscet7i79jkiojr5hv+ldeg/tgkdbz0v2t8whvwhtflpl</latexit> <latexit sha1_base64="hwtjai6zj9txhdv1epw014cbypq=">aaacuhiclvfdi9qwfe3r1zp+zeqjlxchzqrl0c7klokwkiipcis4uwutsazp2gmbjiw5lrlqf6pgm//gdgauddcxlwrozrkfjzdzratdop4zhneu37h5a+f24m7de/cfdhcfnjjtwc6m3chjzzlmhjjatfgieme1fazkldjnzt/0+ulxyz00+houajgvwkllitldt6xd73tbskw4emi6tp3w/m0hz14bbxtuq8qfqawmfpahszpvomw2rniy6waoiiw20bxj5ed2netdhtdconud/q6jdr7bpgdela7+p/fv1kbfy3im1dvvkoequ8jmmrzqpvpnrelwfe/idcbvkgzbigzjob3+8hz5uwmnxdhnzklc49ybqmmv6aa0cajm/jyvyuzh79dn2/xio3jqmrwky/3rcgv2ykxlkudwveyzk4yld1nryx9pswalw3krdd2g0hwzqgguoih+fygxvnbukw8yt9j7bb5glnh0mx34jssxn3wvnoxpknisfhw+ojrcrmohpczpyjgk5iackxfkmewjd14esyaprpgy/bkwodykhsg25hh5k0l7cx4816u=</latexit> <latexit sha1_base64="hwtjai6zj9txhdv1epw014cbypq=">aaacuhiclvfdi9qwfe3r1zp+zeqjlxchzqrl0c7klokwkiipcis4uwutsazp2gmbjiw5lrlqf6pgm//gdgauddcxlwrozrkfjzdzratdop4zhneu37h5a+f24m7de/cfdhcfnjjtwc6m3chjzzlmhjjatfgieme1fazkldjnzt/0+ulxyz00+houajgvwkllitldt6xd73tbskw4emi6tp3w/m0hz14bbxtuq8qfqawmfpahszpvomw2rniy6waoiiw20bxj5ed2netdhtdconud/q6jdr7bpgdela7+p/fv1kbfy3im1dvvkoequ8jmmrzqpvpnrelwfe/idcbvkgzbigzjob3+8hz5uwmnxdhnzklc49ybqmmv6aa0cajm/jyvyuzh79dn2/xio3jqmrwky/3rcgv2ykxlkudwveyzk4yld1nryx9pswalw3krdd2g0hwzqgguoih+fygxvnbukw8yt9j7bb5glnh0mx34jssxn3wvnoxpknisfhw+ojrcrmohpczpyjgk5iackxfkmewjd14esyaprpgy/bkwodykhsg25hh5k0l7cx4816u=</latexit> <latexit sha1_base64="hwtjai6zj9txhdv1epw014cbypq=">aaacuhiclvfdi9qwfe3r1zp+zeqjlxchzqrl0c7klokwkiipcis4uwutsazp2gmbjiw5lrlqf6pgm//gdgauddcxlwrozrkfjzdzratdop4zhneu37h5a+f24m7de/cfdhcfnjjtwc6m3chjzzlmhjjatfgieme1fazkldjnzt/0+ulxyz00+houajgvwkllitldt6xd73tbskw4emi6tp3w/m0hz14bbxtuq8qfqawmfpahszpvomw2rniy6waoiiw20bxj5ed2netdhtdconud/q6jdr7bpgdela7+p/fv1kbfy3im1dvvkoequ8jmmrzqpvpnrelwfe/idcbvkgzbigzjob3+8hz5uwmnxdhnzklc49ybqmmv6aa0cajm/jyvyuzh79dn2/xio3jqmrwky/3rcgv2ykxlkudwveyzk4yld1nryx9pswalw3krdd2g0hwzqgguoih+fygxvnbukw8yt9j7bb5glnh0mx34jssxn3wvnoxpknisfhw+ojrcrmohpczpyjgk5iackxfkmewjd14esyaprpgy/bkwodykhsg25hh5k0l7cx4816u=</latexit> <latexit sha1_base64="hwtjai6zj9txhdv1epw014cbypq=">aaacuhiclvfdi9qwfe3r1zp+zeqjlxchzqrl0c7klokwkiipcis4uwutsazp2gmbjiw5lrlqf6pgm//gdgauddcxlwrozrkfjzdzratdop4zhneu37h5a+f24m7de/cfdhcfnjjtwc6m3chjzzlmhjjatfgieme1fazkldjnzt/0+ulxyz00+houajgvwkllitldt6xd73tbskw4emi6tp3w/m0hz14bbxtuq8qfqawmfpahszpvomw2rniy6waoiiw20bxj5ed2netdhtdconud/q6jdr7bpgdela7+p/fv1kbfy3im1dvvkoequ8jmmrzqpvpnrelwfe/idcbvkgzbigzjob3+8hz5uwmnxdhnzklc49ybqmmv6aa0cajm/jyvyuzh79dn2/xio3jqmrwky/3rcgv2ykxlkudwveyzk4yld1nryx9pswalw3krdd2g0hwzqgguoih+fygxvnbukw8yt9j7bb5glnh0mx34jssxn3wvnoxpknisfhw+ojrcrmohpczpyjgk5iackxfkmewjd14esyaprpgy/bkwodykhsg25hh5k0l7cx4816u=</latexit> <latexit sha1_base64="2vyh+f2j1wnru3hin/gc2jrpn/u=">aaacghicbvbba9swfja9w5bd0u5xl2jh4ddi7djygrqke9ljb0tbinwgy8eoqcuz6xhlup1d97r3/ycpf8bw7odqx3fhhl6ibatdnp0rxffup3j4apb4+otps+cvrgeh5850vsbmmmbyy4i7aksggups4lk1wfxrwevx/wmjx3wd66trx3hdqq54rwulbcdaluy3j5r1ugwoqm9wcch7z0wllqoxmivw3nakr2jfu4awqk8dakdcjhympkmd7hktcx1lmevuwsutrl/swz2pkhudl3t6q//4mjx1eiel0tidptuhd0g2b2oyn7pf6bcrjeguabqnd26eps3m4ukuoof+ydohlrfxviz5gjvzxe63lfx0twbkwhkbnka6zf9oek6cw6siobxhpbutbcj/afmoq+pcs912cfrsflvdq9hqtew0lbyenusaulay3erfklsumnq+zbq+c6mu16vn0m+z3f/5zjxn+j70k91u4y44p5pm6tt78n58erxvakbekdckirn5qe7jz3jgzksqn9egoogo4zho4ndxtrpg0t7zkvwz8cffhqrbkw==</latexit> <latexit sha1_base64="2vyh+f2j1wnru3hin/gc2jrpn/u=">aaacghicbvbba9swfja9w5bd0u5xl2jh4ddi7djygrqke9ljb0tbinwgy8eoqcuz6xhlup1d97r3/ycpf8bw7odqx3fhhl6ibatdnp0rxffup3j4apb4+otps+cvrgeh5850vsbmmmbyy4i7aksggups4lk1wfxrwevx/wmjx3wd66trx3hdqq54rwulbcdaluy3j5r1ugwoqm9wcch7z0wllqoxmivw3nakr2jfu4awqk8dakdcjhympkmd7hktcx1lmevuwsutrl/swz2pkhudl3t6q//4mjx1eiel0tidptuhd0g2b2oyn7pf6bcrjeguabqnd26eps3m4ukuoof+ydohlrfxviz5gjvzxe63lfx0twbkwhkbnka6zf9oek6cw6siobxhpbutbcj/afmoq+pcs912cfrsflvdq9hqtew0lbyenusaulay3erfklsumnq+zbq+c6mu16vn0m+z3f/5zjxn+j70k91u4y44p5pm6tt78n58erxvakbekdckirn5qe7jz3jgzksqn9egoogo4zho4ndxtrpg0t7zkvwz8cffhqrbkw==</latexit> <latexit sha1_base64="2vyh+f2j1wnru3hin/gc2jrpn/u=">aaacghicbvbba9swfja9w5bd0u5xl2jh4ddi7djygrqke9ljb0tbinwgy8eoqcuz6xhlup1d97r3/ycpf8bw7odqx3fhhl6ibatdnp0rxffup3j4apb4+otps+cvrgeh5850vsbmmmbyy4i7aksggups4lk1wfxrwevx/wmjx3wd66trx3hdqq54rwulbcdaluy3j5r1ugwoqm9wcch7z0wllqoxmivw3nakr2jfu4awqk8dakdcjhympkmd7hktcx1lmevuwsutrl/swz2pkhudl3t6q//4mjx1eiel0tidptuhd0g2b2oyn7pf6bcrjeguabqnd26eps3m4ukuoof+ydohlrfxviz5gjvzxe63lfx0twbkwhkbnka6zf9oek6cw6siobxhpbutbcj/afmoq+pcs912cfrsflvdq9hqtew0lbyenusaulay3erfklsumnq+zbq+c6mu16vn0m+z3f/5zjxn+j70k91u4y44p5pm6tt78n58erxvakbekdckirn5qe7jz3jgzksqn9egoogo4zho4ndxtrpg0t7zkvwz8cffhqrbkw==</latexit> <latexit sha1_base64="2vyh+f2j1wnru3hin/gc2jrpn/u=">aaacghicbvbba9swfja9w5bd0u5xl2jh4ddi7djygrqke9ljb0tbinwgy8eoqcuz6xhlup1d97r3/ycpf8bw7odqx3fhhl6ibatdnp0rxffup3j4apb4+otps+cvrgeh5850vsbmmmbyy4i7aksggups4lk1wfxrwevx/wmjx3wd66trx3hdqq54rwulbcdaluy3j5r1ugwoqm9wcch7z0wllqoxmivw3nakr2jfu4awqk8dakdcjhympkmd7hktcx1lmevuwsutrl/swz2pkhudl3t6q//4mjx1eiel0tidptuhd0g2b2oyn7pf6bcrjeguabqnd26eps3m4ukuoof+ydohlrfxviz5gjvzxe63lfx0twbkwhkbnka6zf9oek6cw6siobxhpbutbcj/afmoq+pcs912cfrsflvdq9hqtew0lbyenusaulay3erfklsumnq+zbq+c6mu16vn0m+z3f/5zjxn+j70k91u4y44p5pm6tt78n58erxvakbekdckirn5qe7jz3jgzksqn9egoogo4zho4ndxtrpg0t7zkvwz8cffhqrbkw==</latexit> MLE vs MAP Data: x (1),...,x (n) <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> <latexit sha1_base64="gichi1fa8t4wawaicrfsc/9d7vk=">aaacahicbvdlsgmxfm3uv62vqgsxbojfakgugrf04algxmuf+4b2ljk004zmmknyryzdbpwvny4ucetnupnvzlrdaoubwmk5997khi8sxintf1u5ldw19y38zmfre2d3r7h/0njhrchr0lcequmrzqsxrakcboteiphae6ztja8zv/3aloahvinjxnyadcx3osvgph7x6pe+ktuvtip7gxb0fwd3wun7xzjds6fay8szkxkao9evfpkbna6ybcqi1l3hjsbniajobuslvvizinaxgbkuozietlvjdieunxplgp1qmsmbt9xfhqkjtj4enqkmciz0opej/3ndgpxln+eyiofjonvijwwgegdp4afxjikygeko4uavmi6iihrmzgutgro48jjpnducu+bcnpfqv/m48ugynaayctafqqmb1ebnrfgkntereroerbfr3fqyleasec8h+gpr8wewf5uw</latexit> Maximum Likelihood Estimation ˆ MLE = argmax = argmax f(x (1),...,x (n) ) X log f(x (i) ) Maximum A Posteriori i ˆ MAP = argmax = argmax f( x (1),...,x (n) )! nx log(g( )) + log(f(x (i) )) i=1

39 Gotta get that intuition

40 P(θ D) For Bernoulli l l Prior: q ~ Beta(a, b); data = {n heads, m tails} Estimate p, aka q This is the beta PDF This is???

41 P(θ D) For Bernoulli l l Prior: q ~ Beta(a, b); data = {n heads, m tails} Estimate p, aka q This is the beta PDF Product of thetas and (1-theta)s = argmax = argmax h 1 log a 1 (1 ) b 1i +n log f(heads ) +m log f(tails ) log 1 +(a 1) log +(b 1) log(1 )+n log + m log(1 ) = argmax(a 1 + n) log +(b 1 + m) log(1 )

42 P(θ D) For Bernoulli l l Prior: q ~ Beta(a, b); D = {n heads, m tails} Estimate p, aka q = argmax(a 1 + n) log +(b 1 + m) log(1 ) = n + a 1 n + m + a + b 2 That s the mode of the updated beta

43 Hyper Parameters a b Hyperparameter a, b are fixed p Prior p Beta(a, b) Data distribution X i Bern(p) X 1 X 2 X n MAP will estimate the most likely value of p for this model

44 Where d Ya Get Them P(!)? l l l q is the probability a coin turns up heads Model q with 2 different priors: l l P 1 (q) is Beta(3,8) (blue) P 2 (q) is Beta(7,4) (red) They look pretty different! l Now flip 100 coins; get 58 heads and 42 tails l What do posteriors look like?

45 It s Like Having Twins argmax returns the mode l As long as we collect enough data, posteriors will converge to the true value!

46 Conjugate Distributions Without Tears Just for review Have coin with unknown probability q of heads Our prior (subjective) belief is that q ~ Beta(a, b) Now flip coin k = n + m times, getting n heads, m tails Posterior density: (q n heads, m tails) ~ Beta(a+n, b+\m) o Beta is conjugate for Bernoulli, Binomial, Geometric, and Negative Binomial a and b are called hyperparameters o Saw (a + b 2) imaginary trials, of those (a 1) are successes For a coin you never flipped before, use Beta(x, x) to denote you think coin likely to be fair o How strongly you feel coin is fair is a function of x

47 Gonna Need Priors Parameter Bernoulli p Binomial p Poisson Exponential Multinomial p i Normal µ Distribution for Parameter Beta Beta Gamma Gamma Dirichlet Normal Normal 2 Inverse Gamma Don t need to know Inverse Gamma. But it will know you

48 Good Times with Gamma Gamma(k,!) distribution Conjugate for Poisson Rate o Also conjugate for Exponential, but we won t delve into that Intuitive understanding of hyperparameters: o Saw k total imaginary events during! prior time periods

49 Good Times with Gamma Gamma(k,!) distribution Conjugate for Poisson Rate o Also conjugate for Exponential, but we won t delve into that Intuitive understanding of hyperparameters: o Saw k total imaginary events during! prior time periods Updating with observations o After observing n events during next t time periods... o... posterior distribution is Gamma(k + n,! + t) o MAP estimator for Poisson with Gamma prior is (k+n)/(! + t) o Example: Prior for rate is Gamma(10, 5) o Saw 10 events in 5 time periods. Like observing at rate = 2 o Now see 11 events in next 2 time periods à Gamma(21, 7) o MAP rate = 3

50 Reviving an Old Story Line The Multinomial Distribution Mult(p 1,, p k ) p(x 1,...,x k )= n! x 1!...x k! px px k k

51 Multinomial is Multiple Times the Fun Dirichlet(a 1, a 2,..., a m ) distribution Conjugate for Multinomial o Dirichlet generalizes Beta in same way Multinomial generalizes Bernoulli f(x 1 = x 1,X 2 = x 2,...,X m = x m )=K Intuitive understanding of hyperparameters: o Saw åa i - m imaginary trials, with (a i 1) of outcome i Updating to get the posterior distribution o m i= 1 After observing n 1 + n n m, new trials with n i of outcome i... o... posterior distribution is Dirichlet(a 1 + n 1, a 2 + n 2,..., a m + n m ) my i=1 x a i 1 i

52 Best Short Film in the Dirichlet Category And now a cool animation of Dirichlet(a, a, a) This is actually log density (but you get the idea ) Thanks Wikipedia!

53 Example: Estimating Die Parameters

54 Your Happy Laplace Recall example of 6-sides die rolls: X ~ Multinomial(p 1, p 2, p 3, p 4, p 5, p 6 ) Roll n = 12 times Result: 3 ones, 2 twos, 0 threes, 3 fours, 1 fives, 3 sixes o MLE: p 1 =3/12, p 2 =2/12, p 3 =0/12, p 4 =3/12, p 5 =1/12, p 6 =3/12 Dirichlet prior allows us to pretend we saw each X i outcome k times before. MAP estimate: p = + k i n + mk o Laplace s law of succession : idea above with k = 1 X i + 1 o Laplace estimate: pi = n + m o Laplace: p 1 =4/18, p 2 =3/18, p 3 =1/18, p 4 =4/18, p 5 =2/18, p 6 =4/18 o No longer have 0 probability of rolling a three!

55 The last estimator has risen

56

57 One Shot Learning Single training example: Test set:

58

59 Is Peer Grading Accurate Enough? Peer Grading on Coursera HCI. 31,067 peer grades for 3,607 students. Tuned Models of Peer Assessment. C Piech, J Huang, A Ng, D Koller

60 Is Peer Grading Accurate Enough? 1. Defined random variables for: True grade (s i ) for assignment i Observed (z ij ) score for assign i Bias (b j ) for each grader j Variance (r j ) for each grader j 2. Designed a probabilistic model that defined the distributions for all random variables z j i N (µ = s i + b j, = p r j ) s i N(µ 0, 0) b i N(0, 0 ) = hyperparameter r i InvGamma( 0, 0 ) Tuned Models of Peer Assessment. C Piech, J Huang, A Ng, D Koller

61 Is Peer Grading Accurate Enough? 1. Defined random variables for: True grade (s i ) for assignment i Observed (z ij ) score for assign i Bias (b j ) for each grader j Variance (r j ) for each grader j 2. Designed a probabilistic model that defined the distributions for all random variables 3. Found variable assignments using MAP estimation given the observed data Inference or Machine Learning Tuned Models of Peer Assessment. C Piech, J Huang, A Ng, D Koller

62 Improved Accuracy Before: After: Some students were getting very erroneous grades 99% within 10pp Error is based on ground truth assignments. Results are across all assignments (~10,000 submissions) Parent s Club