TTIC 31210: Advanced Natural Language Processing. Kevin Gimpel Spring Lecture 9: Inference in Structured Prediction

Transkrypt

1 TTIC 31210: Advanced Natural Language Processing Kevin Gimpel Spring 2019 Lecture 9: Inference in Structured Prediction 1

2 intro (1 lecture) Roadmap deep learning for NLP (5 lectures) structured prediction (4 lectures) introducing/formalizing structured prediction, categories of structures inference: dynamic programming, greedy algorithms, beam search inference with non-local features learning in structured prediction generative models, latent variables, unsupervised learning, variational autoencoders (2 lectures) Bayesian methods in NLP (2 lectures) Bayesian nonparametrics in NLP (2 lectures) review & other topics (1 lecture) 2

3 Assignments Assignment 2 due Wednesday for the report, please use either pdf format or a Jupyter notebook (no plain text) 3

4 <latexit sha1_base64="u0xwf3yzhfhquldgyizfugbiywg=">aaa1bxicnvtbc922ezbtw6reknb61e6hrasjhuuqjhonmaseiwkrdiekqtpybqksaukchki8gqdpxqz7w/pp+tbx9qv/obsazyejgjiczdgigg8xi8xuynfrucrmyl29f91643vf/8epf/tmj9d/8tof/fwxb739yy9exvkqpg/zjodfbutqhgx0uwqyov8vnji0soixwevdbp9ysrlgexyifwu9s0mcsteliysq87dvpdvkkml04udembah2hjhizakhk9v+smre55wy4b6tj+loazm9bb+8owesllf8r54pufxsubn1al2p/a2fjypaubuhdzy1bityc5ph5e32pb6/na/u8+6zvepctdy/5lqemnrfwvhg59fdgw94901qvwjqwqqosuzmg0oxz9/a2nvd0/9epbhqpnywgt+js/ffu+wh+vhmdjmqkgfjvykevyrllmy0hrdlwutshhjynoknxljqtir1ptx3hburn445/avk56q7vjujbvikqaarb0isw0rxw2nprx/efaxrcglzuld0bhmpjl7aetexdhoivnabwk5a1m9cei4csvy3hqvmwjssris7w2lknquwzwt+ruxj8wehfn+bvomkvf81q8n8ynlypln/epxcnzltpzrkcwp5ursgzmahhglozaaof/bqd0macpe0xe3vruh/jbhrjcdqsfeheypl+weagiqxwuonfgakck+ulmwpvdhodynqu7sfp0ogzglzdhihpeqec5rsx3gkyydoa2rrmbhszzd/ezse8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkohbhoyixmuidjipkqkgwctqcj5ruz88jmvfmnbosnyegqpqp6hxyevcqyyhl1o8qd72amjl6ixmuxalq4eqmuktl0smjnxlbzrmwaumt/itekdwraqugglcwzwmla+mjhz1tuftutugcgysap+p9nbvm610lfmissyre1cfrjhmh1hyvq0fglowrf5x4wdvsanu+zdfjn8wnvp0dazbsrkotemvjcxfuy09rrzv1nsyets7nx0/pmu0seksefgaxi72n51zwhboqfl9p5s+z/feeg6zvnlwzht2aokob55cjyoooh4dxi0qvy/pr+mrzmlcvwdhpdtnjy3xkxgjak3hl8lqkuklwa6tculo4zlw3psfvsxh0jsmgl4eh0u7voo8eln7d5/tcbuxslivrueyv9odxebxvozlc6xwqui6evfpp8lzuqka2spnmzmrnhis2qqwwz9byriwf7lhuphzizdbig1+uuvbyoilnvfdtxl3miyusu22o8l23gswxqd5rjm+ha69sayjgfoebvxawgfvidda8ytcxtbp0buchjgugrgikuwtp19ud10i0fwhsmnciprbsqa4fopgojobft84ymos4ycxarcwydhbxkthlgoyuqlvzuo9rff3jofasg/cnbzwbksliyg04goyysuwc5t0ngk+cqb785z9akzrg4qlbxbwradd/bc+9bb35naiqmlpcl6ja7uw+rljhfpamgssunojeshham41w5rypexti3nuhx5xv7mkpfznaetf+hq3yxeftz3druy0ccegsw1lezealyrr54pf9a8/j2fhtxx25jc6zb+4i4kqvnkpx10b6qhfk1sfph9eccqxzephp42i4dvq3wxvohstueapnlvecadc61psle29wybq6y9shnr4fhs9n+txbmngsiezdjshftecverauldo6efc2kmhanbenbsxd2jhzzywclbobiscjjc4gyczksycvc2mkgwssifkesu1q/ugeaxsytar6n1itt6msydshdwc4jymexzct8cytisqd1axrlnkqdlsynpwboxccifu91ltcgip0/gxy/bqeowcamczkoo+psnzy+iihccsvink3zecdpgoxmdqynuqkte/rtryvkt6u8jmoscygmy5xktnhhqaud0e6iphystluv0f+cckxnwgknlyedvdw0jyhrflh/mw7k0b/zhewlsbymsaaegbbh+7ydw140gqqfq6gxlql4g5fn7bkthuy1x6yal1znkdg32avc6gzlhpkln97wo5fkiwb8pih3q486pwdtdc5pe8cw2bbkxt72gqzmej30o5hyjkijvjgtbxu7ydvur5tymujzcets1l7ln/3urtciazuhes2xguu1lk9ivnjh1klryldaq80kh4kchmelt1edxcr/h2qb+5um6w6tjr2f3hyflcyjqim3frppdw+eo381alzf08u0rv1vn6izmgzduxdowlg9ysncftb6ektmqpxtbmajgpponfssnvdmxl7b/gkq4sjfqyq1ojwqwps8ytgfvglhx1so6wqpf7vssrfyxpveefdms2vdfjmdmxpoqipfjoeyt2hgyj72mqdcjhhrxs5ncuc1u7ozhxbxnqmi0zadlm7sge3xhdqv3q7yag0h/lq2vjzc/10dzqfhgdnkcimdvgkvtyxzcikcruzedhjzeststparubedrrycm0b2xdaypty0sicdrma7i2pgj4n+6aqpu6jr8cfd3uigdzvkolpohcuo2gokbdckj0mk+z+rn8wqg/tn75ud5jwee9bt9wemacsq66egbtkizwf8zs/6cnzhaf+yy8kzhs5xre5kv1splxcr0yqxoswokohic/ezhk7tne78siqqn8a40ybn1swnyaohrk6rb0r7wx5uqzzmscztl2uegacaeywj+5ip3hpepj1bhp4luc8flxnixh9d3iavr0/vnrkjjgl8otq4p7jhnkochutgynlkeupj6lzuj0fckkzlqmdsj2fxszfcw8oew2jgda0jewxsj2ghxgd7ailo9corme8qwn0huqkr01sh+fqnxtayt8tvjsgqwwahgc0orhcidiziaxgjstzt6kq8fzebgaeelyjnqj76gm11dpxkab1nlcvdr9uosv6hjipfvuqsdixjmyrpolebjultse5usvtnl2mrjnikpu2djcxwd0b6ckrjdejderygllybnoqzqj/lfjcgmhwnsjrnteu6a19g9jvjbvij7lljkjbdobc3riw9iu2glwsh1cgkma5zbzukquho6r7aa/9y2guea1vdawybkgzh5jsovdvclzuih2g0diou9a3c1nlg5runklaodbjvadxopbr91eczlgjhc1qjrhce1v9rltwfbox0inxmrprj2vtmvde67urcdmdbxcz5z63fgnmbqyq2hqvmdrshea6e7cgkrsc8paj4awsax3h43og1bmi4bjordcq1itznmhrinhsfmdmmkjnoi/ystiwuc5tolzwrybuzyqhbmex1+0qeqmlpmu6iloietsmfodngip5pqvkgrjn1c9tulp45x0jvvuyxup6uxu8ptgxafpbn7klebgfq9xg9tbu47gdt1elgrjdun3dhqq3vzmnkexuewkkorluy3wjdmxttbvaa+pm9rneg6uspcadpr9nn8sd8ctdr7q3jqax8n4yejk1bam31mcptlf4lgpiekdvlfwvvbrgkxytjqh8zgo4mffagde2afshh1mfq20c5k+liw7eiyadictfzf2c2mzakjh0uer57knjsggkhw3qmwlcigqantsnb8+6gbfr3bwpcp52okfr7a6hh57sayvwdjfelnebj4rw+sdlicmi0uhcsjctcbwqxubjqgl9mkr5s0cvvv+yb6wg7qkniwk9rlep5koouvgzqnnjhuk9vr2uge2j/rt1sl/rantczst5/atwigpv2stwi+3hbuj3gaxftmhkkltrzrlngxng5lny2aam7hnvsupybhsot0ze5cqyirdphjunal6dgqtq5c2y5von9d7tz07glhkmpgozfegslp77tcq1toxjk8zj8lsjdjr8lqfivxogcaiiikiiqv1pmxx70oxrja28qwzj0orxlnp3p99a/cuzwkgazfqgompzrvu0xygzckvjtzq+2zc4b1qzvfbhs5r1chrt8t80f/wladvftsmc6yxwkyezkoqoq1b11alk40l80r5qs45jnwpzmheqqvmgz7oz8znzzccj27jxfgqumndwaysgbqfwqqaa/sf+ncpzb0xsyrfo/l+6lg0ylnefnnt06xui433e/z9wx8wv+zgbhhndcwre+djswbe583vic6w2c/crhuln0zmui6yuoo0w0cc4bwfbrgn7poxgvv6rszk60z8x5lr0cca7xa1g903klfeqr7my4sqb5y+rfqywkj+ktkf++pjo0r0we79mk09+nnyduyfdiwzdgp1lhctmu7dz9vrxfftt/ukuz0tplxhydc0qkecercwqdit3a9cd65++om38d/hyefwcnq1tnhb863anofbespzqhw4hd7pik7fez6aqsxvtmdup2sbu/v2ttajschvdl+i0yfdb56mnuf2dq9usaedky8dbe38ur5qhvvmxee6hupabiwe5xzsdeq4l1sreqlbpg7igexlobbzcml9xy/ievsfu5f9p48pqk1fmwdjhuaurvnxvvncazchc5hwasx98z2k1krxdnc9lfcymujcbbutjffpwpt3j3i2sum3lzuk8benoyv+dlp4twidk4c9xuhwr3okrfrpemtjdtlivwkq+2ly+yiruiuvk7sbpcvxtkonucrccyi/ww2/fdbcv4tgplymqt825+qusa54hibrboprbu/vfugaxvw9rfaypcb8yggiyxgc1rky hwqhfefztqvldvyfnx6mg6h3nwq9w8lzihvrs6tom8qw67tt24ghoc7b2vuiah67qetce4rvtcztu5r4ek3b+2ce81ru9ulvmdocp2sp117y9wzv/kfnxze+rybgmm9/4rzmj809k7i8v7u2o9nzhf39/45m/n38+8+bab9b+shz7bbt2p7vp1p6sha89xwtv/fpwv2/959z/3//f/v/f/+3932nog7caml+t9x7uv/n/igns0g==</latexit> <latexit sha1_base64="lvj5f1yayqj0pimo0ef8s222bnc=">aaa1fxicnvtbc922ezz6tdvb0j72ha2kqr1lqo4tj5mknoliq3yniqracm6irafjhb5ivbkaz8um+wp7d/ov+tq+tlsazyejgjiczdgigg8xi8xuynfrucrmyp39f63/4ic/+vfpfvrwzzz+/otf/urxb7/zmy9fxvkqvgjzjodfb0tqhgx0hwqyov8xnji0sohxwdujbp9qsrlgexyqfwu9t0mcsteliysqi3fww+0skuyffeyfcrgcjreecelc+ma2nxi54wm1bkjv+fmaympe7+gpx06ojpvd76hnex6nzh5rlwr/e2/bz5mibk7vuy1vltsr5px2ehmlha/p73t2rvsw9ztlsks0bokvn7y3tne98cvlv/a73j2jyp8q5axewz6vaud5xsxbm/t7++rhsz9gzcfmwvnzcvhoe+t+lidlsjopdha22i/keuw4zgfc6w2/flqg4rwj6rl8zisl4rxs5lb721ateeocw79meqq2s1grvihfggasnslmnqx0tz2vcvzrecwyopq0c3vh4zlxzo6hbxkr46cfzaefjoqmzpxcceeklgcbg71ucraxybk8n5rk0rkm87to18acfbmwzvu1azlixvnzvzbmpyylzz71q8cp8c0z2a+lmd+ue0knzmgmyiym2mig/gij9pikkxtn393xvhzww4z73gwenxgtmqvelnscieecl6byrmmmcpvqtseuwt+d2pweoun7daimxiw2r4st3qviuutf9ofngb6etkyagr0lcq7zm0nvg0mpje5nrczavhdqiengejc1wrwqkww5zjdczozk3oedkvjocr6gho9wmcmqlymyykzkiojmeumjee7lqfkifh/jwdfbwhnkked9of4hfwmtshy+apgw+ssros2jm5pnqzqmhapqpk+8kpmy1juf0jhllzrcyk9iameuevaapgkfc5puvpo6wttvpk3rbomimleifkba33tgttkxtcfcsazuqx0srtifrmfxnrrgzfqyekona7wjjvdnxxuxfsn46rprocqo5towdvwqlbrhnfa0crwzb3pk7e5+fvlkvnhgiktgxbtyot54fm4jqzgfyq+ebfmcxrnjomz61ugx0dnrcak+fxo9kqkienx4papupb64oueapxfdhr73uf88scrpyoygtr6/cktnjc1gebtgcagp58bxy3/+qorbb5ka+gp8fo1qqfpw1d4/fe7h1ebiyluwbwfctxcwg1/5mpyvsvojietixkedcs/ncmhqk59ltkbyylbqq1pmfw4py8jcyb5dxc8nqu4onvjlfgupi5baxpc05b1hmllgntmujtt1k8gsn+yrtfp0upyimeqj6hmy1anjhvulqgkwgxwzxbbwgsyroi4oqkflx+ffndcyfacx3irgmjaablmulyryodam5yfosgd0uog1xmrvic4wx7er2zjgemj7wxz1pv+odg0zfxkilyj201gynbymimoofkmbjgneu8dygtnemc/pacwzowreakqv0vkhg1fo8svo+8+d0bkdkywc+ro+xupusyybarjhgmlzyarriyqjcmc+uudxpnsd/yhf9fvozhqh6zwrmyfrmo2t3guwx3ulonhxfiembs387ikwc3e+dxfahn4zvzncvdu40tmg7tc+jefpypcnzf+6pysnuitr3tawmucnit6tfrow74tlp7zufkvnsitjb1ggm2onf2ihhva3o0nwdah18ch50stfsn2zlzkolgqcyrxbtnlrozfptqurnstjbiwp0qjg0n3dmzwgqthywyacilcsctbmomzeoglwhpi4ekeyp6huloaf3yvgf6fbci+dzak7sje8vabq9noy8gb8c0e/mnlioladqpujtzea20mdscaqava4twpnw5xioidp5k2p20btgegzmu5djsqevvwpogbgorlit7ps1dgdxnsfe1gjzqpk5m6lcfl5ddrvksdesmbdkovso7xr0hrxzheyoawkjyblvg/wsqmj0jjzs8ggz0asucoetxxf7hujfdf8wtfpxcwjakmrigwh4f2a4me9eeenwmhca6poinrja1ybywgtv8uwq+ddvd4txkrwqjsy0bzi58elfvrcotgvdtbt4eosgbuzudxbamnmoyox+8rrmjpbdnlvygkzicvsrbyeboyc1vc9ycplcmuhllfr+5z/9ru3wiao1itonccgltw6pjj52ovri66d3wqtnpn8jaoyhi3crg0qq/x/kaxsb5quo7swdbry2xvqsisijt5a6d47onyb/twowrvnf9my9b/dom5imnvnqaaly/widqtzwotlljkjcu4racdotdp37rn0qpkbew3zy0gksxcfedbgveji7fmk4xtwpov9yzmse6t81uolxcgl7rndxjnnlg3xzq7gzkeddratnms+hrkr+9irgaqt4gv1tttfxdxs6s50wmvndoheqw64uludnl13al9xo2adtb3wm9pa8xd/d32ubyrlzjnrjazwplu8tczbcxiamrm5ygrlllpzwkbmxg4y2qpsgtlt28j6wing4yme0ujogfdvqhezzsiq7bnw57oxs+7jcd6jp1rkbqd5as3cg8isnmf0befenu0de/13aqv3powe/xn5uirucshxoysis2h/g5pesr8kudfgmocc8aoscvusv9bqvcqqxok1yifqnkbiiv3gcaorvzovgrkkdcgongazzvv5wmqx6wuq4eku1ve7gn8zjhmk9ftnkcagdusyzsy6zyz3mtdm57ec5avxy8zcbz/xv5c1a8prt/biqwivhlavo+4h7tjhiwlhkhz5njjcej87o/bgljmpudhus9hsxtxnadk3smi4exnizsmug9houaa+yriaa3k6/apklgdyfknkrmbx3gazv6qwm8evc3pkgeg4uatqaq3ykwgcktyywowu+pk/fwxg5jhhi+odsr+uhrtnxr9zojdtsxlhqtvamkey2sdbbxkuk6lcrvq6sbrg2uje0luvmlbtw9jk2y4jgatnessfjdgekjqi7tcxxp2rqw2afwks+gfpzbwibqvjtcup7rlumefrnrv461sswwyyzdqnafwt66svsenig2kb1sh5bmumyd1jaeiagp+wip/gnhr3kova0ngaraml2z76jq7xbxviichthajbjrtwpb54gvvzsgwdowsfslcturavqza82yiixnusuyxrbuf61v1ny2odip8rm5uizlbzfvreu4q3m7gavzmesdtryl5gugkdp2ljg77b5aobcyh5aaafp2moktlwh84efxo9cqcomqaebukdcw2jtb6cfz7hxazdcozdin8lpsmrosbkjvvq6cvwsiow7bzbn/fdgmkpuw5foqizgnrrpbthvyhj2yu1zieitdxpxeij9pcti51lml16wkm7njuxotbzy5/slxmonbvmftuafvoho7chnrkiu5jn/zuub425t5bl1eszooejbttlixzcc2wvqfvd5uap/rovdc3ms4upbrflpcnuzyxap762ged+obislx25p+twmyshedxz4mpg9q3vr3wkulmsnruobmxzichxerhxhhh0hz9jlvnnauzppqmu5amalanbrcxnjnpwuciizjn0eeyzy6ihuocnwils3ciaqgbbn1/vq4g3wf28h3oodpjxyc2+os+u3bmrxiyrrbzhoreezsrxe4gjgulh0mihaxgcctbgswiczyi0wwtwj41vsl+8ju6cp6sijusxcfzqlkfbk1j5y7vijy79vjvtk+6rdbj/+xdrtn7uyc27dibt5eku1qvdwsbo56g8t25yrkjk290prxljvvslaumgfu4z75hqecrbdn9mxmqeiiew6r4lhxpooxqe0oxdosb6lfqo83du5g4jlb4km03xvc6u61w6hw61wwvme8jrovsg/a034kcjvlgcmijoqelntzmf89f12ywtrgl8w+dkv6zt1z//cn3lsyp4gkruibpj+2vrgtc4cspvq08qjv2wmbd0a5xwczoq9rh6zflfnf/ztthrz5btpmsv5qmreyj0lmngxdq5wsnjbme+wpoucwvj86hxgrrppnedqfgted3qqcucvzyufijxufm7iaanxqqqio7bfgqz+w9cyskrsnyvutx9ii5xjtte9nl7qpnn5p2pdtfcd8hhuxxq3xma/tny8lgxmdn1cnottmpwuz7rxemjnrumbcqdmma3gg1366xta6g15ohogxmzctg/edz0zfa2u2w9umdt9irnkj+tive6oewpoqackci/qpzbfsqyfuknehew/ctpbgz9s7nbfqmc4wjnvbqrrqug8/v6913u8/1znm9uyycowfxdoobnfiwmkmyq92vwrdvf7pj97gf48nn9vl6duyyw/nt2p7dm7pkwcb1ef2+cclop2j8nrmzxlzonmnjqvc3bevtulecnl3+bwndhx682ssobk3e3vdohhmerwgaoi39yxyqgfqctfxgac2ype452bnumk9pfrns9jycrahzcodtzhxsov7ehhx2y/yvrw8e3jybfmaarj3mhctyoqvyhlegamdw7qkkomp7ywljr22g5ulvjbjfbii2ggluxrworw/wbovlb9t2vkej2kilfqlmmk/fehyoezpqrj6nsnlpuqdwe6yqxarsopyz5krx4zelxxzod3kgyljgj3djrfkepwzbvjlwxh6jqq3twjq/by5v6emdcxjafxzb1x3z7/uq9io5uohsslsa+ydcxndxrcolpc993aafzrkqiic4jpwze+6om0+zgrv72joe7cc50uu1jjwnzwv59vjp7yhd3oeg7xzrqf8afxxgyyur51a2jeq9ycyhporduazruib2jw3a3zcvm645u3dqvn7nlpqfl8hw2dy/my/mhmzfyjjf3x5f2+0vzf6+/ubn/65+fozt9z+t/ahtttro7up1z5de7p2svzilvz/5/q/1/+z/t/3//dg+8hogz0n/cf6q/pbtd7pgw//d9gm8z4=</latexit> Modeling, Inference, Learning inference: solve _ modeling: define score function learning: choose _ Working definition of structured prediction: size of output space is exponential in size of input or is unbounded (e.g., machine translation) (we can t just enumerate all possible outputs) 4

5 <latexit sha1_base64="u0xwf3yzhfhquldgyizfugbiywg=">aaa1bxicnvtbc922ezbtw6reknb61e6hrasjhuuqjhonmaseiwkrdiekqtpybqksaukchki8gqdpxqz7w/pp+tbx9qv/obsazyejgjiczdgigg8xi8xuynfrucrmyl29f91643vf/8epf/tmj9d/8tof/fwxb739yy9exvkqpg/zjodfbutqhgx0uwqyov8vnji0soixwevdbp9ysrlgexyifwu9s0mcsteliysq87dvpdvkkml04udembah2hjhizakhk9v+smre55wy4b6tj+loazm9bb+8owesllf8r54pufxsubn1al2p/a2fjypaubuhdzy1bityc5ph5e32pb6/na/u8+6zvepctdy/5lqemnrfwvhg59fdgw94901qvwjqwqqosuzmg0oxz9/a2nvd0/9epbhqpnywgt+js/ffu+wh+vhmdjmqkgfjvykevyrllmy0hrdlwutshhjynoknxljqtir1ptx3hburn445/avk56q7vjujbvikqaarb0isw0rxw2nprx/efaxrcglzuld0bhmpjl7aetexdhoivnabwk5a1m9cei4csvy3hqvmwjssris7w2lknquwzwt+ruxj8wehfn+bvomkvf81q8n8ynlypln/epxcnzltpzrkcwp5ursgzmahhglozaaof/bqd0macpe0xe3vruh/jbhrjcdqsfeheypl+weagiqxwuonfgakck+ulmwpvdhodynqu7sfp0ogzglzdhihpeqec5rsx3gkyydoa2rrmbhszzd/ezse8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkohbhoyixmuidjipkqkgwctqcj5ruz88jmvfmnbosnyegqpqp6hxyevcqyyhl1o8qd72amjl6ixmuxalq4eqmuktl0smjnxlbzrmwaumt/itekdwraqugglcwzwmla+mjhz1tuftutugcgysap+p9nbvm610lfmissyre1cfrjhmh1hyvq0fglowrf5x4wdvsanu+zdfjn8wnvp0dazbsrkotemvjcxfuy09rrzv1nsyets7nx0/pmu0seksefgaxi72n51zwhboqfl9p5s+z/feeg6zvnlwzht2aokob55cjyoooh4dxi0qvy/pr+mrzmlcvwdhpdtnjy3xkxgjak3hl8lqkuklwa6tculo4zlw3psfvsxh0jsmgl4eh0u7voo8eln7d5/tcbuxslivrueyv9odxebxvozlc6xwqui6evfpp8lzuqka2spnmzmrnhis2qqwwz9byriwf7lhuphzizdbig1+uuvbyoilnvfdtxl3miyusu22o8l23gswxqd5rjm+ha69sayjgfoebvxawgfvidda8ytcxtbp0buchjgugrgikuwtp19ud10i0fwhsmnciprbsqa4fopgojobft84ymos4ycxarcwydhbxkthlgoyuqlvzuo9rff3jofasg/cnbzwbksliyg04goyysuwc5t0ngk+cqb785z9akzrg4qlbxbwradd/bc+9bb35naiqmlpcl6ja7uw+rljhfpamgssunojeshham41w5rypexti3nuhx5xv7mkpfznaetf+hq3yxeftz3druy0ccegsw1lezealyrr54pf9a8/j2fhtxx25jc6zb+4i4kqvnkpx10b6qhfk1sfph9eccqxzephp42i4dvq3wxvohstueapnlvecadc61psle29wybq6y9shnr4fhs9n+txbmngsiezdjshftecverauldo6efc2kmhanbenbsxd2jhzzywclbobiscjjc4gyczksycvc2mkgwssifkesu1q/ugeaxsytar6n1itt6msydshdwc4jymexzct8cytisqd1axrlnkqdlsynpwboxccifu91ltcgip0/gxy/bqeowcamczkoo+psnzy+iihccsvink3zecdpgoxmdqynuqkte/rtryvkt6u8jmoscygmy5xktnhhqaud0e6iphystluv0f+cckxnwgknlyedvdw0jyhrflh/mw7k0b/zhewlsbymsaaegbbh+7ydw140gqqfq6gxlql4g5fn7bkthuy1x6yal1znkdg32avc6gzlhpkln97wo5fkiwb8pih3q486pwdtdc5pe8cw2bbkxt72gqzmej30o5hyjkijvjgtbxu7ydvur5tymujzcets1l7ln/3urtciazuhes2xguu1lk9ivnjh1klryldaq80kh4kchmelt1edxcr/h2qb+5um6w6tjr2f3hyflcyjqim3frppdw+eo381alzf08u0rv1vn6izmgzduxdowlg9ysncftb6ektmqpxtbmajgpponfssnvdmxl7b/gkq4sjfqyq1ojwqwps8ytgfvglhx1so6wqpf7vssrfyxpveefdms2vdfjmdmxpoqipfjoeyt2hgyj72mqdcjhhrxs5ncuc1u7ozhxbxnqmi0zadlm7sge3xhdqv3q7yag0h/lq2vjzc/10dzqfhgdnkcimdvgkvtyxzcikcruzedhjzeststparubedrrycm0b2xdaypty0sicdrma7i2pgj4n+6aqpu6jr8cfd3uigdzvkolpohcuo2gokbdckj0mk+z+rn8wqg/tn75ud5jwee9bt9wemacsq66egbtkizwf8zs/6cnzhaf+yy8kzhs5xre5kv1splxcr0yqxoswokohic/ezhk7tne78siqqn8a40ybn1swnyaohrk6rb0r7wx5uqzzmscztl2uegacaeywj+5ip3hpepj1bhp4luc8flxnixh9d3iavr0/vnrkjjgl8otq4p7jhnkochutgynlkeupj6lzuj0fckkzlqmdsj2fxszfcw8oew2jgda0jewxsj2ghxgd7ailo9corme8qwn0huqkr01sh+fqnxtayt8tvjsgqwwahgc0orhcidiziaxgjstzt6kq8fzebgaeelyjnqj76gm11dpxkab1nlcvdr9uosv6hjipfvuqsdixjmyrpolebjultse5usvtnl2mrjnikpu2djcxwd0b6ckrjdejderygllybnoqzqj/lfjcgmhwnsjrnteu6a19g9jvjbvij7lljkjbdobc3riw9iu2glwsh1cgkma5zbzukquho6r7aa/9y2guea1vdawybkgzh5jsovdvclzuih2g0diou9a3c1nlg5runklaodbjvadxopbr91eczlgjhc1qjrhce1v9rltwfbox0inxmrprj2vtmvde67urcdmdbxcz5z63fgnmbqyq2hqvmdrshea6e7cgkrsc8paj4awsax3h43og1bmi4bjordcq1itznmhrinhsfmdmmkjnoi/ystiwuc5tolzwrybuzyqhbmex1+0qeqmlpmu6iloietsmfodngip5pqvkgrjn1c9tulp45x0jvvuyxup6uxu8ptgxafpbn7klebgfq9xg9tbu47gdt1elgrjdun3dhqq3vzmnkexuewkkorluy3wjdmxttbvaa+pm9rneg6uspcadpr9nn8sd8ctdr7q3jqax8n4yejk1bam31mcptlf4lgpiekdvlfwvvbrgkxytjqh8zgo4mffagde2afshh1mfq20c5k+liw7eiyadictfzf2c2mzakjh0uer57knjsggkhw3qmwlcigqantsnb8+6gbfr3bwpcp52okfr7a6hh57sayvwdjfelnebj4rw+sdlicmi0uhcsjctcbwqxubjqgl9mkr5s0cvvv+yb6wg7qkniwk9rlep5koouvgzqnnjhuk9vr2uge2j/rt1sl/rantczst5/atwigpv2stwi+3hbuj3gaxftmhkkltrzrlngxng5lny2aam7hnvsupybhsot0ze5cqyirdphjunal6dgqtq5c2y5von9d7tz07glhkmpgozfegslp77tcq1toxjk8zj8lsjdjr8lqfivxogcaiiikiiqv1pmxx70oxrja28qwzj0orxlnp3p99a/cuzwkgazfqgompzrvu0xygzckvjtzq+2zc4b1qzvfbhs5r1chrt8t80f/wladvftsmc6yxwkyezkoqoq1b11alk40l80r5qs45jnwpzmheqqvmgz7oz8znzzccj27jxfgqumndwaysgbqfwqqaa/sf+ncpzb0xsyrfo/l+6lg0ylnefnnt06xui433e/z9wx8wv+zgbhhndcwre+djswbe583vic6w2c/crhuln0zmui6yuoo0w0cc4bwfbrgn7poxgvv6rszk60z8x5lr0cca7xa1g903klfeqr7my4sqb5y+rfqywkj+ktkf++pjo0r0we79mk09+nnyduyfdiwzdgp1lhctmu7dz9vrxfftt/ukuz0tplxhydc0qkecercwqdit3a9cd65++om38d/hyefwcnq1tnhb863anofbespzqhw4hd7pik7fez6aqsxvtmdup2sbu/v2ttajschvdl+i0yfdb56mnuf2dq9usaedky8dbe38ur5qhvvmxee6hupabiwe5xzsdeq4l1sreqlbpg7igexlobbzcml9xy/ievsfu5f9p48pqk1fmwdjhuaurvnxvvncazchc5hwasx98z2k1krxdnc9lfcymujcbbutjffpwpt3j3i2sum3lzuk8benoyv+dlp4twidk4c9xuhwr3okrfrpemtjdtlivwkq+2ly+yiruiuvk7sbpcvxtkonucrccyi/ww2/fdbcv4tgplymqt825+qusa54hibrboprbu/vfugaxvw9rfaypcb8yggiyxgc1rky hwqhfefztqvldvyfnx6mg6h3nwq9w8lzihvrs6tom8qw67tt24ghoc7b2vuiah67qetce4rvtcztu5r4ek3b+2ce81ru9ulvmdocp2sp117y9wzv/kfnxze+rybgmm9/4rzmj809k7i8v7u2o9nzhf39/45m/n38+8+bab9b+shz7bbt2p7vp1p6sha89xwtv/fpwv2/959z/3//f/v/f/+3932nog7caml+t9x7uv/n/igns0g==</latexit> Inference with Structured Predictors inference: solve _ how do we efficiently search over the space of all structured outputs? this space may have size exponential in the size of the input, or be unbounded complexity of inference depends on parts function

6 <latexit sha1_base64="n9u9ohsed6elirzxrk0qrbtqcq0=">aaa7vnicxvtzdxy5debyssztwtzxy17kizlqfsmw5xgbrucmlslszmhorstslkopqgpddbe2ash2t0qvx5m/lx+te4hqrsjsjouxu2egbedd3xbxcbewqjim5phx/332ox//1v//zu/u/xt7z3/7d3//dz///b+/ewxny/omlrosfxcrqtnw0desyyx+v3fk8iij30zxj7h922vkbsul13jv0cucpawbszhiqjp+/pp/3snfvk3+8evqes7f9t6/hdmrc543edxuh1gzjwkvw69m1d4phgvhs5podo6ojg5w07owdupviyqxzwjooq0oiedbg6ymlgrf2bwftmj2e29nu/ewys7bg8dhjsxikwy0slhapusq6sdmklkkg3hkwyceedysiljk9dyjzhio51qsldyv1t9r59mraafcjjblaylyexyaido1q0n4yd2g0tpufbl4fdrew00lakv4mppltfnhmchtmgtnn1pgf4d/tntrrjm3u/k2lguv6comg5ph7f4ksj3vfz+vykzrycuzbkptmngz3f+t+lydegs5s+fyfucgqhkpyi8li8v0cl+xprhrh+r64cqar3ram48dht62hvbzvdze6zmjomhbkmjui/y22v/e3wi+rosdwgie3x/q1cjdstuarg2w0xr7lxaprucluewo/8iqol6zaputats2abrsi6geskgthljzcquoy8c+qop9wxrrsyn85tcqwbl+ab+vxajt5kpp3r/bt/zacypxrsfjazhyl2uobyl7sgz6l2utu+i/b6mqdxbhkhysdfleduboo/u6vl7twoqickzvhywohsoiqtjjjg4mlqxvh/wimmcmzbsgc1ritzcvqgs5zb1oqrhdkpejy1oi6/udzstoibbsthripj+goxpjnjy8cfagnnrrytvd9owjpct6ewdy4nz2+o8wme3fdtmhr5hv4zxidp7byvewdr5rvj39+c7x0bh6cdypsfexs9x9ne8///vnyvlgdu4lqss+mbxx8rkb9ylfgyxlsba0ivevsekffbykp+kyufleg+xbtrlmsg7/ftjqtcmedckftkhaoo7cbsnkx9tflwe/v4thr2oj8u8zmtvzimsak5mgyryskk3gg8scgaxbpcfgxrjsmg2dtuuysggk0lclkxqp4xiyf6m25asas3hp1uz1jhkvf2ztxf6zipvlyvbpcqbbf9kspta+lbnjlcrokmimrnpkgqkdab0wsxp2gf7qinhqwg8zhwulehqezmk1dyoyenazxakikheufqqjae4cxbcsvki1b0hoc7no1ngmpba2oobgfs8lgtyexmb+kbq30gt8ketlgj95/tbspbkylohygaug0jhj4howteuaeslcowsu3wweam7clkgt5yvfucipoiwrifcqgthbjfvfi4she7yi8jr7tkr/lgcgg7dmsfca+cg8qq4y2hqlfnhquxmstnbw0autrkguaidu0w2fef8zmw+bmkipkxo1ujgvrpcniwikqdsj4e7zjlrdr6u2oaz5o2xqhrysgtnthb/9xm6lmevxynaa0jakiszhuciqhqlorcvrt5uemdtyanxhvzxin+jtxkwuidkpqqpd0erztvgrwacne1weo5pg8pcr82exntu2vaoipphk2emrs0cywilifvpyv68hhmoo7z49s5h9mcpqh+c3oxkkqcdpb0bviuobwzjsik/b5umzgfr6msn+lmiebbx+im5ekryc/zwjj4q+xglf3fdv+xqu3rub9d3mufrdzc6n74/+j+r01uxmhfj1vrhg234bilp5fquvkliojpx6ubf0uxdbwuwi8blcbotux9utm6nldnjaiq2cil5ihdxqzpdllwr2tfjudn6gez4y70bq1o12qlsd+rvbyp1dtpez/xdtxc6qukhpinco6aqrsoiopthxfstkid1fefdltwj159vmgq8blhtadm1iqgty1hsvhzwuvhnh4gdp97jmqoq6h8esqlmhge+mfqypakqw42uenj2epn4rvktcnngxm7gsrcyw0hnjo8truih5hgnzuyq4+mouoynwjdavorapydvb0d1fbr8d2of4oujiauoecgnkwjcvczrpdundjsdgkskfeax6bizfhz5y998clewb9mjsvi2cabn+farfa6drfa9y24wdcuguwvlfj+lrfjpiixi2xehzddjtns2bbwzfdkspibdzcabc2radqsoxu4s89wwpjlhqwtu9hr/o9ybvp9vofqf81ubgsqwxhhx0ppwr8chuzmr3xlwffn324nzt2p9izh6tqnqcwgzlf91xk8sib2v06yvupasacy8eu/ad7mumvkvpbwswada3jj73ecjbkgscuurmewxu2bhhmpkc0vbtqwv0pu0r8g21zvnajo63cx4vsp5ado5pjuzpwbtrdla/0qmuxvpai92huwd0ymdriac6l+sqic6fll+/7geewwamczmlb+zsny49iixccsfjxl2xmubemdhiwgre66zbtddbz2ribjd5syelgwizxyavvcydb20oj0ctmjti2g41fv8vfzjoxhmax0egervrd1cm6cj/pczq87hmwfilz8gq5jowepbs1j3asbfnimhhugitsyreygrtw3vnovhn7zbn73znkdh32avc6gzlhdklnz4b8ckvfs346w5uhh519sd6evyfrybjswovi50ehve7im5nbnphbsmrkrizsjkp1kl1ti2xlzsdcmrg6zvz02tvhefu1ibonccfrs26pln5zodri66h3gattpn87bqxdfu4jrrprpl/urkwv+lyd/o60lhibtdaybke2nhnofvji6dvch51zlzfe4ppjpq/26doyrp1tugahbv6pjahrjz1jzxmhnihcibzdcado5skvvdmx95a/n1oj3e+hkq4yqmexnnnscyp4g0p+96zsf6qmq9okuldtvwcgkpkxljsjc7zegyeg2iwmjdcmj3swgliryiq7a5vm6u1kx6qzh87ewkubpua2glsaq7upae71oo+3/snyad0j+d3rbpi4f7v5iglcmfj5jc4gebtr3juuimx5huyl3leydzy5pj1nmyphxgyddh2sueuk5ly28mejzizo108ir8enyde8phu9cn/enw2+uhje3luxk89k6jaa+qenwrpsi75n5u3pzabmo73wq3yys4zm2z9ls3absqm1igipdpw4yt31dfgdx7wo1snpgsyhfeuvvs7l3inczhmpra9rpusrfn5zzr0auedxo9ranfjhhsn2frdczptogb12zxs1t8lcbsssiavgx6teqgialjhsrkphco9l13aurfguqanytmagcz117s7koltxcnlk4frhn82ow8v9zqwlln4tqgp20rspbldtqyoepjkkkcys63d6m463elk1we1oknhynvbah1wsgfyvxnbbzdehxlsxdpbl7t0ynrraf8b7qvn8urcpbfbi1gkbjafg2gmwikifs1nrmkmo27e21c5i3pg+ykyrojhwrs30c2do200d4x0w6/ospigi1kkbjks9bhj3tvit4nagunarvknstprjiytmuormvz3krfu3rnpi6jbpbpq0rn3yhea4kxcqp2idiar1arogqwyomour+5lhgkcz5nkyjdnxoqccoxchriyqtpav8hbv4+qdrrmhnsqdcgxbfskd/xt4a55hlqldrgeqjpdwr6iul26gy8rhsfo/ubcorcq1+ark1d0omg5mmhupxu7kwv0swvniotf3velvpwimv/sd6wudibdoyz7pcxqyzfvree4a3a7gav7mesdtryl9gugqfp2lng7bamgpahpikrgwlg9oxhrcxzfbxjcghqdhhuitcdqlgsxfzpg9jaldyogzjluyz1ghextof0rd9gqqdfbkrheuldg9om/vmhtyu1c82uqizgmvrpbqjg5qa72kfwsrnkwvz134s+zkgyo9vtjdynptw7knkbea8/ufsqvlzatpue2aid3hy298heipmhk0n/lkck852afqvdj6u1uccd3e6xpsno3w7ukecbu2j3x86ceu9fwonyj+w65o5+z6ya9czsph/faknsbby3jkqnxgwlw2mfgmadf1rnxqnqq3r09zcx9rcp4wbxzgvfugtthobojbnok2xmomw0ggafdjj0vmfnt6ykaoini0ihl2udhjamvnhuhukd4rbw0b3box58mg+8tn/ieltwfxiiz11xcvz1yh1cs2skihrgbf8s1+oakiomjmicccb8rcnzqrmanwok7wbrlj4bvvv9yl+zgrqlnq0q9balpiwpokvb3wnnohijj0x+p7faj2e6c/kc60ha3m2np3opv+hhfdqn1eku4mze2ip3lczvm2naloeos6n6q7f51cu7invkbp4ilsocm7exaqiitj3xfo655yxbobqrdvkxzrb8r7rcy9xdwywdruzy3vnjcvq4itwqdw6ewn+pje9ui9ak03uciv9cmy0tcrjwrlxoefqqhnmtsa9/4hrnhodhfec8y/guhd67sysb5lvga6y/ddw7xvlcynipohvv9nyhwdqjk+cbd5zxqkpxjol8mp9ry4m1vn2ho9fklizb9flkkct81vmljy8kyuznv5xzw6kexolfi1t3d0/nmrgn2j3dml8yhbse11hds6gp641nlfxbgv6d+hgyfywyzlkmkthx8w8dyquqsu8xadze6j7tet7j3bszgecun2oqw65gp7s7hkcc+zluqe509m8/krvugn0z2ui6yuoa0w0ba4lwfbngd7pyxgqf6rszk61z8x5fr0caz7xgzgsm3kklzq77m64yqb5yhrfqywqj+kjkeh+pvpfci3x79js7bah72gqflchjjciouwrqittn8+ll5ret/+qmezkpnwpqrloyaji2iqyiwdxvhrf9v6ohntz/xnv7pepk5tx5elrny675v2yuylogalga63a6ffgmn/yocnvjf2z3o7odkhbv7/ru2suil/m5jzmh14ncbj2vnkcbu1q8zyddl8ycgid/gc+vrz9qvbjbjpniipbosg59bcfesajwvycvhqryqqy7uyt5g3zdui7jefshu5ftls6fnbqgjghopu9eiboiqloetinaldkuakv74s6tuxw9kcluunxk5hoqinlpzqz61qnmf4dnkbti37kqot2gmlfrrgmi/vtixstjtxwtzokfkrpvgszwuvz1wluaxf8yunjcs9vb1hz4ml0ma0uqllolqgudpcmmvh01wxkjwuysg1w+bs3xk2dy8jwbqhsdupthutuiepfw9rfayy8d81sekvykeqjxk6nch3x86ykiogucan93vtnndfvwiqt7rxbqwipdvkdss1l50lzfn+adwgmcll8hb+woafrypm6azxwunhvzclq0ijuh8aob5qssmul/mdobp+spt37j9qun+sqztut83wtaydr/xj2ym9p/iub/fpdyahb9n/vvfd778j+7pz+5t/dpwp2/tb022frf15dbzrfotn1vxvz/dm9z74t7jyzcns5p8pntqh33w9fnflvfzsvx/6oq94a==</latexit> <latexit sha1_base64="cs6ponrgsr2an3paq41qcb5kuru=">aaa77nicxvtzd9w4dlzplngubtp5zasnkhj7lckqt2asjojzrmm7dk67nyqx3ks5dkiiwjc4gqbvvaazn5g3nlzmb+ul/yb3aqwisvdlveq+3skad3fdxcxfoqjkmjbhr//7xy9+7/f/4a9//ocptv/4t/70z/78j1/+xteirhlm38dlvvlviijoxgr6xjkz0e8qtkkezftb6ooztn97tblgzfforip6kzo0ydmwewlv0y9//d97pmiqyt/9kqgil2j772/cnmg5z5u8ah+guzklypxdr2bvpgqebmgzmk72dw8p91ft07anq9ujkpfnyk45dsij50ebrkyszexyho1pwnz7tvkxmkku2/3a4rk9dz+8kirmsjfmnlcxig3ivdn+mcwlfpvjynmocrjvrbbxyeltipjlum6pjfqun6p/os6nixjouu1sytdisck4nyxanundega9onlavhws+bxa3ltnjwpfejqt5brz4rgj7zhkzw9byl+p/7tu0ayzt1p5izrlfegqjhuar+3dfza6e/j9vcobryutzxkotmfgz/lhmss37zbyyfk6l2spuc1iaxucwyymk0ek9bfd/0txb8cbelht2nwekps57bca78x2xs8zxuhekhrzw+qpzcplo43gs0q+dozm+aohxo08mlqdsnqgg0239wlxras0imue8l8zatr3fqc+rydafkbdejgueam1hra3w2hvcrnyf1v9zesjemmezxwaejjdd+cjtqt1mnw2vftze2+vftd47xo8hhmp/7/mosbxd4ln/3ktdrf9h1qqagocg4odb0owf5053fh5twopqyoyvruwohoqiwthjjo4mbqwvb3ym8iamzjtgcxqitvdvgct5zt1ogvhdlhcjy5riawfdyj4nbeh2gw1ke1/dep1xjxtebzsdw30uswqu+jzr0qx9xlfsmxt7pdfqtcbxg45oa+yvegcslvshkvz1hle+fb0jzthh0fqj3a/jt3hzlb3czb98udfhekz1zktpjl4fhjuyysg1gswzxrw8frqisrxjkxn8fmqniqlruufbbahnukwkzn8v8ha1q57ncqxocebitokuw0rfw3ntzz94wumf1vlih+a0azoalkgmmwecenghwwfhytmdgqn4jkb75kq8mwbbcqsqcptliyqjarlgzeq6ri1ksfvnmvlszavm8l4utbr4/kafaksc7n6lgpdupbmlyxxozxialfgvxezydumc0waqeuaszqzt/rn+8ggan7l+dbygkdzye6uavcugydoie4qubmgtfadtxpn4lqqeeatpqhynpt1oo5mllw1xqe3qngp0bam8bzzuag9ksbgr1lawvjm8yew6phfdeheclwfly4zb9edos3siejhyigj7c4jvhmtdk5qos/5ihk+z2gmqlycyijzjqkqwiqmneatlsfzm1l9swezhlw1q4icza/mfxkv0ayo4ytwt5vjoiw2gi5ocq2ifdihmm74xmeaixnbhbfnwdgowcwsmilshrfqakizbxean6eaolq1zrhn22gd6rbgccyz5ujwf3yrxsyka4pwgiykswq5jkjwlrrazy1zu630gnmre6glr6ouv0gf9nxyazfotr0yhibkaopazqjtnoa82jkebwdfnb286kyxoviqcu8qpydvlxxhckcg+cmbxb0eea6jvvvi1gl2ay6oh17djeooop6/ublvk47vb+fiizxtmxenburrl474wyortnx6i7h5q/ik9moonhd6eakud8o3h2tqetcg0i8wr9epltlffdz895dtwbmzcujvvuuyb/ttiezlb6xwqui6kvdrquxq7ujsa5wlwksigyxsfvvpzkswm0hrhtqiknohexjfkcevvyjym2g52/mx7vl4vbupu7fbge524o8gi3uo2v5m9so1fztdqqe+gua7oiuskgk6+gbfvzaoi/yuqvyv1alvnx+axz4ewg4aobahcs1gwvnufvbqum0dir883eoao5lrhazlom4ezlwzvja8ohdbjjd5chpy8ux1ujek02dhzobksljysmckjxjh66lkeqxk5rli4a9t5gxakcbuinqmgu1sr/fhkvgcwd7hd0vcdhjyregjx7qsmnnjdxjvkjkbixqphgdqczmpovt+uv/mzghasket9kphdmj1fsguj4fguzzobtmoxcfrbet9p4rvwsyih+dzcgxnm8hmdbzstrev062mif5kxzkkz0n0qcrhtc3y3dm8kocqcw/3eva61wo+nvaj+bxyvqobybjecnrrlfjhxae7o5pdedd+8fxp67o1a9+tnd0mhegeha2wlrrjvohfd8ro0gupekg1514idu0h3ctmriqedhysagxustzvivc2idcecomy9hiosdhczcq5pe2hjxboy9oj4ntqzfkbtrxvfzxeldybhbzttmyfscjwhab+peddjpwbfrsppwdoxccie091ltchep0/wx5/3qm8gs0yf3iwd8ylahx7eauffy7hvb0uy4c8zbdftai8002dmgc2n9aq3w7ykgjlngqyjk0qe407dtocta4nnheqsn1qlp4rkjcdiec0voim9grxhqbm0ux+eeyg5mozsaqwzoihytvhiozpitubys+aqykpodbal1s8wcimturoqqoalzfnl75msjy77fvhdlblwuyft58nejhkihb8xqc3q486e2d9ck7piscx2b7kxy6djqgdrocmg7njgprifdkm2mwha6f625blfspbogkj9z35abj3wiaqakp0mumc12zdn6x858oordddbrnw20k+dooyhi3cro10vvn/kany33ssb30d6sjkthugzflcbbyz0p3m9yv3ol86m6uipcu0rv3fbvbnmd26pidawlo9ustdvs76kkpmq7xdbma5opponu2sxijzy28gfjna49lxgwhhozwqops8ytgfvo1h3zst1gts89vjjx3iqz4ty1szs5an0wuewsxjaw0w85jls4cdk03svusf3efdc7v2xqpv7gnnt8dvbrmqo41nwnwdj2dhetd3e/8e7idubpy+dvy83p/dhgub4tjz/ayna7ztfa8cd/ccve7mry442rrlxlkok/mxi062adto9sp1mhnro9mresdzp4th8xej89alhp+kpuxfjc9gp3x8qo6a651nbvv7gjzgruelyth/s/kmfhid0/0+uufe2tljhq2/skxbgjibglgkr9tqfowo+gz86qff2jrhwcpgukl0pd+9lguiwzqrhegxqmqhysp/pqfto+8k/lgtibsz3gjapuqk02zdaavb5qmy/l6a25cabawi9ouoqeaawd2wlx0svo657nlovqdpbwhqztyezkm/lu9airfnty6sbeyr/tdspfhuu1pqzui1iarse0nxzhtzmjpisjkphgestq6ru+lwcylxh9widh0hvwepdvgphwbftqs93xgv5kkvawdd7tkj6a0dfg20fztfe3h1jaenyjeqqbymopkiiyd2tzwrjtomuhfvq+uuzohnc8qyiodp0d5gnw+otthcmdjtvtojyrumzjg4yujsyyr5vypdjmpnjgkbyz8qaa8xmrzqjkdq2t9jxlj1z6spoaattqhlz96bxt6as3ib9yvsauzuk6ijlsqcdrseupcrpngctsqbxtyze3iofhibymiiaqndiqddpula6zpzuemyhms27ss88e+fu+z5uc00ybaaaq5keybc9epuver4hkp3g3hr3kpcm0doxtgbiufaifov1u1ertfvldqrehz3r5x4vijq/lu/sdrfmxspmuyvmqunxvqvneouwe0afuxlkg/wwizyfxik6tuxyo+wdydwiksbkbob4/ac4q0twhwv4hfj0huq1ihqgqhtrl30aylrq5y8qicy5vcfdrorfrxjdexcraui3gsrxbjd3ylzj/76ok0ln3hnr6ku4pj4zgylxeqcodhdvkkszcnc9cyjpy9lmjjwuyxfpaq3uslzmhilv7kvdewrs4x23qao87gnvt70rjkzwgv5drrhjpzt6drplu26ksju/2wj9kz26haxxwtny1e67nqql3r+fbuufz3xj3nmftdxvnab4p44egqtfbgpntbq4yxocxj40xayqtc69v9sc7o3rkmptyr/czarmdr9wday5zzhmn9bsy51a2g0awarpmeipi5qfsewfer8skuzayj45ibio8n0j1j/cikmjf7jy/ph8g3+du8d0tet6ibaeuuyr+e7aor1iyrraliwivigv1aruq00dlqaqrpiiqunwnwbqebxexkmoead96v+re2i1drc9xlxojetjc1jwioe6e88ajbuempx7z7roz3tn5t3wg7w5n0tk9favw8yrsuuv1lnbnymwq+5ctkpm07upzxlnrvshzveou3mu982nobutgzxnymycercye64phxfpgindafiz9i7klfipcb2xuiectwakjlg+osll7hrbq1tqht2fv1mcmthhphjtdryjx1wxjrmjelzgveh4wqoeea1jr3/iguceh2f95zzl8aqcpruxhihmvuydpj90nbtdwulkcik4e9x03ifaoqot4iepnneqq9fm6xww/2/lgzw+fyc70uoujkh0usqrxpzvuyuljytjtm1xnhnbqr5egswlvpcpt+cysy3yncoaxzicfitxwf2zqcnrju0svcgc/op5+bh/lbjowsypoffxdwpkq5bjtzeanl7qptn5pupdtjmb5y43y6pbrme/uzserwl7ow6otnt2bz8qu+6q3tpa6lphq5rtdqfrgtz9ucz3ulhcaj/pfjmtrvnzhkvhrwbntedocwzessvldvszrjkohlifewrlcon4qoqsh6m+qukjfhv4iztsafvacf8skgomig67bgqjnm3z4uxmt63/6qz5kqmfclcss7jomlyhdihz3fyet/1xowc1/e2/nd48rm3hl4te3lrvlxbw5iwgzosydrcdp8ucad/kwi1ukxznc48zmkkd/f9a22sker93cbikovaotdp1pojjd2rhzlaymrjauetv40xyqoeqsvcymz4sbflzyuhp4ms7ixval7dlo+docbthanfvmg+yagxcb39gt3lyzuxl5rdubma5x6nrkxz0fvzwj2i0cumq4cs/vidpnrdb2rwu5q3ermghag2wlmpnrwo2z/h2cpu2lfsqo7paqwt9gsywt9wolfk2fzpm6r8pgluaxnbrxpvavrrg/ya6em5l28hwfnizvsjrs5a0uivcc4m9wwy+ftnbeqnbtkwbvb5tldcrynjynygruw9tdp2pnsj6kxd1evpgjwpzswsrxkqsqnetw5/vdfzrkqiic4jo33e+2edd9nauq3tfcglckl1up1jlwnnevf83zonaaxyuvwdv5agb+tqkzobnfa6ce9lqvdqgm4fxqghmjkwxqv+yogm/7ym3fup2s6f6ypm263zfbmbh2v/gpzib2n8i4h988ozwchu7+7e93fv3p3z/nj6q62/3nq4ndn6h61fb73aott6vxu/oh5ahlw+udqujv/j+d+p/0tdf/rf1+cvt4yf4//+p9hqupo=</latexit> Parts and Score Functions given a parts function our score function is then defined: each part is a subcomponent of input/output pair score function decomposes additively across parts 6

7 <latexit sha1_base64="hhsevwvbrpxzaew00e9ejadzqzi=">aaag6nicrvvlb9qwee4lu5tl0raoxcyqrvurvkmfbkqkviklxyl1hepv5hi9u1adb7fdn7l8e7hwacgu/cju/bvg2aspxbotfeurx59nmt988thhkrhunvdrbv7w7ubzzsld1r37dx4uli0/2pdjnlg2rxorzichkuzwmo0prgq7tdngolcwg/d4jd0/+mgyyzn4vxup60zkepm+p0qbfcw3ndaoibqgft0ygukvubnksooh6dqlfowmcxjoteabvrdnwgoa0uo5ek622hfweecmbrnii0rke+iz5cayolcgwuipbisj6mkigkljnwskgnn5dx4fumur76hrmjmlhnlllc3bal8epdmmsfybcrjapqdipiglmzkbrfazmmmjagcfxdmxyatvor8uzkrvumnb4ofgank3chyclvsufr1esfdyz93kexumz7srigzcieanfwg3mtqhgapsqj43ky8fymlrftq1hncuhnfh8nf6yzqnk2fxvjupzw3uky/ipj0y7vi8tegbij1qsf5jxsg6ftfgz5c4sqta7tus88hwmro9ylvgtmguy63sp5md969vpd1ruxpr1lpg6kw6iwyz2c6bmdaz6zmhyxhiha6rssvhhmptuafuvrzgacvb98qbpg2/mlacauwesz9xl6f5xgjfbzhyypds1dxw2tkvzlrwlllk6dezscmwyxix2dxlvw1qg5ae6iczplfcjxo+qpni2vla01ytj/cs+ke9o1z1x3u1j9ncszioe/vzgvsc7l2pejxjvikcdeizdlwrhzkmuav/byucp1nytlg/se576/67fyvbw5ucc84t56ntcxznpbptvhv2nd2hngant40vja9n0fzc/nb8pnadn6tihjsxrvphb9l0asc=</latexit> <latexit sha1_base64="yz3htpnwmceleyr2zdoyxl9yqka=">aaahtxicrvvlbxmxen4wkpbwaathlhzvucrsardcalvfqssfeyps+kb1tpi6tmlv+2dtpyks/0eusnz4f1w4gbbivmm26yamlcxsyum55vpnnwqsk4vk77dw7+1u1kdwhxtu3uvfsplpzxhu7loesp26oxidgegmemt2ffechsypi2eg2efw/nruh3xkqerx1fldhlvd0ot4l1oiapjxkrsoq6l6qvcpjk/qk5qvudlgyrrfgowbazz4wq17zq2jsrcahrr6bp5rtfo5fac4kca01ckw0vj6bnkzbiisy7bqahtbldpygmkkseozryxpvaepivdfie+gwyhtewzvzfc2b61+cfdmmctzbyriypxsizmghkrkbtxqtwuooqccprbniseazppcd82a1vjjqakeykjso/q9nozbtsw6szlniz1by09smd7tbkxk80y0cnzoc5tjc5bhna0v8xbysabzonpglcsbtclxfq7x1g/tlcmhxvptwz8bqjnfks17mzqj+1bsaxa7fzl8x7yqdspihm4gcwlx1errwgahrwrme+v/gdmguyq3vj9mdt4/nzhx3clkvlkfktborr/zfskbbykcekbp+ixlibhar8awhxmetds89bfsxe983tj5hlkjtt3fwvh+8qq74eydtrve2fh1xmpxx/6cozhnqhypkoiur56bqlagm8wpykagm8ksqo9jjx2bgzgqybboxwod6ob0uddo4ysuythjce1cacsdt1u1lo9z8g97r5nqvmxrhiwzyhedjepmaqky2acfdxjkqbjdmahnoxbfte9sqhu8qfyer1zytlg/ueg5g96756s722m5fp3hzhon4xjoc2fheepsonsorxyqfkv8qpysfq5+r/6q/h65zs+nyx4558bcwh+oxy94</latexit> <latexit sha1_base64="f6upsv2ea2kjoks95b91mesmeuy=">aaahwxicrvvba9swfha7jem8w7s+7kwszktmdxyzbjqocnvzywe9jsoywvesufkys15jvp3jpqzg/soejjl2mzhr2q4tgj3z6rydtx86vpawyoxr/lxafvcw0wytplifp3n67pnq2osjhmcpjoc4znf6eibogi3ioacckzmkjsgmgdkozj6a9envjou0jg5enpbeiiyrhvcmhib8tubchieso2agx8ox4amoxcqkjlcdyifhodrnvhrbntp0qa0w85aub3tkpt2ewccavxwyhplfnctfxrldptsflagtkoifigz9nod6klcmiebkdb7oifxthshtg/gek6mn4rseqwu3r09emay4+qoycoyeq8atkkbu8b27/briwacas6ffmsow1ntn+7nanfpkyfa0zatuy3pfg2mx5edwjwv3ro88kcohgvbku5li9r1ovhjuu1njui1dwq1u8w7winoyulaldgwhmcfhbtjcwt+is6sedunp6/rcq53iiuyyi+fm7ltnd42yqzlkp9bvvecyjugcejd2ld3egjwlzukwtlfxf7hssntyk/ppyef9845t1y1mzpwtlnttvpwr6zeiuxrsei/l+yikbbcer0czy0eaqek6l/fgurvtly9mqy5uuxbbpbm7rkwekdtf3xc7bjhavogvxozvjn1/9tvsxzglssqwq5yfem4iellfmoozutbmoekqpkndcqrnciwe92txmijq1kgfdojuf5eabtqdivhiztl0pohi62sg/nvaasyg73usrkkmsiqnhqyzayig5pkbfzoslfiudyrtqrkcpeipwki/rkyer37keenou+u5xe/z24293vkofeul9crqwj71ztqzpln71qgfgz8av5unzrp5q7xuwmnzk9dlptjn3zozrfu/eoyrga==</latexit> <latexit sha1_base64="cs2rlstvqqvviaakpmv2nv3uyg0=">aaa6jhicxvtldxy7cea1e8vhhvz1ltkgizlqfslw5pghkzrhtkriovexpvw4lzy1b92n6yhylwfozrranb+rbbljr8kuj4ts8ltsbfrmdwmykvlg1lmxdecrqlwhucg8gjypl+r4+h8/+973/+ip7/zjd/50+8/+/c/+8oc/+vzhx8miehf7exvpib4jqwqpz9kbxvxkvikfo1mysq/dy8fy/vuve5ix+wtvl+wio0nozzyicqqmn9/z3qn5wk4e/iquvci5/xdbrtvcze1wtvtbwksxrdp41dttxfkibe09nrwchr0d1toa0cjailzboym8eio9gctkse8odnqxn8mana7b16qpcuiisjy2ltyxv2etg0+w3bkuaa/xfdf86aettvbwnvlmrucdwwlgz367vbe2f7306vfqqtuxyzbvzbiguztb9i8xu7zbyinswvupfnyaepzq0uppo7uuuhdv19u08eeldeaw0+dht62hzy0+/f9l7fm4qdfsuaz22r3zb7y7trwzeivedgzfo7q70adthpb5cwjwtnt/eivc8jy6mizujxiwgqwwtq3sbazqtgil0ywoiy2uli661qd70m7iuq3rwlkb000mcoayho8ap4eovcnsyfh66+hch8zk+v13jtajzuuax+uobqkkihrgp/cq11g/33lvwnac5hij0kwy4vonn02ummjgrbinzcsuib6q9idubvhyujodoijjgjgjwtfoaqq34egwsvtfm6gwuqsellc+c1npjejxjecah86kjs2pxdthcssqmuffypzdmfgzozry1wa/o0wuoyn7trixarbqpnip+v6+xbyby3d4f/dsls+m7ye/rkucqyu4uxftvii7zkqia2pz/aot461j/e/zh0hztb3c/z9poffhbervrllfda4vpjcakuglhiejsydjuojctpdektdg6foc2yvgj0qtwspaijyawq8f+uik4dujq0kzghayk6srsnk31t55wa/fichr+sfmq/09gssokqcc6bjoycrjdw8eejwufwes0pejechxj71e1ju1go82kksqpyukufljcj2ktqcs6j5bg2q1lfrbey10bffc8tvetj6lkgfktcjwszja8tvdglm7qknemrnl5g2gbae65yxj+wnxyqnqf8vterwycgczknv4zkbzfadokqkkkfy7kmhjs7a9efncjs7ufq82xcj6twxhnbwxzwuzuofbp2ddzhleyab2qx+fgafda+8+y+pxpodbqgsgzxqaujlsd1ygjzjkrkunkoml3ncfviddunlzoxomzuhagyixavpyrktknkkuuxryd4xdrj+piw/5zddadvtzghbpodeawqu9bkbxyx8lhzkltqlrmfy69albwnvnmnn3xfctlvmybkcc8bpbhnkniqcj5eqagypgzcad4eeuhy2tbhlguhdzpgwwoym83x0c/svozzliao2he0ohgsis0o7kqfauhsjgu3tr4wozikdcflmea36noety4g0umpa8pqhgnfukszmcyh52rnqg4pvzh7dtfzy80lp/ituzyevlpwhkgcgeqnl3fnekafjpru01orsw9zrh33/hpuzbd15on1qer3+oaghlmejw3z9hse+vkzi36ayartjf4yal7sriqs3tetqp8ogonu8op9burv2gd2huyo+qe259p3r/8acdzrdiyoq6oskrd6upme5lwgevuwq7wqui7e4mpqauutbtrwkha5lxe2wkz6qp7zmfvgia2vasrq8wuovaeadx75ruhkmgcu8t1dew8zga0sl+zqkb36ywcxzidbs8d0upyimsqu+hmytbt0hvulrgfvs1oydwfk7smcudktzgs+3zb3faiw3abyaenilsoyprl85+ggmzsw33nio0qgkqscbkugzh5kvdoec9xwsjitruzot04ev4anz4lpsgnmccvywsylu9jtxo7wesgycniqjctb76bcgbq8hqkqtk05byko735hly5/ihymgbq54iaeqmkpo7rizgm6ksfcysjxccqc/1pojqbyv6ziccyob5o0n7nncg5optoot3geet38tsmw7eoweis30/iq9bziih4nmyhuyzzkymwx638zqb1regxmqpua5nq3sgkzezwmazz3ggwdut3qz6/klxa76d1lhzc+2riwqmy2bb0ucc2h8rt3z3jrsbxpvpl6cvzlau/yns7dhjzscgblbm0r23xc56umqwxkjzq8q58ekqtb90b2eyzns+wlaaslml0byhqnmgxfhijslwy6dcxshxr0ow1r69b4hejz0cvq3wnbvyxpf2kajfiwliwthnxpwji3jfapubkkp8ew202dscaaaxa4topdw5xieitf5k+f1vd/ainuncqlk0mjeucexblrrwob736qqiapkkwxbtyvdana1iwrj9tilsdp2x5fiyizbxrfpzk9xxsozwcjrhmyoe7vzj9v8zielmngfrjwsgl7v2akwal/apxwr6phypjyvplbikxleowp6eubmy9qipjpgycq11sccbjgx6q+4snlr53br5na8zeucue9uyk225mhvhm2cdxqs2ogv/3chhouefpfb+bfdneedy/ednyg9jx9qoovnxb3phbrlvorknydmfrixqbvsuwygh4yin1nfm57h7jwyiojberdkucgxw1cnknz6mxnq97nzrtz3ki1himwzhnmodsc7/n7ka/u3x9ydwky5bbrsgkp7gzmy7c91vxjx9g/orm7mu2lpmypt/3qz9ftotaxocxvirr2j5so2sl4ninsq7raccozoz3hnm0gvlfuw1zn9tphjno6dsuu4njm4+t3hbag972lfohpwc9hx1ukkfcmrmxcau3gxln/hlhsbcywmdlvncqdgfhsvh2muqszvgbxo5csvd3ezrzp6a9u0teik2tcd61how63pa+61/anzadwj+2zorhu7/ro+yghccbh6nkwhe9ornjjt4qv4n8yi3onkky122jpp5srucba22ney562rjro1kzzc4mttdpiq/3jgpvednu9gn/ovns9ep3zwhn5rrtwcf1xuajojg4rnnmp+z8qyecoox+31wg7y2c8phtv7cfm0n4ipubkv6tk3gf+6or8hvpob3tk541ja4rih86xcv5qridjowuvyyxbiqrek/0zcpnxcsv68oxi0zbjrgu3upwlruaavb5pg2/h7bbqjhhofevakgegia7rgs7gohc89ll3buetwxyksyrriowny9uwur0z7fv7asgm34bbnzx3wm4ej1amklpnjmgt0wu71kfbkszuho6v0ag+nq43shj1qdfo0dfydvbgh1rraptqktnnxisxtyp5oyc5dre3wwf4wmsvwyin4vohbxrbyigblrjedcithkhrosnknjlqwrw1l9s4b54vamvqpsyw7w10/eayg80di91e1rljxwmki8v1rliei6nbgukmutsjkdti/ltjem0oyyvnekrm/j2mpnf3ruyiajdppl707b1yhga4lrzqvygcymgmkxzjqcjzkptivywyg8fzpflyzdnnqg6fwt6gsoyetigukansj5b2uuyc1nauizft+xip/bpprnkevasngaramknvhkjqvbprl5eex+j9rt44t0rx5qgtv3sg0dkwshwldturgfrlc83zmefdusw+lqibfzeprm53jt2jjnultkvryqqlznhx4hyac/yykqzrlrbscwxa9u1yshfyi4d0inqioqwcxu0jw1tbshhn8lirdatsograuh3ktys+ttf6qekqephzdpvbpxfbxjlol0ddtmqrdbcklth0lzh94m8u2nrye1xiipkijonvbddxnzxjd/aqlyqg6tbxpxpiz7ocdo71dml3kelnboqmjdqdt29/ypuvmc0+4tzogpcdjb3wezgapwxo7+0opklvzt6dlupes1rkx3d7re+ym5fgsge8me7kpdfzoks71/cg3kp5brk7m/ppunvnab4p44ehrdbbmnfpkbjkeiphpjzmdnddovdazq+yydftnrmpdqq/k9c78ni9kby4z9brwewhew6lsweei6dbsc9fzvxceiaa6jimersf6qmjsoekz41q1sm8okrwnzvnr0+gwfejg3xpc5enysgpay5p3h6swiuwbbhkyhsbf9s1+oalionjmmcccb8tcnz6rmafwok7worfj4bvs19yl+w2xuxp61k/balylivbkff3wnnohiljst8e2+2tcbtz8p+yqnvdzisfeytw4umv1axwqy3hzms0qexftuhk0ryry7jgefegzpeyu3ax98z3bjm8hj0nstmbbylmtikuj7rmzyhba3my0uzff/029h5j5x4gphksptryixxwd68drq1e5/ap7lx62mzwin0ct/eryouvxxgrc1mmtnbpwwl90hpfauubx3h3obtptfcsg7/p4otshgaalskdmpnyxen2bquvz5js5nhftxmc74akgq8ytf6jd1k/rvlf4kmtd9789hnmzcy1majzryflfvvtq5ecnjyuuz1ttc5hrx5scrrrrrpnecafmxwd3qqc+ixzyufig/ufm6oeanxqqqmo7bf0x13gy1ky46li6ftkhwjpykiotdgjhi59h2m9n3dv28aa5q03yvun1zef3j2pi4f9nbfujzp7dj+1xffbbjjsdv1yqc1ph4ekx2s/0+i63q0vne7mi0zi1q34jiojo4ez2tf6bidvjooignxzvcntdywdils0xqj5kjkeb/ovcvcinx/9lmpaaj975omyhi5xhefxsoiy0wwffq5f6/qffuonmfqzmbmac7umsqvi0jbhxcvr638venj900+8jf89nnzurybxyis9dd+67rvms6inc5got8mneszy30uiryovutoz/yzwulvvx2vtmjb67zy2dlkjhgbzzk05arr79umggex5pcboejf1hfloz+oan5gxhmzik1khwm+ghhtjvzpxsoutia7iviv6mw+qngjmim62x7b7oxn57gmzgxgg4nyburtr1nx1lvajtngshlvcyxz8xlia0ms7ufnka5lcquieg6200m9a9oxp8wxln+hbdjxhexzbc/sshvc3ju6savz0ev2/zlgq0awnwehzyx6rsxuxv0kvn3ca9pbvhzkml2msspgllolqgudpccovhk1wxkjw0ysgm2+bc33k2dyicwfqhsdupthux5astor+ikwxx4d5uyojlxmivcvi0u8puv9mlivqbme1a7rfbfo6+zixtl+jurjbslguhdrlwnvevv40z4paetwqraheluob+pg6bgrngv479baxujycybiulgeyl6ziborx3ahwsv+57ru3nztdx9a0tff7ohggw+43/thmxp4tgffjq/thk+ojye/+cefx/9t9+cwptv rcmw7/y+vxw862zrtdb0z33d/7tzr/f+y8h//ngvx7894p/mddvfdbr/nxw6ofb//0/vhkhjq==</latexit> Structured Prediction Tasks task output structure minimal parts multi-label classification set of N labels, each of which can be true or false set containing individual labels in label set sequence labeling label sequence with same length T as input sequence; each label is one of N possibilities set containing labels at positions in output sequence unlabeled dependency parsing tree over the words in the input sentence; each word has exactly one parent set containing indices of parent words for each word in sentence conditional generation sentence (or a paragraph, document, etc.) set containing each word in the output

8 <latexit sha1_base64="vjkavpghhaxnlamjo+ypm1+x77a=">aaa243icrvtbc922evbixll1lrspfwerasrxkqrjng4mrttvbdfuxffvx9ikorikkjg8seicbsbzmc3+gb51+tof1j/qf9ndgoeqbebjzlqztgjg28visbtyxbqwkzpq8pc/77x74xvf/na33/vo5ne/9/0f/pd9d370uesliojzikdcfbessvow0+ekqzr+uqhksjclfw0v7mh7x2dusmbzz2pz0lomjdmbsigoqdr/4n3/7orpmfr4k2mwbsypa6l4sbkthcqoee4/2dwpzqtxynnyljp4vc3renc8o5gqfvwe+vjwn4o7wbgqpd6v2n1r/aj60yvz1g/qonhdnrngnle4wj6p9uc/23shbwd7sydzhwghyncxckhvjnoel1libqj0nftfs18ae+s8mf3v6f5mv+xmv9blgi3qrvjxvzofgytsihqkcyjqhvkq+u0v5e/qequldza3rxyst5pobkx0bugoiodrzaprdvfsuwjezi7+f+j1pdt/f+vw4fd/bo7hqpny2mh+ts4/+pcdccyjmqo5ilii5enosfbnfrgkrsmtn8elpawjlkhct+ezjxmvz5v2ujrygzo4mhab/3iv6nouruuyiyidminqku02rps1nzzq8vfzxfkivdspteetmg0ud9cdg5gj0hs6ha8scqaybtgucbip8nxjcfschpc94bsovwfvgs6ncmghrtfi36tvmzkib4vf8b8rnle8xzfvuka75lrvq1fcaicqkoxrkglurewrxxhkmejdpgimbsnf3fxrdmgn8qogjmiog0mjizdxiescagcykqqjmlusbsqzvjgkiudgt7etq069fjmpywxb4ttnivsncfiu0dtzeis2oonay7shmo8zpnbltdd61o50quwvrw0betyhowtxkseivdovsyxmegstghnzjstblyuil2bmwrijctftjbjuvb85ihetyl6ii9r4o/5qoag7wmyfcc+kg9ui1twuucvmhxt/pnuenbtuc0n4eootpu1uyfffuyoa3bkwnc8kppbjvosqhlfikgaextcuy0rcz66mhrv4c0q7snmmdoyvcz6vdgi7uvtlrgbhrq96bjesekd6owqxokmgyjwb2pxwhekrgog18ucx7deort0zkj0hoaqjatky5qehjlbkfb1ijy3//05ofzo401l5zit+ryfpt0zbggcaqshz3zhguwtburmovbd46tzl5pefxvo8trmuxu/qexo0rd4/mreqr5lttvg+me6rohjvhpghg0nuoxufwezaukic44ifqjdhy3x09fltdobrtax4gpoh1qdt54dfd3sactamvkscqlgjchpztpwk/gklevsfoliatwlgadimasnndwfp/nxkbyylfqq1pmfw4zyymuvy+h5jcmuu1pnvjlfwupy5a5xlcm5a1hmlimltm+idv3k2vkkfgypn26kygqamsq2ehpyfapjbvwlbjur+i5cvniuwjiirsutfl8ud3yiubzhci+dylpdsua5vkvh4nublh85sgpsogd1attvuksdgcniome5qzzzrlmbm+c46oje4+dghrufokqguwjvhzr0cmk5lez6pyllak2agfx8ktz5kxahomrhhwfmvddd3czvakwnwegymgbv6h22c16igtjhcjdmeyqbagn5ameybjn2ika9n6kvrgjcd2kgrkzibo34zdfwnqyv5xzauyiq8iqlvp2fp8bdqchniyw0dzzxewtb7ptbmlma18ql7iqtiezlnqsk4dulbpmmz2wr9aob1pde9yoa76d1l7zy22rpqqm5gbbmbsltefeb9tbo+0b037w2fhjk5vpvyu7e05y2qjivgwk7rzlct6currwqoowukyff4kk7ar7o5plvowdbqtalclrtiva2ybmcoqsew0xugfjzl8jjsitx9y2qk+sfghfvmuadxtiwlsu0zylghjwtdmxf8kixbfa/qbfmq/rqitniygawjgw6mrtk8s1injkt5bdz1qar7aje1jnoo66di2jd2khsmkxukczhghkkusyyjf4zpo6mahfflxcdrvos3is2rdionap7ax3hltahx2maoygybtvwf0vqcr0jprs6aiy0itte4jszrf7n+bgdv2rpywupbngkdijdiq9pnidgpaikst4gaqtdunhg4xsagox0llodldfnlxzmkzk32qjem23jh9skn0x6k1qiff9ba+6eh0mdbwtudulzvomgzpz2l/szomnpbdnrvyoqyicvkr7yebokd1vc9actlc+ughlfr+45/9rt3wiao1iaynccfrtsknuykrb7wyfsi62g3xqvtjb+jqozhc7dra8q6/x9kafubpusorsmdhdx2dvqsjamndxa6pzx+8bz9q1gzltouzjd6f7dbn8u065qgqbfw6pfyfrj01pdemrvinsiattgyto7cz+mfmj/2euyvbyle+iiidaanexjnn6eyoic3lexlx2g9io2vtirpq54bnxjiyly2bigv8zbmhh0ysjxyofoudqzsyy9ckm2cf9xfyht3dbovo9sbl1c5ibinoedy2g7ydn2h/dlvga3qdcava2ffw/3f5vewei6rts8xmsdbvvhimqcvygtkxusaka2wzgxrgjkfo2bka7btzi9ci+tjbsn7ngbkrrt4bv5s0a+94gfx9a3+2ba3+uhddjmormeefvtvatkcg4whjmp8z8qbesgn+ub32g3yws8p6+n6u1u0nyj1uamdadh2md51z30nfukbv7thhgcnnemk7ku/wylxekftlevzyntjqvdcf6zhujuve78qccsncw40yfn1iwi67ggr6+qu1v5ognuqgausd8yvziahaoaey8o+5jr3xdrp506a5wi04jmva8xn18trsbl16e0zk4hrjf9uw7c194tmvlboxqg520wspbld1p0xkeisqrrowjkxlk83hityuwnydoyhyesoqzkxlpuyyf9njl1aeqxmsqwroytxiwjm6wbf69flmucjul6hrivylcswbywytqtfegkty0tjekpdi7fmch3jwpmfrvndr1+jry4unxyjo6mjpkuogiwps5rksbhmscqjjhvdjv0laqmkzsvjnyozq+llbmuuj9smvzoujfroybxbdzxo6kvp1odlhobtpof6oxeaitwsaiwlntmu6yf7gdfxjrnjjbdljkncdoxc3rqy9is0ga6qhvkpkha65hzukbqhka37ag/8e+muer5udq0ybecafcx3uah2ugsrkr7dao1gxulbhavz0mkrgldohbikutk6ncgm+omfznnmouaoeowudks/1tprpt0amrt5zf4p07k3moqic9zvur3agr1mis5aiwx7aomubtsw7b12dya9cnvdkioaebtp8dywnl4m8lgxaaikdqg0j/quaxttmmd0ufwebtczdcq3tipgeekyxr410sov18eqtstqt2d2ib+5ajtvskgtvfnbpf2wcuoznfhuwpkxqj026ueulen4ecdi71dml3kelnbnhge+kbp9c/5co4umvb3az18l6jscd+ikrylmp4vr2frls92wfqrzx4iqrp2g6l9ul24hlmdmdr42buuz4hjd29hgflhs03y93jlj2vu3eo5vewxr83wm9r+j2lycpddb772jg+qhfr3gsvlcgmr0szmh/rch5srhfghxsgldbn1thaujdtq+mka8ei6hbscpetp5ewcsaajn0enks2oiibqpdccjutwioqpg2zc/56vxt877vb95ilrbmljl11sfp2ybveswslioe9cdwnrty7xebmh5coe0t4medg1jcckxaw3mui5y0cvs1+yb2wg7qkni4l/rjkthnzcoqmmhpofscuhpbt8dbuh/xbnzp/xata5nym4e6twigpv1stwq+2hfuj3gaxftmhk0lbrzrjguxng5lti2aa27hnviwozdhsoqm7e1cqyerdphjuna16dgqbq5c25030g+j9ys49dhwywhy05ntdwn+9dhjvep0lj1emx2a2fuktelqprc5mdgnezgsrkqv+hobv59wk1co2pmfucsjsa+tzjh/awimlexpivqquyozje43btgeowezli4/+vp4qeafebt7imhmmr9s8oxx29sefdmt80wj2apxutipgpz0kh1dv1y0liyslwnmpzdwv3oakasu2qe4zl8ztj0di1w6pfmhwuhddyxbmocqpgppq44sf+ad/nynpiwvfe0qucxacsklhsmmigeln0fab2fco/b+odfftdiyyuuy167ox9havs6b6fpdhbsfpz23wuzyblxdcmkvqcdbpicr/1mi2t0v7zqodivmifbt+i7zoyobs5sr+sz7l6rjhkj+biou6ofwi4h0pilfs1tys54rp+qacw6c/brlnub/owedxyfdiwzdgmnvhcjmu7dz/vrxf/tt/0kuz8tpkjjydc0qkecerkoqtio/a9c9y9/+om38v/jyefoanqntnhb863bnojbbtpzqhw4ht7ky0f/lgm9uzlvtmd2m7le3x37wpvepfa3h1/rmmbhnk/wmqn6u1c/pipblmcdgizxvv8oj36mrnedj/fgq5zmuaa7gxlujfvqxskwt4a4ifmz6sw8qtrx2czizvsfu5ejjw8fvntjwwcme5i7ewxnv2cjb8gelmbajztmx9es0pbe2shvul7kzayjewy0uq6ftwjvt/fszxvctmxrwvs0ghb6guzhnwt0la57uoisx+covensifaq/klf6tig9g/pxx1gkha+qjbu8oqkcy2f4jiijqj+ddf86u4i/rkcm14xqhnbzpupy12jjatx3apx3tus+5astor+ikwxh4c542diiwqc1qpy8ofe889ewqhfefyzqvldv8+aj1nj9tuasx6selzgui9p9wltevaddgp78igldtyulh3wvxvdd5psvhzqyy91uqfbjfxcddbpteup1k65hed9tnltn2+/whceqb5rv83vy2aydjvfm+fvb43sp5fxpz6/fta6pbj95vdbv/9t8+cz7238zonng7sbo41fb/x+49hgycbzjejg4y3pb5zf+nsdeucfd/55518g+u47dc2pn3o/d/79py6rdrk=</latexit> Hidden Markov Model (HMM) transition parameters: emission parameters: each word-label pair forms a part, and each label bigram forms a part note: define score as log-probability to make score function decompose additively over parts 8

9 <latexit sha1_base64="v3dlba9vvjf35p6agesc4csvc7u=">aaa4gxicxvtldx25cebysmizd88ky2zaitkslzlmlwplzxzlmjyuywc0tkzhegbyfa+6g7cvxh4jqn+hwp1fkg2yya/jlifbrpjrkiqg7+3go0nkm1bnhg3gq0khufuopbhvgrpy+ph/pvne92/9yz/+2q9+up3nf/gxf/wjtz/7669fwfoyvo7lrotfretqjbx0twqyo99unji8yugfosuh2p6hoewclcuruaroeu7sgk1ztcruxxz2/f/dk7jq8ssvznl+glptveqicamys8qqh19nscvp2zthtusmywzbsss2chowbkqk5w1wj9r9imzovbloy0uqtjmjm3v7xnaf7dis4qebwxc/vrk934zfwdr5rpn2bpcridh0bda5jwwzrj7qawhcfp4gcalpc7i0r7nqqfgvrceuwpfjcke07jqmtz+kyshjrxqwluso0kwusvmhn4p/bdvvrs6ad0oczfsb+6t4mvw07mgkfeo2b/5ovhdsw5mdtocnfwkxpy7t9+rc+3xm9oh+zfkmyvvqejvtdn+oob3pj46obozx3w0v/ed/z+gld9c7q9shwer27f0wpcrkdkzv27c1y9v7lipsvjfaxsgnk+120po50raxlzoq677kfg/gx8dgxcwnpo0b3c4+3tk+oly/gfsx6t52trqf5xef/eytmcnjoqefvjkety4red4qllmc0xy7rawtshxjunognwxjqthvvkhqgz2osyjpyeg/qgaqdkjrkfzgeacjmgq7dst9bwe1np7yvgffvutaxlqjaz0fsgww7guj4zd6baufjoymza3igyfajse6bhvdvcsd6fiuxlaa9oo4zcuznuwkmrf4adbmdsyzhekzni7nrehlwzjv01zqqgfuujbtyomkfmecwzevvm2ywlucrh0wsxp2nv7kinhwwg8zhwulehqwzmicbkuzccagcykiygwlhsi01z2d/agaye+cnovmnaijkuvt0ekeg1w8lkhye3igs5fqtkktskmslwf+zvk9a+ir1emcibwycg46zhxmd6a2scmihsuhjlkbzfdntdgzqews5ctk+aghoqlxciijzqukqmirmdscl1seza9j9u+fbekw1gwzcla/qffivuabooqvwj1ofhw00fbqbs3mieqbdtv00yfe1uzmyfmmamqkrk1ue2ykgoueevaaphkfc5o1ozo6wrxnmc3byymiwlaefky5pvq53qordg6rhbwtaq1jkshyhivdtvcamwvj2m01dvconebdefml+a3jac8m53ojvqrflncvttqcsnoat2jshh4zfpn5x32thwkyj4sc6njy/phweipyd5yyvdkln0plou3cenvmfvz4j67unvqiqi6thjq1g16vh1nt3sik/b5vgpgfrqisn+lmiebfx4rdy8ihkfqzsztgh9vasou+hldzumetcg0hfgo2ihsp2p3x39c8jptnmzokzqqikmq+kuuvcrukbydvyrixh9sph8ubgexaftbzwlwssigyxwfvxpzoswsyiuhtqykn4hefjascevvyhimbdcjb6rke+ok5e6dckondmhnwyw5rxmagbszqiebcbud/qnmghhxgqrstka+ujboozylok4kqtfqz/fnhd9cndcahjoqxjawlkmuhzn4acaoxbfecjjmvnb/o4lgm5eknapkxjumkqw5wuenj6cphwwvksi/atxzwbksljyg85gi5m4oy5knsfavi4hdn53wzxjkxjwhahtmofq+n5zbr+c5f4ifbk54dfnhbbfjpgsmeokcczjjsegkskfeazj3dhfhz5y03c2ebzdqlw4ezl/ga7fxuyqi7u5rcabocskyknvz/evypcm8hs5gubpnexbat9tyyumw01bvmikmxxohuhqvy6pwus5z3ogt1cob1m9fnapa76dvqp5mbjva4fjsl5w9p4n7rhxwe7ozhfetb9/dfrs+dq0p5kdpsef6ase3beuuvnwieupyujsc6l6sdxjxgis9ppu7uysip4pfiwakzlesx4czrsyj5blxlthqiwnewzhklpavrlngd6lpqk+rdysh+jesxbb40xje8jbmc3e/amlyk0a9smudtfgay02daca6mxaohds7xirkdp1l2p+8bhsgmjas5jqfqujwopoifwgrh4l7oyxgglxlskc0gr3tticay7ac1zlebvktakg2bjgotys5xx0gbw8nrhoyoerzbjdx/igpmz+izjs8ha73ctscou3yxfzxxuv5yziyphbngsdindiq9pxedgpaigst4gaqtdunfg4xscgyx0llovbtfnbxzmkzl32qja623jh9skn0h6k0qiff9xbzdcjdgbyp4prawewbhagcrffbr1to4joza5mjglsilvkm2bth6yf6m1bllsob+gijdp3/s3gmdofabotccf7hwk3yak6z5xodri66h3wattpn8jioyhi3cro0qq/x/laxsb7qub7sodbry2w1qsiyhnt5z6h7z7pfr9k9ozapou5jgqp+7deosplvxfas6sfap1lkqlbo+pjlzeo8uaxcgxqrzz5olf8r82cuyvx2leoujimiznepinh2ezjwc3rawbx2h9ykuvzqppa95ox1ijctz2bixvszdmhl0omfivnjputix0srerltybg+ay7uphqpmx3e2a66uc0akgnpa1y0dsot6qput3we7ooua37boiof7v6ujlcaci5tdywsat63iqwmoxpdxylziesnby5nl1jeyp3beydzg28ieukzmym0jezpizk67eab+atqpvebxv/qn/tw4n/rh4w45qq5xnhvu7qfyghufjyth/m/km1liduzze+8geaxnjbm6/tiwbqnibmpkid3ahsax7qxvwg894lf2mpcsyxjd6eu/eynxekftrbsix6ishsgr/5mgtu28tvyujha3prjrge3vjafzpgesbpshsvt7aw5dahawitaxwygaaib7lcv7wkjcc9mlnxsbngvqojyugchsf729asvent07txiyxfhns3cu9pqtml14yqs80kxzprzwuoqukstovix1kpyxwzmvzdyh3damqmhsn3dfkpyaxgaptqiuj1yqswt6pyoyc5crhtwwf42oxe0brpxnxtpsrbyigalshedyishkhrgsnkzih1i96gy02ma88xlgzvh6zgex1dptlarznhsddrduqsvyjjyngvkqrhsfkmsrpo1e5j0lasp1xsvmnkbnumj9s0vzomperosb9bdzxoqevp3odfayczcgh1i9ibrlszogmwyoiosy/cywhtoc4mlcaum4wjorqkexvkyghpa10hb6qeie10ztmoirlcylv3fr74p8jd8zyofhowcia0h7i8rkh64w6srhgmo7cbca1vva7oiyev6ecrc2cqqurrdilx6bcwmhuji7ujsnweedw/bvxwflyz0snyub1szrwxmkqic9w1ub3agr1m8sfaiwx7aonuftsw7b22araehdqquing3h5rvlufja8cpg4mogjhhqitidrl2kwbjhg9zmmdcphzblvypxfhuttgv8rdtcrqtbbkldhulzh94q8v2lrye9d8tnur58r3nliots6wb3vkkkmibjirpnfiz5osdi71vml3kelnbsqcpsqdz25+ypwx4byfcrs0wpuoxp75isgpshlg7+0oirzvzt6drplus1qjx3z7re+y5y7bxau8ezd3z/u8kohuntwo92i+w+6ez9jfpnvnab4p49xlp822xuwpa1mzyvdyx8ayanwtc69v9scbhc1lzhysi/hptdmbv+6bneef2uqd7uk2d2xtaqqjontjz0vm/vx6jodomjg8yll20rhzqixnrqjuer5rbe2bnfpxr8pg+8gnvqclzwex4nrvl9bvd9bhfuu2cgjhroafcbu+4aji+rgmmcdcxwqct7orwiow4c4wrdkj4fvvl9wlu7gr6nmqui9bqlqimlnk1j1whjqh4ni0x2o7fwk2oyf/qq603e1mwrq3yhu+xpfdar3eeu5mja1i/3jcjzo2xmnoocu6nys7l90ad3hpfjdtwrlycwz2jiahkynhspgoa9yydfqbw5c2klvon9l7tz17gphkspgozrtd2ny9dhi1ap2lptemx2a2eekjelqprc7ndgnewksvkzv6hhaqz51rvmvb+jq5x6fir6xngf64gwex9jsqvmoowpth7ga3aw9qspykth71ftmh8a6o5pggq+c16pd1wzpfdd/y8udnb59htvvsi4mqfrsyphhvgqrkplfkmslp1tmhtfrrjyxyddu9w9p5zltr7ebgzc+zdwtcaqwv2nqvuontsxvwyl8ah/qxbdblmarovmfpa5zxjzeyygzqutr9ppv+wr1vmwhla27evtdcx7x3dz6obpz13lkd6ozz/azsuvd6w2sv64ijpu47dkqfxvvpftfornmhcajfzek2bsv3nbkvdzzzjjczohwjmzq15mu8zqh6yblcpcurloqnkknwqp78aix6xdhp47wn4gcvelysogocyrhrsizo1qwffm5e6/qffqonmeqzmoukc7umsqvikijfxcvr638venj100+8jf8jnnzuradxy4s9dd+q7sw4zaccbvsh2+gtiuh0tgjutbvldzpzjycr3n33r7vjqiq5p/6krgcovxmy1hxp7f79kaegux4pcjr4dx2hpoqzusimadjvg6ltnygzrwl6lw8xq2fbzeaznqsc3mddkgov7e9fyldi8nl548bnzdzqcme5y7fmxdv2ujh8gelmfabzt0xx8lpsa9soprpbysyrwsithozav61qk8p8ozld2wb9lvhi9odc30k5jc31xowcxs6wkyez0jzanko1hoisseq0qj2n9kl48ysxv4ukk7yquspjr5gusilcd4m9zwywct9esibmrfopptc6logdugpxg45gg47sfxa0lyjoxqiblchapmfw4muuwhuk0hrz876p7turzceqtxvol+t82r7unmupwo5tyavbyssqgwtpasqzxvng9qdxhc8hksna8g4iebogoaubx26mhpvnmayblolweyf7rcapvr7gd4qk/c9s3bt5ru72japvt9fqydyfd7++ltnyn9jzlux9f3jibhr5pf//3or3/b/fnmd7b+duvvtu5stbbub/166+nw863xw/gt6a1/ufwvt/7t/r/f/4/7/3n/vzt0e590nh+zzfzc/+//a5ok+ro=</latexit> Inference in HMMs since the output is a sequence, this argmax requires iterating over an exponentially-large set we can use dynamic programming (DP) to solve these problems exactly for HMMs (and other sequence models), the algorithm for solving this is the Viterbi algorithm 9

10 <latexit sha1_base64="qaqdf3ra9risitu3aaknjgbtmxi=">aaa5qxicrvtbd9w4ctbsziyb5biz2cfkgylao9ja0qi92c3ae+es1nbspenrvl7tjevzbytrbfi8gqc7m6azv5dflz+s51qb7cyjgjlse51jiwc+khqkvyxcrugrmcgpj//ns5/89c8+/+ivf/zx23/9n3/7dz//8qu/fyxykof0zzgnof8hiiimlkmvjzmj/ahglkrbqv8cxn7h9j8vkbcsz17iqqdnkykznmmhkvb18dxn/72bjcx0d3fnaeotmkq8ynu7v3y1nz3wqn3vnldc1h6qj5gouvhv+1ssptlbxuw9p2urypy7fkasejcvrvhqt4i6upjs1v8m/r1p9v0sz8o0onz3vwdp25mfktvfxx3t+szdgpyhjkmfni3nj3qm9/xvx8rj1+j418ltfy0iq5+homsvgyczec47mbz34znlec2kt5xttr2jywwtjwlpzqknjohymj8dcicgwui9haq0ofiesqxojt6if+goayxghtksm5l9ashl+td+v6d6ccbhkg2m5yrhjvqzjs9q+qylkoc5i0e2v4pspwd3uk8zpuqtvldpcbho9qez3in6rdbwzkpfnkuoh27mtyai+7brplot6ftlz3t3e9gerentg727vgcjznp48d/vfpxrq8jy5qdm/avkk/9elgri1x7fuxrb3wbvrjxe2do6ojgwpidqfaoqclgwm0fs0un9j/hj04muk80l0fqp50v2y/l+gvwfbovlg31nnhm8efpzr9a0fr9tzowwulj7j+bsdupebvyybj/0+8nntvxbfyucuuxhy5c3x0rh48+2pafuxstt9pl7769wd+lidlcr4tjksis+lxic9rieostgiz7zecfis8jde9g8+mpfsc12rpalxdqim8wc7hh/iqqu1t1cqvocjayvwqzhtwutrosjn73xnnsqjej9ydzcrek7mh65axmq5tkftwquloqfyvnbnoqlicxpagm4iksfpl+waonsotznniwbtzusxzubrwpmuigc+xw9owxzciwhk2rj6lwlfm5lcwwsxrtiq1ompauzgjozza4tono/zgsu3ze/qra2/dab9leasbkyvzb04w1mtytwaxiomfvc4plbxiofr3mnmka3nqmjmg5+mwtptbjmxmahhcb1u8l6jyifamuwmh7zrgyedjnmp8zbxtado9mlerwycg46zbxmd6y2iwmihs2hdjkbzfdjh7azusp5zitk+aghoqlxmiidzqqkkgbbywgez6k8se6t4j8ywftgrqkyfkb+uk+qvulrlicvwtyrv/uaamvokmylecvdh6rb6rpvsibmdxg4jvlwq8mtfbvkkqqugkle0gtftr0tmvqypk2/qreswqq+ux8f/czshtu5jrw6aazqn0srzmdr2fudbrizlqzzvuma70gj1pmxryzfmj7mbhqu442ahjqa9ajhnfo0crwzb2fqhr5+9/treaundzemii/kcnry/nwshnakkp88m+icgxcy9cndu6oz93nevx58nsqiihrw8gpuqxp8eu2pnevjpn54okb9/8gsp4kxgjz6/cksn5g0gftagieepp4dx4n//f0jg56yapoofbttuknz4buj//i5ndu7u4tvwrsectxdwqx+5ctyvsfqjsquzxff9coghvnau1v5mnmywgadvvfvmrtys1kw5kiogcp+phhyqlhbl7cosbmx1ca+pslvjzormrbjdjxzozsm0ua0j2gypjstcaladcmp+gqgzchyyvwjmuhynpegiaalik5isthozzf1lrccnnedhjoqsjphwvncxjs4qoawxwshevhyhkramnjalwa4u15eyuvbcncnvbnpu+/05ot+e68gbev3btwdkchkygw6gy1lzi06y3kaalu5gjj4+ojzk5zf4apbu2ok1pkf3dghb7u/akvgfvhnvcf2mzgsillemmccvhicgslicmewzo1ttoidnx1re37t5oiznsglj+ea2xnim7nbqangnjgkcgcp72bxbwb3b+dzqolm2d7qonq321jfdotqecey4eyfsz7av5vjqhzp6pge2heohzep7j/pxghfvuug+ymy1qecm3w94ohkge0r8d5kzzozme2h358+ebo27y9kz85jjlobyyuti/a8zwlzgrk6ckkkdllmurocpn2kozstvdbxwyibifkscn5bogxcfitbtxahgqoti4ydsu5p8+a2axoxdwj4nlqttkcty9ofd5c5jyahxzqt8ycsijub1i9qlnkydbsynjwcobmdi1y81blci4h0/mty/aidoasbms7kloyps9vy+iagcissi3u+yeoapgdxsgwgl3rtlyym209lyg43eumgjrmcgccmlv3gjopwh9ojku0tjgy3aqp/igpmz8i5ds8ha72cmboukl7y/ww3cuipeckiulglhiqlwkdy0xpbgwevmkccj6hqwjduvmhijucqi62frjw/3ts/dtvd4txmrwqjsy0bzi58cu1gki0a8bctfbh6ih3mlhuzqmrnhho2o1c52ldey9qmovnhb3plbcmrkrinydmxrihqbvmuuyglyymn2rf8c9i9fqzxoczerzk2ck3w9anldy6mwnqd7dzrtznki1hamgzhnmqewox/oyxgf9n23ao1pkoq226akiurnfhwqvehjw9fon+1alzf08u0rv1vn6izmhzduxdowlg9ysnckmt9isuzic4pauacjunnzkowtihzy69g/nau4q2lgghrccohsfz5knekecfrqc1gls/mqlki7khfajma7mzsvg+dihy+bqgqalec7lkmkmlupszucfsfcmvlyb4qfqdnvnomb1nqmi0zgdvjd2wlbrhu2pbgdsgbyd/thykx7u/66osocwjgx+hzeb3rskx5y5oefoi3mir4xsjwu2w8vi3ngri9uatsn7bbvzegsa2emri7pdxthwf6n+6aspu6jr8c/gvdenh3fiuxw9ckygag+qetwopcip5n9g3hrdbja0v/d2kfd6ttha19+zom1abiaaguihnofxnt3rg/bbb/itosy8a+gdv+su9lutcg2xok1yitqmkhmivhcfaejuzune70occwoggw3yvf1ymmx6woqmvqe0v+vhnsrjhss8ftg/ufm1so+lyj1xbdq56+g5apxy2zqbzpxx2xuw4s3z7xmjgvgm39q7txx3mgaus3dncdmbtgi8gv01wzfisjkpholy6jfunxvdnazsmvqjy2gz2woqegyvggpsq4mg1yuvwdypye2p5czetg8d4gszekfjpbfxrylqcqylawxbibotytbewsmyubkbujfibbjcxdjwfefpvvux1gino6snr+tobinpoqpwsfikjrksrlksdchj3lrj14nakkmasnknstpqbhlbmccjnw3vjggxujpsr1a9pxpq0rmzyhea4crfqv0yt4ab1axohku8o33si/syyqgca5nkyjdnxotsc4w99wuzcgkcbsf7pa4hzxtnoqghcujcu/cfhvjhwl7zhkgggjaigdshmj9eobrhbqxeoayjsxtxrw8vts4dk69oqyf1yjcosun2itxozwaazrel26nkmlogqp/yqkz5bgequ+rzc6vmysfiqorwcvfvdgal5jlje2stfswldfj07vgwd9gdghag5aahnqlm7qhfw1vqeoxhcapxegjjeghj1cnxbg2aypsqofckygyyvgacrvhzarldfrbrkis3wsoyxfc3yoajv75ou8lnwpif1uwce9fzyky6ocmezckrjamsfq761io/j/bd1+zyt5vcl5lo5czpauwc8otmp1xpdm7xebdbpbzraoyjm4islmlh/m6oask73swz6ckknusfsgy3w7oke2otlftag+iw9rmeayxsvyydzr/nt8vd0zm72hrvhc3jybx6iltz1gx7ssbu+hg89jexq4dq1rrxkjqqsy6wmmy+xhh8rnjswav7qjqjz2yigxyh04eswq+ceddupomizm4uhrnateygjcdter2ucf40ao7baouqxtmq3z1wf94pvadr6nou97seduvpfqbw/w4rvlpghioyjas2jrvccfxhrx6tfbhisjbg51i7agycfellk8q8c33i/zf3zjv9hzqlavqxyaizjtznsecb5aoeb4ai/hzvt02g6d/mc60paf09on/ur5i0qt11vcnelgg9i9nfdjpklxmjlosvynyesyheae98y3obusgj2nz2ycehkzciwuj7rm9ybby3lo02z5g/1ger+2cwcdlwwgh6x5wra2d689ro1a54lznemxmw1e+gie9iorywxdgbexusskus/d9bpqnau1bbxi9neo0ivrwfn/1mk9s3masfokfgd6y7lbtcwbspzs0cqjvm8mbn4b5rwfzoi8rh2yfljni/4hux68+e0yzjleajermo9cvjtsxeovrdswrbllqtrnmfy/uoirq67az3g6n5m1nd0inlglc2fbsi11bzuyagp8aqkcduwx4em/lvvmljeujcr7xmnpkxojkaanpkvdrxrvj+z7tifgdc2nwhxndcx7e+djswbe563uic6u2u9l1r3xgyzzxrdmkhwaysdo8nppt9hgd80ljrp9ihoydso+48yoagdndrizwf4bysgviv/mzulva0sfii2pskifsvbbvvpzgjtot0e/cdpgg59d7+gqgi5xhvhvdtyqrzr+w8/na13300/1jfm9e6zcywhxng1ahbkwsk04atyvqg+vfvqjt/gf8orzdz29wibsrf1wbc/blt3llka63a6fzbgnxwtpzvswt6czeympchffvdymesnk/vgrgh049objwhpkypfqhvqwmpi4qnder+sl5vhp1bwu5zeobmdiwc7bzqcee0m1mpew5emgdshubmoxr5hw9/uirrdfshs5efboyt3xnqmw7nhuwpqnx5ulfaqtuojpfvdsh58kpsa9soprpbysyqisithojbl61qk8p8gzlynftuwu4qmaqgv9hqbwpwt0rbz2dchzvm6rslaluswn2wwhvavr7b+sywemxb18xahd5bmjyxo9wyurshd89v8q3zuix0jwuysg1w+bc3xk2nq8dsa1d8b1d46bissnyq4eiivmmwb+a2giyxgc1rpy9oud9p+oshckilimdfu7qv+0h2ecqnc05wnywfmif2pja87ayvp6aa92aa9znoo186ohvr+pg0atitdoheyjkg8gmitzyx7mma4yglo1twd80fvuu+btvxjoy8gjmrr9frumg2h7e/viy52p+scy9ser20ft46ppn/5l5/d/bp985mdb/7d1z1t7w9otf936/dbjradbl7fcz//3i3/8yvelx965dedpd36481pdf/jzs/olrchpnfd/alhvclm=</latexit> Viterbi Algorithm for HMMs recursive equations + memoization: base case: returns probability of sequence starting with label y for first word recursive case: computes probability of max-probability label sequence that ends with label y at position m final value is in: 10

11 Backpointers in Viterbi Viterbi only gives us the probability of the max-probability label sequence how do we get the actual label sequence? contains label that achieved max probability in max-prob label sequence that ends with label y at position m 11

12 <latexit sha1_base64="qaqdf3ra9risitu3aaknjgbtmxi=">aaa5qxicrvtbd9w4ctbsziyb5biz2cfkgylao9ja0qi92c3ae+es1nbspenrvl7tjevzbytrbfi8gqc7m6azv5dflz+s51qb7cyjgjlse51jiwc+khqkvyxcrugrmcgpj//ns5/89c8+/+ivf/zx23/9n3/7dz//8qu/fyxykof0zzgnof8hiiimlkmvjzmj/ahglkrbqv8cxn7h9j8vkbcsz17iqqdnkykznmmhkvb18dxn/72bjcx0d3fnaeotmkq8ynu7v3y1nz3wqn3vnldc1h6qj5gouvhv+1ssptlbxuw9p2urypy7fkasejcvrvhqt4i6upjs1v8m/r1p9v0sz8o0onz3vwdp25mfktvfxx3t+szdgpyhjkmfni3nj3qm9/xvx8rj1+j418ltfy0iq5+homsvgyczec47mbz34znlec2kt5xttr2jywwtjwlpzqknjohymj8dcicgwui9haq0ofiesqxojt6if+goayxghtksm5l9ashl+td+v6d6ccbhkg2m5yrhjvqzjs9q+qylkoc5i0e2v4pspwd3uk8zpuqtvldpcbho9qez3in6rdbwzkpfnkuoh27mtyai+7brplot6ftlz3t3e9gerentg727vgcjznp48d/vfpxrq8jy5qdm/avkk/9elgri1x7fuxrb3wbvrjxe2do6ojgwpidqfaoqclgwm0fs0un9j/hj04muk80l0fqp50v2y/l+gvwfbovlg31nnhm8efpzr9a0fr9tzowwulj7j+bsdupebvyybj/0+8nntvxbfyucuuxhy5c3x0rh48+2pafuxstt9pl7769wd+lidlcr4tjksis+lxic9rieostgiz7zecfis8jde9g8+mpfsc12rpalxdqim8wc7hh/iqqu1t1cqvocjayvwqzhtwutrosjn73xnnsqjej9ydzcrek7mh65axmq5tkftwquloqfyvnbnoqlicxpagm4iksfpl+waonsotznniwbtzusxzubrwpmuigc+xw9owxzciwhk2rj6lwlfm5lcwwsxrtiq1ompauzgjozza4tono/zgsu3ze/qra2/dab9leasbkyvzb04w1mtytwaxiomfvc4plbxiofr3mnmka3nqmjmg5+mwtptbjmxmahhcb1u8l6jyifamuwmh7zrgyedjnmp8zbxtado9mlerwycg46zbxmd6y2iwmihs2hdjkbzfdjh7azusp5zitk+aghoqlxmiidzqqkkgbbywgez6k8se6t4j8ywftgrqkyfkb+uk+qvulrlicvwtyrv/uaamvokmylecvdh6rb6rpvsibmdxg4jvlwq8mtfbvkkqqugkle0gtftr0tmvqypk2/qreswqq+ux8f/czshtu5jrw6aazqn0srzmdr2fudbrizlqzzvuma70gj1pmxryzfmj7mbhqu442ahjqa9ajhnfo0crwzb2fqhr5+9/treaundzemii/kcnry/nwshnakkp88m+icgxcy9cndu6oz93nevx58nsqiihrw8gpuqxp8eu2pnevjpn54okb9/8gsp4kxgjz6/cksn5g0gftagieepp4dx4n//f0jg56yapoofbttuknz4buj//i5ndu7u4tvwrsectxdwqx+5ctyvsfqjsquzxff9coghvnau1v5mnmywgadvvfvmrtys1kw5kiogcp+phhyqlhbl7cosbmx1ca+pslvjzormrbjdjxzozsm0ua0j2gypjstcaladcmp+gqgzchyyvwjmuhynpegiaalik5isthozzf1lrccnnedhjoqsjphwvncxjs4qoawxwshevhyhkramnjalwa4u15eyuvbcncnvbnpu+/05ot+e68gbev3btwdkchkygw6gy1lzi06y3kaalu5gjj4+ojzk5zf4apbu2ok1pkf3dghb7u/akvgfvhnvcf2mzgsillemmccvhicgslicmewzo1ttoidnx1re37t5oiznsglj+ea2xnim7nbqangnjgkcgcp72bxbwb3b+dzqolm2d7qonq321jfdotqecey4eyfsz7av5vjqhzp6pge2heohzep7j/pxghfvuug+ymy1qecm3w94ohkge0r8d5kzzozme2h358+ebo27y9kz85jjlobyyuti/a8zwlzgrk6ckkkdllmurocpn2kozstvdbxwyibifkscn5bogxcfitbtxahgqoti4ydsu5p8+a2axoxdwj4nlqttkcty9ofd5c5jyahxzqt8ycsijub1i9qlnkydbsynjwcobmdi1y81blci4h0/mty/aidoasbms7kloyps9vy+iagcissi3u+yeoapgdxsgwgl3rtlyym209lyg43eumgjrmcgccmlv3gjopwh9ojku0tjgy3aqp/igpmz8i5ds8ha72cmboukl7y/ww3cuipeckiulglhiqlwkdy0xpbgwevmkccj6hqwjduvmhijucqi62frjw/3ts/dtvd4txmrwqjsy0bzi58cu1gki0a8bctfbh6ih3mlhuzqmrnhho2o1c52ldey9qmovnhb3plbcmrkrinydmxrihqbvmuuyglyymn2rf8c9i9fqzxoczerzk2ck3w9anldy6mwnqd7dzrtznki1hamgzhnmqewox/oyxgf9n23ao1pkoq226akiurnfhwqvehjw9fon+1alzf08u0rv1vn6izmhzduxdowlg9ysnckmt9isuzic4pauacjunnzkowtihzy69g/nau4q2lgghrccohsfz5knekecfrqc1gls/mqlki7khfajma7mzsvg+dihy+bqgqalec7lkmkmlupszucfsfcmvlyb4qfqdnvnomb1nqmi0zgdvjd2wlbrhu2pbgdsgbyd/thykx7u/66osocwjgx+hzeb3rskx5y5oefoi3mir4xsjwu2w8vi3ngri9uatsn7bbvzegsa2emri7pdxthwf6n+6aspu6jr8c/gvdenh3fiuxw9ckygag+qetwopcip5n9g3hrdbja0v/d2kfd6ttha19+zom1abiaaguihnofxnt3rg/bbb/itosy8a+gdv+su9lutcg2xok1yitqmkhmivhcfaejuzune70occwoggw3yvf1ymmx6woqmvqe0v+vhnsrjhss8ftg/ufm1so+lyj1xbdq56+g5apxy2zqbzpxx2xuw4s3z7xmjgvgm39q7txx3mgaus3dncdmbtgi8gv01wzfisjkpholy6jfunxvdnazsmvqjy2gz2woqegyvggpsq4mg1yuvwdypye2p5czetg8d4gszekfjpbfxrylqcqylawxbibotytbewsmyubkbujfibbjcxdjwfefpvvux1gino6snr+tobinpoqpwsfikjrksrlksdchj3lrj14nakkmasnknstpqbhlbmccjnw3vjggxujpsr1a9pxpq0rmzyhea4crfqv0yt4ab1axohku8o33si/syyqgca5nkyjdnxotsc4w99wuzcgkcbsf7pa4hzxtnoqghcujcu/cfhvjhwl7zhkgggjaigdshmj9eobrhbqxeoayjsxtxrw8vts4dk69oqyf1yjcosun2itxozwaazrel26nkmlogqp/yqkz5bgequ+rzc6vmysfiqorwcvfvdgal5jlje2stfswldfj07vgwd9gdghag5aahnqlm7qhfw1vqeoxhcapxegjjeghj1cnxbg2aypsqofckygyyvgacrvhzarldfrbrkis3wsoyxfc3yoajv75ou8lnwpif1uwce9fzyky6ocmezckrjamsfq761io/j/bd1+zyt5vcl5lo5czpauwc8otmp1xpdm7xebdbpbzraoyjm4islmlh/m6oask73swz6ckknusfsgy3w7oke2otlftag+iw9rmeayxsvyydzr/nt8vd0zm72hrvhc3jybx6iltz1gx7ssbu+hg89jexq4dq1rrxkjqqsy6wmmy+xhh8rnjswav7qjqjz2yigxyh04eswq+ceddupomizm4uhrnateygjcdter2ucf40ao7baouqxtmq3z1wf94pvadr6nou97seduvpfqbw/w4rvlpghioyjas2jrvccfxhrx6tfbhisjbg51i7agycfellk8q8c33i/zf3zjv9hzqlavqxyaizjtznsecb5aoeb4ai/hzvt02g6d/mc60paf09on/ur5i0qt11vcnelgg9i9nfdjpklxmjlosvynyesyheae98y3obusgj2nz2ycehkzciwuj7rm9ybby3lo02z5g/1ger+2cwcdlwwgh6x5wra2d689ro1a54lznemxmw1e+gie9iorywxdgbexusskus/d9bpqnau1bbxi9neo0ivrwfn/1mk9s3masfokfgd6y7lbtcwbspzs0cqjvm8mbn4b5rwfzoi8rh2yfljni/4hux68+e0yzjleajermo9cvjtsxeovrdswrbllqtrnmfy/uoirq67az3g6n5m1nd0inlglc2fbsi11bzuyagp8aqkcduwx4em/lvvmljeujcr7xmnpkxojkaanpkvdrxrvj+z7tifgdc2nwhxndcx7e+djswbe563uic6u2u9l1r3xgyzzxrdmkhwaysdo8nppt9hgd80ljrp9ihoydso+48yoagdndrizwf4bysgviv/mzulva0sfii2pskifsvbbvvpzgjtot0e/cdpgg59d7+gqgi5xhvhvdtyqrzr+w8/na13300/1jfm9e6zcywhxng1ahbkwsk04atyvqg+vfvqjt/gf8orzdz29wibsrf1wbc/blt3llka63a6fzbgnxwtpzvswt6czeympchffvdymesnk/vgrgh049objwhpkypfqhvqwmpi4qnder+sl5vhp1bwu5zeobmdiwc7bzqcee0m1mpew5emgdshubmoxr5hw9/uirrdfshs5efboyt3xnqmw7nhuwpqnx5ulfaqtuojpfvdsh58kpsa9soprpbysyqisithojbl61qk8p8gzlynftuwu4qmaqgv9hqbwpwt0rbz2dchzvm6rslaluswn2wwhvavr7b+sywemxb18xahd5bmjyxo9wyurshd89v8q3zuix0jwuysg1w+bc3xk2nq8dsa1d8b1d46bissnyq4eiivmmwb+a2giyxgc1rpy9oud9p+oshckilimdfu7qv+0h2ecqnc05wnywfmif2pja87ayvp6aa92aa9znoo186ohvr+pg0atitdoheyjkg8gmitzyx7mma4yglo1twd80fvuu+btvxjoy8gjmrr9frumg2h7e/viy52p+scy9ser20ft46ppn/5l5/d/bp985mdb/7d1z1t7w9otf936/dbjradbl7fcz//3i3/8yvelx965dedpd36481pdf/jzs/olrchpnfd/alhvclm=</latexit> Backpointers in Viterbi Viterbi only gives us the probability of the max-probability label sequence how do we get the actual label sequence? similar modification for final label: 12

13 <latexit sha1_base64="bfar6w1md5lmeltbtvgce/tciqq=">aabfdhiczvxbdxw3cqy3tw1zwywpyqm2jffkiury6owmkaycpsvgtkztjetjttjkllohmwoybwlqnbm1ov8tpyo/ik/js55tbfrmdwmyxqyz60jhygp4ufuofaqfaugwj7lue3v/+cmpfu/3/+ap/+jhf7z6j3/6z3/+fz958jdvzfaiil2osjgt34zuspin7lxikmbf5olrjizzn+hvm2z/5pojybp0vm1ydp7qucqhpkikqvophrzdogmc9/7xmqmh22qqskmujqlnb2wgsjysyichapzl8dpmpuu6fkfro+vhelvvxbjmzfwjaswmsqnym/lpdox5yl8wub75jcqkink78ieis3ggz4cly2n14tejts1fcrszbjpzfvnzuxjqmqj5+uxsiql4srssey3lo6rcdjebqk3okvflnr2akvpvm9m+mev4aikfjx2qogwazjxk4rza3mwmo71txrb8nfzbqzqlrriyeqsrwziqclb1s1v26mnenvl49nuvzgaqpssklpmsbf3dmmvzi22pqczbjpjrv5yr5je/ocomtdx8ojdky8v3digwstxw3/9o/mrm7sg9o8j/raqwkj9+tcgxqll+x2megztlouccqwmubny+2dtcwxfp/k4n7fpnffj3i1l/9hytbhxalxpjk/j4wf3bqrzykkq2idbfcjqoynybk3eoeln92olv1cupdekcf4lpie5zrgbd7mic5fqo+rgsajuipnrlklyzqlzjuzyr9fsuqay5ers76/ccoztgdq8yjbexi09rzfdhilsstqt7wve8ftn7iuqex+disv9rcszrz32orer+qffv7mj852ps0zsdh03ifiugjhr3kehecgyiire73tpapytfht0jcu95uitudcz5l/ncrtal27u7u9uz/vfc07gqjmmmgge0gpmk9ytrr9zb7vks5nsxrq0ath9a3c55gt82atxvin5ysz7+bvpyotfiy6dyme6liw+o//9b8a/kco9brnaugowyww4tovsoylobzufl9vp9vxzjf4mefwet64u5gpny7c6aqoymo8l3yzavyqbik8a/onwnwv8tjve+xmzfrupwwi6rvrrabzuykq5kloswzwkpk83v+qptvtibkqba1ffotekbqk2sjinr6dhnsaf7nfks+r0tzfqjxb9v6sktas7oaw1t3f9d/fxjdd/zhbupxdfbcu3htsgx5ez0ayh4fjgpskd5vnuev6kgqmhbwnlcyby53jeepve2yojxr4dizgiutwhq2aiyscogfmlabchkzvqy1vxa+swhbtnsyeya6vohgrbn+gg8nzeheitoeejevitvsnr5fdyisv1q6tcqqifexftytxwx8w9aqzukoosm7alpszyv1eh6ymswkensqam6mdheuguebenbi9g8ykpqjjxtty/kcwythmsclk+b1rtpmzgriajf7kdnhulwwy0nhsvusj6ctwqzh21xvvbpftwhg9yha6rums7gu7qsp9sdxdzodv/ukgh/ss0jbavbvz8kt1abg3b/j2t7u3v6d3e/evxh2kr956t/4ngkewfqlllzb4rlexq/ms9gsexaxadqrjchpd0re7g8+ujkyelzolvjenqbmqysbgv1qrxdvuudje4oieji5r2m1y6ws7k9twh85h+vncgf8zjizftfrgmkveblyafuizfnbicjcvrgmk1qwykksdnjmnqejp1hlkiue8kipytfm7ejqf82jaru2kwhe45nvro+wa4l6vpd3qyqj0i1r1axnmdxnumysymoocsqt2mme0hiyacmus/p79fjsskoc3inbjbaqdkzg9zncehambjtamd5mamjbqjl11j2c610yelt0japx062qrdvnihi1oekdkzaov2wweyvzqgmtka7cjejtb/iytfwvoocv0quumtauhhxebpgdtm45cqqqrhwrju8xqibqwm1qocszmjiptaxme4qvuhdxwsz6zdmdrcf4xdra/o/k/p7dcwvpjjchb/abeqwyxk9mmvlj+thxc6qqjs69blbqxvflpn3xxcdmuyibki56wenixkssjeykgaerjbuy0lgm9dsyvyj2wvo0g3whcb7bmyr3dx9qtdkmoaqyqdjsgg4hklqoqvqufglorrfrtjk+v8hf+w3iqs965yy7oasjduga4qiexyifnzk1hdna+phlxbqdguyrvsex44nw5iwwvdcq/elnesr+shx5bzn6pefx2i5tra4ao54c3owrn8futhfmajkry8lid+uqfi348qg9amfhlqhxjk5ydd7gb4ppebhtxg1fvchj0ug1g72cnoh1qdr6+2/33qlbhudzzsbv5trmommoigqklrnzc4j4yenetinn2w9zwnkm9hldbitvuncs61bioya1uhykaxkcl2tzk8msvcvyc8mtt/mj0fls8gy1gbrcd023h3w026wsivbjbd/de7awbhf4eatwssckumqi6wceuhjg1lwji8rjaqwr8vcgf+rcgurzkx4ymwarbmqby1knbn9ck3nq6r4uq1elyqeswna2ieic85xibi8lqzak5pjh4dkrymmmy7kgztal2fmvqmu3cgtpqnbnjbgtvfpzg2z53ji0dwfiiqxiztzxdzrn5qocc44ciiqcmcskekcu6zljtyxry3lujchpjr+ccyzylrvgjt+f9f5mffsmf9qql1jmq6/tqsb8bjtj9lnhqbildwxc2+mywtyuminvg4xqgzuoite203g3jm25au4j4kbng2p210yguxkzqmsse6yeavy94u9ezo2yc8o20dpqptk12ebgsmw3hppbrhhfp1td660tm+/cr46otuwnfk5w9j6msbyqdlim685bksqok2dqlyrtiphzechztjt3ltm7shg4wlaacbmk0bibqtihxfgkjiduyqhcuzbumlgcsuti3qo9gdqk+rdy4aenesxypokkmbhcdy5ij8rmw5bwd1c/puatl+kcl3ucwadriynhxpyawqxedez9zdn/dadycdbmqahi11kvrhh1qc4uvjsw9us4iglzicms0cjyappzp6lyffxddluksfes2bckorsh7jscovu70dnssczbw3cot/jmmmzyjxiy6ggj0at2eofjtrf6yjtdrmyv5ojdohqhoneug7pgbu4dhlbpdgi+u0nqxtl9bz6ap6s5go5svf82xvmyinovovwnmtoxc7i3p5aa8sk1fc35aw7uur+ceedod81wegdbf1rhye1ittz3oumtg7jggo0p7tg5s6invul1qy2ul5sacsvh7zvrssnfcia7uhpg9xwxo1drc13owetp1kfvs1xaqj51cjm4lj1flouv4fykjon/urft9hekyxlyloolxgnl4z6p7/ojwdawfymg166k9xaxg/+s26fxmva9pclcwdo+rzsezz38zkw5dvfmewdeak4mduys9uo7h3kd8cmmps18pkp3b6ydeoecplhjgbqv7zlmwxpber04o6up2zzpyrpw7zpkyutxdmht0ymbynanv7wh7yi72kmts7nbrxs1ncuc3+9jzc3b22wletssw4ozoc7bm3er7nx8b1kb3ax5xotsenv9u9rkacixsfiorad62ii8cc/ccvebmrs4xsjmwu2qdi/njlxjzamwb2reukxwxtpf9scti3oxigfjp0nxobs9fir7bny5fjx748gw5vf2nnh1unwesigeffztb+m+km0yqg3tn773r5lwey97r9ze2aasq76cwdkrb28p40p31bfjsa760x4s5hla6ivof342uc4jdnm6kbdc6zcgy3j/tmkgddxg/kyj4jseenobqdsvyvoca1vx5vgt/g+axhhdcu2le2bijdgdpwfb0mdgx57qoozci5guyyyzzaiozx5d3icwqs/1zk4drhc/ktx1nfcrsjng0j4subsijzixoq+4yfatjtc1hrmwyzncbwy2k3dhmloyhjusoqzkxzpqy4fxnjbtdetngstmvwl3u0ombown8luyv2qgz4vphncrbiigblcjemjewqexrgsnk1lhqqrwflbsyb+yxtgy1j65ggx3dpdlgr2nhsbf1qpwkblcsreimjsmpktrdlxsbqlwslk0kf6hkrkytsevjtkkze6cxh+k7i5ocai06qs89gwow2wcoswnutzihgdjdig2wlkws3xxxvyzoksc5pfi4zdnlqrafqm5twtpc2kbxyfzxj5b2uoykamimkmjwfy4j/5f09zwpqoigdaigzy7kdlcozrglk5eew2jsrt66tnjx56etv9sg0ekyxlrsup1idpqlhebpged1qhlfsotlv5ghvmdrvfprk71txkvnm9vfj93vuh3agr1nitzeiwx7aopmttsw7bn2bya9cnvbkioaexvo8nywnd4jmg4kdqdpjyemvge51tgmkxoplqmsazhlofirgxgkhwn0avr7q7ryokubjya+bbmz/uaitqc3usgumsninphyg6lmcoyc7cnlfq3jdqx64viffxltpfv0yxcp6q1usosnqace3z3llwswlo5xg9tc+1jjr/5ojkzxbup3mgqpaljzoeh8mpj2yqvjuw3wj9mj2+fao7wu7trn63lq0j1refbuar7e7k7gvl9g76tmmrmvf+9hcazpcorbvnoytpvymc5as3xutfigzhdhs1lmplykfpgvtuc5m5awugyw3sasixsnxcmwbd2gy6shiod+kg0rqnreujsytdxeeclahedgqggqhlela5qd/ovztvn97jrf40wklv9w7kplmrchc/ekjvseoel44al1td6iaml6qlsiimjhbby3vhgyg7dgbhzp1idg25yx3au7zvfr41mux4ieljwfyeioznduok5gr2upe3z7r9vuzp5hxthwtzojzl0vy/hxiqpd6/mrck3xosa2lyd0mgnrlsvc8lr+q7j+vq9whc/mjwstfabntmjhagocmwhzv0x1jcsogcqm0o6bzrx3w8l9vuyeaj4zldpa850hlo5ew4qqvc/hu9gbx2mtw4h0d5rui5gra44+ysblhtozfh4w6ieec1jz23jd3xqo9tfwmw1+wspjlt0nnmljbjdzsb7ardsdvdxhspzhf99nclwdygq+ydbxju6yfpjhi8ehwx68+w0izkhzajeqslmhuxy1s0oxndcwtmo9uk3mye1+baoj1qzqz3gmnhnwzo4ejv2s+bagpmh6ne2rq298aqkddpwx9k884wnzmusxymhu5beej3kmfiayre+xvo+03k+4921dwpswg7hzldcx792tjyobfz031rmddzvpzk57bw5m9r4uudtqtn3akmvrp9pigt0tlzqozitminyt/44zo72bm9vrygbbbyqhwqhxsihiph9ybubp6qyl5qlkgxzo32xhix61+8soqqj82sch0xwy4wzdwmkcylttfvi5ek3rf/qpn2tqz8jmacycmnoviendhtuvu5x/vejozu8/8tb+ezz53jhpr5ejudxfuu0vleuifwuodo/db+lasies6jlkszo7s5nqgz7um9fadebz/xsbs6bcoa5zell2hnu5vfohlqygpb4qninbeke8+pm6xrvwjacb8tewe2bnumkzpn7nczjycrahzcpcvzmx2dcizczujl9wejl4+ekwxa8mafy/gsylbkrz0nvrwj2isclmq4ss+fheupqunzk4xcobivxdqaqhrtjtz1r06o8xt7ienc3ruunzfkel+wqm8oscf1ygz7qill7nkfxq0lksoolvg9wlpdjp46vnni4a+lzcljoxddrig5e4juijnd/ha7962sn1cc5n7xjmvg3odjaxksuohkw5dut3e6/qqplbsoihybqzb5hfozjb1qgc1ryy++l2/z/xsuckwlkmzf2zkk/rjzpj9dua8w4sf1mesb2lvwd15xl50qrtwknmzgdtytycp1vudaaxw2unbnakyx0ibuhiqov5aso6ogbpbqgfn5wrvnn7evn/zk1v1j9vgqezrh/il8307f+rct/e7o/29nz7//z3a7/+rp71mr+v/pxk365srvrw/n7l1ytfrjysvf6jhvzhg/968n8p/uf5/x7+zeha4yab/uitus9frxt+ho7+h0to3pm=</latexit> <latexit sha1_base64="udndqd9a8cf2zszwahnidkhxc=">aabghhiczvzzexw3cqc3ye6goxztp+yfg5ewusk5hcrejjk139i6e9m0i1hywsznmt2ygzb9qyhmzkjv+r/zml8kr6kcerobx/cwvsqrfrybwk8ofaqfwkepsoglubv73x/97c/+8q9+/otf/vxq3/zt3/39r3798sdvrfrkixsdplgafzeggku8ya8llxh7lsszjqcr+3zw8rjbv71kuebpcizngtun6tjhix5scvx9jz85x6djlpx+9qezanlqklbpmmyxtyzlinnsujvbtoukzednij2va1ccrw2vddaqqjlhihvvjqokm0kpyy/ehxwy88b+p3hrpfifiukqsbfkftbio6gyazaqz9x7byty0n/jwua8reab5lnhjkd5gbtql/o7joajucqgncopqgojijasg0cs9csuv4bjwlubsz4ii/kqmquphjckbydmnetujkdebmtbvs1vypl4ijedje2kemdyifhddyzulvoqx9rkvx+iq1bdzzftm1uoupoqwlhsalqqwuvmkoock0fevffb6vpni84ct5qeo9k0lmfbvqridzaot0id3rwcvlfdytbptsskj0o2s7nj2lblu3o53+fwv0tvomsv9zrovs66hbue6y4rjja9qpdx70faoh/600e6zfndu4tx2xtxhkvhjnls5e99shw5ybeqbjjuv1dvr5x7h6xkhe4nfqw2t0cbzqqyqgqeq+tfbvq2uvcmvf+vg/blxh78+4gip3i8vtf3geri8io8akr/v9vznuz5oivaknoajcn2bhn9axptvydzv50dqycehnuidnargroyf9siixnnpyawoajyufrl0nwifdcnouv5pyaidciljdjyeb8x5kojyt0wlwjrofk4rgyvdtluskpdz6z6ont/ahv1wde12fsgprb18z5husw/16tqcwtx8whpqywsw5p5jeoqr4ub6hbrawzhkdlk1/zwjelxb49jzbmef7e7gnfmsl6ycizkc9acrxn/sle0totphowmmbpoybudo+h8k3e3ej6p+vfcawca6x9x10tu8idagdsnxe/ryyf/l55clpz4pkh8rnuu073dp38p/olc4wenfc5ueggas+tummsbldaw+ci9fqp+iilul9gwn9vm+vlcjuxmemnlvaubhvl9ssulsrn4b3ahv93pdl85vvmqvyaz0ylzoql89w2sa6isusxqnyz4hv0cb3fdn1kpgojhct6uy8zvft1wvwz9j18khm5iauj5aifm9tix5o1+/pevkqqlbzwjsl9zeqntdylfe0cte1oiopocnevvms3jhtnorpehmpgm2tzw6/son5fqszlpikhc3fniascdmhyx8ychq3vqc4rohcvkbdwjwueknmkygutsk8ltkxr1rbeesb0wojfobqeoimp4b3fy5u98tzyd3aa8uvxrsyj3tuyp/khaostpdq9c/xwjfp/4bvvw2ae6njd0lx5gltdjdwxrjq6pakrlwfqy6gyiikgou0hmudhqpgwq5dxujecjhijxz8eqk8zn3e9okeicrypzotjire7atxmkjc8ng8o5v+6ibqxjxv8jcsw5smqlpjp9ti6yqebrm2uf4c/t0mo/55vxl6illv7yq3v9undv/xd3z3dtuf4n706o87k/wfo/7h9z8khmlyxcyrsuot3m4mt0tyl3gyswo1katlahhbx+wephmam3faqloxiqxdzzcm0hz+syrrtv2kksycjyqgsy/cbsnkx9tjiuf/cgrdnxus4p8wncoiilocr2xkyhowqjshdxrmhhql4yscd0mwi1vdteyjknmsgl0pcysxppcbgrxjngyths7m2riijidtnlkbppeq98o0mathmfatemnwmhgfllpjlm7okmikrnrhfi8lodeesxbzd+y3w6thgn8y3cftuhrcjvsswx4snmw0xdr8wosusuqrmuaowxuvwt6o7egqk9ise8vgxmd2b3hajao8lwhye3icp6rceymbgh9f4xtgzxlvwv0fweknvmzbvbdtic/b9wbok/gasuhqiqfrtuaoye3ycs3kjm3njozboywrqjdqoec2sbkmjuootvckyf3omc2ejtddwvsjzcja/gbeiecrk5muvlj2qhxi6knyllyckgloqliepvg24gjslcgajxlsqshfrbc3toiaajcngljtpazu0lgskndzxhubfmgud2holls7n9utdlooharubjsgw6fml6nqvauf6lpwrfo1d5mvsjf+q3+qk96pph1rw1aooojhr0zeg4eckds9sr391dhzu/tioqhillwo91+dosrqnihm+y/xoscva08plwhvjoj64cxvqcfd1jonv6mkjfh1nrjzeo+r8umhgfr6uan+nmyiwun+i7b8seomqwyxgrd68hq4rgwv3hbq6tubwn7cheu/voz8+nbnp4ocjco7pydvkwwu51w7dzfjeybvsklcr4b5zadicfphwyudjl5g2gcxaqtazia3menak6tbupelqjbbig1++vvbyigpxej7mvlecjjajf2ybu227sedxtqgbc8y6xdtrwjiknqnmkiw9bvwlwflnjomdbbre4ogloskvlrbz7pyng8blutatm3ikcwwrckup3g4qoaaxq8e8rdi86rznqosdgcdmt4rtzheaakyytoyho7vpz4ggc0wdxbmdk4ek4vv6yjgg6ht6ytn4xdyyhnewr/63bm0zahtyvcvehjb892yk/ce9jl+kdjywjanqhq5jgtohsjnsjxu5hqskowhbem/m0lro7cw9m3jqr/kj3rvweimyovtoct3geez3ittm3buoymblpxtkb7jtz8bhs3m0dyczfrny2ybn3pdairirwy5v+gsiw5u5tjtizj2da8kugrc21spd9p+wlftajqfkf81ejgs6wvhh+2jpykern3prs1x7adfhx4clvz7luzsmulersbsshtrhbdetftqxgzesnzcsknuhsbpo+heywkethywyafgc0vdsqubsg25pjblhkzfqivztw0kkjlj1dmebxo7bhhwbbvgcccmjrelpjym+rbycewzmx/colgqqp0siijzrgmtnk3oancqguunnupcrkymdf5k+f1lc/aonuk5kkowyy5v49idwiiscdzu1wuaauqvhy2mxebyn3vigtl+web22+qlczzscgqctsp7itsovm73dnosdpcw3sot+xmmmj0jjyy8gaz0ys0eoejxrfl4tkdmyxrxysgcbupss8pb2cn9dwldxjscbb9dobuuqxidku1t1z2frjwfn83nvmzinovsvwf0tuxc7ivpnw14kcqkfvy4hpuhr5098hyef2cr4nh8s+vid0nn1a6ie1paxhfbrqwkbazs5invul1q62ur5sactdh6zvw0xtthedtqq/sa4wixzm3vaz0yteh62dvrtz3ki9gay9jcbdqsypx/l2ub+5tadifw0y5bbtsajy+gzmy7c92xb09fv+rpkdkzktpttgpuv26dooup1zufarhw6jg2pgturc9jyw2id4gaoefn0rmzydil5x7sfczpl1kc+yiocdqnehjnnyd5zgbve9j3zot1gtr8dvjjh7kv58qyrtxly5fx5r7g3gmddratnjcmhr0rtezfgamb4ky8wljitmr2ibmn4py6cyheyybg/mytsbbdof3epwfrodsbv6+cfq/3f1dhwua4tja5wskab3vfc8cdvccvk3mrs5xsgeuuw8fj/nglttaabsd74tqzibwd7musj3oni6fjx0vnore8fcr6on+8fdb64csn5fjzhxtwuluhiclufj7tgpm/k28aq25gut87n8gro0fcspvxtmonibuojr1p0xy3vnjhvqgfe8dndp/wrkfzxjh60u9wywxeerqlqrqyvbiqaey/09cpnxcsvy0oxi0rbjrgu3wrs6irgnvv+uhzf53gnorwwhoi3xwtaqea91hw9jfvueestjvxcz4lmjkofgxkqr/ob8aqr072tq0erje+k+/ske5jlrcchwtgynlmmsat0vll9kfckszkesfs92f+sz5cw8rtw3xjh2pgbh+k7snc9qh21vsw642xyz6u8apdchmirrco8nx0xraxnoirh+jobiufalzgec1ewayr1njg1mwwaqnew+umzohnc8qysozp0dzgvw+otthemdj1vlwr5bvgslhczstpmzk8qzguu7u2krsn5e+vtncygvs2wsm17e804mn1z6spodqttqhlz9abxraao3qk9dpuaq6yzswgweot1ixdcs8jtom4m1qku2y6tmmuuiitq4uhpa10leyqeps00zxnoizgcalt22d44d8w7prnqvxqgeeatnmw6tyq1xa38rlhcyzwb8s1cytzbt5w8ooanhaodcjvad1oxbr90klzzmjd+qgs30omyn/td6xo7vtqr032shkvxmkqis5xv+d2aav2mpl31los2bcyngvbswdvsa2a8cckgzaaaep2hogtlvh8tvc4kdqewjoemlj1yrwgpk0xesg0jxkwsxwqsu4jgqrwnc4j62ivfe2wglpqqfsw+8rfx7sp5cys8kumizgmvrpbrak5qqn2kgwsdqjurnrkxj/nke0c66ms71lsm9ykmrttdzy6+slxnmk0umvtuafvoxo78bmxmkqp9n8raedzvpp9bp0nx16itdi+22j9mh25bjcq4jm4s/dcz4ms7l7dg3kp5uvl7mjc+41452ged+pv7z002xptugsu0sxgsy+nmqfkrhovlbugmxw9zzn7wefwo6zzgwfugxsoc6ajbnok2xmognuggafdis0xmfjzazkaomzi8kht2uzhzamvnhuhokv4vbwsbxbox590g+8tn/gepnnciqwhrrmefnuwck9ysluquymct6lr9q4xuhpcmeet4meljvjcachav3sujsfghfer/kxtgtu4qezdp1eirgishyhozqe85tjxtsmv64a7f3zhbn5h+sa219ozno3vuxdb+vydq1xmwj7/smdwl7ckilk7zdwcxzntrvsnyu6g6u4z75xs4eh8kek9izgirejlxgikddk9jgunkcurrjwke/jdkvfe5h4b4qmsb3shuxvtmkruoodpya/sj82suekwpn1hihexhgpekissonp1pcxqdz0xrba+8ya7x6fir7xnfvymhpmlexhoneumyppjrcgt2r2upgai1kd930wjvankc3yqofmadcj6fpevbu9tffdmt80wr3qpxutipgqzsbcdgqrkplf0fqmzqnmoa/vjm+ixelu/w9p5zkgwdinw5nfmhwulndyxbiomqpgppqo4sf9qv/kej2xjieesovor3xeez2kumcukarjufat1fsk9bzmbs2tuxobxxme8c3c+jgb2dd5mneis23lmdt07vwgy13xbhtknhqbgcv776rbx6a55obgvx2rctm7fdxwzfq2c0q6beey+krymbetlerex9caygehl51jutyw74ed9pj9q9pudz8o4ivbnntydzsayrxj6shyqbzruw8/mta7/6ad6kqmecbncywhx1ktahtrgyv2xu/lfhw5f/fqtb+n/xj9cxwap1qwv2t2l7btcrqsodpcdu8nwxzdlcqnvjjwp/obmr0jrv79ru2hdjm/d7gkihxgunvnqw1rxq7vz+kg8guxwocpvw6waipeqaucj0z48eo+hiu5ubnumk9pfrnc9jy5aao6fqm1gjeim0q6evcb79g97l/8vntci3qdpd/7rcuu1al4auyhfswytmyvgel/fgjtnskvwq4xssrmvxcqgqbrshvz1ru7i/wbgutafvwfoetfkil+xqg8jsmj1yke7qqnq9zpcxvask2p8lfi1wlpdgvo4snni5b+kjct5oxddxmw5e4jeijgj/hdb981mn5ccfnrrhmv21o1sljvebjauznlzi6w7uvcymh41w9rfayxcd83seml8yqqbaqnftb9x86ykioguaal/xpqjyup04eu+9otg1ylqdzktssvp3ulafluafwgidpnok35/mo+hftz0ajhtdolexalq0ijuh5rqfzulcyohbn7qcftjwrvnh7bvn/zk1v1j+vgmewrh/il8307f+rct/e7o30dnd6//fpd/74rf3rm79c+yevf1zzwomt/kh1derbytvf4jp/mvt/7n05v3r282dbz+4/+1xdf/zrtfppivhn2r/+fwd/ob8=</latexit> <latexit sha1_base64="hgddodk6gjezjepiyhior0kee0y=">aaaew3icnvnbb9mwfm6yaimm2ea88xje1a1fa2k6jnc0oql4qgkiixuxvfer67pttnywt1ajk3+kxwmmcv4bendw22wguoohwff5ety3sqea7edvvbirlauxb9xtpn69b67tt3nzbvncowfoyfsnalxfmasu65at9bfz1+hglo/yhhzwyxr7l1s89csdcmuphevo/tcecoxezrp5xn822tswtsowjec8cqoyomqm8oe19/fmejr5qm9y8acabjbvpdizqx09rr9h6mdatmwz5kqvwtdpea7mweokrjdiqn2doaqsxyp1nhjdta3ecruzww6qmjnm0jsxxm/kd/nlum2vsuk8yalbsq7yagnnaxae/kkyamyn/absfwjuwoqir1uv9xzcyi6mlut/guedx+e/kiw5e9u7kzwnzb2lo4z5ogk5rnvbdpp7rjrxl5dm78g90p0naee8hiza6ywq1wuqsllujjt8tflbmvgyvyoivwpwx2nqrtvjsfsbzyrva1inhsbpwiw5dfpg+qevtknt2osk+oqjd5plvilhle2qud854oa+pz2vf5qu+hpjndgivcvwfcni0zfpvltm0amw2rvlqwjf+21otx9lyv3cckkqdszjskpcaqsiseq1dwhl6ia8qeq2sfnqgcmtqxzskvuf4xwd9p3fcrfxrd8vgvjy1vxve3yr7amoxgtnoy2y37w9pq4cuiawvgq+orutds45lxalwxjo0tg5lfzk/md/pn6vfkaswqrm+g5krbuw9cgpuhvwgsrmyj</latexit> Following Backpointers in Viterbi full backpointer-following procedure (after Viterbi): 13

14 <latexit sha1_base64="w3k9brscurijmdsmgaj7qrnnjqy=">aaa2t3icnvtdd9041c6u77wvmaox3bismc1nqk6hawsoi9cwljwdupoxzeyczsm2jo8a23il+xzuy37j+2vew7jhkl/chyu9jz9jw5ktlkzvxpkevbw1tffw1keimmnshr39870bx/nq177+jw9+a/t//vfb3/nu+x9871pjkxhtlzhpupgsipjmrkavfvmz/awulorrrv8sxdzh9r/mqzcmfy/uqqrnouklnmuxuvb1/sgnu3tfvk5++atgzpsm+fzewnbfzpocfekdrmpgfwnqmi/4eoo8s3kizsfuqbt2myyackh7nhman6etm4owk6dmgb6oomm9m2mabiiwkpiaohs0uykkql42++zjzt7qrxmrube0n69cxuttmhoonfjw+tklfvbves48gbwcl6o8ogi+eybljkykwmu02p7xpzcn+yvnzks4imxikwa01ywis6vu/rv52wbavqhqjfvbzd8wjcb+gclffmn3ev6vpgxzobuisvy/azowo1n186pog9wh/u7yxifb7qd6oib+vx/ecgvlzwpgwp5vll5obgzx2ghlwuvfg5auy7c+eko23g6fzfn7o0ehr/oncd8m7cfovvvz9pydn70xjjyuclqoocnsnk6osnvwe6fynnfmo6wklul8qvj6cp8fyak8q7wbn8ee1ctblav4v6ha1/ypapjltdraoptsbsnkx9tppaa/pktzuvakfrhpafplaagfvsvimidhzyv4ilfgigsqz4ggsqkf2h50u5imfkrgg6huacxggewwnhwknlf4oaznq0wxwrfd2pjpwzeqxgyrpznwrqo1rkvg3vqqrs3ooldykq3azierbkydmevz9pb+zd/ycmbvfr8gcxb0fszinayfdyqqgzhbrnwc0kitdtwdg/1axikaexlulb/wysqastqelu74oepwhyodak8x4enjyzqbo8psdvmzy+9yq4+sthdersbucnaxe2b6mlvfghelxtlulf1nhioxhj2sss24wfei9gxmeyhxebuxwyvlsyueore8p+o4u0/kpxsqdsfam2qoqf2gxh2f6oldfy3v1b8oiibijkbfheqp0khqdvrkm4rjgutrikasqpxk1hbailgeeaefyjprmkdzheqpo2vth9g86tdoggvlwgfqo8o7diueq5wi6kazqeosjiqpo7crbgowzinzs709wgsuyhs/ytxm+tcua9nqr1lfcvttwcgnoa12jshbwcdph5212thwkyh8ry4nx8/phgeirglnffxs1wruimdwdx+ewj29nshqi8exoxkkqacpl0dvusexv/riizxt6ocna9qnjxzxsxqjagpgl+p6gcllwpcdculoexw47obp31qw6f0bqn+aj6idanu+fhp4v1juzujuiauyjezo6dyrvqhs9qqrtzc48iri3qsi2txm1hzfff5g2gcx6qo6zknuostiltwaoeyxeqn2nrv88ouclanlxelbhvl2obmpupfswjad+mkgpch5qrmh3yxasabiwid1jzborsykq0rsdnougkqzsv0ecwvwycz8vqpv+xcgur7kwiyktibltxp5wsnbn7csvvcqx5xaqbzjhy7bcpache5zwta3lcfu8dw4ot6+/yqosunfr1w1gynbyminoop0n3fgxxcrx8bwlseofnhonekrencfqkkxc2r53lwfxwblwynqzosax9u32a1mjgtfhcjdme6qbagnfcmeybjnxila9p6avrujsm1qdm/svcqcczm/c4finvcoi9u5rcaeocskyknvzvgftu374olybc3tm0vign02tmkmdsiif1kkpsnzhx3qyjfvyzz3ta8kydrhbar7t7pxwlftomh+om11gmas2sw49k5pz7i7ytozl58nrt2u/izp6tqrycmlwbojtvhvx0oiwuvzcyg5qz4yugatfp3s7kquj4ymecekviposgulyhcwk5rew7dftygdnssalkm1d3lncbtepat9wa5t2donyurbzgioechnnmzj+wknceud9curvjnnbi0wgkge4mldrx1orylsix+znl9/mo4bfsyoru07inll3j6inykkxwlo75nmcaec7snfgmxpimxkwytp9ukn1u8picszyemo5nkjvhhqetdyahe5o7snhu1vb/kyoxnylnnl6adpri156gtf/qe4kun/5bllkuksgirmcqnht45db4a9aayjpozca13s8qyjmz4icrya3fx60/za1wyjc5u9aozjtlyyvfczwxovumvrgr9o4cpqow8awomccibhs32di/06ajnaqxq27gdmmcalske2awzqgzvqvw3lzqvlibyxufuofw67d8igdtsgmdxhba7vuj7p+cyh0yuuh91mrbatfcskgiyt3eaneov8f5qf7g/arnu0jnquctsnulesotbeweh+/+ths/svvs26aluywej/3qz9ftouaxocxvirr2pzympzx1lfbih3iga4q2myufoqprfk/nhlml8epxjtoydsgzxosjx9nmkcat62sm8dh/wctl86qaqpew58yowrc9mymb7mw5h5dgdacsafgllysxkivyiotale1rdruzzqzb7ubadcxewasdtmgktro2dbdy/2c78dtkdxat9vnbup93+xr1laoey2u8tiag9bxwphhlwgr5f5ksngtsyyl61jzh7sijftwlarpxanbii1jezxijg57uiz+itrp/scx13rn/gx497oh4875ki6xnhwul0hyalufb6rhpm/k29kitcymt9bn8hrpwdsooupbde2idzajqykq9vd+nid9q34tqf82h4tnjx0jiu4l/3upfxdne4zlmwh0sulwuv/myzj7bxo/kyiedemungatdwfonmmb6xu6nta+3sbbkmcfraimnehgyqaghssk/ty6nxz2aadewgec9cat9cmfddfr6/bsjsnd86sbeyzflxv3nhcu1pqwei1i+rsm0nxzhtzdmegiemmaqrjzcawut4yrmdljme1moawktsgzcaw0moaftwr9grllzgnlazpkvyhijmta3xtri9piifi+s4xlwcxkmawfgi6krzdplwmesubkhuj3obldywdzxe0znufq412orp8coyozo6srqjqlaquuzlf4jilky+s1hwvdjworzkursr/qmsszpcxltm8ujp2tjkw6jsjcwtvczqplz07a5b7am74auox3afg1lciczz4qfukh+5lxfa5ziavwc6bjanzfwp768syenigukl2sd1c2umac1bdmotetu5lbppbvmevanngaqagkofd9aobrhbqxeegyjsxt5pw+vrs4jj69oqzfzyjdpsut2ijfozxaafqlr33vobwjr/ufzpof0z2js5dn7pcyqwwkqi85xv+92aav2mil6ay0w7asmunbtwlb32aoa9cduakemaubtacvbw9d4ksdjxqalknyh0ilou65dtgmc0unxeztazdkowjqncivkmboibqnvuw2cvwkjow/b7bn/c9gmk5u4ennqijgnvrpbqndyij3yu1yqemx9xpwpe38ecdi71tml36wkn7nhou2jb55d/5qr5+aw73ab1mp7jsae+ikoktio4/d2fbhr9wafqzdj6iuqpwo7hdyn2voxyk4d3ha3d8/1pcjp7ju8kpdovl3uns7y+az752ged+obiak225phtxmysh+dxz42zgjq3tr3wmuhssnrusbmxzqcn5abhxjphkgl9jlnndauzptqnu1bmak6nxrc5ntppwmcijbjkafnqouoyayqpddcvyfwicpp1+ycvz7ob98hbva94slvxyitv13svd1yh1cs2slixsacl4ir9r4xehpcceet4melj1jcaahav3ssh4h4bvs19yl+zgrqjnq1k/balpistbkvf7wnnghikjot0e2e2tybtz8p+aqnvezqtcvrur8fgkalpr9zzwzzlyihyvj3qyaeuv5kywon1dsnvrdnax98y3bzusgt1nygccchkzeiwuj7pm9ybby3po0xa8jx4jvv/zuyebtwalj9b8yaibu9ceo0avc9h0ivhyzdyivq95hixzxhjog/xzvps9es1rbxkxopq5few88y/geldy7sasb5mvgamy/ddw7xhqbiozwtr7eklghrax+cdd5dx6kpxldb4yfmnlgze/xyy5nuuteuo8papz0rhzdv1y0liyzlsnmpzdwv3oekasu2qf4zl8pnu3fa1w5pfmhwuhddyxbkosqpgppq44sf+ad/nynpiytfe0qucnactllhsmmigeln0fab2fco/bhodlftdiqyuuy966ox9havs6b6lpdpbsflz23vuzyblxdcmkvqcdbticr/1mi2t0v7zqodyvmifbt+i7zoyobs5sx5sz7l+rthgf+bkomqofwiyqackki+apzb8c6kf7knfhh3fjvangzy94ucyhy5xhgguwhhjv9b9+bl7r+p9+6iez+pkwfrolu6zja+kqimvtxwhjfxv6cpntt7yn/y+efo6tp9fihl2137rtobhloj0fvifb4emiefrdgeizknh7onmzjyvc3xevtulcsnvr/bvn9+chdlpm1wd36of0mjjyeedqjk7qc+xrz9ttp4/wycowtrkao4ms7ix1al7blk+aocbtfenfvebamdslunl+we7l+nmjh/vuabiacy9zn6js+oos4r2y0cvmq4ss+fivpdskl3zwtzsxmpldqgqbryzrzy3a+zm8w9knu5zdtfiaxtbcp4ep/fojjsvhtxetzescpwpdgsukulx0wf0axf4+u3jasnrb1xxgtz6rnkxjm1wsoqtbn+ggx92bof9ccnmrbjvvm7k+zwxqkubgmvvguvthzrcsj6nqd5e15ggwhzmy5ckfqlwghp5sv/1noiyeigiued3+buox7cepppodzdkahn9ewqve0prttvksftqrhcbjljhyu1j1wpc3dqnorvhaqym90eubbjnwcdhdpdmva1c35vaad7rkbd+8/qtdeqp5rfo3vy+dytbsf2+fv78zsf9exv349m7h5ohw8uef7/zuj+2fz3xz6wdbp9q6utxz+sxw77yebz3derkv3/i/g/9/4283/n73h3f/dfffh7xqg++1h9/fgvx89k3/ahglqoo=</latexit> Viterbi Algorithm for Sequence Models (with tag bigram features) score function for label bigram <y, y> ending at position m in x could be anything! linear model, feedforward network, LSTM, etc. 14

15 Approximate Inference exact inference limits us to small parts functions e.g., Viterbi requires parts with only two consecutive labels, and takes time O( x L 2 ) time complexity of exact DP algorithms is exponential in the size of the parts we want to use bigger parts without exponential increase in runtime so, we consider algorithms for approximate inference even when using small parts, approximate inference can help us to speed up inference with little loss in accuracy 15

16 tag set: N: noun V: verb Example: HMM POS Tagging D: determiner J: adjective example sentence: V D N Lower the lights 16

17 Greedy Left-to-Right Inference build a label sequence one word at a time from left to right at each position, choose the tag for a word greedily to maximize a local scoring function 17

18 Greedy Inference Lower the lights <s> starting tag 18

19 <latexit sha1_base64="pkzxjez09gfztrywto6qwaybiyy=">aabbq3iczvt7cxw3cqcvrwst+87jn6lk4siylk4kj6vlxrljsh1puqsukvqrkd/eifewm9hdkpmsgohuajt5dvl4+sb5m93a7m7gsxw4lxkksjuafuhunbqnbga7klmu1f7+f33ykz/64z/50z/76z9v/svffvrzz37++v99i4tkxoxtxksf+g5ejut5zt4qrll2xskyzuyp+3z08qtbv71kqviip1alkp1mdjlzmy+pgqrh55/+5zbn03lwzw/ji5cvczvtrezzzpszwvxkgc2topkfbooogxxzoljsrpsqv5t/0jsndwmb2k9a3jsqetemb+qxiaiyzhq+rbcrz6omqmlm0/qwaui0g3/lyr2nijsrcycvallfm+auofuhpxwwmya++alagu/i4p6pv7qc/miaupz7m9ub2weaeiv5qx6uwh4hrasftbuvopgr6ck4zscpo6s/rkdfnb93b01zdry5tsr7t1i6ql1psf7ku7ikjrvkptzc1h1evusyinutan7skh550dxdndrrqrkrf7sydikhm3aincmu3fmg6ruyodtcxssff3tnjdevbjibhfviqp/ldylosxq46san5/0ltsyplfh2qqpaaxtr4kxyge8as4iqy3vmauv6xb7drr6ll8efiezdznwzx5m5ivnuelimbkdvb29nyww6ovpnhjzkzmmmjopcun6gkxxb2/mwhzza9i2hoaap+4uz7zcn9kbljdrpy4lnz0f2nj9dkiz6mmzlrnrh8e/f+34b/jfr71xofsbbkxgl0ngl2o1br3hc3xl/rfwwxdnriohutskjzi8jzcthpnkqelplduuwlckiejwgpa3f4m1dr99+vv/mukho3ratnuz5eqsmkqukpznoktfui9uvv9upwtshzjbvqbmnkxursk8w3tj2z226wf6gakgd1ohbbdwt3n+pfb/8duk0cenj3x2tu4v24+ni3w8d1duztwhostad51rt6r787vrqsfi/i+szqn9/rjqhvecd3iablxgg8uh0jc8rhimhhocvddwidqggeqvwad6ctqqitmfujeryvfy6wnxxzqpvca2ukjq+oweoj4d3lybujusjbd0fm1tbrbzcapfzcz8q9wopq0pidhlb9uvr6ybjv+sm1sjgxbryi9jnedqqw/evlyxwxqwcunvhzryh1gijxolgshnavlxijyad+ki+yjddqszp8hxozye8gxojyvdlyxrrkwr9r+fe45rkycelnmzmo++qv+ppokdk28ksooom4+w8pc96vmpp4np2+hexjifnfu5ok3zvmxndg1sqyktsxmtm8od39vf29r/ifwzajzsb7z9xw89//umufhgvsvxpiu8g+6u6rwg/4hhkip2qjctpfeen7aq+c5oxevrr5leh21ctkheh4l9cev3b71httokcbcsoubptwblqo6nu+j9oyfrlsoh/m4zgvupuqtatjqkxoiv0ar80fhxkjfgugnupyfc3ltyltsffzqtrkdummzb9lnbtrnbyyuo5xztvqekq5tm1cxejca8qcrt6nahdkld2lyp5yyiq5lbgu5fj6dqmm8zeydsek5bxd+yxo2rfab9vvedmioiutoklg3wsejyayxg+ymrgwk472urowhqhmx817isoawbxywo05hn3tdjhvpuofcrwbp7gyyi01sgsskn0usd8tlmhztbhdtmzlqswfrx0zawyhkxtphlrjx051ci9yqxvwoad0epnc7fgvowimcmql6dqxf2vlcxle45g8iapg/qjlf81hxuo1poiqqnqb/vktuhznrfwxcrh9spsnmmw/bjeyxfb1e30yfcvl9omjkzswvnaty5ghzg6iqikqdrlye7totjtx8qm3mdz02/qhwy8gtvt7+/9xm1lgeujynby0igmisrwo5bvawuys5gs2btpzi7kcx7dejqtwak5n9htui/siugoxsqoh5yqowoyu/vvq+en7glptuutqrwdvdn1hkgcgeqhr7d6xy9bz44czh+mihr34mpuwhd19nnvqepzfhsnr5znk6z+dmshxj73xe8n6eebm34z169pvnlwis4qxj8ogojw9oz9bypecghspaxrteotzmfv9/4jemxc3xl4pkqypfw0xrqijuomq7wqui8k4rjxstz9clvvzpigiwxwshvvhtgbwsyhrjxkiqjprvsqhu0gv8kiym+ez37n+6bn/fxdadxxu+2abrvhbrbzzxdtpxy/3huxmwqu+hmingvgcrtk0tghjtkdpdrdiogr0wppqvhzrl4fqodmepbdf5kwhly1tevdgijubkm8c3spk4gdxmywwilhbepeo+y5x3nfscaiymjrwcgtq1lsesngt81gsrczoinoatzkvfhnhchiikvm4affdbk3axkcs2hu1hhy29k9uyafceq5ysgs8saqj0iyua7lgnudtif1xzwgojf8ai4yxrlafc16z9l/dbiwrpnjqxowexnyersk+x2guef3m1exhoaarbftelxw1gxh3wel6a5jfepizattv4gptww5agshrcu7m+otkv61qtsywwprdg6gxv9npy2i0dvr1wq/lq26qfwgdzbdjmcghteffk685ga41pp3t5dphqadq3jofoss5bashbmkv/3ni560apmwchzqcppyiiwa7dpaezyuxe0cgca4dklsbtdgjlf3jjizaiwyebchcjbuzkmnacpxba7ycdar6d1jtr6cszdiniwsesimexzmt4cyty2qhq1/so8nv9omxtixgaojgw6plte8u1imtet47dx3aaggbjlqqaxz116rpph9rbyyvncw8uixggbzikma6by9pu8wl2+1ef0e0qlsmx5eig4lifspeycbb6d7a3yjmhhhsrdvgvmmrwjp6y+aii0istd4jstrf54zgbxo9fypnkehugopeug7bhb/4chlw0hqafxagzl2l/g55np+rerqobz1fn56fmcjzb6fvjtltlw9ynz5wnbjfaiw78uixbrkefpfbubpdnewdyfefhyo9is9r1olobwdqzquav9mwwbbycnvc96crlcuuhplfr+976tnl7bhah6klmnumdl2rt7msdgl3pbsit9mo3ymdoi45uc9oonz11/l+wboq3letex086brhtcqh4mjax7210vz989hbxv6tmxxsxmmho//sn+iym3dc0bfg4u8feszcft79mfhchwskc4bsnyctonskgliexvxa/x9vjpnsdsm9woidx8jzfbqo8a2hfews2cnlr1qslq8ihwrprqhkflf9hlwci84aodfhoc6hshq6vtlexiybddmf1xdiud3vzij27abfxludstg4blm68afvwvb7fhxdgc/qx4pent+nh/ne1lwwez2ttk4wm8k5vvpdmiqgkglkqucbilljuk/wmlixdy2qrsgtkl3wjs7gukb1yy2t+cgkm/hjtogyc1y/f0ocp16/gmhz9glyrrupadqpzgixiovcczhj/oxhtbgid2/w++e5e6znllq6/ckvbgbifwjoqdu0o4yt/1lfg8wd43b0tnjx0jiukupjdsbmeeeztqsooo0sooijdzxomtasu4vcvbb8xxkqdkqolwdivb6xu6sda+9se0xdccc+jezphjdgazlgc6gomy895g3zuezwxykqylwmownyd34cuae4endobjcz8vt95oklpwm4ej5eekljlziino/pghoocijmpnyyvgcpizmo4hpq/hswambse/deom4afhgpk1vsy65vxypxu8qbx5saduekd4gs1eskmecku31oiehwsesicqgwt6rdevo4xomswulfirajcxdjwfefrvvowndrp6orjmtqaekq6rlerjhwfkhwsvyljbzskbqqk60rtlarcjyvdjwm1vsrwtvfizdg7tfle3xmzi6jeopp60rmzylkd4lsyqf2s8iajzkixbetfzvpd9/zlcfs5xpjkicum64tsc4xc+rjyqrpax8he14cqbrjmhdtqfcgxq/ssd/wn0t/zaqgggtajgds7qthfobrhrqxebgyjsxt57doqfz2pvliiby28a4nuvzq3e5lbv3bqpe943b5v4lujuf1v5ohvyz1b+6jj3sntytpmdde77urddmdb3szfb6/fgnubqcuuhqtuhm0bzachliqycji3pwxvbuhjc4lhjattij1dobnvp1xbanmuvycqbcmbmgnquxmaeewvz3r53hqrvfo5q8qrq9+cusf+5qjnbzdxjs6zkekchm4gc83kbdm4u1yqokr7seorz/88l2jvwe+xqpeswecmyniebudpzu+5sgkwxs1ug3r40nhyybgto3lawpidjezudnzcm+gymqq7ldkz3q4bkuyv3+fsozwbd+mf6wvq0s81aij/al7d7l5n+xdf3juax8n4/vohvvpjc0rbvpoyzxmtzas/xutcoo5hzhs1lnps4r/lhczeclfyatcm2svifzqu4assc9chpan0lhry7dvdoiggij2dskqnxeeclarebgqooqavel65q989enfef71he+r4xier7gyfexng8plu4vs64icmz54bn1td6jamkgqpsiicjebby3vhfygrdgbxz50shg2+rl/oxduqvo6alul0eilstkmctunndueq5g37bhfbd9yld7j/8t42jb25lj4d+klfh4rbwh9tilvdowestu5yqojl29sowlnrdvslyu2gfuyc58xzdje8g5irsjkahk4nvd8ahrwlsegpdcv29etn5vdfdrmqcihgrw6gjnw0ny3b32cdv6n8onsfeoxyw2eukw1hihexhh1ewmwz0ov+hhbph55lukvb+ib7x6hyx1vppfcycx7jto39cbzhxsrxbb7gavz5hs5dhfnxmc74akgq8ytfyjd1k/lupf6kmrd978dhhm2gy1gai5ryfzfndlq5e8mjbou71stczh7h76p4wavfsmz8qz45bzjcbpwliqfoq02jczquroju8ttcobfeh/5akfy5ixtxvdoyk/ijwrc6ewxcr6uvr9ppn+wr9vswhza27eftdcx3zwmx9pavc6b65pdlzdpgu37onjmnx9xxkp1em6gumo136mxte6a15ohjgxmrcto/4dz0z7a2+249um9t9ijkuf8bkouqyfwebgaekci+apzb8c6r/eoks/3ftnndue/mytz/msgommw1jjemju03/4uxqtg376qz9k6mfctggsze2dbsshix63fxtn+fxo7tv/e2/gc8+dxetq+rcbm137rtdszl5q98jabdb3ki2bes6jnki/z05m5gf5jdd6+1aujl/bunnvnuhidjnpw9r1nzaxjsw2diewbbk7iof8qjn6lrxg/fblajphkxauwmsphl6t28gprpgdgguzxsm3mnfapiboim/yls5ed182f1vmqighgvp25ewdhvucitila5tkuekvn4qvkarlcyuf7kk4lcqkaeivza6gctevwnelayfxutw7rjuxzdc3sju/h1iqurgjwupqvxourvurqwk2h+0wf1as329+nfu04nhxxzyzbjazqzsoq1bolqaufpmefxjwe4lsg56r2dmbfnht5lbgoxgchs3iglu7pf2jasmqj9eflj9ghzww+txezaus0he7/eaf8zxhzcetjxrg7/berj9unemv2o5tsclaioc6m3toakrtytx/vqlxhciaksxsx64ceroguamrx26mchumxbmagxfz3ma1nhgbo9twd82lvuhubtl3x7y5qmbv+9cobosp0xfzqzch8i439882bvsl83+pd/upo7l9ufz/x04282/m7j7szg4x83frfxyupvxtun+//uxvp/vis7shuwdvdt4drab6k0/apn+9yf05yp8dfvd3sa==</latexit> Greedy Inference Lower the lights <s> takes O( L ) time (iterate through labels) 19

20 <latexit sha1_base64="pkzxjez09gfztrywto6qwaybiyy=">aabbq3iczvt7cxw3cqcvrwst+87jn6lk4siylk4kj6vlxrljsh1puqsukvqrkd/eifewm9hdkpmsgohuajt5dvl4+sb5m93a7m7gsxw4lxkksjuafuhunbqnbga7klmu1f7+f33ykz/64z/50z/76z9v/svffvrzz37++v99i4tkxoxtxksf+g5ejut5zt4qrll2xskyzuyp+3z08qtbv71kqviip1alkp1mdjlzmy+pgqrh55/+5zbn03lwzw/ji5cvczvtrezzzpszwvxkgc2topkfbooogxxzoljsrpsqv5t/0jsndwmb2k9a3jsqetemb+qxiaiyzhq+rbcrz6omqmlm0/qwaui0g3/lyr2nijsrcycvallfm+auofuhpxwwmya++alagu/i4p6pv7qc/miaupz7m9ub2weaeiv5qx6uwh4hrasftbuvopgr6ck4zscpo6s/rkdfnb93b01zdry5tsr7t1i6ql1psf7ku7ikjrvkptzc1h1evusyinutan7skh550dxdndrrqrkrf7sydikhm3aincmu3fmg6ruyodtcxssff3tnjdevbjibhfviqp/ldylosxq46san5/0ltsyplfh2qqpaaxtr4kxyge8as4iqy3vmauv6xb7drr6ll8efiezdznwzx5m5ivnuelimbkdvb29nyww6ovpnhjzkzmmmjopcun6gkxxb2/mwhzza9i2hoaap+4uz7zcn9kbljdrpy4lnz0f2nj9dkiz6mmzlrnrh8e/f+34b/jfr71xofsbbkxgl0ngl2o1br3hc3xl/rfwwxdnriohutskjzi8jzcthpnkqelplduuwlckiejwgpa3f4m1dr99+vv/mukho3ratnuz5eqsmkqukpznoktfui9uvv9upwtshzjbvqbmnkxursk8w3tj2z226wf6gakgd1ohbbdwt3n+pfb/8duk0cenj3x2tu4v24+ni3w8d1duztwhostad51rt6r787vrqsfi/i+szqn9/rjqhvecd3iablxgg8uh0jc8rhimhhocvddwidqggeqvwad6ctqqitmfujeryvfy6wnxxzqpvca2ukjq+oweoj4d3lybujusjbd0fm1tbrbzcapfzcz8q9wopq0pidhlb9uvr6ybjv+sm1sjgxbryi9jnedqqw/evlyxwxqwcunvhzryh1gijxolgshnavlxijyad+ki+yjddqszp8hxozye8gxojyvdlyxrrkwr9r+fe45rkycelnmzmo++qv+ppokdk28ksooom4+w8pc96vmpp4np2+hexjifnfu5ok3zvmxndg1sqyktsxmtm8od39vf29r/ifwzajzsb7z9xw89//umufhgvsvxpiu8g+6u6rwg/4hhkip2qjctpfeen7aq+c5oxevrr5leh21ctkheh4l9cev3b71httokcbcsoubptwblqo6nu+j9oyfrlsoh/m4zgvupuqtatjqkxoiv0ar80fhxkjfgugnupyfc3ltyltsffzqtrkdummzb9lnbtrnbyyuo5xztvqekq5tm1cxejca8qcrt6nahdkld2lyp5yyiq5lbgu5fj6dqmm8zeydsek5bxd+yxo2rfab9vvedmioiutoklg3wsejyayxg+ymrgwk472urowhqhmx817isoawbxywo05hn3tdjhvpuofcrwbp7gyyi01sgsskn0usd8tlmhztbhdtmzlqswfrx0zawyhkxtphlrjx051ci9yqxvwoad0epnc7fgvowimcmql6dqxf2vlcxle45g8iapg/qjlf81hxuo1poiqqnqb/vktuhznrfwxcrh9spsnmmw/bjeyxfb1e30yfcvl9omjkzswvnaty5ghzg6iqikqdrlye7totjtx8qm3mdz02/qhwy8gtvt7+/9xm1lgeujynby0igmisrwo5bvawuys5gs2btpzi7kcx7dejqtwak5n9htui/siugoxsqoh5yqowoyu/vvq+en7glptuutqrwdvdn1hkgcgeqhr7d6xy9bz44czh+mihr34mpuwhd19nnvqepzfhsnr5znk6z+dmshxj73xe8n6eebm34z169pvnlwis4qxj8ogojw9oz9bypecghspaxrteotzmfv9/4jemxc3xl4pkqypfw0xrqijuomq7wqui8k4rjxstz9clvvzpigiwxwshvvhtgbwsyhrjxkiqjprvsqhu0gv8kiym+ez37n+6bn/fxdadxxu+2abrvhbrbzzxdtpxy/3huxmwqu+hmingvgcrtk0tghjtkdpdrdiogr0wppqvhzrl4fqodmepbdf5kwhly1tevdgijubkm8c3spk4gdxmywwilhbepeo+y5x3nfscaiymjrwcgtq1lsesngt81gsrczoinoatzkvfhnhchiikvm4affdbk3axkcs2hu1hhy29k9uyafceq5ysgs8saqj0iyua7lgnudtif1xzwgojf8ai4yxrlafc16z9l/dbiwrpnjqxowexnyersk+x2guef3m1exhoaarbftelxw1gxh3wel6a5jfepizattv4gptww5agshrcu7m+otkv61qtsywwprdg6gxv9npy2i0dvr1wq/lq26qfwgdzbdjmcghteffk685ga41pp3t5dphqadq3jofoss5bashbmkv/3ni560apmwchzqcppyiiwa7dpaezyuxe0cgca4dklsbtdgjlf3jjizaiwyebchcjbuzkmnacpxba7ycdar6d1jtr6cszdiniwsesimexzmt4cyty2qhq1/so8nv9omxtixgaojgw6plte8u1imtet47dx3aaggbjlqqaxz116rpph9rbyyvncw8uixggbzikma6by9pu8wl2+1ef0e0qlsmx5eig4lifspeycbb6d7a3yjmhhhsrdvgvmmrwjp6y+aii0istd4jstrf54zgbxo9fypnkehugopeug7bhb/4chlw0hqafxagzl2l/g55np+rerqobz1fn56fmcjzb6fvjtltlw9ynz5wnbjfaiw78uixbrkefpfbubpdnewdyfefhyo9is9r1olobwdqzquav9mwwbbycnvc96crlcuuhplfr+976tnl7bhah6klmnumdl2rt7msdgl3pbsit9mo3ymdoi45uc9oonz11/l+wboq3letex086brhtcqh4mjax7210vz989hbxv6tmxxsxmmho//sn+iym3dc0bfg4u8feszcft79mfhchwskc4bsnyctonskgliexvxa/x9vjpnsdsm9woidx8jzfbqo8a2hfews2cnlr1qslq8ihwrprqhkflf9hlwci84aodfhoc6hshq6vtlexiybddmf1xdiud3vzij27abfxludstg4blm68afvwvb7fhxdgc/qx4pent+nh/ne1lwwez2ttk4wm8k5vvpdmiqgkglkqucbilljuk/wmlixdy2qrsgtkl3wjs7gukb1yy2t+cgkm/hjtogyc1y/f0ocp16/gmhz9glyrrupadqpzgixiovcczhj/oxhtbgid2/w++e5e6znllq6/ckvbgbifwjoqdu0o4yt/1lfg8wd43b0tnjx0jiukupjdsbmeeeztqsooo0sooijdzxomtasu4vcvbb8xxkqdkqolwdivb6xu6sda+9se0xdccc+jezphjdgazlgc6gomy895g3zuezwxykqylwmownyd34cuae4endobjcz8vt95oklpwm4ej5eekljlziino/pghoocijmpnyyvgcpizmo4hpq/hswambse/deom4afhgpk1vsy65vxypxu8qbx5saduekd4gs1eskmecku31oiehwsesicqgwt6rdevo4xomswulfirajcxdjwfefrvvowndrp6orjmtqaekq6rlerjhwfkhwsvyljbzskbqqk60rtlarcjyvdjwm1vsrwtvfizdg7tfle3xmzi6jeopp60rmzylkd4lsyqf2s8iajzkixbetfzvpd9/zlcfs5xpjkicum64tsc4xc+rjyqrpax8he14cqbrjmhdtqfcgxq/ssd/wn0t/zaqgggtajgds7qthfobrhrqxebgyjsxt57doqfz2pvliiby28a4nuvzq3e5lbv3bqpe943b5v4lujuf1v5ohvyz1b+6jj3sntytpmdde77urddmdb3szfb6/fgnubqcuuhqtuhm0bzachliqycji3pwxvbuhjc4lhjattij1dobnvp1xbanmuvycqbcmbmgnquxmaeewvz3r53hqrvfo5q8qrq9+cusf+5qjnbzdxjs6zkekchm4gc83kbdm4u1yqokr7seorz/88l2jvwe+xqpeswecmyniebudpzu+5sgkwxs1ug3r40nhyybgto3lawpidjezudnzcm+gymqq7ldkz3q4bkuyv3+fsozwbd+mf6wvq0s81aij/al7d7l5n+xdf3juax8n4/vohvvpjc0rbvpoyzxmtzas/xutcoo5hzhs1lnps4r/lhczeclfyatcm2svifzqu4assc9chpan0lhry7dvdoiggij2dskqnxeeclarebgqooqavel65q989enfef71he+r4xier7gyfexng8plu4vs64icmz54bn1td6jamkgqpsiicjebby3vhfygrdgbxz50shg2+rl/oxduqvo6alul0eilstkmctunndueq5g37bhfbd9yld7j/8t42jb25lj4d+klfh4rbwh9tilvdowestu5yqojl29sowlnrdvslyu2gfuyc58xzdje8g5irsjkahk4nvd8ahrwlsegpdcv29etn5vdfdrmqcihgrw6gjnw0ny3b32cdv6n8onsfeoxyw2eukw1hihexhh1ewmwz0ov+hhbph55lukvb+ib7x6hyx1vppfcycx7jto39cbzhxsrxbb7gavz5hs5dhfnxmc74akgq8ytfyjd1k/lupf6kmrd978dhhm2gy1gai5ryfzfndlq5e8mjbou71stczh7h76p4wavfsmz8qz45bzjcbpwliqfoq02jczquroju8ttcobfeh/5akfy5ixtxvdoyk/ijwrc6ewxcr6uvr9ppn+wr9vswhza27eftdcx3zwmx9pavc6b65pdlzdpgu37onjmnx9xxkp1em6gumo136mxte6a15ohjgxmrcto/4dz0z7a2+249um9t9ijkuf8bkouqyfwebgaekci+apzb8c6r/eoks/3ftnndue/mytz/msgommw1jjemju03/4uxqtg376qz9k6mfctggsze2dbsshix63fxtn+fxo7tv/e2/gc8+dxetq+rcbm137rtdszl5q98jabdb3ki2bes6jnki/z05m5gf5jdd6+1aujl/bunnvnuhidjnpw9r1nzaxjsw2diewbbk7iof8qjn6lrxg/fblajphkxauwmsphl6t28gprpgdgguzxsm3mnfapiboim/yls5ed182f1vmqighgvp25ewdhvucitila5tkuekvn4qvkarlcyuf7kk4lcqkaeivza6gctevwnelayfxutw7rjuxzdc3sju/h1iqurgjwupqvxourvurqwk2h+0wf1as329+nfu04nhxxzyzbjazqzsoq1bolqaufpmefxjwe4lsg56r2dmbfnht5lbgoxgchs3iglu7pf2jasmqj9eflj9ghzww+txezaus0he7/eaf8zxhzcetjxrg7/berj9unemv2o5tsclaioc6m3toakrtytx/vqlxhciaksxsx64ceroguamrx26mchumxbmagxfz3ma1nhgbo9twd82lvuhubtl3x7y5qmbv+9cobosp0xfzqzch8i439882bvsl83+pd/upo7l9ufz/x04282/m7j7szg4x83frfxyupvxtun+//uxvp/vis7shuwdvdt4drab6k0/apn+9yf05yp8dfvd3sa==</latexit> Greedy Inference Lower the lights <s> J error here: model must choose a tag and stick with it; can t change anything later V D N Lower the lights 20

21 <latexit sha1_base64="cadaxhopvynlf/+a8wv+1+uyq7y=">aabcnniczvtbdxy5cey6n4djfk/zmdzaezmvljlm0lgtsksc05iixutxgynag5scy3owmyd7jgdnmvgz85jfk9fkjx8ll3nlywt+qqqanungzxhzn5y1dhbvad5ufqqfqqgagrqjl2p39z8/+t7v/o7v/f4ffp8p1//oj//k3p/+4omffihzustsxzwnufhqqcvlembeka4s9luhge0hcftycpem27+8zelypdtw84kdpnsc8rgpqykq/sf3/mktzknr+7vh5nxr14tnafokbd3k2dto05rmwyqsuayrkb3ksypsbkauqj6vf1/xtq1jadhpdg4zmlbvzet+jzwlerxjlm761tziwzrsnylpuh3unym28g/rr6jbngzlhgglqffrun5gcp/61wf5lilgfwyodk/tdhk1ocfm4akpzd9qx7vd+yvzqwozc3lpgrun64vrm+uq9djisvyeml96+aoudjiajc5fmcxqsn2thhz9xvevrffbxdq+uz4/vnitl7ktabtvuyzhm2lqtsukhky9a1dhbppcsgeue8wb1fuuv5/ra1ukwj5kzc7lzfi4ru2oqsya7k1mrkcyomahyklwfhur+yxetxve9cxcjyjdn7jery1m/7hflv/fgvbtirxb+pss9kqc+cfeqouqjisxebge6swcguc7fbjapvs/j64bljecbtmvvnmc1szgsvwzrs7ezsbm37h0tn40kytphghnf4qmrue66/anerf7kyb8w0mqdq/nf9m+cl/taiclej+g4t+fd/05krhrgfr3xidm+j/jz87bfg78gvhnzc4uiegeec3sscxy5gs4hnr8v1i/1xlml/hyr94rlri4uypcn300s9yggjd6w7vgqxskic8idwn+d9txir/ny5frupb6nquvfvwatygpgqriqwdmcfbvjhd77uq65vyxykggonrjpsdxykvqy7xmxuu7aq3ywusmejw6lfe6hzp3hr90w9eula2xmtny378/qqbrcg7+ns3ttiig9fs4kr8ln1ypzwkfgmky+i1tx52b1xpsbc6eagi7jceho5jwnl4nzixgnlgplwapq3cfc6ahhik4ywyshciet1wpg3xdr6xae+dkwrrtqypa4did70a3h3yulnsk5s676q72y18hqrxczhi6rud6uodylbsgzbl6+t24z/gtnuo4ata2bpsjflaamo31deslgzlwpk1ywp8yzaiyqacrrd5gft1scvqab0kxzjyiqdwndlgdca9xk7jhlf4mvd6lxuypphx4nhczwsj+yd2cxh11uv1vnht8imc8iaqtums/wupuvzlqwlm9nh320y6p93oagt2fxniqlos3cgknmbe+v9h9zf3dnvf4j/0ws+7q81f476h//0o2iyx2xkmqulpuntfuq0gv2cxwmde2mpwuhjczpmj/cz0ztj00qfy2uyctvdmsof/jcpomu7psqaslyqgmqxsrcnk0ntj6ua/e0pth9rkvb/htgotijkcr6kyzal0eiyhw8acw6yknhcwbouhj3xltyftedonoxwuco84sfbulbrx4iwex7p7nq0tbshy55dg+exepeqplorrynqltnl1zkyhyaoyg5lnbu5kk7tciqjbbg14yql/ap78rzzusbvfe+qkqg6irn6yea8cqc2gmmypmbqylimo9rqtsf0l5ky1o4kqelq18lymojjd7q44vavybuq1gaeyf5dgmtkq7cjzjzd/ezspwfoa4fplmo5maooouycta+mnhspqjk+hgqq3gagsmhdtmipjrmymyq2bmwrijdrnecusoqczuoorvcwqf3kgs3+iymvdtaaieej6gf1sjvpwjxl8mwkp9ut0qrxwhyjoms4okpm+ut7kstjxuudnuzzpscxo0i8oieccka0ywbokyrsu8ekutplad1t0b2mfahrptrd+znbyjckescgtqarhq5vvhqfrgoowjinzpw6nsa6kmb4deopt3qnjjuip6eajcxduy2iuq45ifd7zhv7s6oxp26ykapyjlqo99+eesjqwudy/tcbnftlxotdh9mhcak+exu9asgq9fzf9ahsc3x3a0ewpeo6enfoov6/9mrpxuhbazn+gvdvafpw8ceuefyfyihirvt2fqmd3nab7d2subrdrc4x73f+orjsvn3veatkoqbc1o0xri3vgvzrixefgyrltsui++fqk59mquly4jbqq1pinrwuq1grluvq04uovfuadh6frcgeq576nr+zno9wd6pl2o+2bbpth7vbzp1ctjfy/xdvxc4quohpifcvgcvskspazjpmdjbqd4kgrkhktmri51n1kiqazxug2y5kydly1jsvbwiudhnd4pta97guomhfdimlgm4mrlwjnnfmqusyenlkdvf3nx2qghyybntqblocncuzdeltwdabdzalnaayagz+8js+9yytg8jsgnqvzm4bug/m5iraoscmruiegojeccnxdzlzr5ppslqrehq0ko3bbcm4l4uiqw8t+i8zc/2kp+3vz5k3izd3oza9ahpv9ih11dgrhw4gsnw3s3ise/pd8gg2h+yrrkymwrbb+jybvluqiliqxluzljrslatuldm0md0q4oof7/z6dtcoa769vqv5qnuqhcbg2ww4jrwp9oh4sng/t7hctf+8pjw4wpj2hcm5c5ljrka4yjmip2+znlaghm2ckkkffbmrhgdxdtkdzoq8a+lgwqha4zbgkxyczrdyssgwifmlgqoxi21gsjbwn+05opfjfghftmusdntiwbsu8tqxq4jbmcze+amlcteb6lf0klnvfadf7smyafoxsoj5uxplnyihiz8cu79saqhbrlxinyo76ti1nj6og8ikz+lexuyxqn5yogi6bi5nu8cn2o2hjus3y7gkw5ilgyhjgcpe4omdvdu9nr5lpsqctyqh/5xjdgficysviak92hanknf0kt+mcfr6zbm+lkw3ysh6stkie7jvl2a4iyyq4kmrdpyl7w/qs+mjurfr6obzzfn5qbkc5yz61rgtbfkwd+mzuyegumvrgr83cnv16nwdb2dwkysaw+zbohz7qhqirhod2awmngkyqrrns2cjekyg6nvxllcod+gjjdr31qfn3nodofaxyvych7hqa3tfg8dottdabrlxu0e+dhpwdft4jfrrwcf/k0na+azh3enrsccgtl0cfu+gzmv7g90vd168w/xvqfkx3svmmpr/fopoxtt7moyac2f3gdswmvf2l7hiliq4rqccodgz2nkmgytympya4ucre5yheldpdu4hjn45t3hbao9a2nfegg2c9hr1qskqsm/wxcqq3cflv9hlaci8oamdlpnckluh6ytt7mwasbfdwxwxmmvt3rxi5y7a+u0ledutwodzwy/ahnwn79fhbdga/qx4de3tehj+u97lasizssk1rgz41ypeeeyqbawnlihcywqllpfjekywxq4wsixynbjxvphzwnfixq0wmn+5baz+vhidbsgrl2jo8merv2myvnpbrltxcwah1wealojb4svnmf5z4qyxxaa2+x3wnbzwc8itxx/miryecqu1yiat2h3gz/6sl8hnafc5oybmnxtsfxko/g6lxea8pkkuzyvrjqerf+gchgntgov4funbb4zwoaghqgvbkiuhrk6rp1r7mwspiyqtnhhzspbicab4xnkij6mopwdn2lljmc/asd5aefc5+tq/bslrn+ydoggmjnxw3d/t1mcsy4lhc0ji2suyxszorlbhocbizmofy2xgml/dgg4g5y9hvmimdsf/dmqmya7haodqktnnyiswtip43elym07chb3gazl6ycayeddpo1ejdgkjzeehhol0cgjfxxhrmkups/gwvg5jhjhf0jrvpgyntjq6fnkmjiaekm7q1shjxamkh8r1slibjanbkukmurslkvdj4vdjwi0vsruttkkze6cjh+s7i5oc6iw6qs89wwowwwbo8inut3mpogcgfbtikc9yt+uofxlhk8c7pfi4zdnvqnafqm5dwswhxaavzkdrqeg3xpmsntrbsozqvsce/1j6e14avumdoggqzlvl2yhuo9yllciaybr2i29cw4wv84exvzsggzcwshslczurgvqbb82ziy+bvcw+lrhu/2gewj3c7zwpmtydmimtzvqxvxrx53yac+42ktp7lrbccwxato1yce/yfkagempckioaexvo8nywnd4nmg4ktqfpjigmvge5ntcmkxoplqtsadhhodingxflpwd0wdx4q6xcoquhi4a+bxmz/uaitqc3cskumsniniryg5lmcoic3ckrfb0k3vj1ypm/l3pasevpuuhsmhjc5ckb0wa8ux2wk81hwdzhnqidd6xgdskdgm2shmyfzdsgouxm5qcl4tjyqzce7bbykgrhfodl7fbs3kwf1wugph/wckd81hyz3r1neh/j3kvnyzje/+5heayxosvgkl0mpn1cja3qbl17raifud3rulazj5ochxxle3jhj6qfrpmlnzblyf1dyagdqq/om2mpyfgyskseea0rm0aeq9y7ihmocnwils0iikpkbboxf33edb7pfed7miu04wsofxvj8/zg4v6x5iogp5yhnljf6x0qigaisociikjewhhrg4efcav+zphllqk+zxnjv7bbdru9mrf6jujemlkkhosadoe25wp2bxvcddt7drux+r8br9vczoxz/1aswmcrqgmtf0fc+z3rgni+nndbpktxlnlbs+ynyczfm8adm/ekzyizw5irsjkahk4lvdmdu1qswctuuh2zflg++3gvunzameqji4dltmrses7147hgq9z+ft2gvh4xjbinqhmv4jkytljj5iygwr0ll+hhbph54lugvb+il76vdsr61nfv2ogzmldxr0t/0azj42lrgnd4ckp0w28ujv2wmbd0c5waczjq7rsdarrbwyxbny4m1vg2gozfalgzcbcpmxuf0auusfsxsw6jvqyg5n99o/ctssmmd4jp4zncxubu7ckowwiktbhpxnwubvfgqphq6cf/rpnvcxlbnxrde0kvitwtmifwpdtkkns99hou8n/ps2gzc74uzsfsn1zaf/5onj4f7nzxrgz9plm3frppgdk7uvsy61ol03mm7w2s+0+ez3wwunffmie6j1x7/jzghv4m12vjzb7hvjyv5cvczkhokhlhf4wjrhonkq2qvh+re5knhpd34wpzwbp5vkxawaxjjdmfaygbjvdb9+ll/rhp9+6iez+pkwexolp6zedelqay+bip06/cp0+/qnn3gb/y2efg4uptfihnyab932fpbl4rfgeo/d+9lqse8k0tov5u125kfk53i6b19r0yet9o82vky5crzm8otsoco6vyyhhqygpaeqnimbeke8+pm6xnvwtaa74onrlsdooirnsb2bl3dkeyaoyfqo9gzeyd8ompu4ox7b6wx/zcsx1uzkcibxr6zurfns1vhchyiwguyrhjl5+fzsmq7xmrhzymujxejabaetjnfpwvtqtzc3shg1lru643mwqwt7dvp4eyelk4czxuyxr3ouqnrpjvbq7klf6tjk7c+ti+ecjlv4osiskzd0pgbdn7glqgmcp8cdv3raw3ujzk3vgmy8bc51lrguxhgas3mllu7wbm1d0ufy6ifigrmlmj97mohfgbzvarlz063mp+nlwrwbc02r5t+6om4+titt72holvgh8ikxekurt5rk0+qou2vb41zkyo1i3ge/w9zz0ithtvmlo9blc4jbuljoyn6ycgvu7rkd4po2cj00bz+uml/w1fxz73uwdibnv/ijmz77exn/44u9nd7utu+f/vr+lz5tfj7z/bu/x/vltqdrvbw/wfvf2qu1o7v3a/g9f7n3r/f+7d6/7//h/n/t//f+/xjo9z5q+vzzmvvn/3//d5wyr2y=</latexit> Greedy Inference Lower the lights <s> J D uses best label for previous position V D N Lower the lights 21

22 <latexit sha1_base64="ew92qxkjqfpqx2j2coucwzwjuzs=">aabc53iczvtbdxw3cqy3tw2trnbjy16werlly8ty5n1nilk5s10ijazorqj8ezucxxrjzkd2tq00z0atzl/iw05e84pya/jvugx0tdcuzyt9chzp2goah6okhukhcscm8pglubf3p5/85a/+8i/++e9++qfrf/bnf/gzoz//9c+/fllzhoxtmmvz8e2ichbzll2vxmbs27xgnbnf7jvrxvns/+asfyjn6bfc5ow0ozouj3lijvq/3zf2/snm4h//cqvhz1ira5tfkyrqcpm4vzkta0qgkribokkle2rwlj5llk6gvxj3vdmzdmgf1k4zadkzxvoh4oygms0gks0pmwwgq8driqpygnq4o6jse2/s2hvtdk4kgsgfymdcwt63ua8o9/f5dnwphzzwboeeqw9128pcxgfxqawt9v+t/4wfzlln2cu6gnf6nr++tlst7/wzlefdkv4vsk8wye6euau9nqxn9c39wh0jhgnxm4kvfbyfogf1ou2yxax4ltdbkzs+rjcvr1/xfnundnx+ubrld3jkyjtnmjsbn0r+qkp4uud2fyseorvkom3dsczett0i0v6vuotyqiftdiyxfdqvcopu0gjjiptvurxw0yqhgqpoqd6f7wfg9ydx4meypaletbd5jjwb8ycmrv8a1b3bajy38fkkeqg144+yicoxdekeimmtxagufy5na3a0erzvhvwvos8jqnzeloyjkbnjcqginz3t3d3v4md1eaxruym7kceubdkalrt7hkq73tgc9ivhftxgba+8f19zxx+emtwo0m4se15mp/s0uulkz8htq+0z0n6vpw/78f/0iu8ldjcpciidar2huimouankfyoprcv1b/rzkmeyqcfjaxybeug6fkdpqolwwn4yf1h0ptu/fj4b/q+uzikhs37k9wg7awejsupvvqnguyy4mu4rjiyqi2yib1do+7oexalcjgeqjwykrjew9j4pu6s0zvtksluc1iaxucwcygg/uk9so7/oguj48ioduntbnxf1v/rbus5nu6cjenomhb0nrqi3xw3zvfxwk+r+qwmzqn97vjqirecb3iablxag8uhgjc0jclwphqekc8sqdlnqkutgaa0oqhhiiylys4wgulhkthvzju+7y0opvg+nshicszfgdvlu2ogclh07o/1re2wngf1s7ny4tk+wopq0hcfhka9uvq6ybjv2cnvsvgxbjyi9jledqow/wvlyyu2qwculvhpr2h0ggjxgunyfoaqwqqh0eg9jf8xwcgg1js5snqmg95oyyshodl/eislmj6fsejhzevgo8xhdn0r9dvr9rtb4/ippwkqnvnxtl6spf1fnvqxfxs8o8ogq/pu5zqvsyqjpwcnaosijizswj9+po7e7t76g9xpwbnx9215s/r8wmilcwtlkol8clgl5enfewxpiwzplilydknl+iencbnshmmtiuvendke2oims4k+c+vrnv2e1q0ebggayljfhybvvratko5/vttmp68lod/nknxgrozecziscql0ek8ga8afhxkjegugnvjypxxdty5jshxtznjkbvmgzc552btpkd5lidzszypy8khztnrw+ws4l6zpwb1oag6zsrnwgbzwwoqmuuztuwmhvubzfdka0znugqj/8b+uu1wfpbbhrtkbojoyzremsgniwgjiybhiyznjkwqo6nubez3khwj2p4eou3molgoxnxijxyn3kgqmomknyenebajtdwycowonmqwp9pkgtx0kcv0rsuctauhhficta+mnp2mqbsuhhiu32sgyskendbstrniwwixxcacgxgplsnuqytpwrpxnii3to7ht2n+tymscldwgakkud+ov8hfzko0gy+wp64ekey8ikwxieqkc6pqpk+8l7my1luwyhoevmpymsre1akruacimqnzmlabmjqw19ues+pug+ow4xgsmwpv9zd2k8moqgagtqenabtjrb+frgoowji1zpw6ea51ku/wg8ztnwxo9qgkmozqfjcmrzumwjnkhnwdkj2dl49ennqnyykvt6ryup/m1bggfgwk33+90tl+2xh+adh7mexuu5dxoykgqgfpr0aviupbaziynjnu1fnda/xqhsn+peepwuvxi7b6tzocgw+xged6igwobgrv3pcw6a0bwn7dgku7vop8/n7334kcjau7a4dumeeuf3wbhsijpmnqjstui1fx2alynn7y2spmqzcqnlik2qqwmekt4rdwcmkqvpqckus2ionfflgwz8qtt/ow7rnv342we7fbju624+8gm3uc0v5s9so9fztdqke+guddm1byvalapgdm6sim9hjbxb6xeiqmn2fvlg8bmutadmxixfly1hsvdx4kqrkh8c7tpswlhoqyhsesdgctit4xtzmemqsylrkeho7vpz0goc0xdhbudkyeo4s16jgmo8gzdzovsqhk5rz84lshdyytjwapjookz2obuvcw5cobpmcpruiogojeccp7uiy400l36o8qcwoassfggmgcq0xrolex/vcnc8okpx7uz6kzize3ozbuay0q3upsnxbeoamrbpxtlb6rg4guedxfqpmyilmdlzttfmf1qymif5kxxlmzljpqlx2qfknimr4icnwct3s9pwjhad9oq9f8ogzvqgcwrdccffqp9oh4snf3snfj2s9fhr4clu37lutsoulfiyakwlrozlsqzi0oznmvjg8h+btwqrbro+lezmkrtnswyaeguaxhtiva2yzcuoglqtziomlgcjorlbirzx5yopetfghfvmucdhtiwlsowllwrbcdy5ij8rmwxbid1pf0kno+ptbi9ykyafoxsoj4ux3lnyhix0+w3v+2ai9gy14ioq476li1jj6ohcikx+lexgyhqn5wsdetase6qemtzpbdeqlbvvysysmgqmsxcmuvmeng1c5gd8asbwnpvmxxxzcb4uw4zeefrkaxg/yexyou8sdjarues5hhpxa2dekvkqdhd/fdbqy5aawbprpca19s/gy9m0rvny1gnz+vms99zra4n9grwuhoy4xzg58+g/ailryt+hedn12ponvg7qwuzylaspm2isuekyao7usnjoopy4kmsuxzdnjyb6uhet2wyxbkqthio/ad9wmyd9wgdtsg6d3hbs7v2t7xejbq0/wqw+3vdpcpnuyc3ramut2dvfzfswlym4z1p68jhypydgwupi6yjxc2uichz9/i+mrurir2etmy9x+3qz3fuaggala/eywbaycpaxieq2xdtfajyimeny2stphxi/9gri5709zn09qhagpwisj8+tvgcaty3so2fbekfqvtqhpa851guijyp3yfi+utxdmht0omfimhxs7gh7shn7mwlc7nbwxsxncuc1+9jzc3bx3qletn0lchhjbdiw7vt9zr8ag6c7al+rnr0p87+rvswghcobxmfkglet4qvjdl6q18i8yb4jw2k5s9yxmj+2x8hwynvixrpgzmjti3vzy2tucvem/lh3hxrb/uvrn/jj/txoh/cvyf51hxt2ujudjbqthrc0wfjpipsmebuy5vfbdfjkzze3dp2lldokxa1uz0batd2ml91zx4hebze0x41ta5rsh84xcr5rlimi0zivqmklmilpefaejqzrui35cu/myyew1iqi4kfq84yhvdpvba3ysyhhboeer0s0yiwafgjmvfhzmve86bshot4lkai9l4subm+uv8bqsk+utbqrxakmjn1d0hivqepazg4ziqurajtpbkdf6by5aqjdpzw1jqmsxunozrslljwpsmoshkjkhqmszugccvpojdr7wc46sc150un+nedeoax6vrczbbe3h1fhwvyjeqqbyuopkiiyd2tywrjtouuhjvreumxohnc0qzioep0vzhv0+o1thuudj1vrolysuuzjg4sknsoyr5uyvdj2qjjgkryr8qaasxirz8ikdq2t5pzcfqzkgfqxuwnvcxnq0bi20ax9km6mezaxwxtypfwmps1u26615gmmpxklqkwtbte7irfhlrymiiaqndittdpula4zpzuenjhis27nm88j8id8/zogpoqcca0uzibaefaoe7shlhmyzwbss1ayt3dt5y4oognhiodgjvad1ojbr92klznojhc1sjbyvg1t/rb1yndwfnoyz7p4xlyznvree4q3m7gav7myw6ey0w7asmmrftwlazbamgpahpikrgwlw9y3hrcxpfedxuje0hyr0czag65dpam6bopwrwkbyiwqavwqcrqvo6rpegjbdky5wziiwx1c2yfekvl9pucbowxsxtrzwt39lgqpiciad7ynjjr3e3vj1y/m+ljhao9vtjdynpdw6yhe2obx7f/jqrywbz3oi2qip3hy0d+dsxmsyzjn/laesllpt9bp1he2+nxdi222j9kh25hs6vwznxl+65ngcl3fzdg3kp5pvt7mjkhyv2zte8hsarxzqs0hqtuwym0sxgsy+nmqgkrxovlbcguztasp/5wefw43yvgefugxsba2bldfqsstdqpo5a0ao6tfoqyuyn0hibrepepjflsvwosayqpddczyvwicpy2+ycvz7rot9nrvm9ziqk4wsoxxuj/fzg6v6xzisgzoyhnlfx6x0qikapsociinxewhgrg4elcavuzpfmlqk+db7kxtj1xuvpf7l6crkwvjqfq0lnceeo4wr2thvcs9shzrtz8j/rjra5nzlk7q1yjo9xzbnal1pcuwmjqwxftqhg0tyrs3jb0+ynyczfm8anzjm3ciz4bdknssmbcyfm2ncvj7qmluggtil0+6zz4/16uf/l3epaj4nfr2negmlq7rvdqfb7hd6fvxi8nrgvsleg6t4subjk6cmilvkylttzsea99fyswtrg19w9dsx+ynrmws8aolmwp0h9nhfg+mnjhduwbyh5wkqjj/q+mrb4b5qv+cbdxzxqkpxjml4mptry4m1vg2go9valgzb9fdjnybs0vmkjy+k8vitvxxzw7qd+lqlync/wddwzbpjdcbz7jfnhquin9tmbmofe+nrsorzif9rpnvcxlbnzwdi0kvirwpm8kysgmernl7qptn5pupdtjmb+zy3y4prrma9u5uniyf/nzdwjzqbnz2hxfdajk72vcy6uom03menx2k+3uez3zqunff0ie6j1y7/jzchv4mx2ujrb7hvjkcshxi7kmkkhlgf4wrraon4q2quh6sfeknfvd38tjjwbp5vk+twhxjjdmfayhgjvdb9+rl7r+p9+qiez6pkwkxqwsqzbdelqeq+bit3a/yp05+qnn3gb/z2efg4up1flhnyab9x2bpbl8sfratph/tqq2gecqjlks+z05l5cf5jdt6+1aurz9bunninxjkmnt9aei43s1q/pyddk8ycgqbiof8qjnqkrxgffelajphlnbdgzldcxvlt5cslfaekatovibeyv9g0cvynr9auyl/3xl55xg4emambdt12lsqkroorbegfzmfybjf3xvatuxa9kcl2uvxk5hiaieq04u89a1oqp8wxli2hbnlthzyyefvykpvcrhbdwbjldsfevc6ssvkkxw9d0oswqui/2sxzxjnnjc19w6gxymk4mlhqnwykuwplztpjl4wgus3buasdg+m1zpk4z66qyjgbpbsps3d6rtugstqr1eflh9gdzwwctxuzaus0hu59vn/9plwuucjxrujx/1tvx83eimhphc2ra8illm6g2tpqkqtytjjq1bjzmigysvvh0we9xdqy0znjt1miovnmaybbexhqwr3wfawr33a7wwvu57pu3x1bnl2vqqvn3khg6w+zf/nhmwp6jjpvx9ypdwd7u4f9/ffd3xzq/n/np2l+v/c3avbxb2t+t/w7t5drr2tu18m6v77y7e96jnvan//7kp578p4b+5jomz1+tgx+e/nf/arm68o8=</latexit> Greedy Inference Lower the lights <s> J D N </s> V D N Lower the lights for final word in input, include transition to end-of-sentence label

23 Greedy Inference Lower the lights <s> J D N </s> x positions, O( L ) time for each position (iterate through labels) time for greedy: O( x L ) time for Viterbi: O( x L 2 ) faster than Viterbi, but doesn t work as well can we improve it? 23

24 we can convert this greedy algorithm to beam search beam search maintains multiple hypotheses at each position two types of steps: extend hypotheses prune set of hypotheses size of pruned set = size of beam 24

25 Beam Search (beam size b = 2) Lower the lights <s> starting hypothesis score of hypothesis low score high score

26 <latexit sha1_base64="scqy9asdvie3tp/hwf7nml9lhus=">aabhl3iczvxzdxw3dqyn24tzxthju14wewmri5ldpjktrljyhtbmxdtnsjs8smgeddw6g2rtkqdy3spvfkxek5f8mpy85oq1/yl3atvvhaw5wdphix2lbec7cy4uli4wepbfxmjd3f/+6ee/9/t/8id/9om/xv2tp/2zp/+ln3z8l29ewuqhex2muzp/m6ccrtxhrywxefsmyxmnbxh7endxbnu/vms54glyloczo43pooejhlijvf2p7/zvok2irpcpd8kayfadhe3dni5pmiwdmwadqgxikidpbj8h6axcg+jsbxttsfzvlqkxqahqvcdztepzfir+0yexd+zngrf+2uzvkzwmsnk/nac8udjdgpuhvuwclfy7mvxlyjbgqzehlhfwllsqnjtlx//6ij2y/n49qmd8soabwzanl7qa/fwdug3akdl5xiowzrky08xjpkv65aiepr2sqs3v9dx1hvmuxcsi2fvyvst8vr1/woszbjgdjxglvccfqj9opgbtqa/ueegxstswszhcabj/52b9enefejx0qeap5vc2afjmi97ubalczscturkkavlea5yhwep6mkgynepxbowrd9fvq+ebw1mzw6hsk5baudjocnoqxhgtpclh4bmuvg9x1z9z9oa4jsxyy6ahpn7uvub+shg8rrr05hxwvrpotkmvjvf8hlkdnr3duc2fmsuddochxxotxau410hvvay56voe6q6sf5r85mdjk3/vg+xlcbufdmoj/llnwyuqdz5ylqi9tnxfbz6cnssimnbvpg8y234bg/bzxh78+4gip3i8vtf3geri8io8bkr/u9wtnuz5oivaknoajcn2bhn9axptvydzv50dqycexnuidnc9gxoyf9siixnnpxiisako0lggzy1aaxoqu55xaijkiflsii2d93tbrpog2a1tkbawi49rzfbhj00no9p9yqqhs/0hufehhfotvq/sy1e/73muvfl7vaowrxy/h5kewik5ljrhqarxtlijdrc2ybawmu12bw/vkh558itepofxebsjggem5iwjyei2yc3ymvcpg0vjrjhowm4io+eevjgwe86/cner5/oy7yw0dgcw/nf1veqgf6gvun1a/lcefpjb9ory4ctnqevz3xod7x3+7nvwdxqkdzuh0e6or1tblaew2mdio/t6qforiri/rmn+bjpr5ws1fqnrjs1rlqqaync2amclsrn4b3ahv93pdl85wvmqvyaz0ylzoql89w2sa6isusxqnyz4hv0ca3fdn1kpgojhct6vs8zvf11wvwp9j18khm5iauj5aifm9tix5k1+/pevkiqlbzwjsl91eqntdylfe0cte1oiopocnevvms3jhtnorpehmfgm2tzw6/son5fqszlpikhc3fniascdmhyx8achq3vqc4rohcvkbdwjwueknmkygutsk8ltkxr1rbeesb0wojfobqeoimp4b3fy5u98tzyd3aa8uvxrsyj/tuyh/khaostpdq9c/xwjfp/4bvvw2ae6njj0hx5gltdjdwxrjq6pakrlwfqy6gyiikgou0hmudhqpgwrzdxujecjhijxz8eqk8zn3e9okeicrypzotjire7atxmkjc8ng8o5v+6ibqxjxv8iisw5smqlpjp9ti6yqebrm2uf4c/t0mo/55vxl6illv7yq3v9undv8nd3d3dtuf4n706o+7k/wfo/7hdz4khmlyxcyrsuot3m4mt0tyl3gyswo1katlahhbx+wephmam3faqgo8iqxdzzcm0hz+syrrtv2kksycjyqgsy/cbsnkx9tjiud/fwrdnxus4p8wncoiilocp4fkyhowqjshdxrmhhql4yscd0mwi1vdteyjknmsgl0pcysxppcbgrxjngyths7m2riijidtnlkbppeq98o0mathmfatemnwmhgfllpjlm7okmikrnrhfe9qodeesxbzd+xnw6thgn8y3cftuhrcjvsswx4snmw0xdr8wosusuqrmuaowxuvwt6o7egqk9ise8vgxmd2b3hajao8lwhye3icb7pceymbgh9f4xtgzxlvwv0fweknvmzbvbdtic/b9wbok/gasuhqiqfrtuaoye3ycs3kjm3njozboywrqjdqoec2sbkmjuootvckyf3occ2ejzddwvsjzcja/gbeiecrk5muvlj2uhxe6vnillyckgloqliepvg24gjslcgajxlsqshfrbc3toiaajcngljtpazu0lgskndzxhubfmgud2holls7v7bbgwzroqiodghah0ozlkehqaok0getwbvqnntfzgp8hv5uj71tftqihqecraxdxo2ing45ixd7zhv7i6mxp/bpeulflbkc7r86dzshoqpn91+udy5f1p4dwsletrd13edxo4ymuu+fxy0qlmtx10hkstyuymehburlf4760rgjakx7l8lyjy0zdjhebkloy1pguba8eltap9dsahslcxt9ujnz2dudfw1ynirv9hxwrzzrnlftnkso5rlwkyvxhrnml52kxemhba10mngzyypfqq1qmzncyg5przagq8uvoejuktb45vcfkyc8donva8r7y8lomxbjtjxztp8mfusysr3ipmo1f4khovcfwkba0ldyvyaveeqy0efe7smcucwq8jyjp8/k+z4ewk4d2byhq5basqa4fofhojprft95ymmiz6tm26hk0j11yhhhpof49ybike9jcri//+sazdtdnngxm7gsrcxwpymad4zor5m0j0ngk2cqb7/rc2fqkifmhufv4sltzxdjtt4t2of4ocjiauoecgnkmpi5d4g0wxjsmvowsdkgeaz9bczfjd5a0dcnc/2sp+5vz4kzije34zdcbx5lcj+2zdhrhw4gen/xjoix3laa4nfsds0jyex1tmy28tnxrayixmswcxxouuhavs4ztyhjz/bagqsmve315kdtb3w7rubzgfjva4hjsl5w9ne++ipiydrd3tos13725ehb0ck1b8nohpne1arcbksx3xflxlqfrwzmhwqtjjvkxgistopufsbmscshcxyanrc0nlqqknuqswq5rmhadfq4n+qmijkzvp3twac34xyb31zrfhds4ni7ymnpmg8hb8c0e/mnliofadqvosiszttqytpkdactglh04qno5wreuodplt9ftgcpyioeczkko+zsny49qixccsfjxl2miubecdhiwgyodvmnjpjthwvkt01ekmdjhkdg0asyl7jjyov2b6fhygcj263skj9natozcmlcc8hal9bsayoux5spxwrqpqyrhxbcwtakvaqcld3cdycw7eujspaxffrrkoo3gnnuvt1zaftzedn87mugxlnoxhvgz1suzf749nmaf6msamgpa7gzettza29hcheway7nt1qu9ojuto0gojeftbwxzfsqygbarj5ybdwrtl62ug7curut78xpu7wtbrgjnksvos5wydb2vtaduyuuh12zvttjphinoiyt3eytivb5/1iwsl+prxdohe0y5lynupjoygy8s9b9dvdsdaveoskzq6i9xtrg/es2qloyel1tebbhrr5jy0pmzvoyltygeicigbn0jp07myy9uo7hxsh8fcnfuy+ccqdzkifx9nms5wzwtod960xyl0jnvyev9ch7ek4s48pdtnwzx+5hzd020gaxsxnputix0sredjiwcc7ki4urbqtmnzh7as6vm4bitgwczm88awvrhdpv/rowbrot8nvkwffw/3d1lawe42stk5wm8lzxfo64gxfkdtivcomtlbdczes4mr+7xmkaso1kn7tozmjtj/t8izo508xt8eol89alxj4vfz0/xj4b/fdle3kpuy49k6jaa8qunwovaiz5n5u3jse3mn3vnrvklz0jbtj6c1u1bsqn1jkotgi7g1+4o96azz3gc7tpenbqoa5ifel3q+uc4ginuifajcpzidtzn2no1m47id8wfolgcdcaskm6yfnusmdqqnysrl9ocbtcooej0c+jiyaaghssk/uyqtxzvqed6wtpbrhjrwsgmtvf5zdglvcne6dwaqmyn5v39xt3mutyzsmfi+rsmxnjyeismvsgiulmcolgqfswv1kfrmhl9mg+pa81i7cpuvdhrvoa+2oq2pxgyzbpynjvibkjeddbb/hqei/yge/e1zt2nilfqgbbmigwiiygsgs5i2pmglvrr+fye+fa8wvlwsxdtghro6shr9to4hjpoqraspiki1ksrjks9bhj3tri16lag0narvknstprjiwtm+crmvz3gvgxujpsr1cdssfupwfrwgilwfe6hfpp6gahtlniqyylceus7rixeazxne0qhv02xazkvymu1txfunigukp2sd1k2umac1bdi4setu0zafc3fn86aqamaganpsy3qblwq723ij8dhg6zfi2rmvutyfohlfdro4bwarqrruj2knfmmhetlkyx1uiw8lbuu/6qdwj3d79amme6wmcmmxvuxnuktzo4afe5nmo2stfuwldjq17viwd9ggqhgq0kbijybxe8rw1hyspid43ehqamedak2pouvaq5+mgd1kmpmmmfkolvineufsoe6xhhw0soomwa0tndqtmh3iry/avhitfvkl00uce9/zykkenkaee8gysqdrn1c9culpi5r2jvvuyxcp6u1u0pinqqce3fyuk05hwtzinqid9x2nhfijge2ifprvftsgesvopohmmvusyc322xps2oxijlffbm3kv7rudbcxev4ug5r/p1cnc04f1gvhm0j4fx6vcemm2nkskcv+li8njhxpgajda518pake2onrlmfawj+fhw7maz90a6xzntram9hew5fi06eiyarpcwixj5ubrmafezmxmkqwyji7kbcs+nuneipkok1jy7569pu8h3qrt8d9m87sscq9dcqr89wirxlnkqikkrgafutxqhc6jp49jhgggfewjc6kzgackcu1gkayuab71fci/sll1ft+azegkssequoung9qnnthmkdk1/3lxbe2a7c/i/1og2vp0zp+6twiapv2sdwi+2hhd7xgaxftmhkknbrizmou/qnyrrf3uh13dpfd9ngg9hz0nstebcihmui8wjrklpmkhsdl3ajk2j3xlp1wr3mpdpypfrlje60ny9dhhvap3dp7bx9sdm1qh0c57ui5gls44xyshfftg5eh4wqieec1yl33jd3enqpffemwt+wspjht0mnm4ibjd9sdbg1uwosh4zueujvm+mbn4bptk+ynb5jtpkfb/if4p3tj5489tmmco91giizb+fzfjytg1vctjyoovutnu5h7x6srn0wimqn+hpfgzuc7sropjr5sockhrrczzfbtt41fifhngvqf95wseyzmqjydcpym8jj7m0l5hiejvc6j7sej/h3rezgnk1n2lza65j3rk7h0cd+zpvpk501m05c7vund4w2eu64ekz0w4d4wsv/xsl63txvndy1y8yivu34juojiogzmihzqh230go0wly5byimhpgguckpxms6qesxxcg/qceqnevd34rxhwbp+vk2swdwtjc0feyggjtdb9+nq91/u8/1znm9uyy5qolu6zeberqaq/rip3k/yp0++qnn3gb/z2efk4vhlfrhnlqb9x2cqyluzmftifb4f1kmln7gqirstl6dgyjpnpc3bevtemqzur3npymuq4cevnkrtns2l36ir0mpjweedtl18lcfdqzdyxrzbgpdsdhozqhp4ms7ixval7ali8hducnyqaw8xjpg0av4nr7bbux/zcvnpvrgwaa/yokpdy1kg1flsxcggmbwbakkomp76wljr1swpvaxsnkehii2ghfqxrwomz/hgcra0hbsqyin7iqwtixmirfztixutjthbr5nsnlquplstlnllqski3ffhzdpov03mixfxqavktjmru+xcurshd8ow745emezksibmrfypptc6pogasyhw9gam7b1n3aruxipbzn6igywuwc5pcozngxhkc1gow82kr/01ewqhee17isf1blcf1xiph6r3nqwli8zvkhlrtqpk48ly86tqy8tpmuvd2fd8bpmjodgjg8dmphb6psqdajzy86mje6wgc1a24h+lstxpwn28/k+jdrqrl+erumg2h9e39ppmf/ioz78wzvp7e70/uxv737q0/rx5/58cpfr/znyszkb+xvvn618vnk0crrlfdo+zv/duff7/zh8/98/l/p/+f5/2rojz6qae6sgh+e/9//a4ese08=</latexit> Extend Hypotheses Lower the lights <s> N V only one hypothesis here to extend consider all possible ways of extending it D J scores of extended hypotheses: low score high score score of previous hypothesis

27 <latexit sha1_base64="unweat8sv7xv4jiwvqwthg7kux4=">aabhmhiczvxzdxw3dqyn24tzxte5eckljiitckry2frmkkhwztdanjimgymsfxbzg65gd4ostquuu1ulyp/ja/ksx5m85eq1vyl3atvvhaw5wdphix2lbec7cy4uli4wepbfxmjd3f/+6ee/9/t/8id/9om/xv2tp/2zp/+ln3z8l29ewuqhex2muzp/m6ccrtxhrywxefsmyxmnbxh7endxbnu/vms54glyloczo43pooejhlijvf2p7/zvok2irpcpd8kayfadhe3dni5pmiwdmwadqgxikidpbj8h6axcg+jsbxttsfzvlqkxqahqvcdztepzfir+0yexd+zngrf+2uzvkzwmsnk/nac8udjdgpuhvuwclfy7mvxlyjbgqzehlhfwllsqnjtlx//6ij2y/n49qmd8soabwzanl7qa/fwduo3ndeclgvk4fggam+smi5n5vvxlrqlse1lvm6vrq+s98imjsrcxt+r9l/usev+wvjtyigcb8rboqd6bttf8dladbt0j0dfi9yxii7kbjp7ezfrxokmkeoiajc7n720c5m1gvn3buowcjydym0jspighla+c1fvgqmu5h67zylcfoqtwwze4pma2ukumibwipneu9fs8iizjry7bnvx9h66uf7lozxflsrcxtun5vn2rgpxg43ilnojnk+b7nzxsk15lovg4zds7o4zxwz41lwcddg+69ca6vngvg96rthpu9t3shst/nnt5nw/t4hy5ctdgvphkf9s5qcc1ueflaqexqd+l7apavtlhfgskt53jc4/tyg7bofj3a0v/8hitru8dyzhgfxnysp+mqic9zfn0soak5zqzr+wmjvoc9kz6d7ouojugtgiitxed4ii3jjavtkvgmppl8rbjrr2lsqyag4ec5lsnpk/uujrbpkqrgzvv94jcnugwg6zsbc1hhtfiomjomhp2tpveva9n+w+gog44piz7h6thon7e5yiq5pd7vs2olh4+jd21unjyni9rcejocrmdbm3aicwym+za3qpr/plgcyl5wumidkcwzkzjcxpbkgzbwra17x82lsajmj2wnbfgwwmsymannh/l7lbp5zhfs2jtagd94+p6yq3+uctwu4h4yt358lfpyexcic+dyue65zrfo/zd9+afkbqedkkhnvbfqyjzdmcxgcvh6fvt9vcucx+jhv3cztbpf2osutcbw6jacdsu72wagetlc/fp4o/q6vq8l5fmk181s7hwejaqj1vfnugytxqvlywlb7doaqujjw/7sutvih1j4frv6xnn4uuqy6xosevhuc3iaslolbh93qys/ciu39p15bgbyoe0dhburya9r7fa7mktbmp1vfjck4mt8lm5mdtsfj8h8j4xmieb3x6vevp7fjjy5bu6mos9qvgspkowpzqukg6sqhgnaowtgia0aiglhsxwqlxthxktpwnfqbygljai1ub4zkehxbl+ao8uxkjuibpdurxxiiu8djee96mup5q5fcrsiahpx661btl/daur2ganrsceka7i09ocbqy8zkhuirwmqgpthq2vrugwymkiiwcmvy2ebbukkbwrmmvolpjybhjn+rztewlciov+dfpifl3zcejhrixgo3lhn/3rddwneargevm3nhlqdzn+tphsft3bmmxxvtj8u01g/foujpqvs6resw6uthvt/k/u7u7sqj/e/ejvh3dx6j9h/y8ffbqm07ciwskvxie93uyelrbe8dbi1wpqcjbr8iko2ql8jjrm4rruj3gvwyeairmlofyxskjquxqljqvosebih4xdhpw+tpncjv7+fiy/kytepy1overepgspa8mq52cfaa4fnmw56erccqxvkiwxq4ayjezu8iq1ullifi9mitc1asc5zsy8njm1crfjdts8szzmlyhvlwlivo9i4fsk0qxlmd4sp5jznnfvxehytcmphqfcrwmxlobv2m+2smmbv2w4q6ag6irm6cwd/srs2giiafiaysljisi0zr2d616yffdzgyansvknisgij+3e4oabvxkq0bam8arpdix2rjyep4rgkyzpjn6zuj6whe6pmiorykddnoprwdam4wgvwtvddqkbjfcrm7atwshjms8zzbdygcnql6fywg2tzbllhhyd4bwt+9etmj1pyiadt0biuid5wbxczinwjil8sexx+yjub8ysuqrvepxqzt184m3bxaqqgweb86rug4tziw6deaefyboxckdjgaihy1lv7rk46jyogikfwpwpd3d+ybcyzkjyefaz0iaohzjdjkjrfrsgz1qzatu8+l7ixvgn/aloeqf6deqnqzmicoa9ghfthhjc7vbi9vyxry9o7epzhipbcjncf3xqkenzbprvv1zrhl+sptu0hl2bioq7z69gdrminj67gluoia+vkcisefyvzw4n1jcvhpwjmubqsvdfhovlgmccyognhncxp8bxlxj1tobor9ka9hbmkpqhmueztzv/gursvn7toayklkm8r9ptibzlm6xwsui6mswvoxwl0w/bwuk08tlcbotvw9uym7nfhnjaiq2gil9ahdxsbpdlrwpsdnnset/xlpexk9fi7jjta7jtpxks1jfke5fjh2svekncot6bqbwkr7cqdcvz1dghg4jauwrxwvtgnrn+npx3fqiwxaeybuoglifltxh5zsnbndcsvvoqh0wev822uzwgo+uq8y54wvhytzbrnsbkch//yqhjaizpsgnmccvywaxorzqedj1ej2keh8bwziaoftfnzqalq5gkg6rek9ua78acvcewz/fdkzedhjwr0shljhpuegmc5aqyp2crzawhgprztioavbwgb04w+iv/3kvoemdalm/dibkmrkfmybsamohqzwvq8zxwo8ojtao9kcmkeqmeps2wzjc65btuw8ycznkpx10ygqxgzqecee4yeev014m+rjqdsp+hzajeyd5asgapnlvedoo330r8sttbu9tswu/ezlw4ojhwvfkp09jomofyqnli6645aiaquk2mwlyvpinsm9ecrtb90rtmytlg8wlabsbmk4asfqtigxfhkjkluyqhcu1e1bmmesotuzqg/hlqk+rdyo7tje8xarh9m0h0iojmkm5k9yfascqf9custlakdfpskzafo1sojeu53l1yihzp8sv79sar7frjwxchr2zkvqhhtqc4uvjse9ukxdglzismw0gbzrpk5mmnspc8hum7wkwzingyyjswuvccfbyu3eto/fdhk2w6ulf84epjphhiuxkifernkdfcm+kb+pcdr8tcm+liszyeh6stkoe7jvtmdyi0aq4gmotnylfw8wsqmturpqqobzpvnc1wyjc529kozotlyyvfdpswevulnrgh/xcdp0qlmh3o7g4iwchjtvqvzseamz2kf0ygqyoc7iqfsrzycnflakqldtvwylhisjnlrfmz+meccmykdtif5zxodcro19rqejfl0pu2attpn8jbpwdfu4jvpcrpl/psxgf1ol7ta62jhibrug5ngq2xhnofvs4nnrsj1zumzwrxukayz6121qzzh1uqygimjapcawh8yd9wusuq3xdheaj+hmonc2wxqh3i+9gvn5uopzhwuvtuduquls8ytpgebtd/vwmbbekjqvtirpq/b1nfjglbts+tk+3moye2ygwwks5tkksgolib0ymgetnjuxc1fcvs0+cfyenf83azfo2qsc33gc1qi7tn/6j2andcfgt5wz4uh+7+oocwjhyszxobngba/43hehl8jrzf7keidbylnl1neyp3ajkzvg28k+d53mxnpo9vksj3oni6fjx0vnore8fcr6on+8fdb64csn5fjzhxtwuluhicluff7qgpm/k28aq25gut87n8gro0fcspuxtmonibuojr1p0xy3vnbhvqgfe8dndp/wrkfzxjh60u9wywxeerqlqrqyvbiqaey/09cpnxcsvy0oxi0rbjrgu3wrs6irgnvv+vhzf53gnorwwhoi30ctaqea91hw9jfvueestjvxcz4lmjkofgxkqr/ob8aqr072tq0erje+k+/uke5jlrcchwtgynlmmsat0vll9kfckszkesfs92f+sz5cw8rtw3xjh2pgbh+k7snc9qh21vsw642xyz6u8apdchmirrco8nx0xraxnoirr+1obiufalzgec1ewayr1njg1mwwaqnew+umzohnc8qysozp0dzgvw+otthemdj1vlwr5bvgslhczstpmzk8qzguu7u2krsn5e+vtncygvs2wsm17e804mn1z6spodqttqhlz9abxraao3qk9dpuaq6yzswgweot1ixdcs8jtom4m1qku2y6tmmuuiitq4uhpa10leyqeps00zxnoizgcalt22d44d8w7prnqvxqgeeatnmw6tyq1xa38rlhcyzwb8s1cytzbt5w8ooanhaodcjvad1oxbr90klzzmjd+qgs30omyn/sd6xo7vbqr032shkvxmkqis5xv+d2aav2mpl31los2bcyngvbswdvsa2a8cckgzaaaep2logtlvh8tvc4kdqewjoemlj1yrwgpk0xesg0jxkwsxwqsu4jgqrwnc4j62ivfe2wglpqqfsw+8rfx7sp5cys8kumizgmvrpbrak5qqn2kgwsdqjurnrkxj8xke0c66ms71lsm9ykmrttdzy6+slxnmk0umvtuafvoxo78bmxmkqp9n8raedzvpp9bp0nx16itdi+22j9mh25bjcq4jm4s/dcz4ms7l7dg3kp5uvl7mjc+41452ged+pv7z002xptugsu0sxgsy+nmqfkrhovlbugmxw9zzn7wefwo6zzgwfugxsoc6ajbnok2xmognuggafdis0xmfjzazkaomzi8kht2uzhzamvnhuhokv4vbwsbxbox592g+9tn/gepnnciqwhrrmefnuwck9ysluquymct6lr9q4xuhpcmeet4meljvjcachav3sujsfghfer/kxtgtu4qezdp1eirgishyhozqe85tjxtsmv64a7f3zhbn5h+sa219ozno3vuxdb+vydq1xmwj7/amdwl7ckilk7zdwcxzntrvsnyu6g6u4z75fs4eh8kek9izgirejlxgikddk9jgunkcurrjwke/jdkvfe5h4b4qmsb3shuxvtmkruoodpya/sj82suekwpn1hihexhgpekissonp1pcxqdz0xrba+8ya7x6fir7xnfvy0hpmlexhoneumyppjrcgt2r2upgai1kd930wjvankc3yqofmadcj6fpevbu9tffdmt80wr3qpxutipgqzsbcdgqrkplf0fqmzqnmoa/vjm+ixelu/w9p5zkgwdinw5nfmhwulndyxbiomqpgppqo4sf9qv/kej2xjieesovornxmez2kumcukarjufat1fsk9bzmbs2tuxobxxme8c3c+jgb2dd5mneis23lmdt07vwgy13xbhtknhqbgcv776rbx6a55obgvx2rctm7fdxwzfq2c0q6beey+krymbetlerex9caygehl51jutyw74ed9xwlqo1/u/ckmkwj/1smzwqaccyshr2qb0abppvxsxuv6n36qj5nqmtdlfrz2tb0k1kedhtyvo5x/vej21u8/8tb+ezz5xf8mr9yjpdxfqu0vteuijguydrfd+8kwz/ceusovppxpzezm57i7b19r0yhn1o9tlblyhtj05slac6sxe/vdohhmetwgamqvk4x6qgfqctezmr7sgi9haq5+bixcs6rvviatxw7qge7fqc3mjdiggv7e9fyldi/7l188k9cczqd/txrluwtvgr4qs7gfezadyrvq0h/fs0tneqwa67w8kskljesw0yps9axfzf4iz1bwgrzltre+zsg0sc9hcl/kcgklskclafm6r8pslzzic5pctfhvwor9llp4yum4hs8q9dr5scdjnnyjsykuiphz3pdlxz2clxdc1irb9nvmvj0yvmu+hsdu3ikpu7vbmzb4om7vq2sf2qxmlx3m8gimgwob2xmwvf+noyyeigiucvn/rmrj+unempwo5tsazxmapuitadvjxxlahnvqdxiy5il4ez7vgj80dqy0ynjt1miovnmaybkex3qwl3wfawrx3a7wavu56hu3n5x1b9zuzf3zkhggw/on/tjmz/4vgffjzd5ob3en9y9/e/dxn9a/pvpjlb9e+zuvjzxeyt+t/grl85wjldcr4z3qzr/d+fc7//h8p5//1/p/ef6/gvqjj2qaoyvgn+f/9/9riht+</latexit> Extend Hypotheses Lower the lights <s> low score N V D J scores of extended hypotheses: high score only one hypothesis here to extend consider all possible ways of extending it because score of starting hypothesis is fixed to 1

28 Prune Hypotheses (b = 2) Lower the lights <s> N V D J keep top b hypotheses low score high score

29 <latexit sha1_base64="wkokru6zbz3tjasu9w/vkgbdg=">aabdkxiczvtbdxw3cqy3m2tdza7l7gneeldklk1ilkfo3iqrxzslsikpmpeo39jkllohmwoybwlqnbm1ov8zl/ktqqj6phux5su+ismdww3gq1whucgucpiosjiqu7v/9cfp/uqnf/03f/uzv1v/+d//4pcf3vnoh74seclj+ibok5x/exfbe5brn5ljhh5tcerskkffrxf72p71jewc5dmxxbt0ncwtji1ztcrudt/6clzjsqqy/plr8olly4dosvokdd3m6czo05rkoyquerhvvzgsoc0l+dfk59ugfocb2xvrrl3xjlzkom5qoarzkwx1qfgxb0y9sn9p3gy4jbja1mnb8cqicz8ar2g1cfmmhihjuh3udrbu4d9iwivrnozeamglqfesur6ncf/llwf5jpkwzwykbb4r6ll6qe/eidupj7lf+hp5i/nnjbcde08e913rhlkx8sssimuym+rzrpqzq+djixdc31z+8nqgoufs3m/3tw1jf3ahukwb08wrh8bohwhdnhpj8f0jmzkk0sela9x4g+qt9vd+r67bgkf/rtkjaiv9qkypjswxrbqbguj2aysak1ugctsszeutofte1pdrcvwykg6bjv0mbyfxtvthmsql2ophhjfismzbmhj2ppsnu7xhnfgxraxdv5khp0iozf1iyjbvxr4m6ozdspz3utaie9alcz4njlgiwjhijjlcwofhxbz2t2thq2bx5cf+klkmpwxqzmbamjyxkoip2drz2dlada9xmsylm5tstgnk4mlqo55w0vw7wwofxxwvpo0bgpap6+s5r/chwodbtcspa8mh/5uwxc2n+dysfaz7hqjpw///fvwjucldjitciidinrrddliezlsomoux6t/ooyhfziooq2ssffisgy4haaqjcmdeghvzdkbvpxseaf0prmyih7q4uvv/uvljoav9n6km9wrw50fzmvtspywohu0b1vh7jrvsifs6bag5go5b0lia/rljg92y4trdujbexxyreydu4r1o/t+oe6pngcglnzwjsb95f1+7rbar6nk3ftikmiezjnbzhpqnvz+yvb54h8ebjn0/vdcvusioqojfrgfdqbi4ciofmcjyh/zahq3lia8hobslsadwnfuekmqc2eufoli4zlr7841pu2nkkvrvjuosgbm/0a3l2audos96z1v69vbbxicqtdzscdiuwppq4fivtibnqxq62bjp+envstgbnjyi+dlsvtrh2ur7yksdqufzsqcjptdzvggndmsqybb0xvg3wpmqcpzcsgmxzeki7lgnz8ynoxbwemvtypzkujro93nhicechyengrtx90xfvkpxx4oni0thzqdznxtp7szt3fmnrxptv8ux2mh+ddtmgrzlvzilspj+a8k9oi8vxhnbu7o7vqt+b+djqpu2vnn6phr598ei7yuexpjpxej4pdqp5wsf+wokfwui4fluh8qsb0bd4zkljxwqkuqr1sqs0ogocc/stkogq7psqsclyqgmqxcrsnk31tj6uc/+kupr8ojfg/zwhcjohma8w3bcpgqqvjaj5izbnigsrtatylkrfrbpucjesyldeguuh5m84hnjvqj5wuuxbpzdq0tcsdy55zg+exepfkpdorxynqltnp1lkyh8qjpbzlnbuxflbtaiy5ghh1wcrn2tv6261grqg/zbwtzedqatallxtok3bgizgg4rgvm0oz1dfudwqme0l5vnutikepwsfkamwm9mhxwo0qnkturak8wzsr0nzir2bhyssh+zmmd62hrxbtgrelmbucdmw4mb5mbtajibsuhdjkbjjdjtdhj6su05wvkofbhoyixmuijjitkqkg2yihebymci/zj8w/zrdcwvotjcha/abeircjrbicvmjxphocnkkoml2ckbkuqkqzpvg2zgjav2fejyyr1orivihhj0raaygmfmxpwovq6mhrv7s0rbsnqsomjwdnvls7v7dbkuzrhbjubjqko5hm+1hiqoycjfllvq+bkbwlyolfmj76zhcquzdqgqooksmoahxo5qqnwd1bsl39xdhzuzsxlxfxsyqhe69phweipyd53quntvpl49mhxezdffhfvbganakievrsalspol65hipn0kldpts0uc+fo+ine/sgtr6/iktxjc0y+babck6p50bxi3z9torbb9ga+hbwknqhuueztzv/gxi6ru4ohfjlurdg6/yyiifydkuvklipjphlp2kz/bc1lc8ylyfssei1vs0xk1rkikwv0ico6iveyc1fbr/8omdpeuvdzg90zwf93ug5cbtt627b/m6wwacq7svmp9x7stmefootcnq9y4vdzvsbicamramxlwjiiqtebdz+nlupfajqxaeybungninttvh5zknbntckvvn0j0vo69wxuzvgojsq8y5zxjc7l4ixz9pgcg9v/yaosifhsknmmcxywaxbjysnrs6os5ynmzcvc/cd3w2zm2nzcjzcvfeyrg3o3lse7wm45/ihsmgbq5wiywrflwvzouko/v0lj6crbaiugma5whqnemvzf5vzgfbsyaa+y5wjubwnhewb0kiyb6mtxo44jipgq29n8vhdfntb4/kcmscqmepo2wxjc6zbtug8yiiz5c666fbv9qlapklneidavqve7rv/0i4dvp1wo/la2aqbwgbzbzj63ghtefhbxt3bro9pp3t5ehc0no1bkrpnjbonghda0kv33jixa0ejnxshrqspptwlwa7tphuziuxw0sgcbyddlymnlqtknussqcwr0xydfc5lnclickrrs4cw6o2krcc31zqkhz041i54pmv5cgjwddmxfskiwhaa+p4ezdbxb1rsppwcobudi44/1bfcgxjp+mmy+8sw4bfszliq47ijllxj6inykkxwlo71zr4d5dwdi6zf4fg3dxyc2x5yqns7iksylnkqidhwoewlnjhott3ygdduqcjxq7l4l6jaccae0vgcitcldxucekux+woaqk3hpggjujgbhisxhigwh3vuaoazaaibprpca19s/gy9mzqqoxunaj5fnz/7mifwbqjxhdhrlguznz6dg/ailryt+hedn12pyj2wdgaxuqgwblalyrhhqupudqjtk8humufkppjsbmzsg9vqvw7lzqvlibyxufvo+jtzo24qb2pd9j7japdqbe9rpri16xrirfzqo8jhthfdt4xhqj7okv7vjqhnm4z1p68jhyxydgwulblrg+9sdj8fphttqwcuss2qac8xjvh/dxtulqbz1xqewfi7x8sykiwzv0wksyhekqlgfi1jx84mss+u+bfxed/v7xhu60gemzgvkdjnpmk4bbxtyd86c9yluuvvcsv9ykfee31umuuw9dflhslmowmnftocs7oh7stn7evehd3hrlpymuk2avaxsxfg4rofij36fudixguwyd3p+61/atzadwf+wzs7hp7/rvayghcmbhqfkqhetooxjjl4qv4j8yj7jgyjzs5zx8j82b4jw4fti3vhgpmjty3sry+rucvfm/dj3nxobfcvrd/gj/txox/evyb71xxs2uhvgsalefb4tlkm/6y4aqkxgwl+71wnr/scmepxx9iirudmqpumpexbw/jcnfuv+nwdprfhhlmgtroi94xfrzrlimm0yyvomapkiflcn9pqoz13eb8tcfinmr404fb1wwmy4odvdfveax8zwgniwakwbfobzidaaeazy4o+zir2nddh52aaeqea5omlaznrr/mbkol1ycntk4brhm+quw8v9qnnkgfxkhbstolmmdm6r80xsaisqexhlpuyfjcbwzwk3desesbqehlhipuyfmomck4mgl6vvaljpilvns436ct00qg+vqmxdiizcfvqfpvgkrbafhsimqilipa1jbelm5s6em9f5sbggfkfpvnfw9roq6orj0fraooo6bpejzlkfuqysfyljolrkrypkq4ttvgstjxkd5w01rgrwzhfljq2d5kwiboz0imozqit6tkznwcxbeakn0h9lheaedwk6ahleua7xxfcywhtoc4hlcapm/qj2rukuxvlmys0ga6qna4eie1wzunukaqhsa37ahp+e+huer5udq3obecabzlvo1dtcfdwijyg0dqnuhztfa7oiyeuaecrkzbivkv1o5fq9cslzbiri5tujb6vikp/0w+stu4omkdn9k6zlmzmqopouqtzo4afe5vknb0wc/yfbinadizyj2wdidwiasckrsc8pav4awsaxwsybgyadsjkas2iynjtoe0t9b4y50ebxcydyqxsrjivjtflceotsnllreawgoowzm7464s2fdzejb+kuohz4ssnzorjcxkwp6yqjeq6seqr43+e56st1lml36wkn7jjuzohhnhy8yxxmsoyumvtuafvs40d+dtrkiu5jn/lkck85wbnoivrxnupei7ttlifzeduh0vl8ezcpzvx86cee9bwonzuflpdhu3zcmxesc1jml791gf1rde5jwaqxqymfwymcvbsnxutogxz3dfs+szhssgpi9ujvhat0hzxzmob6cvkr6fk3iggb3sztfte2e+ljqkihohji89l6x2rdfr4botkfuervdc22cm/pu0636eu8z3medrxbyeuuor+e7b0r1iyrrazwwikv1dhuq00elqwqrpilgunwnwbkebxezypiwad/6vore2pvdru8xhxojetjmljwiosbdue24gl3thnft9ohz7mt+j9rrnrczk9y9fsvw8ypsquvlkfduwdggti8nvdbp65wmjloseuoycdemcaza84fwwez87ajgqkbdjxx1dmdu0rg0btu+j2zflg+/vwv5a5h4bpbouo0rwxhnxda4dqrfy5fap75xhsyiurbkhtfsryccnqr4yykbkyum/dqvxqc0lqartfmtcdiv2v9czdxzfw4mkebvursodpj40vbsmeogqpfy086vtmquadum7xqyaoa1ss9f0yxgzf2/lgzw8byy71voubkj0kmdo4xrqq5isxzj6olapjdmv3uz8gvayaz3g6nhk3zg4etvys+bagpmb6ne1zqg98aqkcdpwx1e+e8lfsmgajpghuwe8clhy5lxhibmq61h2k9x7cvw8zafnrbsqw11zhvhpo4e9nxexgv0nm0+c7vunt4w2fu6yekp03ydkwyv/xsla3txvndy0y8yivq3/dvojpigzmzhqxnsvpec5sxey7xmqhpggyknjqss6qesxxcofvamev1h5/dxwgfwzzn4ni+amc4wjdvyqrrqug8/v691/u8/1znm9uyycowfu9ogbnfixokmyqf2vwrdvvrpj97gf48nn5vl6duyibfmw7w9hmw5/bm3wopwxjbi9gmrqjnk8iy7cy8lczzdt6+1yygu6ncbpvouhyc+pfl7jjror35ib4mhjwcetfw6xiipeqaucj0v48fgbdloodgzlpasqxbzeo58hmqbmcpibeyv9g1dvynr4xecxvzepx9wbysaap7gvze6fmvfv0ujtybc5zctakr643tjqbteyeb6ka8kcgkbery0klw9a1grp8hcykbytmyojk9pdc30juzhlwuurbzoddfzvc6rslklxiwn2uwlvave7ofjxvngji18wagxyssymddrk9wsoqton+gbxz4z4loe56z2dkrfnucqy1hxfblb0tycpbu1w5uqddth6igywuwc5g8oznqxaue1hox8stx8p70sucjwrmnv/ftxx83hiadqhc2past4xurcbwn1svn5wh11ag14npmcrj0voud9vz0btsheo7wwa1u2ibie84so5pwumedtntsbpm0r133z9tuq+wvnxtx/xgvdz9j8iz+agdg/kxe/vnq4m9jdgfzhp9/97m5zm/w/vhtd+s3vsbrp1x7bo1f2tha2/w4g//+85p7/zizi/3f7x/5/3p9j/x0j980pt51zrxz/+l/wgeos1c</latexit> Prune Hypotheses (b = 2) Lower the lights <s> V J = set containing the top b hypotheses ending at position 1, along with their scores note: this is a set; the items do not have to be sorted low score high score

30 <latexit sha1_base64="wkokru6zbz3tjasu9w/vkgbdg=">aabdkxiczvtbdxw3cqy3m2tdza7l7gneeldklk1ilkfo3iqrxzslsikpmpeo39jkllohmwoybwlqnbm1ov8zl/ktqqj6phux5su+ismdww3gq1whucgucpiosjiqu7v/9cfp/uqnf/03f/uzv1v/+d//4pcf3vnoh74seclj+ibok5x/exfbe5brn5ljhh5tcerskkffrxf72p71jewc5dmxxbt0ncwtji1ztcrudt/6clzjsqqy/plr8olly4dosvokdd3m6czo05rkoyquerhvvzgsoc0l+dfk59ugfocb2xvrrl3xjlzkom5qoarzkwx1qfgxb0y9sn9p3gy4jbja1mnb8cqicz8ar2g1cfmmhihjuh3udrbu4d9iwivrnozeamglqfesur6ncf/llwf5jpkwzwykbb4r6ll6qe/eidupj7lf+hp5i/nnjbcde08e913rhlkx8sssimuym+rzrpqzq+djixdc31z+8nqgoufs3m/3tw1jf3ahukwb08wrh8bohwhdnhpj8f0jmzkk0sela9x4g+qt9vd+r67bgkf/rtkjaiv9qkypjswxrbqbguj2aysak1ugctsszeutofte1pdrcvwykg6bjv0mbyfxtvthmsql2ophhjfismzbmhj2ppsnu7xhnfgxraxdv5khp0iozf1iyjbvxr4m6ozdspz3utaie9alcz4njlgiwjhijjlcwofhxbz2t2thq2bx5cf+klkmpwxqzmbamjyxkoip2drz2dlada9xmsylm5tstgnk4mlqo55w0vw7wwofxxwvpo0bgpap6+s5r/chwodbtcspa8mh/5uwxc2n+dysfaz7hqjpw///fvwjucldjitciidinrrddliezlsomoux6t/ooyhfziooq2ssffisgy4haaqjcmdeghvzdkbvpxseaf0prmyih7q4uvv/uvljoav9n6km9wrw50fzmvtspywohu0b1vh7jrvsifs6bag5go5b0lia/rljg92y4trdujbexxyreydu4r1o/t+oe6pngcglnzwjsb95f1+7rbar6nk3ftikmiezjnbzhpqnvz+yvb54h8ebjn0/vdcvusioqojfrgfdqbi4ciofmcjyh/zahq3lia8hobslsadwnfuekmqc2eufoli4zlr7841pu2nkkvrvjuosgbm/0a3l2audos96z1v69vbbxicqtdzscdiuwppq4fivtibnqxq62bjp+envstgbnjyi+dlsvtrh2ur7yksdqufzsqcjptdzvggndmsqybb0xvg3wpmqcpzcsgmxzeki7lgnz8ynoxbwemvtypzkujro93nhicechyengrtx90xfvkpxx4oni0thzqdznxtp7szt3fmnrxptv8ux2mh+ddtmgrzlvzilspj+a8k9oi8vxhnbu7o7vqt+b+djqpu2vnn6phr598ei7yuexpjpxej4pdqp5wsf+wokfwui4fluh8qsb0bd4zkljxwqkuqr1sqs0ogocc/stkogq7psqsclyqgmqxcrsnk31tj6uc/+kupr8ojfg/zwhcjohma8w3bcpgqqvjaj5izbnigsrtatylkrfrbpucjesyldeguuh5m84hnjvqj5wuuxbpzdq0tcsdy55zg+exepfkpdorxynqltnp1lkyh8qjpbzlnbuxflbtaiy5ghh1wcrn2tv6261grqg/zbwtzedqatallxtok3bgizgg4rgvm0oz1dfudwqme0l5vnutikepwsfkamwm9mhxwo0qnkturak8wzsr0nzir2bhyssh+zmmd62hrxbtgrelmbucdmw4mb5mbtajibsuhdjkbjjdjtdhj6su05wvkofbhoyixmuijjitkqkg2yihebymci/zj8w/zrdcwvotjcha/abeircjrbicvmjxphocnkkoml2ckbkuqkqzpvg2zgjav2fejyyr1orivihhj0raaygmfmxpwovq6mhrv7s0rbsnqsomjwdnvls7v7dbkuzrhbjubjqko5hm+1hiqoycjfllvq+bkbwlyolfmj76zhcquzdqgqooksmoahxo5qqnwd1bsl39xdhzuzsxlxfxsyqhe69phweipyd53quntvpl49mhxezdffhfvbganakievrsalspol65hipn0kldpts0uc+fo+ine/sgtr6/iktxjc0y+babck6p50bxi3z9torbb9ga+hbwknqhuueztzv/gxi6ru4ohfjlurdg6/yyiifydkuvklipjphlp2kz/bc1lc8ylyfssei1vs0xk1rkikwv0ico6iveyc1fbr/8omdpeuvdzg90zwf93ug5cbtt627b/m6wwacq7svmp9x7stmefootcnq9y4vdzvsbicamramxlwjiiqtebdz+nlupfajqxaeybungninttvh5zknbntckvvn0j0vo69wxuzvgojsq8y5zxjc7l4ixz9pgcg9v/yaosifhsknmmcxywaxbjysnrs6os5ynmzcvc/cd3w2zm2nzcjzcvfeyrg3o3lse7wm45/ihsmgbq5wiywrflwvzouko/v0lj6crbaiugma5whqnemvzf5vzgfbsyaa+y5wjubwnhewb0kiyb6mtxo44jipgq29n8vhdfntb4/kcmscqmepo2wxjc6zbtug8yiiz5c666fbv9qlapklneidavqve7rv/0i4dvp1wo/la2aqbwgbzbzj63ghtefhbxt3bro9pp3t5ehc0no1bkrpnjbonghda0kv33jixa0ejnxshrqspptwlwa7tphuziuxw0sgcbyddlymnlqtknussqcwr0xydfc5lnclickrrs4cw6o2krcc31zqkhz041i54pmv5cgjwddmxfskiwhaa+p4ezdbxb1rsppwcobudi44/1bfcgxjp+mmy+8sw4bfszliq47ijllxj6inykkxwlo71zr4d5dwdi6zf4fg3dxyc2x5yqns7iksylnkqidhwoewlnjhott3ygdduqcjxq7l4l6jaccae0vgcitcldxucekux+woaqk3hpggjujgbhisxhigwh3vuaoazaaibprpca19s/gy9mzqqoxunaj5fnz/7mifwbqjxhdhrlguznz6dg/ailryt+hedn12pyj2wdgaxuqgwblalyrhhqupudqjtk8humufkppjsbmzsg9vqvw7lzqvlibyxufvo+jtzo24qb2pd9j7japdqbe9rpri16xrirfzqo8jhthfdt4xhqj7okv7vjqhnm4z1p68jhyxydgwulblrg+9sdj8fphttqwcuss2qac8xjvh/dxtulqbz1xqewfi7x8sykiwzv0wksyhekqlgfi1jx84mss+u+bfxed/v7xhu60gemzgvkdjnpmk4bbxtyd86c9yluuvvcsv9ykfee31umuuw9dflhslmowmnftocs7oh7stn7evehd3hrlpymuk2avaxsxfg4rofij36fudixguwyd3p+61/atzadwf+wzs7hp7/rvayghcmbhqfkqhetooxjjl4qv4j8yj7jgyjzs5zx8j82b4jw4fti3vhgpmjty3sry+rucvfm/dj3nxobfcvrd/gj/txox/evyb71xxs2uhvgsalefb4tlkm/6y4aqkxgwl+71wnr/scmepxx9iirudmqpumpexbw/jcnfuv+nwdprfhhlmgtroi94xfrzrlimm0yyvomapkiflcn9pqoz13eb8tcfinmr404fb1wwmy4odvdfveax8zwgniwakwbfobzidaaeazy4o+zir2nddh52aaeqea5omlaznrr/mbkol1ycntk4brhm+quw8v9qnnkgfxkhbstolmmdm6r80xsaisqexhlpuyfjcbwzwk3desesbqehlhipuyfmomck4mgl6vvaljpilvns436ct00qg+vqmxdiizcfvqfpvgkrbafhsimqilipa1jbelm5s6em9f5sbggfkfpvnfw9roq6orj0fraooo6bpejzlkfuqysfyljolrkrypkq4ttvgstjxkd5w01rgrwzhfljq2d5kwiboz0imozqit6tkznwcxbeakn0h9lheaedwk6ahleua7xxfcywhtoc4hlcapm/qj2rukuxvlmys0ga6qna4eie1wzunukaqhsa37ahp+e+huer5udq3obecabzlvo1dtcfdwijyg0dqnuhztfa7oiyeuaecrkzbivkv1o5fq9cslzbiri5tujb6vikp/0w+stu4omkdn9k6zlmzmqopouqtzo4afe5vknb0wc/yfbinadizyj2wdidwiasckrsc8pav4awsaxwsybgyadsjkas2iynjtoe0t9b4y50ebxcydyqxsrjivjtflceotsnllreawgoowzm7464s2fdzejb+kuohz4ssnzorjcxkwp6yqjeq6seqr43+e56st1lml36wkn7jjuzohhnhy8yxxmsoyumvtuafvs40d+dtrkiu5jn/lkck85wbnoivrxnupei7ttlifzeduh0vl8ezcpzvx86cee9bwonzuflpdhu3zcmxesc1jml791gf1rde5jwaqxqymfwymcvbsnxutogxz3dfs+szhssgpi9ujvhat0hzxzmob6cvkr6fk3iggb3sztfte2e+ljqkihohji89l6x2rdfr4botkfuervdc22cm/pu0636eu8z3medrxbyeuuor+e7b0r1iyrrazwwikv1dhuq00elqwqrpilgunwnwbkebxezypiwad/6vore2pvdru8xhxojetjmljwiosbdue24gl3thnft9ohz7mt+j9rrnrczk9y9fsvw8ypsquvlkfduwdggti8nvdbp65wmjloseuoycdemcaza84fwwez87ajgqkbdjxx1dmdu0rg0btu+j2zflg+/vwv5a5h4bpbouo0rwxhnxda4dqrfy5fap75xhsyiurbkhtfsryccnqr4yykbkyum/dqvxqc0lqartfmtcdiv2v9czdxzfw4mkebvursodpj40vbsmeogqpfy086vtmquadum7xqyaoa1ss9f0yxgzf2/lgzw8byy71voubkj0kmdo4xrqq5isxzj6olapjdmv3uz8gvayaz3g6nhk3zg4etvys+bagpmb6ne1zqg98aqkcdpwx1e+e8lfsmgajpghuwe8clhy5lxhibmq61h2k9x7cvw8zafnrbsqw11zhvhpo4e9nxexgv0nm0+c7vunt4w2fu6yekp03ydkwyv/xsla3txvndy0y8yivq3/dvojpigzmzhqxnsvpec5sxey7xmqhpggyknjqss6qesxxcofvamev1h5/dxwgfwzzn4ni+amc4wjdvyqrrqug8/v691/u8/1znm9uyycowfu9ogbnfixokmyqf2vwrdvvrpj97gf48nn5vl6duyibfmw7w9hmw5/bm3wopwxjbi9gmrqjnk8iy7cy8lczzdt6+1yygu6ncbpvouhyc+pfl7jjror35ib4mhjwcetfw6xiipeqaucj0v48fgbdloodgzlpasqxbzeo58hmqbmcpibeyv9g1dvynr4xecxvzepx9wbysaap7gvze6fmvfv0ujtybc5zctakr643tjqbteyeb6ka8kcgkbery0klw9a1grp8hcykbytmyojk9pdc30juzhlwuurbzoddfzvc6rslklxiwn2uwlvave7ofjxvngji18wagxyssymddrk9wsoqton+gbxz4z4loe56z2dkrfnucqy1hxfblb0tycpbu1w5uqddth6igywuwc5g8oznqxaue1hox8stx8p70sucjwrmnv/ftxx83hiadqhc2past4xurcbwn1svn5wh11ag14npmcrj0voud9vz0btsheo7wwa1u2ibie84so5pwumedtntsbpm0r133z9tuq+wvnxtx/xgvdz9j8iz+agdg/kxe/vnq4m9jdgfzhp9/97m5zm/w/vhtd+s3vsbrp1x7bo1f2tha2/w4g//+85p7/zizi/3f7x/5/3p9j/x0j980pt51zrxz/+l/wgeos1c</latexit> Prune Hypotheses (b = 2) Lower the lights <s> V J = set containing the top b hypotheses ending at position 1, along with their scores note: this is a set; the items do not have to be sorted so, this step only takes O(N) time if there are N hypotheses to sort; cf. unordered partial sorting low score high score

31 <latexit sha1_base64="7t1squwmqspokyo7k32a0jtjl9y=">aabh0hiczvx5cxy5dec6l8nc3nj/s6ukjkijtiy0h4qdrfqlzjuuyvmks0uu9mcty0wpzgzkx2qgyrm1oqn8mw+u75jvk/eanu7gmtxwca2lwreb/n6bh4ehh4m7zciu5m7o/37yoz/64z/50z/78z+v/svf/tvf/81v3btyit8pc9cdmozb8zuseinra3ksuifzpljmbdih09ph+c7v9fsfzwndms84ydxhss8depqysqwac//z91mkrz/18ekigqrqyjuwztokbjqaxkmg2rmoipnkyz/byms3iniro1rbxhwlvvjlykoqprgwqzkwx5ufhxb8y8sf9o3poxg/1n8pgoyynyhvbesq9pvo5xfql6+hd1pruuwtcnrmiobcjgkwlvbwiev/vdfnrj8nv3ab7zezlvbj0bl2kb+lkgon6b68ayjevxkci0z8hnf6fzrbquixiy96kqnlfvxq2cnlkuashq3udymalypszzr06h8lbqra+u98nnjczbxn6rd12rs+rdw9qqqy/sb8q90jdcbaptfajjc2a/r8dwxlz8xmzya0j8pz8n4kv3sfdwmn7e2wti2414q99thzxppnizsnkkiicsh16vb1mqyzl0zva++hhdtro+5zm1s4uqnqmpfswnpqcn4huxsspybk7r6vtwdf3uojdhlsnbxjyn5dfwvwly/y2jhmnqm1fad2s42sl9lktxcci2t7cn47z8ai4pohwedolndk3ibge9w5nmqov7pdti/nn8/z+lw/m7hy3fmslxsdc86t/vffyo8ucjvynq68t31eypxpelbbima543dl6/j0h7ymmx/v1i0r89xqvre0bykhhfhjbsv6nqsu/zpl0kqjjzmkwidgotfqf6w32awvedhkenbmtbia5xqz4qmbdbiimzzav4ilgijtjybs2nqafzqloev2ooyibs0oinnq/6qa6aininuil6y5fhntkosjomhh3t7prq4wz/avtuacfuzpej9fjuzwccrzx8fr+qxzhfz4ekr1zkdovmespbhc1uosotdrierwatxdtbnyov95+qmcc8lmj3bopmllwweqxjd9ac3nxw0fgaj8lllowmmbpoyrudo+h8k3d6fz/hfc+htqoa9y+q6yu3+aotwo0g4of15ipfpyexcyc+cyqf657pfo/gd9+df6bqenajhxzcfaukzjmaxqywh6xxz9rfucsdjroocpvzif+osuhdb2yjaihqul6zaqaulc3fp4g/q6vr84fcmq981szfwuvhujxwa3mkyzbrvvywlb7cogusdje+hciuv4l0lifrvafnnyuvqy6zoswuh0c1iwekolzedpqbsvqju35x15nhbikd09pzul+c9lzgarkntbip1vfjcu2mtsqn5czss1f8hsj7xgiebnb7vurm71vicjfx6ggudgvhsziowp7qukc4sqlfnqoutgia0qqhlbgxwwlxtrxmtzsofaxagljaifyb4tohhhbn+ag8s3chuifoduvwxiuu8nrfenynuv5q5lcqsiwepn+51rpj/zesrmodhbsde0s6ik9qc7ix8okfuodumkoqxopoqcxcgnfoq1g8ykhqjgzbvyzkjybl7mscl0+a12za2z6riiry7kenhutrm50ghkzucd6ed3tth91q3zinch6rdwutavu36wcbkv3rs55hi+vf4d8tmh6cdswhr5hv9u5yc7xdaa9+cmdne0f9ie5hv/64s1l/orx8+uctyjsgrcwsqtq+7u9k8qse9ykheatwg0kwjibndmko4tohmrmnptphrmg61iziom3hv0qsvdulkgksceicevso7das9ludf3l8zycw/fkhif5pqemiijilefxjrjwhk0rz+kbhzkfxek4pejdkuvg1xgq0opinqdgverd4yqq5qve7ywk25ehmri2lshly5pm1yxobea9me7n6hapfipfmlypxytmvzokmrilgwqodxeirl/sacmli/p79vecadvgtw21ycypoyzremnhpwoanhpigd5m8zcxrhka5y3ddc5ypk3sq5dq260qxhpoj3vscckmqtyua1gqe44mz0n7irubh0ssf8zngu1bxh5bqsyrm4cry6zdn4howtmlkskvw9zdd6cyjvoqm7jgwcprmc0bzxg5jboolva65bzisy8mioxo8znivekkzf0tghoo3rshqgpnbveloi1ymkxyx7hh5inqh2iy5afusnfblpxzixchftcqdizvwpfsdi1khbp0qaqvggjfwp2kzqkfjwvxusljqniicsz6covpubp/sbmwyreugodkgar2nzlochaiqkecftwbvqnkwf55n8bv6ux33t/tpibqgchgvdhs1jto45jjc6zotrs8pn5/yx/sjfbfkcrd3+srrhuymnn97tdy5fll7dmajez9f1hcvrkangkkeprsavsijb66ryjj4upxpdgzuy+eo+teei2il+y/c8hwnmw4xxazc6mtt4lgwvh5xqkfxbab7b3mu/vcz89m77f8icjyu7/qdvkwwuz5x7tzetuqpvislcr0z5redisxph22t9dlxmsigi1vb1tizucuc0lohdyakx0cf7ck2+ovxbslhphaj72vk+8vjadfxyby02zafdbbrglk9yktdtrejiafqn8cgwtjxwfvglxnqmnbhro0pgrgskvaadd9py/s+bfiua9myisowwlkmuhzn4acaaxbfecjdis+rztuostcddch4xzzhedkoydhpy3kwt/dkn2q0wztymto4eqwsvqcjgg9htq+tni9dyctneae/g3bn0jirtivhveljbc13y04+enjn+khiyafdnghp5dksoxeinmfyupltsegygram/wwmry3uleibk4vbyr/3q9pegzcl23bi7gopmrkf22bsqeohq7zva0bxck9qdfb4nofmmwsmols22/ic61ztes8yy7kkz110ocqxmvreswd4imfve96merlf9go+nvajev/5qohazfkvoppoh/0r8wtttn9tiws/e3mwf7hw7vuys8ckebwcsmhsrxfcenhzgii280kyfpjncy8esdtb9wot86tlgwulajtbgk5bcjrtyawfxcjklqyqnct/u5dmgatody3qu0mlgg+rnyo7nng8xethzzqpiafhnbpzjyykbqhul6eokmu00glt5awarrpydokpzuvqxejnt5bfx7qaj2jjngs5djvmujwopaifwgrh415fpcfaxnpyylomjnrtjyay7qcfzldnxpjgyyzaxtgkshe442dlvn+7z2ihcdut0pi/zwltmxdkwnpiqm/x7agkff+uj8ceaj6mer8vwlkwjl2ghjq92hmnmoxfi0jwmrra65kknxjz1fbdwwhu81ntfozrhss5zl4vrmdblsxe+ptzgbeprgjbj2q4gxru2qnvr3bxfggozxsqf3teaqz2ej2aaqaocziqvwqzygmfrilqvvsvwykh4ain1nfmpynecypyurui1xwxudbre1/rwahf18ouwavtjb+jhhzdfm6jlhcr/h8pc9jf1ki7ti52dhlbbih5ngi23lnovth/9qzsz7cumvxrnmiao/51g9rztl2ukqiisfapieuhc2d9muhuq7xdbmapopponu2wxij3y69gfrau4sxhqyxtozwqops8yxmgenvdvnumrbek5qutsvqqaz0nlnhlllu+jc/3moyeg2iwmka5ncnswgliz4dm2asn5fncfbdus0+cpqhn101ajfo2aec3noc16a7tt/4jwapdcfht5ax4up+7osocwngy6rvobnjbk1447uafez3mi1zizassd9k6tubhlngybmw72qvxyuys7wqvljizo108ht9aog+94ovt0df5o+wz0q9fpigxmuvis4kqpubmcapwnmay/1l50wrya9p93rtbxtk54oatv7rva0dcqc3psiu2u/glo+on+mwdprp7hgcnneok1jd+t1ouii7qkbwixaishugz/5mgtu28k/hdqsfujhgjazuq85xfjqssrsrhyvrrblchhboeep1+mwgialjhsrkps5v7zuq0c53guqaj6xjbqkb66+wgrplqepfesmau49pyzq7ipmejy3m4yiscbsytpbmdvwyfjctjtc4rlhuf5jfrwzws3d7ml/shzut2qeo+zfufyf9nbbveebnmsrmvoiq3iej66wbfte8fm+cjuhp0j0awwahgcwbrqotfegktz0tndkm26jvcbuicel6glktkmbztbxt14ggbtr0jxudvg0leysslxvvgkh4jyzsa6tpvaynj20j+vel7jzgxzvm8utp+tim+uxdg+giqm+meuvrshvj0abyll1b/mtraidos2ahlack6pnvuzyrphgetsmgxtzcp2vukpxv1mzs0ga6shvkpkna65hzu0aghow37da/8j8jd8zyochowcia2wzldqqxa7jzeijyo0fqnuhzuza7nh05euyogzofbpcqt24lyo19zaj6mefgfvejbiwh5g/3a6vhov37uzk+uuwespqrohhd1bgewyc+tewetxyj9gugzth0l9g7baagpqhoiqrewbk8z3tqcxecejxtjtscsq6blqk651tcnkuypmeyka2awqyxwausqfi7tjwedrzkicvyjsw11c2af+oulnpxchev+wxqrx8r3npgoiccowr6ytnjh1m1vd5348zylnwm9vfjdsnqtmzrme+qbrzc/5yptmba3ua3q4h1hy/t+ikatkix+ewunad5ks8+gs9hes5qjx3dbre+zq5fgqgu8e3fhnut5umlda3hq7tf8vdwdtvmgee8czenhvpq9h2zby0qkwfw6gdz2stemqil17rwyfmsto6cscx/rch6cntvwzd2qznhonnfatyf7dkxjdgqjocuk5slgfi4te0duteweasrb6ihsomjzi1s0ci+qgrxnzvnr027wfeog34m0jzux4ma1l9bvdxbhfuu2culsimcx1lv6hwuo6epsyyiihxmi3opgyahcgrtyjgmlgg+9x3iv7jzdru/nmxojerbefdldrvuj55ytcnzmf9yx2/tmu3pyp9gbtr6dmaturvigj1dknvovtor3+sygsx05ozjj264s5jlp6jcka+d1b9dwz3w/z4kpym9j7exaqiitlipfo65patcoba5d2isto98s6dck9zdw6wdxuzy3utdcvxyyvwqdw6ewv/bhztaodaue7ior8wuomwlerrbruxoefqihngtwc994y93jukrx3jmlflbdybk9ddtoigyw/bhw4nbsdqpfda71ud83uwlvgnich2toveydsn5y5ivbb1sfvpltm8yxxmoxebkpqmysbkegkjlplj1faqbqnmna/dgmeqxe1c/wdd4zroxdcbz5nfnhqumn9qwbignqfgqpag7sf9svpofjwtlmkwtovoqxhmdzmktmmykalnufab2fco/btmdsmhux+txxme/dny+jgx2dn1mnouu2nlld915vmox1xxchzgmhgumc1366xxw6a15o7okxmzctw/edr0zfa2e0w2yeu28kr2kb+xjerew9sawg0ti5fvvtys44up/xbntov9u/dookwj918mywgwacyegrwuc0abo5vxuv6nn+pjpnomzhkfhv1tvwj16jchdcv25x8vunx100+8jf8etz7xf8ordujp9bdqew3tkqjjaqbd7fbemsrzpuhuscvpftqzedm57u7b19p0rdp1extlhlwhdr15stycaexe/zaoblmedwia8utkot7qmbrcdwambzvkk3gag59bcfesajuvymuxgzqguzfui3mjteggf3g9/yldy96r58/ktuazwp9txlluwpwgr8osbsgezwbybzt0x/fsupnekeb6la9kcgejewy0olq9a1gzp8kzlbwgbvlthe9zcc3sjqzhvxlorbt2dcftxudiwarsumxok/mwq0plsv9e5085nbtwrywejq/ozmjgr3bjhbief44bfvm4j/msgptamzh+25yqu8aqzcddmjo9mlq9ncqexknjrh4ik8woyh7lyebnewhuc8j2g179n46yeiogumzl/bmqj+qpy8huo5ota5blazyktarvx3xlsxnyqtxgyzqn4o35van+0tqz0ijhtvml21dla4jjeh7ewbzsfqaoxxm7wkdt5apv3h5e1r9zu5x1z6tggazrn/hlm337v2tcj7e72/2d7f5v//horz+vf33mxyt/t/ipkxsr/zv/wvn1youvw5u3k+fnf//zk8/2p3v5/nxz2fp/fp5fgvqjt2qan64yf57/9/8bnzc2da==</latexit> Extend Hypotheses Lower the lights <s> V N scores of extended hypotheses: J V b hypotheses to extend consider all possible ways of extending them D J N V last label from previous hypothesis

32 <latexit sha1_base64="6dc1wp+pjf701yzke51tyrpgp68=">aabefxiczvtjdxw5cmaptzg9jno++pjjfd3uiosw2j6xlbx8hk3jkt0umynrvthjgmqwqgpkbgkqrcql0mdf7yt/jw9+vvrs3+kli4csxaakl9az29lrvgl4ebeibakbacriiibk7u5/ffsj3/rt3/nd3/vx76//wr/+0r//5n7hf/kvshme0jdhgqx8m4aigrgevpfmrvsbjfmsbxh9org8wpavrygxle1o5dkjzzgzjmzcqikhavtxt/57gyrrnvybr96lly89uibxftf1p6hzmi1jkowlx6zzubz+toqszedfif0uaygubtudyfcwzrcuulfwkf/shzsy+ez8rqwjdtjpnw7dnxfzlmvr0hvi+yrpgdeowposuukejcooy9lzt/bvnir8ii3gygm0iqarsvluask/+c1hoqf8k08ahloxt3xg4yxjab/uabt+qcn/74p5yxm9a/eonv6hlfuifprbokrsoppi+0rzfktp64bzwdfc3/h8c/gazuwr+b+ctm8394d7b3l+4ola39ihlsecld6jmvs3qjgj9gnn6dydlpyrzbrr8+j9dfqqfl89lmvzexieog+9ias4qqdekibbivmywqv4baabhxhcqdji3hjvuw557fkyvdnr+jhi5pny5wjoc9w05ufjviqtfurjhfrlhgrxwpz0f9evb1qmfwgrfd+wzo+d5fjvl0cmwrxs4bcs2hmrfx95qz8cvtdvm+8ehfaskw4jw50v6id5zna7cbrqfl8ehngxs1icx/ymxlmx80pfmcvbozs7w8vrua1pxajzgexuoyscesxqcddfsbs1dfnm92jagqbo/6s8mxo9ic3g0iflhlpvrftorizcqxfuky3qtpfr79/7fgh8gvhrvc4uoeeaac3irct+zdzgobj5lrp+qvyonrj4qjbhib8zuylphupolyxbaj/khouylnki4j3ana31iozg+48mw9fwupg0kxk0fyhldtt1cxwda4gh0wsb1vfjusbass6uqc1rkmdci3vys+ltve9g67shdvjbswx6nbyjr8ofg/nuv3dl332annz7w2nu4vy/dlhdv8z2p3u4mjghkszeyrz4vnxyna8auih3qd5tmd9rgkcrfccxks4grtzcwe8ggspmpkh3ueyg8tqh6dalkpgiy0yighxtquqlyvhwxfpavfhooduxrrscnc6taqz5p+ao+utkgayfuudb8v72y14hqrxc3hqylld6uobqkbsni1l1tbtxn/pjfurydtzsaee22wz5u6bf95ruopxsoovrxm2hsqjxj+hjmqng+yqgr5hmrah8lqbnmcxious4dwyssamxl+cl7cju5ziawftjx4gbeh2gtzkk1/tf11rz7sf+xtgicmqlrnobtpabnebhv0ctm7/lvttuyxbu5ok92q6id5ae7hazlhaexro3v3d3d21r/p/hhwh/fxqj/ho48//cgfp2ee00qqiu+hu5k8k2c/ygfe4ysfc5qr8jjm6sl8jism4qxqayrs24castdjofyxse/vtnsujba4iagjyxrmg1a62k5zofnrm5j+ljfg/zsjsr55mvuw5+gngqctrev4icfniksxzghyl6rcrhfyzcqikivpzygfhvhcfglttu2uk2zgwkw3ns4jyecy160n0yuie2wadksnmddne9mtpta/lbnjdcpokmiijnrxhpnemgqpsrqzd/rnw15nab9luopnqdcznynxfm6tcgajiybhazvzshpvsavugez3ivkgncdbzujuncidczuao8uj71txvkjiu8bttfsjby10dhyutvoyn1m8zww9mjjoivicqecgq8bb9gbqk2lapldlkef0mxnkerd2sni5s/mser7fyy5avitigjkqytkajbkawwsq96mdkv1daiscrdvcgglud+ovchnriknhi2zpisdelq6jyrwikucckqrpe29zjmzl4qd0ypjcts5ghxh0qgqugghewzxmha+mjmzlsuvjst2goszzgnzmsbvzc7ovyhtfguhzgfpkpjzppwpzlvcamwvjyvvuwu8ym+i3jkc8hz7p7iiahikicoqjmnhaod6pd3/obw9/cfz8zewojktckcrr/uszsxjckui+/2rqsr8mnh0zzn7nepxdi+try4qop8+ur+wk45sboniknpbfs6mo6uvzs/xoih601omxyfgkxbkdh2icwfxxfcgo/ndvcxj0watqt7bg0q6vop+93flhn9njcx9okcqzjdbenscqozxnwk2exh1kzk9afavsh6mtdj44cwgdqaqpaoh1qcumwlohowqvpz8iuaxi4jdbfow5zrhd+ahu+bc/g8mndrdt3w3b3q026xiivajbd/de7awbhfoeamxpwgfxgzfdrgncgoiyswrxwzrjeh0/z4uhlgrorgxznifjmsc2pqh856cgmiss3zm6hznnzx1svcuyzgzevbowmlxhen6ep7f3tl9/cohljmmw2fizmblslmagixihy2vuscrjemjkbfjb70bmmrrkdeshkavm3fz0n5feew/oow4oerlaecdcgnnxzcmstrpdf1fjcwgkmyilhnhwi6jcb63615mfucgedmvzxjqqq7tqsb4cjsj5gjtqbildggc2+mywt9qftbs8wsyheqkrqy6wu21syxrrvxanmunmubm22levfaowceyyhghw1yi3ex0cnuoab6u103yoblwdwgbzbzg6ty/2ihhvch846dhtzy+pdo9xpn1hcuacjkisea5yumjpwylmdsiicyckaydzjdsh2lwzdcczsuwaolgwahc4jegsgudzhfwricvc2mcgwrow7dksnnlyfm8avz02cpg2wqo4prpl2gup5ykfqwyoysbgt1guqw5q39mjt/r6qivzh1mangjg0fknoparegmdpxl2f9uahijngbdyerbupwosfradhrwwxb2+skoavgzwxdqinoimlk/oth/len3wcumcjrmceucdyl7hiymw28odiy0tjby3cop/kgomz8izds8har0cmbmukbrih9meaj2mervnwtowjlkidiq92rcxmjxfiwjw0rua+5lyn+jz1fhd2mhu80xdfofqhsc5il4vrkdbnszc+hruwiluwjtgjxw863pu7oe1m7jkrybhsy0viz32kqkme511gcwse6reks/wgu1csbkq1025tkeshcu2at9an132lhvegzoqvefywjvam/tab0ztug5y9v5tbvnykwdotvay1dnzxf+9job8u7fu9bwkoxdb1kdjoje18dzg9/nhszelessh1kyk5hltgpv/u0hlyqp9tugahbf7ta0lwvr7y1qye+kcigdo0jh07nwl6yqyn/ya4he9ps5cpyiwbqcceuucjxmngdct7ftrwtpbar1aoaqlodjroo8qs8myprrmqzg5dkdbypzy2e1h+sou9jkgwuxwxlyuthfbnbvymqtwedmcxe59c3b56wvysw71/da9acugvqc/la0dd89/13tzqfhgnrvgyabvwsulyxycikeroze9rrbcmpuszwrubi+r1wdtyf7yrtbfmkb2osfi7oxigphj7zp0gvuxomvwj/2r0q3vx5c96jpx7kdqdbatpcg8jzhgf0bcnixyogt+72wnr/qcsy6uvzbfq0gsg+ozsim2h/gfpes1+mibvjdhhlmgvroidyxfjzqrimu0sovomkpkinlmndpqoz1zeb/ncfincr404fb1ywluc8dqsniitl/h4theyx5l0i1bpgapcmzuqfcxv7lqqwc8pdvad10smkgez118utsphydo/mcgau4fpi/p6ipquj5sxceulkjpepzkyxzxcmeojkknsysz2g5e3gcampewzlnjfuhowxsd2gprodnkujodcrl8m4kwnlq8ttojf9dicvevsctjejrl7uohimegligki0e2eqxl6gmajkhaxw4tvubmmcmf9qmlu8uhptdht95ggdzswl3dsrupk8rkkgieuujb1kkrdv0k2ivkqspplcozk2mk7els0wpabtnursqu6mdaqqteieuvrsdfhsathk51a/ty1gqduposzsmtb21x37mqkrhouqsucutfqebauf3nqydiq0gbaqra4oic1wzurukaghoan7dbp+u2hveq5ucq3obecabzluo1dncgsreq7daoxg3li2mlvngrvxvkdashheqtk4nyg1+pwbzsmyhvwqet9kbmxf6wdwp/eh1ammc6em8s5mqopwuqt1o4afc5vkrb0wc+yfbsmadiyyj+woqdgqsooqgghz9ptirs1ofolhutgroggjctqnkss1qjsm6d1kyr0mibkglrjzcd/jlanlwspbjxntrmaggoowzmz464s2fdyeob+iuohz4sonjorjkxiwpyytjijaseqx5x+ep6sv1lml16wkm7hjyzoldnh0+yxxnmkyumntuavvso0dujtrkkqpjn/jkcc84wbmolpx1nkpe5btnlixzmd2hyvl8lq4kzuv50aj+6zhqnmp+wq7o56xuc3ess1jml797qe+1nq5rwaqbqymfuxmf6dywvdawqmyu6ol9jmpkykfzpujplmt0hzxto0n9biy11a0auhqa9pmgipi4qbseaferarli01l4x2rdfq4botybueqvdcm2cq/pm0736e28z1kedzybue2uor+e7byr1gyrrdzjgeee1vrlsogpotkiwgixetacasbgruic/zmkaqnar71ecm+sou7ip4tm/usxkejydlfqlwgc9tybbtde9w124fddivzp9wotrqdmab2rvigj1dkfvqvjot3h50dyvnyqgwtpl5pzdhlqjckg8tqgam8mz/kvlaxndk9mxkqemiefv0x1turogrkk0k7b5jw3q+h+43mhqrcmhh0loy7q6jvxluesrxp4vpya8djeqtfugnn+5hi5rvdhzfmioviuj0p89vdzxwplw18xex0kpzx1rpwf1rbvutzgtqvoqgmpwy1bmaoulkyikoe9x07ifaokox4iephnsrj+n4vl/rvtxnw5rejmcd6q8vayeyflgjyla1vssjysojustuxh7h7qr+xklbvmzwdz0wqzrccr27jxfgqumndzibpodc+tvqob84l6idp+fjwm7biujqq7+cei7ouswwxptvd6j7sed9h37d1aysbbsswn1zhvlppye5nxeqmv0nkw+s7punt4wmfu6yekp03qd0wsv/xslbxq3vndy1y8yivo3/dvojpig1myh9qy230io0xzizz5hvd2w9mhtkiuw9vpjnthxp71gix6584swlj34s+e9w2taggcyxuqtifo17yef9wtd999srtpromxghh1dqsqrwsslcq2cndr0k3r3/6ibfx3+pj58zqervmyk36vm2vyvmufkou8di8n4w5/ur4aqastmrobmzkiaf75ru2gznm/w6jz8q149chj2kz3tqxvswmdi4wbbe7+jf8qjnqkrxgvfolabm05sdnygjtxlqt08hymfb3fapjxqm3mbfx1fb+l6+awnl/1xz58va18twn/z91lxotr0vzrwbyj0admqokq/vpeuuuu1dg6w8loivxaqwuerstwzfrx6i8ytdpymzaa6pquhtncxmivfzriwujjthar+nsnlouq9we6sywarsr3yz6plp4xmg/iqqi+tv2q6penxucvcczw/wwo/fdledqnote0yvl9ttlwwssz4nicluqvld2u37eli8zsrh8gkswuyx1qy8euuhnuksvppvvwf9rlgisc5xkx1b1mcvb+ngqp3ngcdwmbtlbvqsytpq8qz4rhv24ghku/b2vmybt6o6zrqiok1uwm7vouobinwftncvnivhvcz57aat5vkdde8/ayoflltftw/18hqgvb/4o9mhuzpzoypr/z2hrs7w1//5f1ffvb9fobha3+29udrm2vdtb9a+9xai7xjttdr4b3g3j/d++d7/3lwrwf/dvdvb/+hot/6qorzp2udpwf/+t/v72j4</latexit> Prune Hypotheses (b = 2) Lower the lights <s> V J D D = set containing the top b hypotheses ending at position 2, with their scores note: using backpointers, we can recover the entire hypothesis

33 <latexit sha1_base64="6dc1wp+pjf701yzke51tyrpgp68=">aabefxiczvtjdxw5cmaptzg9jno++pjjfd3uiosw2j6xlbx8hk3jkt0umynrvthjgmqwqgpkbgkqrcql0mdf7yt/jw9+vvrs3+kli4csxaakl9az29lrvgl4ebeibakbacriiibk7u5/ffsj3/rt3/nd3/vx76//wr/+0r//5n7hf/kvshme0jdhgqx8m4aigrgevpfmrvsbjfmsbxh9org8wpavrygxle1o5dkjzzgzjmzcqikhavtxt/57gyrrnvybr96lly89uibxftf1p6hzmi1jkowlx6zzubz+toqszedfif0uaygubtudyfcwzrcuulfwkf/shzsy+ez8rqwjdtjpnw7dnxfzlmvr0hvi+yrpgdeowposuukejcooy9lzt/bvnir8ii3gygm0iqarsvluask/+c1hoqf8k08ahloxt3xg4yxjab/uabt+qcn/74p5yxm9a/eonv6hlfuifprbokrsoppi+0rzfktp64bzwdfc3/h8c/gazuwr+b+ctm8394d7b3l+4ola39ihlsecld6jmvs3qjgj9gnn6dydlpyrzbrr8+j9dfqqfl89lmvzexieog+9ias4qqdekibbivmywqv4baabhxhcqdji3hjvuw557fkyvdnr+jhi5pny5wjoc9w05ufjviqtfurjhfrlhgrxwpz0f9evb1qmfwgrfd+wzo+d5fjvl0cmwrxs4bcs2hmrfx95qz8cvtdvm+8ehfaskw4jw50v6id5zna7cbrqfl8ehngxs1icx/ymxlmx80pfmcvbozs7w8vrua1pxajzgexuoyscesxqcddfsbs1dfnm92jagqbo/6s8mxo9ic3g0iflhlpvrftorizcqxfuky3qtpfr79/7fgh8gvhrvc4uoeeaac3irct+zdzgobj5lrp+qvyonrj4qjbhib8zuylphupolyxbaj/khouylnki4j3ana31iozg+48mw9fwupg0kxk0fyhldtt1cxwda4gh0wsb1vfjusbass6uqc1rkmdci3vys+ltve9g67shdvjbswx6nbyjr8ofg/nuv3dl332annz7w2nu4vy/dlhdv8z2p3u4mjghkszeyrz4vnxyna8auih3qd5tmd9rgkcrfccxks4grtzcwe8ggspmpkh3ueyg8tqh6dalkpgiy0yighxtquqlyvhwxfpavfhooduxrrscnc6taqz5p+ao+utkgayfuudb8v72y14hqrxc3hqylld6uobqkbsni1l1tbtxn/pjfurydtzsaee22wz5u6bf95ruopxsoovrxm2hsqjxj+hjmqng+yqgr5hmrah8lqbnmcxious4dwyssamxl+cl7cju5ziawftjx4gbeh2gtzkk1/tf11rz7sf+xtgicmqlrnobtpabnebhv0ctm7/lvttuyxbu5ok92q6id5ae7hazlhaexro3v3d3d21r/p/hhwh/fxqj/ho48//cgfp2ee00qqiu+hu5k8k2c/ygfe4ysfc5qr8jjm6sl8jism4qxqayrs24castdjofyxse/vtnsujba4iagjyxrmg1a62k5zofnrm5j+ljfg/zsjsr55mvuw5+gngqctrev4icfniksxzghyl6rcrhfyzcqikivpzygfhvhcfglttu2uk2zgwkw3ns4jyecy160n0yuie2wadksnmddne9mtpta/lbnjdcpokmiijnrxhpnemgqpsrqzd/rnw15nab9luopnqdcznynxfm6tcgajiybhazvzshpvsavugez3ivkgncdbzujuncidczuao8uj71txvkjiu8bttfsjby10dhyutvoyn1m8zww9mjjoivicqecgq8bb9gbqk2lapldlkef0mxnkerd2sni5s/mser7fyy5avitigjkqytkajbkawwsq96mdkv1daiscrdvcgglud+ovchnriknhi2zpisdelq6jyrwikucckqrpe29zjmzl4qd0ypjcts5ghxh0qgqugghewzxmha+mjmzlsuvjst2goszzgnzmsbvzc7ovyhtfguhzgfpkpjzppwpzlvcamwvjyvvuwu8ym+i3jkc8hz7p7iiahikicoqjmnhaod6pd3/obw9/cfz8zewojktckcrr/uszsxjckui+/2rqsr8mnh0zzn7nepxdi+try4qop8+ur+wk45sboniknpbfs6mo6uvzs/xoih601omxyfgkxbkdh2icwfxxfcgo/ndvcxj0watqt7bg0q6vop+93flhn9njcx9okcqzjdbenscqozxnwk2exh1kzk9afavsh6mtdj44cwgdqaqpaoh1qcumwlohowqvpz8iuaxi4jdbfow5zrhd+ahu+bc/g8mndrdt3w3b3q026xiivajbd/de7awbhfoeamxpwgfxgzfdrgncgoiyswrxwzrjeh0/z4uhlgrorgxznifjmsc2pqh856cgmiss3zm6hznnzx1svcuyzgzevbowmlxhen6ep7f3tl9/cohljmmw2fizmblslmagixihy2vuscrjemjkbfjb70bmmrrkdeshkavm3fz0n5feew/oow4oerlaecdcgnnxzcmstrpdf1fjcwgkmyilhnhwi6jcb63615mfucgedmvzxjqqq7tqsb4cjsj5gjtqbildggc2+mywt9qftbs8wsyheqkrqy6wu21syxrrvxanmunmubm22levfaowceyyhghw1yi3ex0cnuoab6u103yoblwdwgbzbzg6ty/2ihhvch846dhtzy+pdo9xpn1hcuacjkisea5yumjpwylmdsiicyckaydzjdsh2lwzdcczsuwaolgwahc4jegsgudzhfwricvc2mcgwrow7dksnnlyfm8avz02cpg2wqo4prpl2gup5ykfqwyoysbgt1guqw5q39mjt/r6qivzh1mangjg0fknoparegmdpxl2f9uahijngbdyerbupwosfradhrwwxb2+skoavgzwxdqinoimlk/oth/len3wcumcjrmceucdyl7hiymw28odiy0tjby3cop/kgomz8izds8har0cmbmukbrih9meaj2mervnwtowjlkidiq92rcxmjxfiwjw0rua+5lyn+jz1fhd2mhu80xdfofqhsc5il4vrkdbnszc+hruwiluwjtgjxw863pu7oe1m7jkrybhsy0viz32kqkme511gcwse6reks/wgu1csbkq1025tkeshcu2at9an132lhvegzoqvefywjvam/tab0ztug5y9v5tbvnykwdotvay1dnzxf+9job8u7fu9bwkoxdb1kdjoje18dzg9/nhszelessh1kyk5hltgpv/u0hlyqp9tugahbf7ta0lwvr7y1qye+kcigdo0jh07nwl6yqyn/ya4he9ps5cpyiwbqcceuucjxmngdct7ftrwtpbar1aoaqlodjroo8qs8myprrmqzg5dkdbypzy2e1h+sou9jkgwuxwxlyuthfbnbvymqtwedmcxe59c3b56wvysw71/da9acugvqc/la0dd89/13tzqfhgnrvgyabvwsulyxycikeroze9rrbcmpuszwrubi+r1wdtyf7yrtbfmkb2osfi7oxigphj7zp0gvuxomvwj/2r0q3vx5c96jpx7kdqdbatpcg8jzhgf0bcnixyogt+72wnr/qcsy6uvzbfq0gsg+ozsim2h/gfpes1+mibvjdhhlmgvroidyxfjzqrimu0sovomkpkinlmndpqoz1zeb/ncfincr404fb1ywluc8dqsniitl/h4theyx5l0i1bpgapcmzuqfcxv7lqqwc8pdvad10smkgez118utsphydo/mcgau4fpi/p6ipquj5sxceulkjpepzkyxzxcmeojkknsysz2g5e3gcampewzlnjfuhowxsd2gprodnkujodcrl8m4kwnlq8ttojf9dicvevsctjejrl7uohimegligki0e2eqxl6gmajkhaxw4tvubmmcmf9qmlu8uhptdht95ggdzswl3dsrupk8rkkgieuujb1kkrdv0k2ivkqspplcozk2mk7els0wpabtnursqu6mdaqqteieuvrsdfhsathk51a/ty1gqduposzsmtb21x37mqkrhouqsucutfqebauf3nqydiq0gbaqra4oic1wzurukaghoan7dbp+u2hveq5ucq3obecabzluo1dncgsreq7daoxg3li2mlvngrvxvkdashheqtk4nyg1+pwbzsmyhvwqet9kbmxf6wdwp/eh1ammc6em8s5mqopwuqt1o4afc5vkrb0wc+yfbsmadiyyj+woqdgqsooqgghz9ptirs1ofolhutgroggjctqnkss1qjsm6d1kyr0mibkglrjzcd/jlanlwspbjxntrmaggoowzmz464s2fdyeob+iuohz4sonjorjkxiwpyytjijaseqx5x+ep6sv1lml16wkm7hjyzoldnh0+yxxnmkyumntuavvso0dujtrkkqpjn/jkcc84wbmolpx1nkpe5btnlixzmd2hyvl8lq4kzuv50aj+6zhqnmp+wq7o56xuc3ess1jml797qe+1nq5rwaqbqymfuxmf6dywvdawqmyu6ol9jmpkykfzpujplmt0hzxto0n9biy11a0auhqa9pmgipi4qbseaferarli01l4x2rdfq4botybueqvdcm2cq/pm0736e28z1kedzybue2uor+e7byr1gyrrdzjgeee1vrlsogpotkiwgixetacasbgruic/zmkaqnar71ecm+sou7ip4tm/usxkejydlfqlwgc9tybbtde9w124fddivzp9wotrqdmab2rvigj1dkfvqvjot3h50dyvnyqgwtpl5pzdhlqjckg8tqgam8mz/kvlaxndk9mxkqemiefv0x1turogrkk0k7b5jw3q+h+43mhqrcmhh0loy7q6jvxluesrxp4vpya8djeqtfugnn+5hi5rvdhzfmioviuj0p89vdzxwplw18xex0kpzx1rpwf1rbvutzgtqvoqgmpwy1bmaoulkyikoe9x07ifaokox4iephnsrj+n4vl/rvtxnw5rejmcd6q8vayeyflgjyla1vssjysojustuxh7h7qr+xklbvmzwdz0wqzrccr27jxfgqumndzibpodc+tvqob84l6idp+fjwm7biujqq7+cei7ouswwxptvd6j7sed9h37d1aysbbsswn1zhvlppye5nxeqmv0nkw+s7punt4wmfu6yekp03qd0wsv/xslbxq3vndy1y8yivo3/dvojpig1myh9qy230io0xzizz5hvd2w9mhtkiuw9vpjnthxp71gix6584swlj34s+e9w2taggcyxuqtifo17yef9wtd999srtpromxghh1dqsqrwsslcq2cndr0k3r3/6ibfx3+pj58zqervmyk36vm2vyvmufkou8di8n4w5/ur4aqastmrobmzkiaf75ru2gznm/w6jz8q149chj2kz3tqxvswmdi4wbbe7+jf8qjnqkrxgvfolabm05sdnygjtxlqt08hymfb3fapjxqm3mbfx1fb+l6+awnl/1xz58va18twn/z91lxotr0vzrwbyj0admqokq/vpeuuuu1dg6w8loivxaqwuerstwzfrx6i8ytdpymzaa6pquhtncxmivfzriwujjthar+nsnlouq9we6sywarsr3yz6plp4xmg/iqqi+tv2q6penxucvcczw/wwo/fdledqnote0yvl9ttlwwssz4nicluqvld2u37eli8zsrh8gkswuyx1qy8euuhnuksvppvvwf9rlgisc5xkx1b1mcvb+ngqp3ngcdwmbtlbvqsytpq8qz4rhv24ghku/b2vmybt6o6zrqiok1uwm7vouobinwftncvnivhvcz57aat5vkdde8/ayoflltftw/18hqgvb/4o9mhuzpzoypr/z2hrs7w1//5f1ffvb9fobha3+29udrm2vdtb9a+9xai7xjttdr4b3g3j/d++d7/3lwrwf/dvdvb/+hot/6qorzp2udpwf/+t/v72j4</latexit> Prune Hypotheses (b = 2) Lower the lights <s> V J D D = set containing the top b hypotheses ending at position 2, with their scores computational complexity of beam search?

34 x positions Complexity of Beam Search extend hypotheses: O(b L ) time for each position (O(b) hypotheses, for each we have to iterate through labels) prune set of hypotheses: O(b L ) time for each position (unordered partial sorting takes O(N) time for a set with N items) time for beam: O( x b L ) time for greedy: O( x L ) time for Viterbi: O( x L 2 ) 34

35 Beam Search beam search alternates between extending hypotheses and pruning hypothesis sets the design of these steps depends on the structure being predicted at the end, just return the highest-scoring hypothesis the final set of hypotheses can also be used as an approximate n-best list (where n = b) 35

36 Beam Search if we set b = L, do we get Viterbi? no beam search still operates left-to-right greedily and can t recover if the best path is pruned early Viterbi doesn t prune recombination can improve the diversity of hypotheses in the beam (and therefore improve the search), but is only applicable for certain parts functions 36

37 Beam Search for Generation Let beam size = 2: X Pranav Khaitan, Google Research Blog: Chat Smarter with Allo 37

38 Beam Search in Generation in generation tasks, using too large of a beam size may hurt performance why? 38

39 greedy beam search coarse-to-fine heuristic search Approximate Inference 39

40 Coarse-to-Fine use a series of models of increasing complexity earlier models are faster than later models each model is used to prune away potential structures for subsequent models to consider downside is that this requires training additional models but these additional models are usually fairly simple and efficient to train 40

41 Coarse-to-Fine this is popular for tasks like parsing Petrov (2009): Coarse-to-Fine Natural Language Processing 41

42 Coarse-to-Fine also can be used for generation tasks (by clustering words and training coarse models to predict clusters) Petrov (2009): Coarse-to-Fine Natural Language Processing 42

43 Coarse-to-Fine remember the local predictors we discussed for dependency parsing and machine translation? while they don t work very well by themselves, they can be useful as coarse models e.g., for dependency parsing: train a local predictor use it to get top k head candidates for each word restrict next model to trees that use those candidates 43

44 References for Coarse-to-Fine Procedures in NLP Petrov (2009): Coarse-to-Fine Natural Language Processing Weiss and Taskar (2010): Structured Prediction Cascades Rush and Petrov (2012): Vine Pruning for Efficient Multi-Pass Dependency Parsing 44

45 Heuristic Search Algorithms beam search can be improved by using heuristics to favor certain hypothesis extensions over others e.g., in phrase-based machine translation this is called future cost estimation (see Koehn et al. (2003): Statistical Phrase-Based Translation) if using a particular form of beam search (cf. agenda algorithms ) and the heuristics satisfy certain conditions, search can be exact cf. A* search for parsing, see Klein & Manning (2003): A* Parsing: Fast Exact Viterbi Parse Selection 45

46 Non-Local Features efficient exact or even approximate inference requires relatively small parts but intuitively, this limits modeling power how can we combine efficiency with some long-distance or non-local information in the scoring function? lots of work on this 46

47 Non-Local Features in Named Entity Recognition The Chicago Bears needed a win in Sunday night s game. But in the end, Chicago came up short. organization? location? 47

48 Non-Local Features in Named Entity Recognition organization The Chicago Bears needed a win in Sunday night s game. But in the end, Chicago came up short. organization first mention of a named entity may have more information 48

49 Non-Local Features in Named Entity Recognition this type of non-local feature was used in several papers focused on approximate inference for NLP 49

50 Skip-Chain CRFs with Inference via Loopy Belief Propagation Sutton and McCallum (2004): Collective Segmentation and Labeling of Distant Entities in Information Extraction 50

51 Inference via Gibbs Sampling Finkel et al. (2005): Incorporating non-local information into information extraction systems by Gibbs sampling 51

52 Non-Local Features in Beam Search Lower the lights <s> V J D D note: using backpointers, we can recover the entire hypothesis à we can compute any feature or scoring function using the entire hypothesis! same idea can be applied to Viterbi and other exact DP algorithms!