TTIC 31190: Natural Language Processing Kevin Gimpel Spring 2018 Lecture 17: Machine TranslaDon; SemanDcs
Roadmap words, morphology, lexical semandcs text classificadon simple neural methods for NLP language modeling and word embeddings recurrent/recursive/convoludonal networks in NLP sequence labeling, HMMs, dynamic programming syntax and syntacdc parsing machine transladon semandcs 2
3
Google s NMT System 4
Google s NMT System 5
Google s NMT System they use a procedure that determinis1cally segments any character sequence into wordpieces vocab: 8k- 32k wordpieces they first learn a wordpiece model 6
Google s NMT System 7
8
9
Noisy Channel Model from Noah Smith
<latexit sha1_base64="pw8oji3ujegpi9vjlgbxuppo7ly=">aaa0ohicnvvrct22ezbtw6rekuzn2xk2kqzylkk6jpnmkvfmffu1o3euxzzzsshrqbkhbxjvbsbzmcp+7tp2ifoo3qv4dkkaloroxhybfltylhyxi4ucimfc7u//99zbp/v5l375q7d/vf6b3/7u9394590/fivykof0ezgnof8+iiimlkppjzmj/b7glkrbqr8llh9g+3dtygxlsxo5kohzsukmjvlijfsdv/ofrswprp986mu5e4twqrl1rdfn7gw1/ep27d33yj+hy7kd6krdevte8xxrpufvxf1r/blye+gsiucvnna19x3us3qffx1+sfz2oysn8vb68by/pagsfvxtdt/lszinkf8/f2djf29f/xj2x6j52fhrfo7p3/3wlh/lyznstiyjeej0tf/is4pwycke1ut+kwhbwkss01p4zehkxvmlnfz7w1ateeocw79meqq2s1grvihfggayjxiizdasdlwdlnl8yvnfsqkunat1r+my8wtuofq9iheyflkadxjybrj6ohroqgmttn7rpiajtfkw94zsstqxyz4w/dqyk2lcwnm/ni0tyxg+69eg+zrlscyzfvu4bb5ljvu1foahcikpwrkgzsvyglxrde0wru0xsvp2mn6w46044lcm97wzcdrxjmrkweg8acqgjhdqoam0u4r9dac5p2dcqw1ogpyk/tprbmmwm6pfce9v8vyiyvvau/qgoa2rrmbhszzd/ezsu8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkoe7hoyixmuidjipkqkgwctqcj5reza8imu/m1n5yk0jmhsgflcvkiuevlkox7s4x33m1dcw0rnnpibkhg5vndmnxpvmtorkd2jmskpnlnp8apegevaapgkfc5puvpo6wttvpk3rbomimleifkba3/vibiuakxiohdq9qe+isobdkoyqhgkmwutwr6txghekbor8yylgbxhpfto6q3ztojanvzcufec19rryvfnvy+tt7n55/ois0caks0beg3i5onh2zgldoaxjd55u6jhiomz65ugr0dnrcajepl4afvfepty8glwqhp9f0ypn0riudo96qk8eweinmubqwo9fhnvtkhawgljjhndhc+c46t97vckgn00afqu+inao1hn4au/fpqfjamnkssqlgjcupjulwa98gcuxwk2erhc8p+lttoxncwu0tzxpmicjbdbytvutsz63lgvhlmspoelneyf3fbv8couclbflbei7mvlombkpy5tsv5ptuslsmk/zicz9ugmbkade9y7+bab1wfhhvynqwpmkwsuwt9abgosylok4iilfrt9fvndccnbcb7jrqikawbkmulxwcfdndysxdvkw5dhittcllsrbcla8ii5zxjdzed6y56l3dhdw4ilxkilyt201gynbymimwqc4vmdierocwzmhopjinfmtlkxgckfdfeev3/ddxng/evpba1bkziehf2k36ygsbboinmewqeqenjlfeijhncunana7s/rgjvz6vgky6wxwhezfhen2b7pf7e5qqrejdgkcworbwtwb7fyppj4vohm8pd9z3lbb2ilp1r4gtmtbmqpnxbsvkoduldlumt073ki5ven14ek7dvi2wnvnt5st9hcyi+sfb2xwru0r8d7mxmhzwlqpvzp6crw07tdkz85jjhobmax00z63tmxauelntkjrqoojd0kqtj10z2dikbv8sgaaijqknlqqkjuqkyfciqqtbipmjoh3jdml9cu7buhv3clg22hn0o5olgsxpjzlpiichnnmzj+wkjyeud9auwzdnnbi0nakgfymlfrxvodydsls+znh99mw4bbszliq47cjllvj6ymykkywlo7zna8b8ozfkteynoimtkzotx+vkn2u8pimsyyemo5vkjvfhqetdkd7i5pasnhuvub/cyownqknnlyedpry05ygrphf/se4kun/zbmwlcjamcszegbchh3ydgx70qqsfayfxrqk4g1gnjmbggktnkr5ytv84wqgxlnjxhvgz1s2zfz4fnqnvfo04ccnvb96ih3mlluzqmr1nho221g52gdew9qmopn+bxplbcmrkrl1ygmxribqdvmuuyglyymn2rf8s9+9fqzxoczerzk2cklw7dqksfw9c6mwxqe71vptjvlifdamw7ingibw+f8gc9jfnf13ac3pkos2k6bksurnvlxqffhk8dn6v6nmvtrdtgpu/3adootp1jufgs6m1sm2lgrhrs+xzcbeouuankax6dy5z9ijzw7sfcwvbikuxbrewintcym1z5omu8cbfvad5bbokpjxk5v0ic+1twxxztzbnssxorgzhw40wexylvsuzqzsyy8dkkycl9xl0hr3vboro9mbf9c5ibinoedixg7ydn2h/chtga3qdsafamvfw/3f1vewejarta4wmscbvvhymgcnyglktusaks2xzgzrgzkbo2bkk7bpzi9ti+tjtsn7pgbktrs4bn4y6ido8larugz/muynbviwqw6q68sxgqo9qepwo/cipjj/gxltdllb3/xe20fe6tlhpv1/ayq2areeamagldocxpf2rk/afw7whtkmpgvohffkrvs7lxijstpncifajcozilxwn2no1m7pxk9kanfjjbsn2frdcpqsesdqurqvtl/l4tbeyx7lph0f5qkialjhmrkpmco9503auexhuqd18vgsgcz118unwph69o6zkcaoxi+rjbuke0wzylm4zisctsyxnozo6/4yjctjva50lpuyfjcbwzws7desbsbqmllhipuyfmomsk8mgl6vvalzpilvhzkbedg9dycv1egfjffexf0iohimfglygkj0j8jgilsgmajkpawuxftxuylx4pmc0qzqo6/rvkdxt47w0crs0nvujzlkfuoywfyljolqkrypkq4ttvgsnjxktpw01fqytmkcr2ra3kncynvnpi+gok4n1kvna8bib8bjpop6ww4ba6pz0qhleua7phv2zurfodymlcaumwwj2ruke+vk0hpsbnpcdkgdqprpmnvqqxkehkbuczzwj4w95jlqntrgeabpdmw+i0k1w11zixayrms34lrfkmydb1ze0yac68aguzxg7usq0u8nnmsifjzhlwb0qvd9q1zz8+ngskfiz+zkmzs9xvqvreouzu0afsxlknfwwiyyfxikanuxyo6wewdhqmgeqmoezntdire2opgfh8enxkmgjeoggvgnxjto0wsjh8y5vwazw6ay4ztcz0rl6lkwivzzuqpwksgguguzt/z1rztkbskst6ku4py4zgyz1ckp9mbowsfjkhrz1wmr/jzksedyt5vcl5lo5czpauwc8otmp1xpdm7xbrdbhbzraoyjm4islmlh/m6oaslb3swz6ckknusfsgy3xboko7yjpirg9xft+1zpgrl2xsobso/mm+xuemlov91br/n4ga8msbna1vr7ssbuuhg89jexfydq1rrxklqqsy6wmmq+xhh8ufjtwav7qjqjz6yigxyh04escqeceddupouixm4ulrnanez6ppjctter2ucf40aobbeouqvtm63z14fd4pvqdr5hou87sedivpfqbw+w4rvlpghi1ovam2jrvcmfxhrx6tbbhisjbg51i7aeycfellk8rcc33i/zf3zdv9gtrafegvg0eywnykg54dy1qsf+3x73zfzrv906+y91og1uz+lcvhur8pgkbflr5zzwy9tbilyvj1qyaeqvpoyzrhldsnnzdhat98x3obusgj2nz2ycehkzciguj7omvy9bbxlo0mz5e/0ger+2cwcdlwwgh6x53hbwd68drrva54lxnemxma1eegoe9ioryyndgbexusrkoz6hyfv5tws1ti1vmx0civtkeub+xxu4d2loa0mlhajmf2yucjvmacvlqwjkud83ewlvghkodzj0xqmowx9c5ov+j6y8eppbzphjvdriimqehcxp2lqgkllpljknyln1zmgsfnqoi1zdnc/wdd4zbjq7ethxs+bcgpaa6wo2zqhu+nrsbrzyl8chfizrjvkikrqv93eppuxojaaynpoudr9pvj+w79v6gpk1n2kla65jxts7h0sc8zpvrk50tsx+fmbda71hmtd1wyrspxkg4gyv/xslbxtxvna40c8yivs34jvojiog1myhqxnsvpgm8hlyzv4mvd2w9chskguw9vpjlthxr8broo/3pgrt2oofle9wxkdhommwvm8j0arppvxcvdz1p/1utzlvm2hkfrz2tamaxcebc5ukvdr9knt36qefebv/e558bi2nv8uevtxfqu0zukwnnavuh9vhgyzi9g/cuzov5c3pzhzkfri7b19rk4gu8vbwkxodoptmyvhzzg/36oz0mjjyoedqxk/rc+vrz9qvrumxdmza4nhowc6ghhtjtzqxsoxjia7ivazqma+q1vf1iq63x7b7oxj66lda9dudmo5h7lqufv+vjbwbezqharvq0h8/supnemuh10t5jzmpjesw0upy9axfex+czyubftuyqqgf0hba6fcwhv8x6fg57olcsnqdi2wlsonytrllfqtkg9gvksuhjmqtffmh3eqpiwmapculeuoq/blu+ox9efolbde1yld9tjlxp4x1xemaxhmhxhdnv+5d0ijm6igywuwd5mmle13gekiwkl0pd5p/ospckilgmlbn77o6at5obvxvam56silnrs7uklafnpvn1xgntgcpc56dtfnfb/xgvdedjhsvnvrye1xuqtaj55cdzfnd0qo1a24h+lctxhfn2wcyzmpii+qq+x0vdinh8xv/agzk/omm/fht3b3r/t7om3sbn3/s/pnm22t/wnt/bxtttpaptc/xhq8drz1fc9f+d+u9w3++9zd77997fo/re99o6fu3gpr31no/9178h/jgm6y=</latexit> <latexit sha1_base64="hzphb84eohhsi2oxga9qnsqx7tu=">aaa0t3icnvt7c9y2ezf7tnvx0v7zf9hkmtqxpoocp03b8uwu27u7cvtvj7xewqosob4kkqab8hrnhv0o/v79nt0feecsacu5mrffal9dlba7i8vduzkxqq4o/nfrbz/80y9/8tp3frb581/88le/fv+d33wpesvi+irmgrdfr0tsjbx0lwiqo1+xgpi8yuhx0cvdbp9qtovkvhipliu9zulascmliykqs/f/u1nk5estvwufz3izz0ixufp2jl2ub7+50wqpgjtm6ftdjkzv3gqq7wahrpkz+vzbphldhxl0lpczghqbjvgk6lmzxr49o199h4klm3vnc+f4djinsaoxtzizjdcfzxnnc/w1gra8qpkiis2z97co9g/0t+b+tnqpry325/jsg49uhqmpq5wwks6ilcetg1kd1kqofmcuefeslis+ick9gc+c5fse1lqrtbadnukw5ql+fsrqtx2kmurslvmikdlrm2m3yawv7ars009oa1aulajfbdqavlmgeiczeirmwnizjxyqwdcqnyc5ecrwmhebg25kkshcfxwwlfrrhyp5xg5ru0hkgysxw9q8yhqt/hjyg/m5k1lfi2h1nae+vaggtrtmhwqiqmuzbi7kvfq1ysvamow6yirm7c39cddyc8bvfe8hlydoljiroqvbdcqqgzhbrnulpyumhko754kczuenpqlqlg/rzbvnwwqpfid8ucw4qsuogsfownjyi03ajrkuw/zm8nvw0cor00sil2aqooiycta9mnoijyisrhwqym4yq5uywk5ipwzclckruwlmcmqriiqyrzkypexc0aheuhwypstlpwpvh2ctgtkuoh5qr1tljnyfhy9apqj/hjtoipssfnmqpuchqtvpk28qjmdnhuy0zuwtjxcdqqrhcbfqakyzbxoa1ageolo29qhnm36djrhkcfhmfbd/sd0kcscnajpobtcqjini4yjsqoecjnli1mzqcyb35atqwosyxw8yt3myoa1d4zowdxwuvrrhnq2mcoktygss7o19fvzktnxgmktb5dtyotp8ceoiqwqfyq+fb5vwsatm+vbji6uztzneffv0alrcefxo8dwosvf46poeazgntf34aid64okjfpzibg3m+gvcpyd5cqulm04ifyidx+3wxzskbr1ta+gb8fg0q63ox2/2/xmkoq23jg6rqiwje3q6ixt8klspeo3vwshmnwgtme9vbkhqqftb/llwmsigi1vx1tebcstzexopbgw1v5bitavz4jdffkrmwo4s3zwud8fjsjw6zhugbm9plpnfzhoadelmbeicede75hmyncnjhvulqqpnwyklic/qg6km2c6cudkrzgm+x9d3fqjqxa+yz0mswscyprl86+ggm1sw33ri40rgipxclne6bmpzcri6zqxdheggu8hz4ojw8ogzocqlfx9z1qymbculnwit+tibgys8brzqaxhw2zpmtfqrgctetv2e1whl9/yy+c5y3bmbiimhdgkvu9omksyzq2qixkmvikcriouqdonco0wl3l3rtzyrnmc1wwswcczk/i4ciruyrbz3c1unpxfifmfs383is8dudmdtxrkap7cxu8s7bhtbmtm6fmsllaxt4aypdnxlmkplnnumzzdyaoe3qr4+68yb307ropmzttubalnks+cads6mpsi+2n6abi+y9umvjp4dr0z7hdnzc1livkcmaln0562qlx0oowsvpowg5ux4iujatbq3m7ksoj5ysabekrlpogiubcicqc4r0w4dftzgdjtsgtlm9t0l9cbtepbttwz5tyeotusrx3krqa6oasbmt1iukwkoh6goijeaalfpbavajwywnxhqcrkwkzj8ybl7eqfwcdzlqqpp3foxrnh0qswuvjgw92loy4c8yglolayvtvmvjgzbjyribtd5syelgwizxzqvneoog9z7k/0jzr0kbldqq/8llzjoxdmax0agerftt1cm+wl/u9ziot/yjcwvdbymreaegbbhh64dw140gwqfq6g1lul4g5fnzsbgogunbj5fn5/7mifxbrnxjthzlguzfz6n9io1fi34yxy+dd2qpgvkuhnu5ey2gjbb1bny34owqr1ez8mozo4juqj0zbvapj5ya9wbtly2ua7ceru17/jnshsndojabyhzc1zgsq3yauyy+msfri+6hnbrtdpo8peoyhi2cbs1qqzz/1ewsl9pu+7rotjryg3xqmwyhnp4z6h77nnjv+hfrzp10xyxg9h/uw36lkzd1zqeurbwj9sykkwzvqsk2rdvfafwhszkcuchsy+u+bfxmd8fptj3urdpde4njm4+tzfbaw9b2deow3pb2l+dvnkhpdm+mcavuwzzgf/mycw8ojbgoencdsnswdnexkru2gsv64uvke7pzl93tgmur3najbpzwownhbdtukf7jd8bw6drgn80zoqh+7+roywghcobxwfkglet4qljdl6q18i8ybejw2gzy9yxmj92xmjwynvinrpgnstarvz0xmhcd/em/owoh3rb467og/zlcw/0w8cdclrdlz0rqn4d5aq3ck9ijvmfltelkbsmze+tg+s1njm20pxntmhrebugrgbsoe1hfo7o+hp87ggf22pcs4becqx3pd+dlcui02ngpewwumqheok/0zcpndej31qe4syunxqwqboqnfv3gnvn/ubrfyfabujaalye5poqkbaaci9lzr+xovdctgnntodnajtg0xudxc3x+q1yiebk3qmvwgjgr+ute5p7sgsqwlxihjxtjimejc6a4rgujmlujxsszbiwnxvdnazcmsxhxtaycsegzbiwegywryasxq+8evokkju9kpsqmbn1gk/16cvn8urc3y2ieiwweticqkwn0mkitjyxomqdpa7fw3o5ixhg+ylwro5jqnfor1dpjthrzfhsdvstktqvsrjyxkuk5vgsuqmsrho1vzkylerplyzvddk2cozhasbesczsfwdkjqb6tif1pwdnwhixwbm/hppl7gajaljrbeu8oh3sffcyyqgcz5nkyjdnxotsc4w99wuzcgkdxsf7pb4h7xtnoaghgujiw/clhvin0l3zpkgggjaigdr7iu+hun1w11yipybr2y281rdkv+erk1e0omg5mmh0pxu7krv0cwvniotf7velgf0u1f9sdnz8sjuxkfkpvvjm1wax1uxnukt3o4afe5kuvbuwc/yfbim7dizyo+wbqhoqaobqbghz9ojirs1ofbngcwpqekjreois6foubbrpgtfdcrguwmwyqmi6jqilyjg6im6jvvgtg1viiafvwewtf3prppobubjzaoo4j76zwuj3coi92fnwkhjl/vz12ik/tzjphevpku9s0pvc8jymxappbn7klxnwi3e4derhfudjz/xelbqzh/f7o4qi6hqzz6dljpusldkx3q7rk+zyjzjrgdfezd1zpq9kunsnd8o9mm+xu+mzo1t37xzn42e8mctvelsz7ckdu+lj8njhxgwbulvnxqvsqdy5wsqy+vhh8nnyvqmv3qnpgt6zjgbghwwfyqy9cezat5ooi5z6uxrmaneygflgxhxredlahedgqooqhlel7zqd89dh/ed7ya2+r1zkvvhw5kplmrchq/ckjvseetmiwjfe1xqpc4jp49jjgggfewjc+kzgbckcu1guvepat9kvurd2y1frs2wpx4ketjcvomiopehcc0lbwdaed+z2ybddoflptabtb2ds7t6klfh4rbwp9wpluduzbbc7lxm6mbt1snmmwng+idm+aae4jxvmu4jklscem7azaqwjtdxgikdds3raole59gkl3ka/kd6v7dzdwcedxudrfjce9d1rj1gj17loes14bgzrkd6bp/ti5glomeyktjyzwernyvh9vq9yrwzjs+yehyk9tp5f+pswhlzy00dymqmamx/ba9y2pudfcipbeft3zytaoyau8eggywv0iet3q3wx/m6wb29+uwxzapzatitso5afjtvx0cunjswlthuqytms1y8uyms6q/yznslnpm1nnwjnfsl8wbdsyh3bpiqbgp9a6oad+wx4mi9lgynlfewjcv4uslzkqmgkgejp0ver1vsj975tcfhccyo2voy65q2783eksk/zfvpez8fuz2nxvtubjntdl0xqddphic3w2s+0uez3zqunq/mie7j1k77jzoho4mx2vj7b/hvjhfeql4sqo/qbzqirliyxaj5k9sghfhyoev15/+m4bwl42qkel0roggcyxhqsiey1/yef69e6/qef+kmmfizmhcbcrmnsgdgkynfbsd/4x4xuxf30e2/jv8etz53v9bqzslf2w7e9alcmtloa6na7ffgkgv5rbnqmct6eztzoyrj3991rbzkqeh+xj025crxm82stowqwe/vdehhmetwgablx9yxy6gfqgtfzga82yumuc7azkofeuq/mfwz5bigdmlwrxsxrpa1ds4ib7rfsxg6fp3lcb4egarj3ohcjypqvzhlegqldwlrkkjmp7ywlib2yg+ulvjljhbii2ghlxd9r0d6f4dnkdti1bgvcrzsgfvoftog/snqsdnu6mkxf5yhv69iovpdiospq0ij2s+ziesnpb19vgdd5ttkujs9xsyqsbh+gg371yij+ccfnrxjuvg3m+psxquuagwvuguvuhjrdsj6kqj9e1pgdwpzzwsqxkqsqfwt/o932n4myeioguoz1+7upx7yfj5lqdzsna1gpemmlxtkak7bytd7u1q7gmrccrf0se+ch67obnkn47dtbnunyaijjuljoyz6biggow3n7wedd5azv3j7ecj5chthu7e+rybgm29/4rzmt+09k3i8v7+1pdvyn/76/9ekn7z/pvlfxu40/bnzemgz8zeptjacbxxuvnujbp7r14a2pbt2//9f7z/en92cg+onblc1vnwy/99/8h6itpg8=</latexit> <latexit sha1_base64="3ahrn4ufn5n9i409tvza2imd2fq=">aaa1l3icnvtbc922ezbdw6rekqyvnb6wldtksatqohwascbtklzrd+koii3nylhsgcqodytedidnyob9n33vd+kuwhniaqakrzo2codbxwkxu1hcfbqje3j//7+3bv/oxz/56c/e+fn6l375q1//5t33fvu1yese0pdhnut824aimrcmvprmjvtbglosbgn9jrh8io3ftckxlm+o5akgpymjmzzmizfqdf7erxors4rrx594wc7eipyqbh3rztk7q7zf36m9b17sj3qstwndttucyruel8r0vlp4mkrpkj+b3ijyxkfjxxvfqd2rv9g35xflb5+zeclvrg8dbfttgspqxvspizxdwntqsqxfx9ad+uri788ppj/wocxz82rzuntlnova6/cz13f8lm/knkdc99fdpp6yk7bq5fd6gnry36k7mbjdtaxvxhtf5vzdjf29ffxj2r+j5mnjrfk5on/vw1t+lidlsjmzjksik9f+iu8rwiulewqakwutshhjynocnxljqtitlkhu3hburn445/avk56q7vjujbvikqaatimcclmnk11tj6ucf3xasawojc1c3dg4tdyze2h1xsq4dd1zwacjoqnzpta1maqjtrne66ygcdhmlvegukk6l2gefv3amjniwsj5vzyte8l4puvxhvmuzbhms371oaw+zsb7trsmh3iiqcezbs7fwbi10qw9fubtmult9oz+sootooc3dpe8gqg68szksshvpaheii4xudmjnfoefxwnoafguufttokcpp06uqzjfpujxqnvvffcoml7wbmmheibi43ajpi4h/mzppemoqdgpzmifmaqooiqcta9mnosdoguthwysg4yqyxvw05iksc5x1dcdzjmeyixerkwuyvfqboiorg8opigeuikf2ay8sfae2qoqp2gxiexca2yhl5o8ad61kuhlamzmk1blawdqmqmt7wumzjulr/qmgwvmlymcwgewuraazgmfmxpuvlq6mhrv/s0rbsnimdgivczan/vvtkkytwf+mgyugf1srtjfbifxdvqgdfryep1nq7wjpranx9zxpgn46lprqevr10hpqekkpliqmaevo534m2mvn3dl46endbawhhjihhllochl04tyqinipnb800d/qmhwd98fgh09macqfdpr0zfffgphl+nklwpl6/pkwzpxfepd3uol59y4icxrtbaj1+e1xosfrbwwuoe0mdz4ljpv3hdwqa3tqb9dt6kdqju+fj13n98tsfvxshivryfyvxndxadx/kylmdyryssdzw/4tnohedzbts1lc8yjynsmfi1vs2zpreuzweuzi+h4ocl9i5ig19uuzayyqlnffdt3h0mi2vsk+1qsl03gsziar7rpe83irasglje0z/aob4yk6wqerp4nks4goyjekoqennfefckpaj151l114uazxuguyykohksa4rlkwch1dyweougd0uog1qlu1qpegxny4vomgumczzhjxmeeochbw+fequpkp/evjoyeqwsxqb1ymcfhpynibcvc4idr86znwlzbk4q1fubavhdd3vhfe/n7wxakzefhqyr3amhsbbmitiew6sse9bifkmihngunkjb7yzpg5vw6/okyx6bwrmyfrso2v3mkbo7qangjjgkcgcpb2fxglbbpfb4vodm8fz8z3hhbmmlplv7gjirbwcqnhxrvqocurviu8f07hgl5fam1cnn7tjg22rtnt9tttpdyiasfxywwbm2r8r7mxujzqhtfvzl4bojpwm/jttztjlrcmhkpyv2vgvi1oisondcihzstlgtgqttpds7e4us5ymfa0ckjogkhudzhewj5bihbtfqywjevypjka3p7hmg13glgg+jnuk7orgsxfbwlvmicnbmmzf/wqjyekd9aewzddfai0ndkqbambboxvodyzwisodpht1pw4bdsdhjqo7djrpujaupyqcwwrk4f9m8bmglbttig8gxburehh77yqnz7sovybbkqidjwkwyu9xx0gp3tdeiqywe7vzl9l+gatozcelds8halzfncuoux+wfd9pkh/oerawwfgxjposbsichtgpdxjsbbb9dobeuqxidku1oiezac41qvlg1x7iaixfuslef0dmwdtmxpov2i5uwdfhxa++hhkifowxtdkpyvy2gzxzulvap1za1g+ik38hemkfkpipspdjybauhet2uyxtkqlhio/yt/+x3b4vbhkgj0wuodvyqftsnsvj968kordfbbrvwm0k+egumwxzuowaivf4/yal2n03xhvplogq57qoowrjre28tdj8/e/ws/atrsyqalqyx6n+7qz3fnouagkaxxuorgxaysnaxwdit4pwiae7qmhtu3gfphdi39grmf4mufy4kiqzbqyte2udjxingtqv7znjyj0j5q5vkupdn2iegudkblrviyxymmumhgiwmozd9cjnw9rgxarumwvl1uttfxdxs6s50wmv1doheqw64ulednl13al9zo2adtb3wu9pa8xd/d3wubyrlzjmrjazwplu8tczbcxiamrm5ygrlllpzwkbmxg4y2qpsgtlt28j6wnping4yme0ujoefd/qhezzsiq7bhw97oxs+7jcd6jp2rkbqd5as3cg8isnmf0befenu0de/n3aqv3powe/xx5iirucshxoysis2h/gfpesr8iudfggocc8aoscvusv9bqvcqqxok1yifqnkbiiv3gcaorvzovhrkkdcgongazzvl5wmqx6wuq4eko1vebgn8zjhmk9fw3kcagdusyzsy6zyz3mtdm55ec5avxy8zcbz/xvxa1a8prl3aiqwivfztxfpcy9prjkll4yqs8kkxppred0fg4qkmcqbjquew+jmy7iglt2gxcaygkb2gkqew0knafbvrndrlvdgnlswuknyeykmtw7wtrq9odgeiku7u1scwuiaw1ci7kqydjhwmeaurkfulxgrljcxdjxfujpvffq12uro6snroppysrqoqlgsvejjbourlcqdspi3vdj1ojzkkqas3kmstppexlzm8ehn2ztjwkzujpqrvmfphlr0ba1y7aa4ywdqp8styea1kxphkc9ol3tpvozok8fapblyzzmhibtcyw9dwxpcmkbbya6pq0gzxbmoakickndufyeh/rgw1zwhqoygdaigza7md1godrgrkxeow2jtrlzrw4wt88dkkxpqyb0yjkrsuj1infq5gwzzxmlmqbkmlgiqf9uqaz7zgoku+drckzoyt5iqonxc1bkdwik5tplowosf8wkdfg07fswddg8ghajzq0inghl7rphwfjs+8pc40wsihheincpqlgstbzpg9ja59wpgzhhuzpxg+flpgv0wnteqk1fbkjleuldg5om/vmhtyu1y8invrzwt19lgpjo5wr7mksskczjurnpkxz8noekc66ms61lsmdzkky2ja57c/jqrzcet3ui2qin3hy09cxnrkiu5jn/zuub425t5bl1eszooejbttlixzec2wvqfvd5uap/rovdc3ms4upbrflpchu3y+ap762ged+obislx25p+twmyshedxz4mpg9q3vr3wkulmsnruobmxzichxerhxhhh0hz9jlvnnauzppqmu5amalanbrcxnjnpwuciizjn0eeyzy6ihuocnwils3ciaqgbbn1/vqog3wf2ch3modpjxyc2uos+u3bmrxiyrrbzhoreezsrxe4gjgulh0mihaxgcctbgswiczyi0wwtwj41vsl+8ju6cp6sijusxcfzqlkfbk1j5y7vijy79vjvtk+6rdbj/+xdrtn7uyc27dibt5eku1qvdwsbox6g8t25yrkjk290prxljvvsdyvmwfu4p75lqecrbdn9mxmqeiiew6r4lhxpooxqe0oxdosb6lfqo/xdu5g4jlb4km03xvc6u61w6hw61wwvmy8jrovsg/b034kcjllgcmijoqelntzmf895lyywtrg18w+dkv6zt1z/48n3ls0p4gkruibpj82v7hnc4cspvq08qjvmwmbd0a5xwczoq9rh6zfl/nf/3tthrz5btpmsv5qmreyj0lmngxdq5wsnjbme+wpoucwvj86hxgrrppnedqfgted3qicucvzyufijxufm7iaanxqqqio7bfgqz+w9cyskrsnyvurx9ii5xjtte9nl7qpnn5p2pdtfcd8mhuxxtxxmw/sny8lgxmdn1cnoltmpwuz7o3emjnrumbcqdmma3gg1366xta6a15ohogxmzctg/edz0zfa2u2w9umdt9irnkj+tive6oewpoqackci/qpzbfsq8fvknfhe/fdtpbgz8t7pc+gy5xhgku3hgjvdb9+rl7rup9+qiez6pkw5qolu6zrdekqgivnxv7tfhw6e/xtt7yn/wfpprew06tlwt6ab9x2atzsu84cqspt8eewcfox4amzyvlmdgy7jqvc3bevtulecnyjpztlondozzox5sje7tun6waw5xgaoilf1xfko56pk1zhyxzygmxjniodqqn3kmo1l2hlx0eckkkm1gjeia3v60vcb79g93lw/mnjatpxdmc4h7lruvz8vzbwfkzohkzvqel//capnemvhvwv5zvmppaqwuyrydwzfux9cz6tbppty6yiferdakffwht+u0dhymfpf5lv6xwpk1uaxhksxbzyvrrefp5cpmikbuhlcu0mz0kc0+g5lolqgudpcmmvh4zqlyg4qrwd6rfnutplrcseb+cao+c6o/t1h5jgmvcpkrvmhzafwzjomozatytsfbjt/nnrfkirbne0an7x1xhzcskoekdz2omvpc9yoza0+qsppk2ooru9ejjzhkydlzrgh6u6hjsheo3uwp6pcg+csti/7gce64oeqf1zo8bhbew6a94+qjq/nqmr5vdvmaygze/183c3ruafyngfx9/bg+3vjb7628znhzd/pvpo2h/w/rs2vtza+/vaz2tp147wxq6ft/53e/32+7d/d//39/9x/5/3n2ro7vsnzftrvz/7x/0fykp43q==</latexit> <latexit sha1_base64="pim2buniwj2/wyvvvvkikbweovg=">aaa0rxicnvtbc922ezz7tdvb0j72ha2kqrrlio7jpjlkpbpfvu1ohfw15dxesqosodyqedmanosz9lf0z/wv9km7am8hcycshm3yiobvf4vf7mjxuvaktmidg//e+8lpf/bzx/zynv+t//o3v/3d79997w9fi7zkix0v5knovw2ioanl6cvjzek/ltglazdqb4lrx9j+zzrywflsvc4kep6sognjfhijvzfv/mcrs4rrj596wc7eipyqbh3rzsw7qlzf79teiy/2ezqw24gu2htb5x3pf2v6wv09gtuxlz9azxg5rkcxrr1voj7vk+yby6vlt89zpje76yfb/psgsprxfsoitxcw9c66n+vzmqaur1++u3gwf6b+pptj1hxsrdu/j5fvfxjpj/kwtgkmw4qicty6kor5rbhkyulrdb8utcdhnynpgxxmjkxivflqq70tqim8cc7hxyy9vdulqegqxcinajksorfmg1a62s5kof7kvgjzuuqahbqjczl4mvdwlryicrh8soapenigsnowa5yeemzsvddnqrkysszvdawsdc7dpc36ttenxysf835twias8xzwrw3zkctimwf96nekfmtm9mspza/lrfkdmwyci7ewaqmz2i+m2mospuwnfx/xw3habxnuezmqdojnyjscbxocieecl6byrmmmcpvqtnnowz6d2pweoun7daimxiw2r4st3qviuutf9ofn6e5cwyonwi6soif5maqpjkehrqczihzgkjjokhewpzjala6iflycmkjumkml78poscknov9qwnc5zbgilxezmfmlrugzikervktymhlmir9nsvlbwhnkked9of4hfwmtshy+apgo+syros2jm5pnqzqmhapqpk+8lpmy1juf0jhllzpcdp8agg8ioabmewrmnkl8nxw0qksdmtbdbkuwyxh4thww/5hzctemjrw6qhtqn0srzidr2funbrizlqxev+ma70gj1pnxryzfmj76bhre+dp1ybqqickpjmrsaev4z97gynvb+/lk6xmjjrwxjii35hj8+plceozwcpifvtjuqzfwmpxno2ojszctrh3/7gzurbh15ohmvkl6fhvljzrl47o6ou6hvnpqiz/egefrpx4rvi9iwsbyyo0tqh/pgeom//j1cypenah0nfgo2qfs59hr/x/7ni6rjzhfqiwkwria7iwgv/jllc+wwglz73p+xkedcs/ncmhqk59ltkbyylbqq1pmfw4py8jcyb5dxc8nqu4qnvjlfgupi5baxpc15f1hmllgntmejttzk6vknuyrtfp0ewihayjrxf0jdoqbsckqeqgb50mei2geodcectfdbhffuopaf15u910i0fwhsmdciprbsqa4fo/gojobft87ymos4ydvwprwqgtd2fiiomyzw8xhegoep97x4ehj515bcso/tdumpgqrizfone9ygshnaqhs5rzi4pexzjq0lajxcoqqukz8hu/2wvvbm+8mqjgrbyzkiu3uqyqlzhfpgmfsyqlojishbmm4v07rohex9i1n+pvlxtdty6wjmb4nh+w+po7z/druy0ccegsw1lezearyrr54pf9a83h7vrvysdvygunwvibozmgzcmddtk8qh1qt0tqxpbveqjm8sfx4etso+lzae83pla32ejgj6wuhbhcu7rhx3ubgahpati++on5+sjttt2rnzkkmgggzrhtrnrdmzfpqqudosnfcigl3qpc0nxrnz2krtxywyacilcsctbaom5apgvwipc0gkkym6hckoax1xqmd9dpuefbttczpryewtqseznieqq6oasbmt1guswkoh6aosyeaadfpoavakwywrxiqc7kgeen8ybd7aqtwcdzmxmhx2fgxqrh0qqwuvlgw93kahwb5yekugaxodvmnjvtbj0vibld5syylewizxyqvnekog1z7o/0rts0kblcqo/8ffzjohbmaxkmger1ptlci+gl/y9ziot/mcytkys0ykkwja2gpd20hhr1oagk+hkjjxvlxbiobnecmtbya1xy1ar5ynupi3gsvcqozlrtmlnwk7uyqlrrw0wbedz2qpnpk2hlu5xobdzvtqlzsm69hagbrsb+diwwclegv2xqwsqtwq/w6kzcpliwwxebtw/7z794kgzhqe6lxhbu4vct2gpkk+tafuyuug91qrtatfcqkgiyt3eynekv8f5af7g+arju0lnquctsvulikoibewui+eh70cv2rubmqmi6mmep/u0gdxttrmojaf8bqerswsrdwl1gye+kcigbo0jh07txn6yqyn/yg5ledffcucikswamexnrnscyp4e0l+85ywcdi+auvsrqql9onhrgymy0b4sscjjldbxosjjmxfqozvvax1wevjsffdb00xt3v7orodmdfbq6iremoulizazzdd2i/cztga7qd8lvawvfw/3dzlawezwstg4wm8kzvplpmwqlygpktowbksyyz2vpg5syognkkbbrzm9vi+ljtyj4ngjntlo6bnw76orm87iquwz8oe6mbpuyqg+o6daygag+qetwopcup5n9g3hrdbta3vzd2kfd6tlhp11+aoq1aricageilnofxpt3rk/cva3xljgnpgjrhfbkr/w6lxekstpncibajsgyil9xngjq1czrx65ja3bjjrgm2vdecjqseslquhintb3m4dfgyxzjpx055agia7rgm7gomcs95k3zuexguql18vgqgc/11dqdwvd57cg4kmirxrbxxqhgpauy5c5emklpjjmat0xndh4oejjnkgy6lhspibmo4hzu9hsxagbpg9hikhsncjqh21utq25vxyj5uslpdchciprco8luavaaxnoirg0vugsfcaftqio5egayr1jbglkyn1jv4ky53mq48x1cavx30ndrq6obj0tqaweq6japrkrxbsqalm5qkhuqsd1xsbai2spkmktypkraaxszwtpbitds7svis7oz0evth6ys69gwnwowcomlnud/llwbanssaysnpajd0376m6cvh2qqs2gwtisg7qmfvxvl6qppaw8goqunim12zdmpigpdq1h2bb/6xsnc8b6qgbgwcim2ezpdqqha4kysrdsno7ubc6lufrfpayisaugadgcsq0ridsdx6hyfmwctc5qgsjc4iqn/ukms+2xjpfpncxcmtsreyqqj13nw5hcccuuzyzlqlbfmcgxrtoxbmhxypibwi2unijyb5e0lx1hy0vvdwunfrcirxcdqj6prre22aypsqofckygyyvgacrvhzarldfjbrkitxwsoyxfc3yoajv75ou8lnwpip1uwce9fzyky6ocmezckrjamsbq56yswfpznphoupkuts0pnc5cmniqoe3p2uk83bld7inqiddx2npxctuziloyzf2vfaenubeqzdrlgtqbcw7bzyl2qnnsfubbw+bmqf6zlqwt5rofd20xyz3j1m2owqe+tohg/jgsepv9uafk8jmeoxg8c+jqypun1a91pfczlj0vkgzmc4gh8xqx14yr9ic/szvttqlmt6udluqdac2p20xmtyzavlaoigsz9hnss2oiibqhdccjutwigqog2zdf76pbt8n9jb9zjnascwhnvqevrtwtk8yskuqcx6exhgbk13uicyli4djohwmyharw4elias2itflrci+nb7jfvcbugqerio1esqn2ai5bqznsece1yooojb44hzpuq3wyf/sq60ze1mnnu3ygu+xpfnar3cem6mehve9uwesiznvdkuczy1b0g2r5sbbuke+t6ngkww5/tmtebcihmokejr16tqmahndl3alg+i30dvt3buyocswecjnn8bwurutcoovutcml5lpcazluhvwdn+jhi9zrgjiiakhczu8zcsvqywrja28twzj0orxlnp3p9za/euzwkgazfqgompzrvu0xygzckvjtzq+25c4b1qzvfbhs5r1chrd8t80f/bladvftsmc6yxwkyezkoqoq1b11alk40l80r5qs45jnwpzmheqqvmgz7oz8znz3ccj27jxfgqumndwaysgbqfwqqaa/sf+ncpzb0xsyrfo/i+8fha5fxiiump6vl3kcb7cfu+rq+y33ijtrjlouanvfoxjdcv8+bqrgfl7gdh1r3rgyzzxrdmkhwaysdo8nppt9hgd8sljup9ihoydso+48yoagdndriawe4bysgviv/mzulva0sfii1zyfe/leycffukhcx6ep+jmk09+nnyjuyfdiwzdgp1lhctmu7dz9vrxfftt/ukuz0tplxhydc0qkecercwqdiv3a9c925++om38t/iyefwcnq1tnhb863axojbespzqhw4ht7mik7/kjw1u1nenm5sp2sbu/v2ttajscf3hl/r6mchn0/gmin7u1c3pipblmcbgiz+w18oj3qmrnadj3fgaxapcw52bixcs6rvviqthwdxqkyyuit5hbs+rxdxvf2c3cvhi6dh1aavgybxd3pxoqz4qizhlzjqouyrgjl++ffsatibo7hdyhuztcehgo1wkqtnlcr7ezxb2ftblk1f+isg0ek/gin8z4golcoelisr1zlsvqo0iouku26xqjsi/sk5fsji3mkxfdpnxpa4ptelxbkhbmgf4yzfphqhx0jwuysg1w+bc3xkwfc8dsa1d8f1dw/qpisnyq4eiivmawa+tjdrdqybagnz/3c3+aejliqick5p1fyuq9pm40xq9y7mvacref7kqi1p9vltev6ddgp78ddnovg7x3taj1d1pwhc8dqpht1x5r4ek3b+3cg80bu9ulvmdobp2sp117y9j+e8hjyirprfn8ewgda/8y9mruafyngfxz/yhx3sj/71copzt5o/n3ln7u9rf1nbxhut/w3t87vnaydrr9bctf/d8+7t3hv/4qcpxz30h15o6e/untr/xov9piz/dws7ovw=</latexit> <latexit sha1_base64="wu5pkq/f1oibhpdr2sfc7aj8t44=">aaa0t3icnvt7c9y2ezf7tnvx0v7zf9hkmtqxpoocp03b8uwu27u7cvtvj7xewqosob4kkqab8hrnhv0o/v79nt0feecsacu5mrffal9dlba7i8vduzkxqq4o/nfrbz/80y9/8tp3frb581/88le/fv+d33wpesvi+irmgrdfr0tsjbx0lwiqo1+xgpi8yuhx0cvdbp9qtovkvhipliu9zulascmliykqs/f/u1nk5estvwufz3izz0ixufp2jl2ub7+50wqpgjtm6ftdjkzv3gqq7wahrpkz+vzbphldhxl0lpczghqbjvgk6lmzxr49o199h4klm3vnc+f4djinsaoxtzizjdcfzxnnc1w9gra8qpkiis2z97co9g/0t+b+tnqpry325/jsg49uhqmpq5wwks6ilcetg1kd1kqofmcuefeslis+ick9gc+c5fse1lqrtbadnukw5ql+fsrqtx2kmurslvmikdlrm2m3yawv7ars009oa1aulajfbdqavlmgeiczeirmwnizjxyqwdcqnyc5ecrwmhebg25kkshcfxwwlfrrhyp5xg5ru0hkgysxw9q8yhqt/hjyg/m5k1lfi2h1nae+vaggtrtmhwqiqmuzbi7kvfq1ysvamow6yirm7c39cddyc8bvfe8hlydoljiroqvbdcqqgzhbrnulpyumhko754kczuenpqlqlg/rzbvnwwqpfid8ucw4qsuogsfownjyi03ajrkuw/zm8nvw0cor00sil2aqooiycta9mnoijyisrhwqym4yq5uywk5ipwzclckruwlmcmqriiqyrzkypexc0aheuhwypstlpwpvh2ctgtkuoh5qr1tljnyfhy9apqj/hjtoipssfnmqpuchqtvpk28qjmdnhuy0zuwtjxcdqqrhcbfqakyzbxoa1ageolo29qhnm36djrhkcfhmfbd/sd0kcscnajpobtcqjini4yjsqoecjnli1mzqcyb35atqwosyxw8yt3myoa1d4zowdxwuvrrhnq2mcoktygss7o19fvzktnxgmktb5dtyotp8ceoiqwqfyq+fb5vwsatm+vbji6uztzneffv0alrcefxo8dwosvf46poeazgntf34aid64okjfpzibg3m+gvcpyd5cqulm04ifyidx+3wxzskbr1ta+gb8fg0q63ox2/2/xmkoq23jg6rqiwje3q6ixt8klspeo3vwshmnwgtme9vbkhqqftb/llwmsigi1vx1tebcstzexopbgw1v5bitavz4jdffkrmwo4s3zwud8fjsjw6zhugbm9plpnfzhoadelmbeicede75hmyncnjhvulqqpnwyklic/qg6km2c6cudkrzgm+x9d3fqjqxa+yz0mswscyprl86+ggm1sw33ri40rgipxclne6bmpzcri6zqxdheggu8hz4ojw8ogzocqlfx9z1qymbculnwit+tibgys8brzqaxhw2zpmtfqrgctetv2e1whl9/yy+c5y3bmbiimhdgkvu9omksyzq2qixkmvikcriouqdonco0wl3l3rtzyrnmc1wwswcczk/i4ciruyrbz3c1unpxfifmfs383is8dudmdtxrkap7cxu8s7bhtbmtm6fmsllaxt4aypdnxlmkplnnumzzdyaoe3qr4+68yb307ropmzttubalnks+cads6mpsi+2n6abi+y9umvjp4dr0z7hdnzc1livkcmaln0562qlx0oowsvpowg5ux4iujatbq3m7ksoj5ysabekrlpogiubcicqc4r0w4dftzgdjtsgtlm9t0l9cbtepbttwz5tyeotusrx3krqa6oasbmt1iukwkoh6goijeaalfpbavajwywnxhqcrkwkzj8ybl7eqfwcdzlqqpp3foxrnh0qswuvjgw92loy4c8yglolayvtvmvjgzbjyribtd5syelgwizxzqvneoog9z7k/0jzr0kbldqq/8llzjoxdmax0agerftt1cm+wl/u9ziot/yjcwvdbymreaegbbhh64dw140gwqfq6g1lul4g5fnzsbgogunbj5fn5/7mifxbrnxjthzlguzfz6n9io1fi34yxy+dd2qpgvkuhnu5ey2gjbb1bny34owqr1ez8mozo4juqj0zbvapj5ya9wbtly2ua7ceru17/jnshsndojabyhzc1zgsq3yauyy+msfri+6hnbrtdpo8peoyhi2cbs1qqzz/1ewsl9pu+7rotjryg3xqmwyhnp4z6h77nnjv+hfrzp10xyxg9h/uw36lkzd1zqeurbwj9sykkwzvqsk2rdvfafwhszkcuchsy+u+bfxmd8fptj3urdpde4njm4+tzfbaw9b2deow3pb2l+dvnkhpdm+mcavuwzzgf/mycw8ojbgoencdsnswdnexkru2gsv64uvke7pzl93tgmur3najbpzwownhbdtukf7jd8bw6drgn80zoqh+7+roywghcobxwfkglet4qljdl6q18i8ybejw2gzy9yxmj92xmjwynvinrpgnstarvz0xmhcd/em/owoh3rb467og/zlcw/0w8cdclrdlz0rqn4d5aq3ck9ijvmfltelkbsmze+tg+s1njm20pxntmhrebugrgbsoe1hfo7o+hp87ggf22pcs4becqx3pd+dlcui02ngpewwumqheok/0zcpndej31qe4syunxqwqboqnfv3gnvn/ubrfyfabujaalye5poqkbaaci9lzr+xovdctgnntodnajtg0xudxc3x+q1yiebk3qmvwgjgr+ute5p7sgsqwlxihjxtjimejc6a4rgujmlujxsszbiwnxvdnazcmsxhxtaycsegzbiwegywryasxq+8evokkju9kpsqmbn1gk/16cvn8urc3y2ieiwweticqkwn0mkitjyxomqdpa7fw3o5ixhg+ylwro5jqnfor1dpjthrzfhsdvstktqvsrjyxkuk5vgsuqmsrho1vzkylerplyzvddk2cozhasbesczsfwdkjqb6tif1pwdnwhixwbm/hppl7gajaljrbeu8oh3sffcyyqgcz5nkyjdnxotsc4w99wuzcgkdxsf7pb4h7xtnoaghgujiw/clhvin0l3zpkgggjaigdr7iu+hun1w11yipybr2y281rdkv+erk1e0omg5mmh0pxu7krv0cwvniotf7velgf0u1f9sdnz8sjuxkfkpvvjm1wax1uxnukt3o4afe5kuvbuwc/yfbim7dizyo+wbqhoqaobqbghz9ojirs1ofbngcwpqekjreois6foubbrpgtfdcrguwmwyqmi6jqilyjg6im6jvvgtg1viiafvwewtf3prppobubjzaoo4j76zwuj3coi92fnwkhjl/vz12ik/tzjphevpku9s0pvc8jymxappbn7klxnwi3e4derhfudjz/xelbqzh/f7o4qi6hqzz6dljpusldkx3q7rk+zyjzjrgdfezd1zpq9kunsnd8o9mm+xu+mzo1t37xzn42e8mctvelsz7ckdu+lj8njhxgwbulvnxqvsqdy5wsqy+vhh8nnyvqmv3qnpgt6zjgbghwwfyqy9cezat5ooi5z6uxrmaneygflgxhxredlahedgqooqhlel7zqd89dh/ed7ya2+r1zkvvhw5kplmrchq/ckjvseetmiwjfe1xqpc4jp49jjgggfewjc+kzgbckcu1guvepat9kvurd2y1frs2wpx4ketjcvomiopehcc0lbwdaed+z2ybddoflptabtb2ds7t6klfh4rbwp9wpluduzbbc7lxm6mbt1snmmwng+idm+aae4jxvmu4jklscem7azaqwjtdxgikdds3raole59gkl3ka/kd6v7dzdwcedxudrfjce9d1rj1gj17loes14bgzrkd6bp/ti5glomeyktjyzwernyvh9vq9yrwzjs+yehyk9tp5f+pswhlzy00dymqmamx/ba9y2pudfcipbeft3zytaoyau8eggywv0iet3q3wx/m6wb29+uwxzapzatitso5afjtvx0cunjswlthuqytms1y8uyms6q/yznslnpm1nnwjnfsl8wbdsyh3bpiqbgp9a6oad+wx4mi9lgynlfewjcv4uslzkqmgkgejp0ver1vsj975tcfhccyo2voy65q2783eksk/zfvpez8fuz2nxvtubjntdl0xqddphic3w2s+0uez3zqunq/mie7j1k77jzoho4mx2vj7b/hvjhfeql4sqo/qbzqirliyxaj5k9sghfhyoev15/+m4bwl42qkel0roggcyxhqsiey1/yef69e6/qef+kmmfizmhcbcrmnsgdgkynfbsd/4x4xuxf30e2/jv8etz53v9bqzslf2w7e9alcmtloa6na7ffgkgv5rbnqmct6eztzoyrj3991rbzkqeh+xj025crxm82stowqwe/vdehhmetwgablx9yxy6gfqgtfzga82yumuc7azkofeuq/mfwz5bigdmlwrxsxrpa1ds4ib7rfsxg6fp3lcb4egarj3ohcjypqvzhlegqldwlrkkjmp7ywlib2yg+ulvjljhbii2ghlxd9r0d6f4dnkdti1bgvcrzsgfvoftog/snqsdnu6mkxf5yhv69iovpdiospq0ij2s+ziesnpb19vgdd5ttkujs9xsyqsbh+gg371yij+ccfnrxjuvg3m+psxquuagwvuguvuhjrdsj6kqj9e1pgdwpzzwsqxkqsqfwt/o932n4myeioguoz1+7upx7yfj5lqdzsna1gpemmlxtkak7bytd7u1q7gmrccrf0se+ch67obnkn47dtbnunyaijjuljoyz6biggow3n7wedd5azv3j7ecj5chthu7e+rybgm29/4rzmt+09k3i8v7+1pdvyn/76/9ekn7z/pvlfxu40/bnzemgz8zeptjacbxxuvnujbp7r14a2pbt2//9f7z/en92cg+onblc1vnwy/99/8h25tpg4=</latexit> Noisy Channel Model for TranslaDng French (x) to English (y) from Noah Smith
Modeling for the Noisy Channel we need to model two probability distribudons: should favor fluent transladons should favor accurate/faithful transladons <latexit sha1_base64="pw8oji3ujegpi9vjlgbxuppo7ly=">aaa0ohicnvvrct22ezbtw6rekuzn2xk2kqzylkk6jpnmkvfmffu1o3euxzzzsshrqbkhbxjvbsbzmcp+7tp2ifoo3qv4dkkaloroxhybfltylhyxi4ucimfc7u//99zbp/v5l375q7d/vf6b3/7u9394590/fivykof0ezgnof8+iiimlkppjzmj/b7glkrbqr8llh9g+3dtygxlsxo5kohzsukmjvlijfsdv/ofrswprp986mu5e4twqrl1rdfn7gw1/ep27d33yj+hy7kd6krdevte8xxrpufvxf1r/blye+gsiucvnna19x3us3qffx1+sfz2oysn8vb68by/pagsfvxtdt/lszinkf8/f2djf29f/xj2x6j52fhrfo7p3/3wlh/lyznstiyjeej0tf/is4pwycke1ut+kwhbwkss01p4zehkxvmlnfz7w1ateeocw79meqq2s1grvihfggayjxiizdasdlwdlnl8yvnfsqkunat1r+my8wtuofq9iheyflkadxjybrj6ohroqgmttn7rpiajtfkw94zsstqxyz4w/dqyk2lcwnm/ni0tyxg+69eg+zrlscyzfvu4bb5ljvu1foahcikpwrkgzsvyglxrde0wru0xsvp2mn6w46044lcm97wzcdrxjmrkweg8acqgjhdqoam0u4r9dac5p2dcqw1ogpyk/tprbmmwm6pfce9v8vyiyvvau/qgoa2rrmbhszzd/ezsu8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkoe7hoyixmuidjipkqkgwctqcj5reza8imu/m1n5yk0jmhsgflcvkiuevlkox7s4x33m1dcw0rnnpibkhg5vndmnxpvmtorkd2jmskpnlnp8apegevaapgkfc5puvpo6wttvpk3rbomimleifkba3/vibiuakxiohdq9qe+isobdkoyqhgkmwutwr6txghekbor8yylgbxhpfto6q3ztojanvzcufec19rryvfnvy+tt7n55/ois0caks0beg3i5onh2zgldoaxjd55u6jhiomz65ugr0dnrcajepl4afvfepty8glwqhp9f0ypn0riudo96qk8eweinmubqwo9fhnvtkhawgljjhndhc+c46t97vckgn00afqu+inao1hn4au/fpqfjamnkssqlgjcupjulwa98gcuxwk2erhc8p+lttoxncwu0tzxpmicjbdbytvutsz63lgvhlmspoelneyf3fbv8couclbflbei7mvlombkpy5tsv5ptuslsmk/zicz9ugmbkade9y7+bab1wfhhvynqwpmkwsuwt9abgosylok4iilfrt9fvndccnbcb7jrqikawbkmulxwcfdndysxdvkw5dhittcllsrbcla8ii5zxjdzed6y56l3dhdw4ilxkilyt201gynbymimwqc4vmdierocwzmhopjinfmtlkxgckfdfeev3/ddxng/evpba1bkziehf2k36ygsbboinmewqeqenjlfeijhncunana7s/rgjvz6vgky6wxwhezfhen2b7pf7e5qqrejdgkcworbwtwb7fyppj4vohm8pd9z3lbb2ilp1r4gtmtbmqpnxbsvkoduldlumt073ki5ven14ek7dvi2wnvnt5st9hcyi+sfb2xwru0r8d7mxmhzwlqpvzp6crw07tdkz85jjhobmax00z63tmxauelntkjrqoojd0kqtj10z2dikbv8sgaaijqknlqqkjuqkyfciqqtbipmjoh3jdml9cu7buhv3clg22hn0o5olgsxpjzlpiichnnmzj+wkjyeud9auwzdnnbi0nakgfymlfrxvodydsls+znh99mw4bbszliq47cjllvj6ymykkywlo7zna8b8ozfkteynoimtkzotx+vkn2u8pimsyyemo5vkjvfhqetdkd7i5pasnhuvub/cyownqknnlyedpry05ygrphf/se4kun/zbmwlcjamcszegbchh3ydgx70qqsfayfxrqk4g1gnjmbggktnkr5ytv84wqgxlnjxhvgz1s2zfz4fnqnvfo04ccnvb96ih3mlluzqmr1nho221g52gdew9qmopn+bxplbcmrkrl1ygmxribqdvmuuyglyymn2rf8s9+9fqzxoczerzk2cklw7dqksfw9c6mwxqe71vptjvlifdamw7ingibw+f8gc9jfnf13ac3pkos2k6bksurnvlxqffhk8dn6v6nmvtrdtgpu/3adootp1jufgs6m1sm2lgrhrs+xzcbeouuankax6dy5z9ijzw7sfcwvbikuxbrewintcym1z5omu8cbfvad5bbokpjxk5v0ic+1twxxztzbnssxorgzhw40wexylvsuzqzsyy8dkkycl9xl0hr3vboro9mbf9c5ibinoedixg7ydn2h/chtga3qdsafamvfw/3f1vewejarta4wmscbvvhymgcnyglktusaks2xzgzrgzkbo2bkk7bpzi9ti+tjtsn7pgbktrs4bn4y6ido8larugz/muynbviwqw6q68sxgqo9qepwo/cipjj/gxltdllb3/xe20fe6tlhpv1/ayq2areeamagldocxpf2rk/afw7whtkmpgvohffkrvs7lxijstpncifajcozilxwn2no1m7pxk9kanfjjbsn2frdcpqsesdqurqvtl/l4tbeyx7lph0f5qkialjhmrkpmco9503auexhuqd18vgsgcz118unwph69o6zkcaoxi+rjbuke0wzylm4zisctsyxnozo6/4yjctjva50lpuyfjcbwzws7desbsbqmllhipuyfmomsk8mgl6vvalzpilvhzkbedg9dycv1egfjffexf0iohimfglygkj0j8jgilsgmajkpawuxftxuylx4pmc0qzqo6/rvkdxt47w0crs0nvujzlkfuoywfyljolqkrypkq4ttvgsnjxktpw01fqytmkcr2ra3kncynvnpi+gok4n1kvna8bib8bjpop6ww4ba6pz0qhleua7phv2zurfodymlcaumwwj2ruke+vk0hpsbnpcdkgdqprpmnvqqxkehkbuczzwj4w95jlqntrgeabpdmw+i0k1w11zixayrms34lrfkmydb1ze0yac68aguzxg7usq0u8nnmsifjzhlwb0qvd9q1zz8+ngskfiz+zkmzs9xvqvreouzu0afsxlknfwwiyyfxikanuxyo6wewdhqmgeqmoezntdire2opgfh8enxkmgjeoggvgnxjto0wsjh8y5vwazw6ay4ztcz0rl6lkwivzzuqpwksgguguzt/z1rztkbskst6ku4py4zgyz1ckp9mbowsfjkhrz1wmr/jzksedyt5vcl5lo5czpauwc8otmp1xpdm7xbrdbhbzraoyjm4islmlh/m6oaslb3swz6ckknusfsgy3xboko7yjpirg9xft+1zpgrl2xsobso/mm+xuemlov91br/n4ga8msbna1vr7ssbuuhg89jexfydq1rrxklqqsy6wmmq+xhh8ufjtwav7qjqjz6yigxyh04escqeceddupouixm4ulrnanez6ppjctter2ucf40aobbeouqvtm63z14fd4pvqdr5hou87sedivpfqbw+w4rvlpghi1ovam2jrvcmfxhrx6tbbhisjbg51i7aeycfellk8rcc33i/zf3zdv9gtrafegvg0eywnykg54dy1qsf+3x73zfzrv906+y91og1uz+lcvhur8pgkbflr5zzwy9tbilyvj1qyaeqvpoyzrhldsnnzdhat98x3obusgj2nz2ycehkzciguj7omvy9bbxlo0mz5e/0ger+2cwcdlwwgh6x53hbwd68drrva54lxnemxma1eegoe9ioryyndgbexusrkoz6hyfv5tws1ti1vmx0civtkeub+xxu4d2loa0mlhajmf2yucjvmacvlqwjkud83ewlvghkodzj0xqmowx9c5ov+j6y8eppbzphjvdriimqehcxp2lqgkllpljknyln1zmgsfnqoi1zdnc/wdd4zbjq7ethxs+bcgpaa6wo2zqhu+nrsbrzyl8chfizrjvkikrqv93eppuxojaaynpoudr9pvj+w79v6gpk1n2kla65jxts7h0sc8zpvrk50tsx+fmbda71hmtd1wyrspxkg4gyv/xslbxtxvna40c8yivs34jvojiog1myhqxnsvpgm8hlyzv4mvd2w9chskguw9vpjlthxr8broo/3pgrt2oofle9wxkdhommwvm8j0arppvxcvdz1p/1utzlvm2hkfrz2tamaxcebc5ukvdr9knt36qefebv/e558bi2nv8uevtxfqu0zukwnnavuh9vhgyzi9g/cuzov5c3pzhzkfri7b19rk4gu8vbwkxodoptmyvhzzg/36oz0mjjyoedqxk/rc+vrz9qvrumxdmza4nhowc6ghhtjtzqxsoxjia7ivazqma+q1vf1iq63x7b7oxj66lda9dudmo5h7lqufv+vjbwbezqharvq0h8/supnemuh10t5jzmpjesw0upy9axfex+czyubftuyqqgf0hba6fcwhv8x6fg57olcsnqdi2wlsonytrllfqtkg9gvksuhjmqtffmh3eqpiwmapculeuoq/blu+ox9efolbde1yld9tjlxp4x1xemaxhmhxhdnv+5d0ijm6igywuwd5mmle13gekiwkl0pd5p/ospckilgmlbn77o6at5obvxvam56silnrs7uklafnpvn1xgntgcpc56dtfnfb/xgvdedjhsvnvrye1xuqtaj55cdzfnd0qo1a24h+lctxhfn2wcyzmpii+qq+x0vdinh8xv/agzk/omm/fht3b3r/t7om3sbn3/s/pnm22t/wnt/bxtttpaptc/xhq8drz1fc9f+d+u9w3++9zd77997fo/re99o6fu3gpr31no/9178h/jgm6y=</latexit> <latexit sha1_base64="pim2buniwj2/wyvvvvkikbweovg=">aaa0rxicnvtbc922ezz7tdvb0j72ha2kqrrlio7jpjlkpbpfvu1ohfw15dxesqosodyqedmanosz9lf0z/wv9km7am8hcycshm3yiobvf4vf7mjxuvaktmidg//e+8lpf/bzx/zynv+t//o3v/3d79997w9fi7zkix0v5knovw2ioanl6cvjzek/ltglazdqb4lrx9j+zzrywflsvc4kep6sognjfhijvzfv/mcrs4rrj596wc7eipyqbh3rzsw7qlzf79teiy/2ezqw24gu2htb5x3pf2v6wv09gtuxlz9azxg5rkcxrr1voj7vk+yby6vlt89zpje76yfb/psgsprxfsoitxcw9c66n+vzmqaur1++u3gwf6b+pptj1hxsrdu/j5fvfxjpj/kwtgkmw4qicty6kor5rbhkyulrdb8utcdhnynpgxxmjkxivflqq70tqim8cc7hxyy9vdulqegqxcinajksorfmg1a62s5kof7kvgjzuuqahbqjczl4mvdwlryicrh8soapenigsnowa5yeemzsvddnqrkysszvdawsdc7dpc36ttenxysf835twias8xzwrw3zkctimwf96nekfmtm9mspza/lrfkdmwyci7ewaqmz2i+m2mospuwnfx/xw3habxnuezmqdojnyjscbxocieecl6byrmmmcpvqtnnowz6d2pweoun7daimxiw2r4st3qviuutf9ofn6e5cwyonwi6soif5maqpjkehrqczihzgkjjokhewpzjala6iflycmkjumkml78poscknov9qwnc5zbgilxezmfmlrugzikervktymhlmir9nsvlbwhnkked9of4hfwmtshy+apgo+syros2jm5pnqzqmhapqpk+8lpmy1juf0jhllzpcdp8agg8ioabmewrmnkl8nxw0qksdmtbdbkuwyxh4thww/5hzctemjrw6qhtqn0srzidr2funbrizlqxev+ma70gj1pnxryzfmj76bhre+dp1ybqqickpjmrsaev4z97gynvb+/lk6xmjjrwxjii35hj8+plceozwcpifvtjuqzfwmpxno2ojszctrh3/7gzurbh15ohmvkl6fhvljzrl47o6ou6hvnpqiz/egefrpx4rvi9iwsbyyo0tqh/pgeom//j1cypenah0nfgo2qfs59hr/x/7ni6rjzhfqiwkwria7iwgv/jllc+wwglz73p+xkedcs/ncmhqk59ltkbyylbqq1pmfw4py8jcyb5dxc8nqu4qnvjlfgupi5baxpc15f1hmllgntmejttzk6vknuyrtfp0ewihayjrxf0jdoqbsckqeqgb50mei2geodcectfdbhffuopaf15u910i0fwhsmdciprbsqa4fo/gojobft87ymos4ydvwprwqgtd2fiiomyzw8xhegoep97x4ehj515bcso/tdumpgqrizfone9ygshnaqhs5rzi4pexzjq0lajxcoqqukz8hu/2wvvbm+8mqjgrbyzkiu3uqyqlzhfpgmfsyqlojishbmm4v07rohex9i1n+pvlxtdty6wjmb4nh+w+po7z/druy0ccegsw1lezearyrr54pf9a83h7vrvysdvygunwvibozmgzcmddtk8qh1qt0tqxpbveqjm8sfx4etso+lzae83pla32ejgj6wuhbhcu7rhx3ubgahpati++on5+sjttt2rnzkkmgggzrhtrnrdmzfpqqudosnfcigl3qpc0nxrnz2krtxywyacilcsctbaom5apgvwipc0gkkym6hckoax1xqmd9dpuefbttczpryewtqseznieqq6oasbmt1guswkoh6aosyeaadfpoavakwywrxiqc7kgeen8ybd7aqtwcdzmxmhx2fgxqrh0qqwuvlgw93kahwb5yekugaxodvmnjvtbj0vibld5syylewizxyqvnekog1z7o/0rts0kblcqo/8ffzjohbmaxkmger1ptlci+gl/y9ziot/mcytkys0ykkwja2gpd20hhr1oagk+hkjjxvlxbiobnecmtbya1xy1ar5ynupi3gsvcqozlrtmlnwk7uyqlrrw0wbedz2qpnpk2hlu5xobdzvtqlzsm69hagbrsb+diwwclegv2xqwsqtwq/w6kzcpliwwxebtw/7z794kgzhqe6lxhbu4vct2gpkk+tafuyuug91qrtatfcqkgiyt3eynekv8f5af7g+arju0lnquctsvulikoibewui+eh70cv2rubmqmi6mmep/u0gdxttrmojaf8bqerswsrdwl1gye+kcigbo0jh07txn6yqyn/yg5ledffcucikswamexnrnscyp4e0l+85ywcdi+auvsrqql9onhrgymy0b4sscjjldbxosjjmxfqozvvax1wevjsffdb00xt3v7orodmdfbq6iremoulizazzdd2i/cztga7qd8lvawvfw/3dzlawezwstg4wm8kzvplpmwqlygpktowbksyyz2vpg5syognkkbbrzm9vi+ljtyj4ngjntlo6bnw76orm87iquwz8oe6mbpuyqg+o6daygag+qetwopcup5n9g3hrdbta3vzd2kfd6tlhp11+aoq1aricageilnofxpt3rk/cva3xljgnpgjrhfbkr/w6lxekstpncibajsgyil9xngjq1czrx65ja3bjjrgm2vdecjqseslquhintb3m4dfgyxzjpx055agia7rgm7gomcs95k3zuexguql18vgqgc/11dqdwvd57cg4kmirxrbxxqhgpauy5c5emklpjjmat0xndh4oejjnkgy6lhspibmo4hzu9hsxagbpg9hikhsncjqh21utq25vxyj5uslpdchciprco8luavaaxnoirg0vugsfcaftqio5egayr1jbglkyn1jv4ky53mq48x1cavx30ndrq6obj0tqaweq6japrkrxbsqalm5qkhuqsd1xsbai2spkmktypkraaxszwtpbitds7svis7oz0evth6ys69gwnwowcomlnud/llwbanssaysnpajd0376m6cvh2qqs2gwtisg7qmfvxvl6qppaw8goqunim12zdmpigpdq1h2bb/6xsnc8b6qgbgwcim2ezpdqqha4kysrdsno7ubc6lufrfpayisaugadgcsq0ridsdx6hyfmwctc5qgsjc4iqn/ukms+2xjpfpncxcmtsreyqqj13nw5hcccuuzyzlqlbfmcgxrtoxbmhxypibwi2unijyb5e0lx1hy0vvdwunfrcirxcdqj6prre22aypsqofckygyyvgacrvhzarldfjbrkitxwsoyxfc3yoajv75ou8lnwpip1uwce9fzyky6ocmezckrjamsbq56yswfpznphoupkuts0pnc5cmniqoe3p2uk83bld7inqiddx2npxctuziloyzf2vfaenubeqzdrlgtqbcw7bzyl2qnnsfubbw+bmqf6zlqwt5rofd20xyz3j1m2owqe+tohg/jgsepv9uafk8jmeoxg8c+jqypun1a91pfczlj0vkgzmc4gh8xqx14yr9ic/szvttqlmt6udluqdac2p20xmtyzavlaoigsz9hnss2oiibqhdccjutwigqog2zdf76pbt8n9jb9zjnascwhnvqevrtwtk8yskuqcx6exhgbk13uicyli4djohwmyharw4elias2itflrci+nb7jfvcbugqerio1esqn2ai5bqznsece1yooojb44hzpuq3wyf/sq60ze1mnnu3ygu+xpfnar3cem6mehve9uwesiznvdkuczy1b0g2r5sbbuke+t6ngkww5/tmtebcihmokejr16tqmahndl3alg+i30dvt3buyocswecjnn8bwurutcoovutcml5lpcazluhvwdn+jhi9zrgjiiakhczu8zcsvqywrja28twzj0orxlnp3p9za/euzwkgazfqgompzrvu0xygzckvjtzq+25c4b1qzvfbhs5r1chrd8t80f/bladvftsmc6yxwkyezkoqoq1b11alk40l80r5qs45jnwpzmheqqvmgz7oz8znz3ccj27jxfgqumndwaysgbqfwqqaa/sf+ncpzb0xsyrfo/i+8fha5fxiiump6vl3kcb7cfu+rq+y33ijtrjlouanvfoxjdcv8+bqrgfl7gdh1r3rgyzzxrdmkhwaysdo8nppt9hgd8sljup9ihoydso+48yoagdndriawe4bysgviv/mzulva0sfii1zyfe/leycffukhcx6ep+jmk09+nnyjuyfdiwzdgp1lhctmu7dz9vrxfftt/ukuz0tplxhydc0qkecercwqdiv3a9c925++om38t/iyefwcnq1tnhb863axojbespzqhw4ht7mik7/kjw1u1nenm5sp2sbu/v2ttajscf3hl/r6mchn0/gmin7u1c3pipblmcbgiz+w18oj3qmrnadj3fgaxapcw52bixcs6rvviqthwdxqkyyuit5hbs+rxdxvf2c3cvhi6dh1aavgybxd3pxoqz4qizhlzjqouyrgjl++ffsatibo7hdyhuztcehgo1wkqtnlcr7ezxb2ftblk1f+isg0ek/gin8z4golcoelisr1zlsvqo0iouku26xqjsi/sk5fsji3mkxfdpnxpa4ptelxbkhbmgf4yzfphqhx0jwuysg1w+bc3xkwfc8dsa1d8f1dw/qpisnyq4eiivmawa+tjdrdqybagnz/3c3+aejliqick5p1fyuq9pm40xq9y7mvacref7kqi1p9vltev6ddgp78ddnovg7x3taj1d1pwhc8dqpht1x5r4ek3b+3cg80bu9ulvmdobp2sp117y9j+e8hjyirprfn8ewgda/8y9mruafyngfxz/yhx3sj/71copzt5o/n3ln7u9rf1nbxhut/w3t87vnaydrr9bctf/d8+7t3hv/4qcpxz30h15o6e/untr/xov9piz/dws7ovw=</latexit>
<latexit sha1_base64="pim2buniwj2/wyvvvvkikbweovg=">aaa0rxicnvtbc922ezz7tdvb0j72ha2kqrrlio7jpjlkpbpfvu1ohfw15dxesqosodyqedmanosz9lf0z/wv9km7am8hcycshm3yiobvf4vf7mjxuvaktmidg//e+8lpf/bzx/zynv+t//o3v/3d79997w9fi7zkix0v5knovw2ioanl6cvjzek/ltglazdqb4lrx9j+zzrywflsvc4kep6sognjfhijvzfv/mcrs4rrj596wc7eipyqbh3rzsw7qlzf79teiy/2ezqw24gu2htb5x3pf2v6wv09gtuxlz9azxg5rkcxrr1voj7vk+yby6vlt89zpje76yfb/psgsprxfsoitxcw9c66n+vzmqaur1++u3gwf6b+pptj1hxsrdu/j5fvfxjpj/kwtgkmw4qicty6kor5rbhkyulrdb8utcdhnynpgxxmjkxivflqq70tqim8cc7hxyy9vdulqegqxcinajksorfmg1a62s5kof7kvgjzuuqahbqjczl4mvdwlryicrh8soapenigsnowa5yeemzsvddnqrkysszvdawsdc7dpc36ttenxysf835twias8xzwrw3zkctimwf96nekfmtm9mspza/lrfkdmwyci7ewaqmz2i+m2mospuwnfx/xw3habxnuezmqdojnyjscbxocieecl6byrmmmcpvqtnnowz6d2pweoun7daimxiw2r4st3qviuutf9ofn6e5cwyonwi6soif5maqpjkehrqczihzgkjjokhewpzjala6iflycmkjumkml78poscknov9qwnc5zbgilxezmfmlrugzikervktymhlmir9nsvlbwhnkked9of4hfwmtshy+apgo+syros2jm5pnqzqmhapqpk+8lpmy1juf0jhllzpcdp8agg8ioabmewrmnkl8nxw0qksdmtbdbkuwyxh4thww/5hzctemjrw6qhtqn0srzidr2funbrizlqxev+ma70gj1pnxryzfmj76bhre+dp1ybqqickpjmrsaev4z97gynvb+/lk6xmjjrwxjii35hj8+plceozwcpifvtjuqzfwmpxno2ojszctrh3/7gzurbh15ohmvkl6fhvljzrl47o6ou6hvnpqiz/egefrpx4rvi9iwsbyyo0tqh/pgeom//j1cypenah0nfgo2qfs59hr/x/7ni6rjzhfqiwkwria7iwgv/jllc+wwglz73p+xkedcs/ncmhqk59ltkbyylbqq1pmfw4py8jcyb5dxc8nqu4qnvjlfgupi5baxpc15f1hmllgntmejttzk6vknuyrtfp0ewihayjrxf0jdoqbsckqeqgb50mei2geodcectfdbhffuopaf15u910i0fwhsmdciprbsqa4fo/gojobft87ymos4ydvwprwqgtd2fiiomyzw8xhegoep97x4ehj515bcso/tdumpgqrizfone9ygshnaqhs5rzi4pexzjq0lajxcoqqukz8hu/2wvvbm+8mqjgrbyzkiu3uqyqlzhfpgmfsyqlojishbmm4v07rohex9i1n+pvlxtdty6wjmb4nh+w+po7z/druy0ccegsw1lezearyrr54pf9a83h7vrvysdvygunwvibozmgzcmddtk8qh1qt0tqxpbveqjm8sfx4etso+lzae83pla32ejgj6wuhbhcu7rhx3ubgahpati++on5+sjttt2rnzkkmgggzrhtrnrdmzfpqqudosnfcigl3qpc0nxrnz2krtxywyacilcsctbaom5apgvwipc0gkkym6hckoax1xqmd9dpuefbttczpryewtqseznieqq6oasbmt1guswkoh6aosyeaadfpoavakwywrxiqc7kgeen8ybd7aqtwcdzmxmhx2fgxqrh0qqwuvlgw93kahwb5yekugaxodvmnjvtbj0vibld5syylewizxyqvnekog1z7o/0rts0kblcqo/8ffzjohbmaxkmger1ptlci+gl/y9ziot/mcytkys0ykkwja2gpd20hhr1oagk+hkjjxvlxbiobnecmtbya1xy1ar5ynupi3gsvcqozlrtmlnwk7uyqlrrw0wbedz2qpnpk2hlu5xobdzvtqlzsm69hagbrsb+diwwclegv2xqwsqtwq/w6kzcpliwwxebtw/7z794kgzhqe6lxhbu4vct2gpkk+tafuyuug91qrtatfcqkgiyt3eynekv8f5af7g+arju0lnquctsvulikoibewui+eh70cv2rubmqmi6mmep/u0gdxttrmojaf8bqerswsrdwl1gye+kcigbo0jh07txn6yqyn/yg5ledffcucikswamexnrnscyp4e0l+85ywcdi+auvsrqql9onhrgymy0b4sscjjldbxosjjmxfqozvvax1wevjsffdb00xt3v7orodmdfbq6iremoulizazzdd2i/cztga7qd8lvawvfw/3dzlawezwstg4wm8kzvplpmwqlygpktowbksyyz2vpg5syognkkbbrzm9vi+ljtyj4ngjntlo6bnw76orm87iquwz8oe6mbpuyqg+o6daygag+qetwopcup5n9g3hrdbta3vzd2kfd6tlhp11+aoq1aricageilnofxpt3rk/cva3xljgnpgjrhfbkr/w6lxekstpncibajsgyil9xngjq1czrx65ja3bjjrgm2vdecjqseslquhintb3m4dfgyxzjpx055agia7rgm7gomcs95k3zuexguql18vgqgc/11dqdwvd57cg4kmirxrbxxqhgpauy5c5emklpjjmat0xndh4oejjnkgy6lhspibmo4hzu9hsxagbpg9hikhsncjqh21utq25vxyj5uslpdchciprco8luavaaxnoirg0vugsfcaftqio5egayr1jbglkyn1jv4ky53mq48x1cavx30ndrq6obj0tqaweq6japrkrxbsqalm5qkhuqsd1xsbai2spkmktypkraaxszwtpbitds7svis7oz0evth6ys69gwnwowcomlnud/llwbanssaysnpajd0376m6cvh2qqs2gwtisg7qmfvxvl6qppaw8goqunim12zdmpigpdq1h2bb/6xsnc8b6qgbgwcim2ezpdqqha4kysrdsno7ubc6lufrfpayisaugadgcsq0ridsdx6hyfmwctc5qgsjc4iqn/ukms+2xjpfpncxcmtsreyqqj13nw5hcccuuzyzlqlbfmcgxrtoxbmhxypibwi2unijyb5e0lx1hy0vvdwunfrcirxcdqj6prre22aypsqofckygyyvgacrvhzarldfjbrkitxwsoyxfc3yoajv75ou8lnwpip1uwce9fzyky6ocmezckrjamsbq56yswfpznphoupkuts0pnc5cmniqoe3p2uk83bld7inqiddx2npxctuziloyzf2vfaenubeqzdrlgtqbcw7bzyl2qnnsfubbw+bmqf6zlqwt5rofd20xyz3j1m2owqe+tohg/jgsepv9uafk8jmeoxg8c+jqypun1a91pfczlj0vkgzmc4gh8xqx14yr9ic/szvttqlmt6udluqdac2p20xmtyzavlaoigsz9hnss2oiibqhdccjutwigqog2zdf76pbt8n9jb9zjnascwhnvqevrtwtk8yskuqcx6exhgbk13uicyli4djohwmyharw4elias2itflrci+nb7jfvcbugqerio1esqn2ai5bqznsece1yooojb44hzpuq3wyf/sq60ze1mnnu3ygu+xpfnar3cem6mehve9uwesiznvdkuczy1b0g2r5sbbuke+t6ngkww5/tmtebcihmokejr16tqmahndl3alg+i30dvt3buyocswecjnn8bwurutcoovutcml5lpcazluhvwdn+jhi9zrgjiiakhczu8zcsvqywrja28twzj0orxlnp3p9za/euzwkgazfqgompzrvu0xygzckvjtzq+25c4b1qzvfbhs5r1chrd8t80f/bladvftsmc6yxwkyezkoqoq1b11alk40l80r5qs45jnwpzmheqqvmgz7oz8znz3ccj27jxfgqumndwaysgbqfwqqaa/sf+ncpzb0xsyrfo/i+8fha5fxiiump6vl3kcb7cfu+rq+y33ijtrjlouanvfoxjdcv8+bqrgfl7gdh1r3rgyzzxrdmkhwaysdo8nppt9hgd8sljup9ihoydso+48yoagdndriawe4bysgviv/mzulva0sfii1zyfe/leycffukhcx6ep+jmk09+nnyjuyfdiwzdgp1lhctmu7dz9vrxfftt/ukuz0tplxhydc0qkecercwqdiv3a9c925++om38t/iyefwcnq1tnhb863axojbespzqhw4ht7mik7/kjw1u1nenm5sp2sbu/v2ttajscf3hl/r6mchn0/gmin7u1c3pipblmcbgiz+w18oj3qmrnadj3fgaxapcw52bixcs6rvviqthwdxqkyyuit5hbs+rxdxvf2c3cvhi6dh1aavgybxd3pxoqz4qizhlzjqouyrgjl++ffsatibo7hdyhuztcehgo1wkqtnlcr7ezxb2ftblk1f+isg0ek/gin8z4golcoelisr1zlsvqo0iouku26xqjsi/sk5fsji3mkxfdpnxpa4ptelxbkhbmgf4yzfphqhx0jwuysg1w+bc3xkwfc8dsa1d8f1dw/qpisnyq4eiivmawa+tjdrdqybagnz/3c3+aejliqick5p1fyuq9pm40xq9y7mvacref7kqi1p9vltev6ddgp78ddnovg7x3taj1d1pwhc8dqpht1x5r4ek3b+3cg80bu9ulvmdobp2sp117y9j+e8hjyirprfn8ewgda/8y9mruafyngfxz/yhx3sj/71copzt5o/n3ln7u9rf1nbxhut/w3t87vnaydrr9bctf/d8+7t3hv/4qcpxz30h15o6e/untr/xov9piz/dws7ovw=</latexit> <latexit sha1_base64="pw8oji3ujegpi9vjlgbxuppo7ly=">aaa0ohicnvvrct22ezbtw6rekuzn2xk2kqzylkk6jpnmkvfmffu1o3euxzzzsshrqbkhbxjvbsbzmcp+7tp2ifoo3qv4dkkaloroxhybfltylhyxi4ucimfc7u//99zbp/v5l375q7d/vf6b3/7u9394590/fivykof0ezgnof8+iiimlkppjzmj/b7glkrbqr8llh9g+3dtygxlsxo5kohzsukmjvlijfsdv/ofrswprp986mu5e4twqrl1rdfn7gw1/ep27d33yj+hy7kd6krdevte8xxrpufvxf1r/blye+gsiucvnna19x3us3qffx1+sfz2oysn8vb68by/pagsfvxtdt/lszinkf8/f2djf29f/xj2x6j52fhrfo7p3/3wlh/lyznstiyjeej0tf/is4pwycke1ut+kwhbwkss01p4zehkxvmlnfz7w1ateeocw79meqq2s1grvihfggayjxiizdasdlwdlnl8yvnfsqkunat1r+my8wtuofq9iheyflkadxjybrj6ohroqgmttn7rpiajtfkw94zsstqxyz4w/dqyk2lcwnm/ni0tyxg+69eg+zrlscyzfvu4bb5ljvu1foahcikpwrkgzsvyglxrde0wru0xsvp2mn6w46044lcm97wzcdrxjmrkweg8acqgjhdqoam0u4r9dac5p2dcqw1ogpyk/tprbmmwm6pfce9v8vyiyvvau/qgoa2rrmbhszzd/ezsu8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkoe7hoyixmuidjipkqkgwctqcj5reza8imu/m1n5yk0jmhsgflcvkiuevlkox7s4x33m1dcw0rnnpibkhg5vndmnxpvmtorkd2jmskpnlnp8apegevaapgkfc5puvpo6wttvpk3rbomimleifkba3/vibiuakxiohdq9qe+isobdkoyqhgkmwutwr6txghekbor8yylgbxhpfto6q3ztojanvzcufec19rryvfnvy+tt7n55/ois0caks0beg3i5onh2zgldoaxjd55u6jhiomz65ugr0dnrcajepl4afvfepty8glwqhp9f0ypn0riudo96qk8eweinmubqwo9fhnvtkhawgljjhndhc+c46t97vckgn00afqu+inao1hn4au/fpqfjamnkssqlgjcupjulwa98gcuxwk2erhc8p+lttoxncwu0tzxpmicjbdbytvutsz63lgvhlmspoelneyf3fbv8couclbflbei7mvlombkpy5tsv5ptuslsmk/zicz9ugmbkade9y7+bab1wfhhvynqwpmkwsuwt9abgosylok4iilfrt9fvndccnbcb7jrqikawbkmulxwcfdndysxdvkw5dhittcllsrbcla8ii5zxjdzed6y56l3dhdw4ilxkilyt201gynbymimwqc4vmdierocwzmhopjinfmtlkxgckfdfeev3/ddxng/evpba1bkziehf2k36ygsbboinmewqeqenjlfeijhncunana7s/rgjvz6vgky6wxwhezfhen2b7pf7e5qqrejdgkcworbwtwb7fyppj4vohm8pd9z3lbb2ilp1r4gtmtbmqpnxbsvkoduldlumt073ki5ven14ek7dvi2wnvnt5st9hcyi+sfb2xwru0r8d7mxmhzwlqpvzp6crw07tdkz85jjhobmax00z63tmxauelntkjrqoojd0kqtj10z2dikbv8sgaaijqknlqqkjuqkyfciqqtbipmjoh3jdml9cu7buhv3clg22hn0o5olgsxpjzlpiichnnmzj+wkjyeud9auwzdnnbi0nakgfymlfrxvodydsls+znh99mw4bbszliq47cjllvj6ymykkywlo7zna8b8ozfkteynoimtkzotx+vkn2u8pimsyyemo5vkjvfhqetdkd7i5pasnhuvub/cyownqknnlyedpry05ygrphf/se4kun/zbmwlcjamcszegbchh3ydgx70qqsfayfxrqk4g1gnjmbggktnkr5ytv84wqgxlnjxhvgz1s2zfz4fnqnvfo04ccnvb96ih3mlluzqmr1nho221g52gdew9qmopn+bxplbcmrkrl1ygmxribqdvmuuyglyymn2rf8s9+9fqzxoczerzk2cklw7dqksfw9c6mwxqe71vptjvlifdamw7ingibw+f8gc9jfnf13ac3pkos2k6bksurnvlxqffhk8dn6v6nmvtrdtgpu/3adootp1jufgs6m1sm2lgrhrs+xzcbeouuankax6dy5z9ijzw7sfcwvbikuxbrewintcym1z5omu8cbfvad5bbokpjxk5v0ic+1twxxztzbnssxorgzhw40wexylvsuzqzsyy8dkkycl9xl0hr3vboro9mbf9c5ibinoedixg7ydn2h/chtga3qdsafamvfw/3f1vewejarta4wmscbvvhymgcnyglktusaks2xzgzrgzkbo2bkk7bpzi9ti+tjtsn7pgbktrs4bn4y6ido8larugz/muynbviwqw6q68sxgqo9qepwo/cipjj/gxltdllb3/xe20fe6tlhpv1/ayq2areeamagldocxpf2rk/afw7whtkmpgvohffkrvs7lxijstpncifajcozilxwn2no1m7pxk9kanfjjbsn2frdcpqsesdqurqvtl/l4tbeyx7lph0f5qkialjhmrkpmco9503auexhuqd18vgsgcz118unwph69o6zkcaoxi+rjbuke0wzylm4zisctsyxnozo6/4yjctjva50lpuyfjcbwzws7desbsbqmllhipuyfmomsk8mgl6vvalzpilvhzkbedg9dycv1egfjffexf0iohimfglygkj0j8jgilsgmajkpawuxftxuylx4pmc0qzqo6/rvkdxt47w0crs0nvujzlkfuoywfyljolqkrypkq4ttvgsnjxktpw01fqytmkcr2ra3kncynvnpi+gok4n1kvna8bib8bjpop6ww4ba6pz0qhleua7phv2zurfodymlcaumwwj2ruke+vk0hpsbnpcdkgdqprpmnvqqxkehkbuczzwj4w95jlqntrgeabpdmw+i0k1w11zixayrms34lrfkmydb1ze0yac68aguzxg7usq0u8nnmsifjzhlwb0qvd9q1zz8+ngskfiz+zkmzs9xvqvreouzu0afsxlknfwwiyyfxikanuxyo6wewdhqmgeqmoezntdire2opgfh8enxkmgjeoggvgnxjto0wsjh8y5vwazw6ay4ztcz0rl6lkwivzzuqpwksgguguzt/z1rztkbskst6ku4py4zgyz1ckp9mbowsfjkhrz1wmr/jzksedyt5vcl5lo5czpauwc8otmp1xpdm7xbrdbhbzraoyjm4islmlh/m6oaslb3swz6ckknusfsgy3xboko7yjpirg9xft+1zpgrl2xsobso/mm+xuemlov91br/n4ga8msbna1vr7ssbuuhg89jexfydq1rrxklqqsy6wmmq+xhh8ufjtwav7qjqjz6yigxyh04escqeceddupouixm4ulrnanez6ppjctter2ucf40aobbeouqvtm63z14fd4pvqdr5hou87sedivpfqbw+w4rvlpghi1ovam2jrvcmfxhrx6tbbhisjbg51i7aeycfellk8rcc33i/zf3zdv9gtrafegvg0eywnykg54dy1qsf+3x73zfzrv906+y91og1uz+lcvhur8pgkbflr5zzwy9tbilyvj1qyaeqvpoyzrhldsnnzdhat98x3obusgj2nz2ycehkzciguj7omvy9bbxlo0mz5e/0ger+2cwcdlwwgh6x53hbwd68drrva54lxnemxma1eegoe9ioryyndgbexusrkoz6hyfv5tws1ti1vmx0civtkeub+xxu4d2loa0mlhajmf2yucjvmacvlqwjkud83ewlvghkodzj0xqmowx9c5ov+j6y8eppbzphjvdriimqehcxp2lqgkllpljknyln1zmgsfnqoi1zdnc/wdd4zbjq7ethxs+bcgpaa6wo2zqhu+nrsbrzyl8chfizrjvkikrqv93eppuxojaaynpoudr9pvj+w79v6gpk1n2kla65jxts7h0sc8zpvrk50tsx+fmbda71hmtd1wyrspxkg4gyv/xslbxtxvna40c8yivs34jvojiog1myhqxnsvpgm8hlyzv4mvd2w9chskguw9vpjlthxr8broo/3pgrt2oofle9wxkdhommwvm8j0arppvxcvdz1p/1utzlvm2hkfrz2tamaxcebc5ukvdr9knt36qefebv/e558bi2nv8uevtxfqu0zukwnnavuh9vhgyzi9g/cuzov5c3pzhzkfri7b19rk4gu8vbwkxodoptmyvhzzg/36oz0mjjyoedqxk/rc+vrz9qvrumxdmza4nhowc6ghhtjtzqxsoxjia7ivazqma+q1vf1iq63x7b7oxj66lda9dudmo5h7lqufv+vjbwbezqharvq0h8/supnemuh10t5jzmpjesw0upy9axfex+czyubftuyqqgf0hba6fcwhv8x6fg57olcsnqdi2wlsonytrllfqtkg9gvksuhjmqtffmh3eqpiwmapculeuoq/blu+ox9efolbde1yld9tjlxp4x1xemaxhmhxhdnv+5d0ijm6igywuwd5mmle13gekiwkl0pd5p/ospckilgmlbn77o6at5obvxvam56silnrs7uklafnpvn1xgntgcpc56dtfnfb/xgvdedjhsvnvrye1xuqtaj55cdzfnd0qo1a24h+lctxhfn2wcyzmpii+qq+x0vdinh8xv/agzk/omm/fht3b3r/t7om3sbn3/s/pnm22t/wnt/bxtttpaptc/xhq8drz1fc9f+d+u9w3++9zd77997fo/re99o6fu3gpr31no/9178h/jgm6y=</latexit> Modeling for the Noisy Channel we need to model two probability distribudons: should favor fluent transladons should favor accurate/faithful transladons let s start with <latexit sha1_base64="pw8oji3ujegpi9vjlgbxuppo7ly=">aaa0ohicnvvrct22ezbtw6rekuzn2xk2kqzylkk6jpnmkvfmffu1o3euxzzzsshrqbkhbxjvbsbzmcp+7tp2ifoo3qv4dkkaloroxhybfltylhyxi4ucimfc7u//99zbp/v5l375q7d/vf6b3/7u9394590/fivykof0ezgnof8+iiimlkppjzmj/b7glkrbqr8llh9g+3dtygxlsxo5kohzsukmjvlijfsdv/ofrswprp986mu5e4twqrl1rdfn7gw1/ep27d33yj+hy7kd6krdevte8xxrpufvxf1r/blye+gsiucvnna19x3us3qffx1+sfz2oysn8vb68by/pagsfvxtdt/lszinkf8/f2djf29f/xj2x6j52fhrfo7p3/3wlh/lyznstiyjeej0tf/is4pwycke1ut+kwhbwkss01p4zehkxvmlnfz7w1ateeocw79meqq2s1grvihfggayjxiizdasdlwdlnl8yvnfsqkunat1r+my8wtuofq9iheyflkadxjybrj6ohroqgmttn7rpiajtfkw94zsstqxyz4w/dqyk2lcwnm/ni0tyxg+69eg+zrlscyzfvu4bb5ljvu1foahcikpwrkgzsvyglxrde0wru0xsvp2mn6w46044lcm97wzcdrxjmrkweg8acqgjhdqoam0u4r9dac5p2dcqw1ogpyk/tprbmmwm6pfce9v8vyiyvvau/qgoa2rrmbhszzd/ezsu8bqa6ptgrelmbucdmg4mb5mbryhrapbdhkkn5mhkvdhp6suk5wvkoe7hoyixmuidjipkqkgwctqcj5reza8imu/m1n5yk0jmhsgflcvkiuevlkox7s4x33m1dcw0rnnpibkhg5vndmnxpvmtorkd2jmskpnlnp8apegevaapgkfc5puvpo6wttvpk3rbomimleifkba3/vibiuakxiohdq9qe+isobdkoyqhgkmwutwr6txghekbor8yylgbxhpfto6q3ztojanvzcufec19rryvfnvy+tt7n55/ois0caks0beg3i5onh2zgldoaxjd55u6jhiomz65ugr0dnrcajepl4afvfepty8glwqhp9f0ypn0riudo96qk8eweinmubqwo9fhnvtkhawgljjhndhc+c46t97vckgn00afqu+inao1hn4au/fpqfjamnkssqlgjcupjulwa98gcuxwk2erhc8p+lttoxncwu0tzxpmicjbdbytvutsz63lgvhlmspoelneyf3fbv8couclbflbei7mvlombkpy5tsv5ptuslsmk/zicz9ugmbkade9y7+bab1wfhhvynqwpmkwsuwt9abgosylok4iilfrt9fvndccnbcb7jrqikawbkmulxwcfdndysxdvkw5dhittcllsrbcla8ii5zxjdzed6y56l3dhdw4ilxkilyt201gynbymimwqc4vmdierocwzmhopjinfmtlkxgckfdfeev3/ddxng/evpba1bkziehf2k36ygsbboinmewqeqenjlfeijhncunana7s/rgjvz6vgky6wxwhezfhen2b7pf7e5qqrejdgkcworbwtwb7fyppj4vohm8pd9z3lbb2ilp1r4gtmtbmqpnxbsvkoduldlumt073ki5ven14ek7dvi2wnvnt5st9hcyi+sfb2xwru0r8d7mxmhzwlqpvzp6crw07tdkz85jjhobmax00z63tmxauelntkjrqoojd0kqtj10z2dikbv8sgaaijqknlqqkjuqkyfciqqtbipmjoh3jdml9cu7buhv3clg22hn0o5olgsxpjzlpiichnnmzj+wkjyeud9auwzdnnbi0nakgfymlfrxvodydsls+znh99mw4bbszliq47cjllvj6ymykkywlo7zna8b8ozfkteynoimtkzotx+vkn2u8pimsyyemo5vkjvfhqetdkd7i5pasnhuvub/cyownqknnlyedpry05ygrphf/se4kun/zbmwlcjamcszegbchh3ydgx70qqsfayfxrqk4g1gnjmbggktnkr5ytv84wqgxlnjxhvgz1s2zfz4fnqnvfo04ccnvb96ih3mlluzqmr1nho221g52gdew9qmopn+bxplbcmrkrl1ygmxribqdvmuuyglyymn2rf8s9+9fqzxoczerzk2cklw7dqksfw9c6mwxqe71vptjvlifdamw7ingibw+f8gc9jfnf13ac3pkos2k6bksurnvlxqffhk8dn6v6nmvtrdtgpu/3adootp1jufgs6m1sm2lgrhrs+xzcbeouuankax6dy5z9ijzw7sfcwvbikuxbrewintcym1z5omu8cbfvad5bbokpjxk5v0ic+1twxxztzbnssxorgzhw40wexylvsuzqzsyy8dkkycl9xl0hr3vboro9mbf9c5ibinoedixg7ydn2h/chtga3qdsafamvfw/3f1vewejarta4wmscbvvhymgcnyglktusaks2xzgzrgzkbo2bkk7bpzi9ti+tjtsn7pgbktrs4bn4y6ido8larugz/muynbviwqw6q68sxgqo9qepwo/cipjj/gxltdllb3/xe20fe6tlhpv1/ayq2areeamagldocxpf2rk/afw7whtkmpgvohffkrvs7lxijstpncifajcozilxwn2no1m7pxk9kanfjjbsn2frdcpqsesdqurqvtl/l4tbeyx7lph0f5qkialjhmrkpmco9503auexhuqd18vgsgcz118unwph69o6zkcaoxi+rjbuke0wzylm4zisctsyxnozo6/4yjctjva50lpuyfjcbwzws7desbsbqmllhipuyfmomsk8mgl6vvalzpilvhzkbedg9dycv1egfjffexf0iohimfglygkj0j8jgilsgmajkpawuxftxuylx4pmc0qzqo6/rvkdxt47w0crs0nvujzlkfuoywfyljolqkrypkq4ttvgsnjxktpw01fqytmkcr2ra3kncynvnpi+gok4n1kvna8bib8bjpop6ww4ba6pz0qhleua7phv2zurfodymlcaumwwj2ruke+vk0hpsbnpcdkgdqprpmnvqqxkehkbuczzwj4w95jlqntrgeabpdmw+i0k1w11zixayrms34lrfkmydb1ze0yac68aguzxg7usq0u8nnmsifjzhlwb0qvd9q1zz8+ngskfiz+zkmzs9xvqvreouzu0afsxlknfwwiyyfxikanuxyo6wewdhqmgeqmoezntdire2opgfh8enxkmgjeoggvgnxjto0wsjh8y5vwazw6ay4ztcz0rl6lkwivzzuqpwksgguguzt/z1rztkbskst6ku4py4zgyz1ckp9mbowsfjkhrz1wmr/jzksedyt5vcl5lo5czpauwc8otmp1xpdm7xbrdbhbzraoyjm4islmlh/m6oaslb3swz6ckknusfsgy3xboko7yjpirg9xft+1zpgrl2xsobso/mm+xuemlov91br/n4ga8msbna1vr7ssbuuhg89jexfydq1rrxklqqsy6wmmq+xhh8ufjtwav7qjqjz6yigxyh04escqeceddupouixm4ulrnanez6ppjctter2ucf40aobbeouqvtm63z14fd4pvqdr5hou87sedivpfqbw+w4rvlpghi1ovam2jrvcmfxhrx6tbbhisjbg51i7aeycfellk8rcc33i/zf3zdv9gtrafegvg0eywnykg54dy1qsf+3x73zfzrv906+y91og1uz+lcvhur8pgkbflr5zzwy9tbilyvj1qyaeqvpoyzrhldsnnzdhat98x3obusgj2nz2ycehkzciguj7omvy9bbxlo0mz5e/0ger+2cwcdlwwgh6x53hbwd68drrva54lxnemxma1eegoe9ioryyndgbexusrkoz6hyfv5tws1ti1vmx0civtkeub+xxu4d2loa0mlhajmf2yucjvmacvlqwjkud83ewlvghkodzj0xqmowx9c5ov+j6y8eppbzphjvdriimqehcxp2lqgkllpljknyln1zmgsfnqoi1zdnc/wdd4zbjq7ethxs+bcgpaa6wo2zqhu+nrsbrzyl8chfizrjvkikrqv93eppuxojaaynpoudr9pvj+w79v6gpk1n2kla65jxts7h0sc8zpvrk50tsx+fmbda71hmtd1wyrspxkg4gyv/xslbxtxvna40c8yivs34jvojiog1myhqxnsvpgm8hlyzv4mvd2w9chskguw9vpjlthxr8broo/3pgrt2oofle9wxkdhommwvm8j0arppvxcvdz1p/1utzlvm2hkfrz2tamaxcebc5ukvdr9knt36qefebv/e558bi2nv8uevtxfqu0zukwnnavuh9vhgyzi9g/cuzov5c3pzhzkfri7b19rk4gu8vbwkxodoptmyvhzzg/36oz0mjjyoedqxk/rc+vrz9qvrumxdmza4nhowc6ghhtjtzqxsoxjia7ivazqma+q1vf1iq63x7b7oxj66lda9dudmo5h7lqufv+vjbwbezqharvq0h8/supnemuh10t5jzmpjesw0upy9axfex+czyubftuyqqgf0hba6fcwhv8x6fg57olcsnqdi2wlsonytrllfqtkg9gvksuhjmqtffmh3eqpiwmapculeuoq/blu+ox9efolbde1yld9tjlxp4x1xemaxhmhxhdnv+5d0ijm6igywuwd5mmle13gekiwkl0pd5p/ospckilgmlbn77o6at5obvxvam56silnrs7uklafnpvn1xgntgcpc56dtfnfb/xgvdedjhsvnvrye1xuqtaj55cdzfnd0qo1a24h+lctxhfn2wcyzmpii+qq+x0vdinh8xv/agzk/omm/fht3b3r/t7om3sbn3/s/pnm22t/wnt/bxtttpaptc/xhq8drz1fc9f+d+u9w3++9zd77997fo/re99o6fu3gpr31no/9178h/jgm6y=</latexit> how do we compute the probability of an English sentence? language modeling is an important part of MT
Noisy Channel
Noisy Channel predicted transladon source sentence
Noisy Channel assumes we have the right model, and that we esdmate it perfectly
Noisy Channel assumes we have the right model, and that we esdmate it perfectly
Noisy Channel assumes we have the right model, and that we esdmate it perfectly extra parameters to tune, can tune to opdmize BLEU
Noisy Channel assumes we have the right model, and that we esdmate it perfectly extra parameters to tune, can tune to opdmize BLEU tuning
Noisy Channel à Linear Model? since we re not using idealized decoding rule anymore, why not add more feature funcdons? word count feature :
Noisy Channel à Linear Model? since we re not using idealized decoding rule anymore, why not add more feature funcdons? word count feature :
Noisy Channel à Linear Model? since we re not using idealized decoding rule anymore, why not add more feature funcdons? word count feature : reverse transladon model feature :
African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦
African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe
African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score model score
African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score African NaDonal Congress opposidon sancdons against Zimbabwe African sancdoning to Zimbabwe s opposing predicted transladon opposidon to sancdons against Zimbabwe African NaDonal Congress model score
African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 BLEU score Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe learning moves transla1ons leb or right in this plot model score
African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score ideal model model score
African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score Where s the gold standard transla1on? model score
African National Congress opposition sanction Zimbabwe 非国大反对制裁津巴布韦 Gold standard: African NaDonal Congress opposes sancdons against Zimbabwe BLEU score Issue: gold standard transla1on is oben unreachable by the model Why? limited transla1on rules, free transla1ons, noisy data model score
Free TranslaDons Machine transla1on: Sharon's office said, leader of the main opposidon Labor Party has admifed defeat and congratulatory telephone calls to Sharon. Human- generated transla1on: According to a representadve of Sharon's office, the leader of the main opposidon Labor Party has admifed defeat and made the obligatory congratuladng telephone call to Sharon.
Free TranslaDons Even if gold standard transla1on was Machine reachable transla1on: by model, we might not Sharon's office said, leader of the main opposidon Labor Party has want admifed to defeat learn and from congratulatory it directly telephone calls to Sharon. Human- generated Applicable transla1on: to other tasks: According to a representadve of Sharon's office, the leader of the main opposidon summariza1on Labor Party has admifed defeat and made the obligatory congratuladng telephone call to Sharon. image cap1on genera1on
Loss FuncDons name loss where used cost ( 0-1 ) issue: gold standard transla1on is oben unreachable by the model intractable, but underlies direct error minimizadon perceptron hinge log perceptron algorithm (Rosenblaf, 1958) support vector machines, other large- margin algorithms logisdc regression, condidonal random fields, maximum entropy models 33
Loss FuncDons name loss where used cost ( 0-1 ) intractable, but it doesn t need to compute model score of gold standard! intractable, but underlies direct error minimizadon perceptron hinge log perceptron algorithm (Rosenblaf, 1958) support vector machines, other large- margin algorithms logisdc regression, condidonal random fields, maximum entropy models 34
MERT, Och (2003)
Minimum Error Rate Training (MERT) minimize the cost of the decoder output how bad are these transladons? e.g., negadve BLEU intractable in general how can we solve it? set of source sentences references decoder outputs
Minimum Error Rate Training (MERT) minimize the cost of the decoder output how bad are these transladons? e.g., negadve BLEU intractable in general how can we solve it? set of source sentences generate k- best lists of transladons, approximately references minimize cost decoder on k- best lists, outputs repeat with new parameters (pool k- best lists across iterates)
BLEU model score
BLEU each point is a transla1on for the same sentence Arabic- English, phrase- based model score
BLEU 10,000- best list, default Moses weights 1- best: 28 BLEU model score
BLEU same sentence, 10,000- best list aber MERT 1- best: 34 BLEU model score
BLEU another sentence, default Moses weights 1- best: 46 BLEU model score
BLEU same sentence, aber MERT 1- best: 62 BLEU model score
Why are there horizontal bands? BLEU model score
Why are there horizontal bands? BLEU latent deriva1ons, different transla1ons with same BLEU model score
Minimum Error Rate Training (MERT) What are some issues with this loss funcdon? DisconDnuous & non- convex opdmizadon relies on randomized search No regularizadon leads to overfinng As a result, MERT is only effecdve for very small models (<40 parameters)
Many researchers tried to improve MERT: Regulariza+on and Search for MERT (Cer et al., 2008) Random Restarts in MERT for MT (Moore & Quirk, 2008) Stabilizing MERT (Foster & Kuhn, 2009) Issues remain: Be;er Hypothesis Tes+ng for Sta+s+cal MT: Controlling for Op+mizer Instability (Clark et al., 2011) They suggest running MERT 3-5 Dmes due to its instability
Perceptron Loss BLEU score reference model score
Perceptron Loss BLEU score reference model predicdon model score
Perceptron Loss for MT? (Collins, 2002) BLEU score reference model predicdon model score
k- Best Perceptron for MT (Liang et al., 2006) BLEU score model predicdon model score
k- Best Perceptron for MT (Liang et al., 2006) BLEU score model predicdon model score
k- Best Perceptron for MT (Liang et al., 2006) BLEU score BLEU oracle on k- best list model predicdon model score
Ramp Loss MinimizaDon BLEU score model score
Ramp Loss MinimizaDon BLEU score model predicdon model score
Ramp Loss MinimizaDon BLEU score model predicdon fear transladon model score
Fear Ramp Loss (Do et al., 2008) BLEU score model predicdon fear transladon model score
Fear Ramp Loss (Do et al., 2008) BLEU score model predicdon gold standard fear transladon model score
Hope Ramp Loss (McAllester & Keshet, 2011; Liang et al., 2006) BLEU score model predicdon model score
Hope Ramp Loss (McAllester & Keshet, 2011; Liang et al., 2006) BLEU score hope transladon model predicdon model score
Hope- Fear Ramp Loss (Chiang et al., 2008; 2009; Cherry & Foster, 2012; Chiang, 2012) BLEU score hope transladon fear transladon model score
Hope- Fear Ramp Loss (Chiang et al., 2008; 2009; Cherry & Foster, 2012; Chiang, 2012) BLEU score hope transladon fear transladon model score
Experiments (Gimpel, 2012) averages over 8 test sets across 3 language pairs Moses Hiero (%BLEU) (%BLEU) MERT 35.9 37.0 Fear Ramp (away from bad) 34.9 34.2 Hope Ramp (toward good) 35.2 36.0 Hope- Fear Ramp (toward good + away from bad) 35.7 37.0
Pairwise Ranking OpDmizaDon (Hopkins & May, 2011) BLEU score model score
Roadmap words, morphology, lexical semandcs text classificadon simple neural methods for NLP language modeling and word embeddings recurrent/recursive/convoludonal networks in NLP sequence labeling, HMMs, dynamic programming syntax and syntacdc parsing machine transladon semandcs 67
SemanDcs we talked a lot about lexical semandcs: word sense, WordNet, word embeddings/clusters how about semandcs beyond the word level?
ComposiDonal SemanDcs how should the meanings of words combine to create the meaning of something larger? there s currently a lot of work in producing vector representadons of sentences and documents simplest case: how should two word vectors be combined to create a vector for a bigram? lots of work in this area in the neural era, but earlier work began ~2007 69
EvaluaDng ComposiDonal SemanDcs compute similarity of two bigrams under your model, then compute correladon with human judgments: BigramSim BigramPara television programme tv set 5.8 1.0 training programme educadon course 5.7 5.0 bedroom window educadon officer 1.3 1.0 (Mitchell and Lapata, 2010) 70
71
Bigram ComposiDon FuncDons 72
Bigram Similarity Results 73
Why does muldplicadon work? these vectors are built from co- occurrence counts (like in the first part of Assignment 1) so element- wise muldplicadon is like performing an AND operadon on context counts when using skip- gram word vectors (or other neural network- derived vectors), addidon oven works befer 74
75
Results SDS = simple distribudonal semandc NLM = neural language model 76
currently there is a lot of work on designing funcdonal architectures for bigram, phrase, and sentence similarity e.g., word averaging, recurrent neural networks, LSTMs, recursive neural networks, etc. our recent results find that, for sentence similarity, word averaging is a surprisingly strong baseline 77
on similar data to training data, LSTM does best 78